Harald Sitter's KDE Blog

Åkademy, Okademy, in Berlinemy!

Fri, 19 Sep 2025 17:29:19 +0200

Akademy 2025 is history. What an Akademy it was - and it was grand!

Akademy is KDE’s annual conference. This year it happend to be in Berlin, so I hopped on a short 🤥 10 hour train to the north to meet friends - old and new - and discuss the latest ventures of KDE.

The biggest topic of them all was, of course, KDE Linux. Once again we sent the conference buzzing. This year with the news of it entering alpha status. There were bananas a plenty, inspired by the original codename of the distribution: project banana. We even had a self defense course against fruit: How to defend yourself against someone armed with a banana.

Talks were great all around. Well done everyone!

As is tradition the ad-hoc hallway track was well attended and yielded many useful results.

The weekend closed out with a great social bash on Sunday at c-base, a crashed space station in the middle of Berlin! We had fun, pizza, and discussed input methods as well as the ultimate question of life, the universe, and everything, before ending the night at a Späti, Berlin’s characteristic late night one-stop shops for everything from cheap beer to expensive beer.

During the week we had Birds of a Feather sessions, our informal discussion format. Every day we had naturally something KDE Linux related to keep our fresh fruit intake up.

On Monday we mused on shared immutable distro topics and resolved to lean more onto kde-builder respectively our repo-metadata as original source of truth for packaging information. We discussed automatic data migration onto a different machine, considered the various types of data a user might want to migrate, and how to even implement this. Plasma’s solution for backups is a bit wanting. We think it’d be a good idea to double down on Kup, an existing backup solution, to produce a tidier, more integrated backup experience.

Volunteers welcome!

On Tuesday and Thursday we rendered a whole bunch of decisions on KDE Linux issues that were in need of decision making. No large changes from the status quo though. Well, the biggest change is that we now have manpages. For now 🥸

Wednesday was the traditional day trip. It took us on a scavenger hunt through Berlin in an attempt to secure as many scavenged points as possible. I regretfully don’t know which team won as I got distracted by lunch, but it was great fun all the same. In the afternoon Aleix and I took to a tea house for some Ostfriesentee and light afternoon hacking.

To close the event on Thursday, Eike, Aleix and I headed to a pub for tomfoolery and baby guinesses.

Thanks so much to everyone who attended, the Akademy team for organizing, the sponsors and the KDE e.V. for financing, and Techpaladin LLC for sponsoring my attendance.

Maybe next year we will learn how to defend yourself against a pointed stick.

Filelight Speed on Windows

Fri, 08 Aug 2025 19:25:17 +0200

As of a couple days ago Filelight on Windows is performing its search many times faster than before. Go check it out!

https://apps.kde.org/filelight/

It’s shocking that it was slow to begin with… A few years ago I rewrote the Filelight file walking logic to use native API for Linux and Windows. I thought I did a pretty good job of it and speed wasn’t too shabby. You can imagine my surprise when a friend recently pointed out that a similar piece of software performed maybe even better. Obviously this couldn’t stand. I immediately went to profile the situation and indeed Filelight wasn’t as fast as expected.

It turns out there are multiple competing APIs for querying file information on Windows and depending on which gets used performance may not be what you would expect. There’s a good write-up at https://www.cppstories.com/2024/cpp-query-file-attribs-faster/ in case you care to read more.

Long story short: by not calling slow API, Filelight is now heaps faster than before. Scans that took multiple seconds before now finish in the blink of an eye.

In case you are wondering just how much faster it is:

New

Old

Enjoy!

Plasma in Graz

Mon, 21 Apr 2025 18:37:37 +0200

This week Plasma developers have descended on the lovely Austrian city of Graz! 🍦

During the 2025 Plasma Sprint in Graz we’ll be discussing a range of topic both for desktop and mobile. Of course there’ll also be a fair amount of hacking. If you are in the area, feel free to stop by at Grazer Linuxtage on Saturday April 26th where we will have a booth!

Plasma Crash Course - Sentry

Fri, 10 Jan 2025 14:11:31 +0100

A while ago a colleague of mine asked about our crash infrastructure in Plasma and whether I could give some overview on it. This seems very useful to others as well, I thought. Here I am, telling you all about it!

Our crash infrastructure is comprised of a number of different components.

KCrash: a KDE Framework performing crash interception and prepartion for handover to…
coredumpd: a systemd component performing process core collection and handover to…
DrKonqi: a GUI for crashes sending data to…
Sentry: a web service and UI for tracing and presenting crashes for developers

We already looked at KCrash, coredumpd, and DrKonqi. This week it’s time to look at the final piece of the puzzle — Sentry.

Sentry

Sentry is the service we have on the developer side of a crash to manage incoming reports.

As we’ve learned in the DrKonqi post, a Sentry-specific trace created by the preamble is eventually sent off to our Sentry server. In Sentry, yet more processing happens!

Symbolication

If the user hasn’t explicitly opted into it, there usually aren’t debug symbols available on the user system. This would ordinarily render backtraces useless since they are just a bunch of memory addresses without names or references to our source code. One of the reasons we adopted Sentry is because it can fill in the gaps through a process called symbolication. For symbolication it essentially needs to know all the loaded libraries and can then fetch the relevant debug symbols via the various debuginfod instances out there. Once it has the debug symbols it can resolve that the address 0x2343244 is really the function int main(int argc, char **argv)

Fingerprinting

When a trace has been symbolicated it’s ready for fingerprinting. This tries to merge together the same crash appearing in different submission into one single sentry event. Generally speaking if two submissions have the same trace frames, they are considered the same crash and merged into one event (in practice the rules are a bit more complicated).

Result

Eventually the crash event appears for us to fix.

In a future blog post, I’ll also give a more detailed guide on how to use Sentry to its full potential.

Plasma Crash Course - DrKonqi

Wed, 04 Sep 2024 22:18:38 +0200

Our crash infrastructure is comprised of a number of different components.

KCrash: a KDE Framework performing crash interception and prepartion for handover to…
coredumpd: a systemd component performing process core collection and handover to…
DrKonqi: a GUI for crashes sending data to…
Sentry: a web service and UI for tracing and presenting crashes for developers

We’ve already looked at KCrash and coredumpd. Now it is time to look at DrKonqi.

DrKonqi

DrKonqi is the UI that comes up when a crash happens. We’ll explore how it integrates with coredumpd and Sentry.

Crash Pickup

When I outlined the functionality of coredumpd, I mentioned that it starts an instance of systemd-coredump@.service. This not only allows the core dumping itself to be controlled by systemd’s resource control and configuration systems, but it also means other systemd units can tie into the crash handling as well.

That is precisely what we do in DrKonqi. It installs drkonqi-coredump-processor@.service which, among other things, contains the rule:

WantedBy=systemd-coredump@.service

…meaning systemd will not only start systemd-coredump@unique_identifier but also a corresponding drkonqi-coredump-processor@unique_identifier. This is similar to how services start as part of the system boot sequence: they all are “wanted by” or “want” some other service, and that is how systemd knows what to start and when (I am simplifying here 😉). Note that unique_identifier is actually a systemd feature called “instances” — one systemd unit can be instantiated multiple times this way.

drkonqi-coredump-processor

When drkonqi-coredump-processor@unique_identifier runs, it first has some synchronization to do.

As a brief recap from the coredumpd post: coredumpd’s crash collection ends with writing a journald entry that contains all collected data. DrKonqi needs this data, so we wait for it to appear in the journal.

Once the journal entry has arrived, we are good to go and will systemd-socket-activate a helper in the relevant user.

The way this works is a bit tricky: drkonqi-coredump-processor runs as root, but DrKonqi needs to be started as the user the crash happened to. To bridge this gap a new service drkonqi-coredump-launcher comes into play.

drkonqi-coredump-launcher

Every user session has a drkonqi-coredump-launcher.socket systemd unit running that provides a socket. This socket gets connected to by the processor (remember: it is root so it can talk to the user socket). When that happens, an instance of drkonqi-coredump-launcher@.service is started (as the user) and the processor starts streaming the data from journald to the launcher.

The crash has now traveled from the user, through the kernel, to system-level systemd services, and has finally arrived back in the actual user session.

Having been started by systemd and initially received the crash data from the processor, drkonqi-coredump-launcher will now augment that data with the KCrash metadata originally saved to disk by KCrash. Once the crash data is complete, the launcher only needs to find a way to “pick up” the crash. This will usually be DrKonqi, but technically other types of crash pickup are also supported. Most notably, developers can set the environment variable KDE_COREDUMP_NOTIFY=1 to receive system notifications about crashes with an easy way to open gdb for debugging. I’ve already written about this a while ago.

When ready, the launcher will start DrKonqi itself and pass over the complete metadata.

the crashed application
└── kernel
    └── systemd-coredumpd
        ├── systemd-coredumpd@unique_identifier.service
        └── drkonqi-coredump-processor@unique_identifier.service
            ├── drkonqi-coredump-launcher.socket
            └── drkonqi-coredump-launcher@unique_identifier.service
                └── drkonqi

What a journey!

Crash Processing

DrKonqi kicks off crash processing. This is hugely complicated and probably worth its own post. But let’s at least superficially explore what is going on.

The launcher has provided DrKonqi with a mountain of information so it can now utilize the CLI for systemd-coredump, called coredumpctl, to access the core dump and attach an instance of the debugger GDB to it.

GDB runs as a two step automated process:

Preamble Step

As part of this automation, we run a service called the preamble: a Python program that interfaces with the Python API of GDB. Its most important functionality is to create a well-structured backtrace that can be converted to a Sentry payload. Sentry, for the most part, doesn’t ingest platform specific core dumps or crash reports, but instead relies on an abstract payload format that is generated by so called Sentry SDKs. DrKonqi essentially acts as such an SDK for us. Once the preamble is done, the payload is transferred into DrKonqi and the next step can continue.

Trace Step

After the preamble, DrKonqi executes an actual GDB trace (i.e. the literal backtrace command in gdb) to generate the developer output. This is also the trace that gets sent to KDE’s Bugzilla instance at bugs.kde.org if the user chooses to file a bug report. The reason this is separate from the already created backtrace is mostly for historic reasons. The trace is then routed through a text parser to figure out if it is of sufficient quality; only when that is the case will DrKonqi allow filing a report in Bugzilla.

Transmission

With all the trace data assembled, we just need to send them off to Bugzilla or Sentry, depending on what the user chose to do.

Bugzilla

The Bugzilla case is simply sending a very long string of the backtrace to the Bugzilla API (albeit surrounded by some JSON).

Sentry

The Sentry case on the other hand requires more finesse. For starters, the Sentry code also works when offline. The trace and optional user message get converted into a Sentry envelope tagged with a receiver address — a Sentry-specific URL for ingestion so it knows under which project to file the crash. The envelope is then written to ~/.cache/drkonqi/sentry-envelopes/. At this point, DrKonqi’s job is done; The actual transmission happens in an auxiliary service.

Writing an envelope to disk triggers drkonqi-sentry-postman.service which will attempt to send all pending envelopes to Sentry using the URL inside the payload. It will try to do so every once in a while in case there are pending envelopes as well, thereby making sure crashes that were collected while offline still make it to Sentry eventually. Once sent successfully, the envelopes are archived in ~/.cache/drkonqi/sentry-sent-envelopes/.

This concludes DrKonqi’s activity. There’s much more detail going on behind the scenes but it’s largely inconsequential to the overall flow. Next time we will look at the final piece in the puzzle — Sentry itself.

Plasma Crash Course - coredumpd

Wed, 28 Aug 2024 00:08:38 +0200

Our crash infrastructure is comprised of a number of different components.

KCrash: a KDE Framework performing crash interception and prepartion for handover to…
coredumpd: a systemd component performing process core collection and handover to…
DrKonqi: a GUI for crashes sending data to…
Sentry: a web service and UI for tracing and presenting crashes for developers

We’ve looked at KCrash previously. This time we look at coredumpd.

Coredumpd

coredumpd collects all crashes happening on the system, through the core_pattern system. It is shipped as part of systemd and as such mostly available out of the box.

It is fairly sophisticated and can manage the backlog of crashes, so old crashes get cleaned out from time to time. It also tightly integrates with journald giving us a well-defined interface to access crash metadata.

But before we dive into the inner workings of coredumpd, let’s talk about cores.

What are cores?

A core, or more precisely: a core dump file, is a copy of the memory image of a process and its process status (registers, mappings, etc.) in a file. Simply put, it’s like we took a copy of the running process from RAM and stored it in a file. The purpose of such a core is that it allows us to look at a snapshot of the process at that point in time without having the process still running. Using this data, we can perform analysis of the process to figure out what exactly went wrong and how we ended up in that situation.

The advantage is that since the process doesn’t need to be running anymore, we can investigate crashes even hours or days after they happened. That is of particular use when things crash while we are not able to deal with them immediately. For example if Plasma were to crash on logout there’d be no way to deal with it besides stopping the logout, which may not even be possible anymore. Instead we let the crash drop into coredumpd, let it collect a core file, and on next login we can tell the user about the crash.

With that out of the way, it’s time to dump a core!

Core Dumps

We already talked about KCrash and how it intercepts crashes to write some metadata to disk. Once it is done it calls raise() to generate one of those core dumps we just discussed. This actually very briefly turns over control to the kernel which will more or less simply invoke the defined core_pattern process. In our case, coredumpd.

coredumpd will immediately systemd-socket-activate itself and forward the data received from the kernel. In other words: it will start an instance of systemd-coredump@.service and the actual processing will happen in there. The advantage of this is that regular systemd security configuration can be applied as well as cgroup resource control and all that jazz — the core dumping happens in a regular systemd service.

The primary task here is to actually write the dump to a file. In addition, coredumpd will also collect lots of additional metadata besides what is in the core already. Most notably various bits and pieces of /proc information such as cgroup information, mount information, the auxillary vector (auxv), etc.

Once all the data is collected a journald entry is written and the systemd-coredump@.service instance quits again.

The journal entry will contain the metadata as entry fields as well as the path of the core dump on disk, so we can later access it. It essentially serves as a key-value store for the crash data. A severely shortened version looks like this:

Tue 2024-08-27 17:52:27.593233 CEST […]
    COREDUMP_UID=60106
    COREDUMP_GID=60106
    COREDUMP_SIGNAL_NAME=SIGSYS
    COREDUMP_SIGNAL=31
    COREDUMP_TIMESTAMP=1724773947000000
    COREDUMP_COMM=wine64
    COREDUMP_FILENAME=/var/lib/systemd/coredump/core.wine64.….zst
    …

Example

Since this is all rather abstract, we can look at a trivial example to illustrate things a bit better.

Let’s open two terminals. In the first we can watch the journal for the crash to appear.

journalctl -xef SYSLOG_IDENTIFIER=systemd-coredump

In the second terminal we run an instance of sleep in the background, and then trigger a segmentation fault crash.

sleep 99999999999&
kill -SEGV $!

In the first terminal you’ll see the crash happening:

Aug 27 15:01:49 ajax systemd-coredump[35535]: Process 35533 (sleep) of user 60106 terminated abnormally with signal 11/SEGV, processing...
Aug 27 15:01:49 ajax systemd-coredump[35549]: [🡕] Process 35533 (sleep) of user 60106 dumped core.

                                              Stack trace of thread 35533:
                                              #0  0x0000729f1b961dc0 n/a (/lib/ld-linux-x86-64.so.2 + 0x1cdc0)
                                              ELF object binary architecture: AMD x86-64

So far so interesting. “But where is the additional data from /proc hiding?” you might wonder. We need to look at the verbose entry to see all data.

journalctl -o verbose SYSLOG_IDENTIFIER=systemd-coredump

This actually already concludes coredumpd’s work. In the next post DrKonqi will step onto the stage.

Plasma Crash Course - KCrash

Tue, 20 Aug 2024 18:42:58 +0200

Our crash infrastructure is comprised of a number of different components.

KCrash: a KDE Framework performing crash interception and prepartion for handover to…
coredumpd: a systemd component performing process core collection and handover to…
DrKonqi: a GUI for crashes sending data to…
Sentry: a web service and UI for tracing and presenting crashes for developers

We will look at them in turn. This post introduces KCrash.

KCrash

KCrash, as the name suggests, is our KDE framework for crash handling. While it is a mid-tier framework and could be used by outside projects, it mostly doesn’t make sense to, because some behavior is very KDE-specific.

It installs POSIX signal handlers to intercept crash signals and then prepares the crashed process for handover to coredumpd and DrKonqi. More on these two in another post. Once prepared it sends the crash signal into the next higher level crash handler until the signal eventually reaches the default handler and cause the kernel to invoke the core pattern.

Before that can happen, a bunch of work needs doing inside KCrash. Most of it quite boring, but also somewhat challenging.

You see, when handling a signal you need to only use signal-safe functions. The manpage explains very well why. This proves quite challenging at the level we usually are at (i.e. Qt) because it is entirely unclear what is and isn’t ultimately signal-safe under the hood. Additionally, since we are dealing with crash scenarios, we must not trigger new memory allocation, because the heap management may have had an accident.

To that end, KCrash has to use fairy low-level API. To make that easier to work with, there are actually two parts to KCrash:

The Initialization Stage
The Crash Stage

The Initialization Stage

Initialization is generally triggered by calling KCrash::initialize. You may already wonder what kind of initialization KCrash could possibly need. Well, the obvious one is setting up the signal handling. But beyond that the init stage is also used to prepare us for the crash stage. I’ve already mentioned the serious constraints we will encounter once the signal hits, so we had best be prepared for that. In particular we’ll do as much of the work as possible during initialization. This most important includes copying QString content into pre-allocated char * instances such that we later only need to read existing memory. The second most important aspect is the metadata file preparation for use in…

The Crash Stage

Once initialization has happened, we are ready for crashes. Ideally the application doesn’t crash, of course. 😉

But if it does the biggest task is rescuing our data!

Metadata

Inside KCrash we have the concept of Metadata: everything we know about the crashed application: the signal, process ID, executable, used graphics device… and so on and so forth. All this data is collected into a metadata file on-disk in ~/.cache/kcrash-metadata at the time of crash.

Here’s an example file:

[KCrash]
exe=/usr/bin/kwin_wayland
glrenderer=
platform=wayland
appname=kwin_wayland
apppath=/usr/bin
signal=11
pid=1353
appversion=6.1.80
programname=KWin
bugaddress=submit@bugs.kde.org

The actual fields vary depending on what is available for any given application, but it’s generally more or less what is shown in the example.

This metadata file will later be consumed by DrKonqi in an effort to obtain information that only existed at runtime inside the application - such as the version that was running, or whether it was running in legacy X11 mode.

Handoff

Once the metadata is safely saved to disk, KCrash simply calls raise(). This re-raises the signal into the default handler, and through that causes a core dump.

What happens next is up to the system configuration as per the core manpage.

The recommended setup for distributions is that a crash handler be configured as core_pattern and that this handler consumes the crash. We recommend an implementation of the coredumpd and journald interfaces as that will then allow our crash handler to come in and log the crash with KDE.

So that was KCrash, the first in our four-step crash-handling pipeline. In the next blog post I’ll tell you all about the next one: coredumpd.

LLVM and dot graphs

Wed, 01 Nov 2023 15:14:10 +0100

The other day I had to dive into the ksmserver code, the core of session management in KDE Plasma, but found it all a bit difficult to read. If only I could get a callgraph I thought…

Turns out it actually is possible!

In fact it is almost too easy with clang/llvm. There are already solutions that attach LLVM bitcode to existing binaries. e.g. gllvm. gllvm basically is a shim sitting between ninja and the actual compiler and injects some behavior for us. This enables the use without having to refit anything in our existing code. You might note that clazy, our most excellent code analyzer, works in a similar fashion.

Here is how to use it:

# install gllvm (NOTE: as a pre-requisite you should have Go properly set up)
go install -v github.com/SRI-CSL/gllvm/cmd/...@latest

# clone plasma-workspace
git clone https://invent.kde.org/plasma/plasma-workspace
# change directory
cd plasma-workspace
# export environment variables
export CXX=gclang++
export CC=gclang
# You may need to set some additional vars so gllvm can find your compilers
## export LLVM_CC_NAME=clang-16
## export LLVM_CXX_NAME=clazy
# Configure
cmake -S . -B build
# Build
cmake --build build/
# extract the bitcode (into build/bin/ksmserver.bc by default)
get-bc -m -v build/bin/ksmserver
# Run through llvm's opt program to generate a callgraph
opt --disable-output --passes=dot-callgraph build/bin/ksmserver.bc
# copy into working directory
cp build/bin/ksmserver.bc.callgraph.dot callgraph.dot
# prettify the c++ names (this may produce invalid labels, so I'm skipping it here)
## cat build/bin/ksmserver.bc.callgraph.dot | llvm-cxxfilt > callgraph.dot
# generate an SVG of the graph
dot -x -Tsvg -ocallgraph.svg callgraph.dot
# we now have a callgraph.svg \o/

Depending on the complexity of the software and how many libraries it uses this graph may be incredibly large and verbose though. We’ll need to apply some filtering to make it useful. Or at least I thought so when working with ksmserver.

To further filter the graph you can use the gvpr helper. It is a bit clunky but gets the job done. You can build a filter expression to only include interesting functions like so

N{
    if ($.label!="@(*_ZN9KSMServer*)" || $.label=="@(*metacall*)")
        delete($G, $);
}

and then filter the dot graph with

# Filter out uninteresting functions (filter.prog is the filter expression from above)
gvpr -c -f filter.prog build/bin/ksmserver.bc.callgraph.dot > intermediate.gv
# Filter out all empty nodes (they call nothing and are of no interest)
gvpr -c "N[$.degree==0]{delete(root, $)}" intermediate.gv > final.gv
# generate an SVG
cat final.gv | dot -x -Tsvg -ocallgraph.svg

Final result

DrKonqi Developer Notifications

Thu, 07 Sep 2023 07:00:00 +0200

For a while now DrKonqi has a special developer notification system when used in combination with coredumpd. I just realized I never told anyone about it 😅 though.

It’s terribly simple: When used with coredumpd, parts of DrKonqi implicitly look at all crashes that pertain to your current user. Because of that it can also notify on all crashes, not just the KDE-related ones. Obviously it can’t report bugs or anything but sometimes, as a developer, it’s nice to know when things explode.

This is a purely opt-in feature and shouldn’t be enabled unless you know your way around the debugger GDB or really want to know about all the crashes.

To enable the feature simply add KDE_COREDUMP_NOTIFY=1 to your environment. For example using an environment.d file:

mkdir -p ~/.config/environment.d/
echo 'KDE_COREDUMP_NOTIFY=1' >> ~/.config/environment.d/50-$USER.conf

Debugging App Crashes on Windows

Mon, 04 Sep 2023 08:19:58 +0000

The other week the question came up how one can debug an application crash when the Windows Store crash tracking system is unable to produce a usable stack trace. Seemed a good enough opportunity to share some wisdom :)

Generally speaking in order to get a stack trace you first need a minidump. minidumps are kind of like core dumps on POSIX systems, well, except, mini. Acquiring that is should be your first goal.

There are a million ways to get a dump, I’ll highlight two of the easiest that I know of.

Partner Center Dumps

Ideally the Microsoft Partner Center will have a dump for you. You can find it usually where the stack trace is as well. To get access to KDE’s applications you need to be a KDE developer and file a sysadmin request. Once you have access you have to head from the Dashboard to Insights then navigate in the left hand pane to Health there use the drop-down to select the application you want to look at. This should give you every bit of information about the application health your heart could desire. You’ll probably also want to switch from viewing the last 72 hours to the last month, unless the application is particularly faulty of course.

Now all you need to do is click on the crash you want to look at, and not get too annoyed over the unknown crashes you can’t do anything about 😡.

At this point you should be able to find a stack trace link and an additional download link. Sometimes the download link is not there, I have no idea why but I’m sure it’s documented somewhere. The download link is what we are after, it contains the minidump along with some other metadata.

User-Mode Dumps

Now, sometimes the partner center is not able to help us for whatever reason. Maybe the download link is missing, maybe it just doesn’t show the crash we are after, maybe the dump on the partner center is useless. Who knows. In that case we need some help from the user. Thankfully it’s not too painful. They need to enable collection of user-mode dumps by creating the registry key HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\Windows\Windows Error Reporting\LocalDumps, which then causes the Windows Error Reporting to throw a minidump into %LOCALAPPDATA%\CrashDumps. The user then needs to reproduce the crash and obtain the dmp file from the aforementioned location.

Debug Symbols

Once you have obtained a minidump it’s time to find us some debug symbols. The sad truth here is that I can’t really help with that. Depending on how your application was built you’ll be able to get PDBs somehow hopefully. They will either float around as PDBs somewhere or at the very least will be available inside the .appxupload or .appxsym zip files. As a general best practice for KDE software I would advise that when you do a binary release to the Windows Store you also release the x86_64-dbg.7z file to download.kde.org so we can get the relevant PDBs when needed.

Tracing

Alright, I hope you had luck with finding your debug symbols, because now it’s time to do some tracing! Whee. You’ll need Microsoft Visual Studio. Any edition will do. File->Open->File... the minidump and you should be greeted by a nice overview of metadata about the crash.

First we’ll want to setup our debug symbols. For that you first want to place your PDBs somewhere in convenient in your file system. I’m lazy and usually just slap them on the Desktop. In Visual Studio you should find the option Set symbol paths in the right hand list of actions. The option opens the settings window on the correct page. Simply hit the ➕ and type out the path where you extracted the PDBs.

Once the symbol paths are set up you can hit Debug with Mixed and off the tracer goes. Slowly, because it needs to download a million symbols. But eventually you’ll arrive at your stack trace.

(nevermind my crazy setup, I was doing some wonky multi threaded debugging earlier and don’t know how to restore the UI 😅)

Hope this helps some!

Hugging Websites

Fri, 18 Aug 2023 16:33:36 +0000

…very hard.

KDE relies heavily on web services, and many of them need to be kept responsive even under strenuous load. I’ve recently had the opportunity to spend some time on load testing one of our websites and would like to share how that worked out.

To properly test things I wanted to have multiple computers make concurrent requests to the service and ensure that the service still performs within acceptable limits. To that end I needed a bunch of servers, software that can pressure the web service, and software that can make sure the service works.

The server bit is the easiest task there… any cloud provider will do.

The software also seemed easy. After very quick research Locust seemed as good a choice as any to poke at the service and make sure it responds. Except, after some pondering I came to realize that this is actually not so. You see, Locust does HTTP performance testing. That is: it makes HTTP requests and tracks their response time / error rate. That is amazing for testing an API service, but when dealing with a website we also care about the javascripty bits on top being responsive. Clearly a two-prong approach was necessary here. On the one hand Locust can put pressure on the backend and then something else can poke the frontend with a stick to see if it is dead. Enter our old friend: Selenium. An obvious choice given my recent work on a Selenium-based application testing framework. The advantage here is that Selenium can more or less accurately simulate a user using the website giving us a fairly good idea about perceived performance being up to spec. Better yet, both Locust and Selenium have master/client architectures whereby we can utilize the cloud to do the work while a master machine just sits there orchestrating the show.

The three building blocks I’ve arrived at are:

A cloud to scale in
Locust for performance testing (that the thing stays responsive)
Selenium for interaction testing (that the thing actually “works”)

I actually thought about showing you some code here, but it’s exceptionally boring. You can go look at it at https://invent.kde.org/sitter/load.

At first I needed to write some simple tests for Locust and Selenium. They were fairly straight forward, a bit of login, a bit of logout. Just to start putting pressure on the server-under-test.

With simple tests out of the way it was time to glue everything together. For this I needed a couple more puzzle pieces. I’ve mentioned that both Locust and Selenium have “server” components that can manage a number of clients. For Locust that is distributed load generation, and for Selenium it’s called a Grid. For convenience I’ve opted to manage them using docker-compose.

The last piece of the puzzle was some provisioning logic for the cloud server to install and start Selenium as well as Locust Workers.

When all the pieces were in place amazing things started happening!

On my local machine I had a Selenium Grid and a Locust master running. Magically, cloud servers started connecting to them as workers and after a while I didn’t have the Selenium and Locust power of one machine, no, UNLIMITED POWER! (cheeky Star Wars reference).

By starting a load test in Locust it was distributed across all available nodes, simulating more concurrent access than one machine would or could ordinarily do.

A simple loop also starts a number of concurrent Selenium tests that get distributed across the available grid nodes.

for i in {1..5}; python3 test.py& done

The end result is a Locust making hundreds of requests per second while Selenium is trying to use the UI. Well, for a while anyway… I naturally wanted to know the limits so I kept bumping the request numbers. At first all was cool.

Response times in the sub 100ms at 300 users is pretty good I think. CPU usage was also at a comfortable level.

So I increased it to 600 users, which was still OKish. But when I started going towards 1200 users the problems started to appear.

In the bottom graph you can see the two bumps from 300 to 600 and then to 1200. What you can see super clearly is how the response time keeps getting poorer and poorer the difference is so enormous that you almost can’t make out the response time changes from 300 to 600 anymore. Eventually the service started having intermittent interruptions when the Selenium tests were also trying to get their work done. Yet CPU and memory were not fully at capacity - In particular the intermittent failure hike is very suspicious. A look at the failures gave the hint: it was running into software limits. I bumped up the limits because the hardware still had leeway, and presto: no more failure spikes even when at 2048 users. Hooray! Response time does suffer though, so in the end there would need to be more reshuffling if that was an expected user count.

Conclusion

Knowing the limits of our services is very useful for a number of reasons, ranging from knowing how many users we can support, to how oversized our servers are for the task they are performing, to whether our service is susceptible to malicious use. Knowing the capabilities and weaknesses of our systems helps us ensure high availability.

To discuss this blog post check out KDE Discuss.

resvg for SVGs in Qt

Sat, 12 Aug 2023 17:18:34 +0000

People keep lamenting how lackluster Qt’s SVG renderer is. It leads to poorly rendered icons and wallpapers and it mostly only implements the SVG Tiny specification. As a weekend project I put together a resvg based image handler replacement. It was super easy because resvg is amazing!

The Plugins

An application that uses QSvgRenderer actually can, for the most part, just use resvg’s Qt header and call it a day. It has a very convenient interface that lets you replace most uses of QSvgRenderer with resvg’s. That’s all lovely, except who wants to port every application to a new library.

Luckily, most applications don’t actually use QSvgRenderer directly, they use it through 2 plugins. The iconengine plugin, providing a QIconEngine. And the imageformats plugin, providing a QImageIOHandler. Understanding when which is used is the first step.

Duh! The icon engine is used for icons! - Well, yes, usually…

The SVG QIconEngine confusingly isn’t actually used on KDE Plasma, instead we route all icon lookups through KIconEngine which does some magic. Ah! But where does it then get the icon from? If we follow the code paths a bit we end up here and after some reading on QImageReader we learn that on KDE Plasma icon lookup is actually not running through the SVG iconengine plugin but the imageformats plugin. Slightly confusing but actually beneficial because it means that icon rendering is ultimately running through the same code paths as regular SVG image reading when e.g. loading an Image in QML.

By creating an imageformats plugin we can replace most uses of QSvgRenderer with resvg without having to touch every application. Hooray!

QImageIOHandler

And the good news keeps on coming. A basic QImageIOHandler is super easy to implement.

    bool canRead() const override;
    bool read(QImage \*image) override;

All we need to do is implement reading

    \*image = ResvgRenderer(device()->readAll(), ResvgOptions()).renderToImage();

That gets us a working plugin. There are some extra features one can and should implement but for the most part that is all that’s needed. You can check out the complete source to see where things are at.

Duel of Plugins

Now that we have a plugin we just need to make Qt actually use ours instead of its own. Unfortunately this is where our luck runs out. Qt appears to have no facilities for manipulating which imageformats plugin is used when there are multiple candidates for the same format. In our case we have qsvg and qresvg both supporting SVG and it appears undefined behavior which of the two gets used. So, for the time being we’ll have to overwrite qsvg to get our plugin to reign supreme. Somewhat unfortunate.

Conclusion

Plugins are cool. Resvg is also cool. Making a resvg plugin is double cool.

You can give it a try at https://invent.kde.org/sitter/qresvgimageiohandler

Here’s what it looks like. On the left hand side you can see the image from a bug report where the image viewer icon is misrendered, on the right you can see the same icon rendered correctly using resvg.

To discuss this blog post head on over to KDE Discuss.

Windows Store Crashes in Sentry

Sat, 05 Aug 2023 08:18:56 +0000

At KDE we make software for many different platforms. One of them is Microsoft Windows. But what if an application crashes on Windows? New tech enables us to track crashes right in Sentry! Time to learn about it.

When an application crashes on Windows the user can submit crash data to Microsoft. Later KDE, as publisher of the app, can retrieve the crashes from there. This is the standard crash handling for the platform and it works incredibly well. What’s more, it means we don’t need to engineer our own custom solution for the entire process. So, that is all lovely.

Alas, since we are rolling out a KDE-wide crash tracking system called Sentry it would be even nicer if we had Windows crashes in there rather than third party service. That is just what I’ve built recently.

Crashes for our Windows applications now get imported into Sentry!

There are two pieces to this puzzle. One is a symstore and one is the actual importer.

Symbol Storage

In the Linux world we not so long ago grew debuginfod, it’s basically a tiny service that lays out debug symbols in a standardized file path so it may be consumed by any number of tools by making HTTP GET requests to well-known URIs.

Windows has something like this called symbol storage. It’s incredibly simple and merely lays out Windows debug information in a well defined path setup for consumption in tools.

To maintain a symstore of our own we use the excellent python module symstore along with some custom rigging to actually retrieve our debug information files.

Import

The second piece is our importer. It’s a trivial program that uses Microsoft’s APIs to retrieve crash data along with the minidump (the dump format used by Windows crashes), converts it into a Sentry payload, and sends it off to our Sentry instance. Sentry then consumes the crash, does symbolication using the debug symbols from our symstore, and generates a crash event.

Support

If you are shipping your application on the Windows Store you should consider publishing your dbg.7z files on download.kde.org (for example here is Okular), simply file a sysadmin ticket same as with any other upload to get this published. Once that is done you can submit a change to our symstore to publish the debug symbols, and our importer to import data into Sentry. If you need help just give me a poke.

If you are publishing builds for Windows but are not yet on the Window Store, you definitely should change that. It increases your application’s reach considerably. Get in touch with sysadmins to get yourself set up.

Importing crashes into Sentry adds another piece towards higher quality, more actionable crash reports. It’s going to be amazing.

Discuss this blog post on KDE Discuss.

KDE Discuss Notifications and Mailing List

Tue, 01 Aug 2023 15:35:41 +0000

Every once in a while I hear developers having trouble staying up to date with KDE Discuss(ions). No great surprise when I then find out they aren’t tracking things (in other words: not subscribed to categories)!

Discourse, the software that powers KDE Discuss, has a number of handy notifications settings that we can use to have the software bring posts to us instead of having to go to the website every once in a while. We can use these options to tailor a good experience for ourselves.

Three setting types are relevant for the notification experience:

Tracking

Tracking preferences control which discussions to track. Settings here range from when automatic tracking happens - such as when you view a topic for a couple minutes it automatically becomes tracked - to ignoring things. You can track categories as well as tags. Say you only care about kdenlive you can simply track the kdenlive tag and you are good to go.

Emails

Emails contain the second most relevant pool of preferences. There are two principal modes of operation:

Forum mode
Mailing list mode

The names should be obvious enough, the functionality probably less so. In forum mode tracking preferences apply and you get emails for posts or threads as configured. In mailing list mode only negative tracking preferences apply! You get an email for every post on the entire site unless you’ve opted to mute something. In both modes you can simply reply by email to post a reply.

Notifications

These preferences have to do with live notifications i.e. if you have a browser open it will throw up a system notification when new posts happen. The settings here aren’t terribly interesting unless you always have a browser tab pointing to KDE Discuss. If you do, you can set a time frame for notifications such that you don’t get notifications when you are at work-work for example.

Recommendations

So, what do I recommend you do?

First off, if you want to keep things low-volume make sure you have the email preference Activity Summary enabled. It sends a summary email once a day with tracked topics. You can disable most everything else. You definitely want to set up tracking for what you want to receive in your summary (unless of course you want to see all threads). e.g. if you only care about dev topics then set the Development category to tracking.

One step up from the summary is actually the default mode of KDE Discuss. By default if you simply configure tracking as desired you’ll get a reasonable mix of notifications and a summary if you haven’t been around in a while.

The most intense setup is to activate mailing list mode. This operates like a mailing list and causes the highest volume of emails. You’ll want to set up email filtering to go along with this. I would definitely not recommend this unless you really care about all discussions on the entire site.

You should also note that you can reply by mail to threads, regardless of whether mailing list mode is enabled. It’s amazing!

And lastly: you can configure most anything, so if you find that you are getting too many emails or too few just head to the preferences and play with some of the options. I’m confident everyone is able to find a good mixture that makes the most out of KDE Discuss :)

Discuss this blog post on KDE Discuss.

Writing Selenium/Appium Tests on Windows

Mon, 24 Jul 2023 12:03:13 +0000

Akademy, KDE’s annual conference, recently took place in Thessaloniki, Greece. Lots of people were super excited about the prospect of getting GUI Testing off the ground based on the Selenium tech I built last year. Since KDE produces cross-platform applications an obvious question arose though…

What about Windows?

It’s surprisingly easy! Indeed the most time consuming part is probably getting your hands on a Windows Development Virtual Machine. Once you have a Windows installation we need to only spin up our toolchain and off we go. Here’s a handy command list:

\# Download and install WinAppDriver: https://github.com/microsoft/WinAppDriver/releases

winget install openjs.nodejs
winget install python.python.3.11

npm install --location=global appium
# restart terminal to apply PATH change
set-executionpolicy -scope currentuser remotesigned # allow script execution
appium driver install --source=npm appium-windows-driver

pip install appium-python-client
appium # start server, needs firewall exception on first start

Before we go further into the nitty gritty of testing on Windows I suggest you read the earlier blog post Selenium + AT-SPI = GUI Testing, since a lot of the concepts are the same regardless of platform.

First let us get our ducks in a row.

What Accerciser is to Linux is inspect.exe to Windows, namely an inspector tool for applications. You can find it in your Windows SDK folder %ProgramFiles(x86)%\Windows Kits\10\bin\10.0.22000.0\x64\inspect.exe or there abouts. Opening it greets you with this beauty:

Ignoring the verbosity for a moment we’ll note that it contains similar information to Accerciser on Linux, albeit in a more flat overview. Most importantly what is called the AutomationId is constructed from QObject objectNames, similar to the Accessible IDs on Linux. This is insofar interesting as it means we have a couple of avenues for cross-platform element locating - specifically we could match elements by their name (e.g. the text of a Label or Button), or more uniquely by their objectName-based ID (applicable to all QObjects).

For the purposes of this post we’ll do some trivial testing on Filelight and try to make it work for both Linux and Windows by using the element names. Relying on objectNames is more reliable but unfortunately requires some retrofitting in the source code. To avoid having to build Filelight on Windows we’ll work with what we have got: names. Let’s write our test. Don’t forget to install Filelight first :)

First thing, as always, is our setup boilerplate

#!/usr/bin/env python3

# SPDX-License-Identifier: MIT
# SPDX-FileCopyrightText: 2023 Harald Sitter <sitter@kde.org>

import unittest
import sys
from appium import webdriver
from appium.options.windows import WindowsOptions
from appium.options.common import AppiumOptions
from appium.webdriver.common.appiumby import AppiumBy
from selenium.webdriver.support.wait import WebDriverWait
from selenium.webdriver.support import expected\_conditions as EC

class SimpleFilelightTests(unittest.TestCase):
    @classmethod
    def setUpClass(self):
        options = WindowsOptions()
        options.app('KDEe.V.Filelight\_7vt06qxq7ptv8!KDEe.V.Filelight')
        self.driver = webdriver.Remote(
            command\_executor='http://127.0.0.1:4723',
            options=options)

    @classmethod
    def tearDownClass(self):
        self.driver.quit()

if \_\_name\_\_ == '\_\_main\_\_':
    unittest.main()

The only really interesting bit here is how we specify the application on Windows. Since Filelight is a store application we can start it by the Application User Model ID (AUMID) instead of a path; this is pretty much the same as starting by-desktop-file-id on Linux.

Now then. With the boilerplate out of the way we can write an incredibly simple test that simply switches pages a bit: If we click on ‘Scan Home Folder’ it should take us to the scan page and clicking there on ‘Go to Overview’ should take us back to the overview page.

    def test\_scan(self):
        self.driver.find\_element(by=AppiumBy.NAME, value="Scan Home Folder").click()
        overview = WebDriverWait(self.driver, 120).until(
            EC.element\_to\_be\_clickable((AppiumBy.NAME, "Go to Overview"))
        )
        overview.click()
        WebDriverWait(self.driver, 4).until(
            EC.element\_to\_be\_clickable((AppiumBy.NAME, "Scan Home Folder"))
        )

Cool. We now can test Filelight on Windows. Next we should try to make this test also work for Linux. Thankfully we only need to switch out our app name for a desktop file id.

    def setUpClass(self):
        if sys.platform == 'nt':
            options = WindowsOptions()
            options.app = 'KDEe.V.Filelight\_7vt06qxq7ptv8!KDEe.V.Filelight'
        else:
            options = AppiumOptions()
            options.set\_capability('app', 'org.kde.filelight.desktop')

Putting it all together we get our final test which runs on both Linux and Windows.

\# Windows
# start appium in a terminal
python .\\test.py

# Linux selenium-webdriver-at-spi-run ./test.py


The complete test code:

#!/usr/bin/env python3

SPDX-License-Identifier: MIT

SPDX-FileCopyrightText: 2023 Harald Sitter sitter@kde.org

import unittest import sys from appium import webdriver from appium.options.windows import WindowsOptions from appium.options.common import AppiumOptions from appium.webdriver.common.appiumby import AppiumBy from selenium.webdriver.support.wait import WebDriverWait from selenium.webdriver.support import expected_conditions as EC

class SimpleCalculatorTests(unittest.TestCase):

@classmethod
def setUpClass(self):
    if sys.platform == 'nt':
        options = WindowsOptions()
        options.app = 'KDEe.V.Filelight\_7vt06qxq7ptv8!KDEe.V.Filelight'
    else:
        options = AppiumOptions()
        options.set\_capability('app', 'org.kde.filelight.desktop')

    self.driver = webdriver.Remote(
        command\_executor='http://127.0.0.1:4723',
        options=options)

@classmethod
def tearDownClass(self):
    self.driver.quit()

def test\_scan(self):
    self.driver.find\_element(by=AppiumBy.NAME, value="Scan Home Folder").click()
    overview = WebDriverWait(self.driver, 120).until(
        EC.element\_to\_be\_clickable((AppiumBy.NAME, "Go to Overview"))
    )
    overview.click()
    WebDriverWait(self.driver, 4).until(
        EC.element\_to\_be\_clickable((AppiumBy.NAME, "Scan Home Folder"))
    )

if __name__ == ‘__main__’: unittest.main()


Discuss this blog post on [KDE Discuss](https://discuss.kde.org/t/writing-selenium-appium-tests-on-windows/3145).

Firefox and KeePassXC Flatpaks

Fri, 07 Apr 2023 12:12:13 +0000

Flatpaks are amazing and all that. But application sandboxing, so an application cannot do anything it wants, is a challenge - even more so when you have two applications that need to talk to each other. Perhaps it shouldn’t come as a surprise that native-messaging sandboxing support for Flatpak has been in development for over a year. To celebrate its anniversary I thought I’d write down how to drill a native-messaging sized hole into the sandbox. This enables the use of native messaging even without portal integration, albeit also without sane degrees of sandboxing.

First off, please understand that this undermines the sandbox on a fairly fundamental level. So, don’t do this if you don’t keep your Firefox updated or visit particularly dodgy websites.

For the purposes of this post I’m assuming Firefox and KeePassXC are installed as Flatpaks in user scope.

First order of business is setting up KeePassXC so it writes its definition file in a place where Firefox can read it. Fortunately it has a setting for this:

~/.var/app/org.mozilla.firefox/.mozilla/native-messaging-hosts/ is the path inside Firefox’ home where the defintion file will be written. Naturally we’ll also need to adjust the Flatpak permissions so KeePassXC can write to this path.

flatpak override --user --filesystem=~/.var/app/org.mozilla.firefox/.mozilla/native-messaging-hosts org.keepassxc.KeePassXC

At this point Firefox knows about the native messaging host but it won’t be able to run it. Alas. We need some rigging here. The problem is that Firefox can’t simply flatpak run the native messaging host, it needs to spawn a host process (i.e. a process outside its sandbox) to then run the KeePassXC Flatpak and that then runs the NMH.

Fortunately the NMH definition files are fairly straight forward:

{"allowed_extensions":["keepassxc-browser@keepassxc.org"],
"description":"KeePassXC integration with native messaging support",
"name":"org.keepassxc.keepassxc_browser",
"path":"/home/me/.local/share/flatpak/exports/bin/org.keepassxc.KeePassXC",
"type":"stdio"}

The problem of course is that we cannot directly use that Flatpak bin but need the extra spawn step in between. What we need is a way to manipulate the definition file such that we can switch in a different path. systemd to the rescue!

systemctl edit --user --full --force keepassxc-native-messaging-mangler.path

# SPDX-License-Identifier: GPL-2.0-only OR GPL-3.0-only OR LicenseRef-KDE-Accepted-GPL
# SPDX-FileCopyrightText: 2023 Harald Sitter <sitter@kde.org>

[Path]
PathChanged=/home/me/.var/app/org.mozilla.firefox/.mozilla/native-messaging-hosts/org.keepassxc.keepassxc_browser.json

[Install]
WantedBy=default.target

and the associated service file…

systemctl edit --user --full --force keepassxc-native-messaging-mangler.service

# SPDX-License-Identifier: GPL-2.0-only OR GPL-3.0-only OR LicenseRef-KDE-Accepted-GPL
# SPDX-FileCopyrightText: 2023 Harald Sitter <sitter@kde.org>

[Unit]
Description=keepassxc mangler

[Service]
ExecStart=/home/me/keepassxc-native-messaging-mangler

lastly, enable the path unit.

systemctl --user enable --now keepassxc-native-messaging-mangler.path

Alright, there’s some stuff to unpack here. KeePassXC on startup writes the aforementioned definition file into Firefox’ NMH path. What we do with the help of systemd is monitor the file for changes and whenever it changes we’ll trigger our service, the service runs a mangler to modify the file so we can run another command instead. It’s basically an inotify watch.

Here’s the mangler (~/keepassxc-native-messaging-mangler):

#!/usr/bin/env ruby
# frozen_string_literal: true

# SPDX-License-Identifier: GPL-2.0-only OR GPL-3.0-only OR LicenseRef-KDE-Accepted-GPL
# SPDX-FileCopyrightText: 2023 Harald Sitter <sitter@kde.org>

require 'json'

file = "#{Dir.home}/.var/app/org.mozilla.firefox/.mozilla/native-messaging-hosts/org.keepassxc.keepassxc_browser.json"
blob = JSON.parse(File.read(file))
blob['path'] = "#{Dir.home}/Downloads/keepassxc"
File.write(file, JSON.generate(blob))

It simply replaces the path of the executable with a wrapper script. Here’s the wrapper script (~/Downloads/keepassxc):

#!/bin/sh

# SPDX-License-Identifier: GPL-2.0-only OR GPL-3.0-only OR LicenseRef-KDE-Accepted-GPL
# SPDX-FileCopyrightText: 2023 Harald Sitter <sitter@kde.org>

exec /usr/bin/flatpak-spawn --host --watch-bus "$HOME/.local/share/flatpak/exports/bin/org.keepassxc.KeePassXC" "$@"

flatpak-spawn is a special command that allows us to spawn processes outside the sandbox. To gain access we’ll have to allow Firefox to talk with the org.freedesktop.Flatpak DBus session service.

flatpak override --user --talk-name=org.freedesktop.Flatpak org.mozilla.firefox

And that’s it!

➡️ KeePassXC writes its NMH definition to Flatpak specific path ➡️ systemd acts on changes and starts mangler ➡️ mangler changes the path inside the definition to our wrapper ➡️ Firefox reads the definition and calls our wrapper ➡️ wrapper flatpak-spawns KeePassXC flatpak ➡️ Firefox (flatpak) talks to KeePassXC (flatpak)

Selenium + AT-SPI = GUI Testing

Wed, 14 Dec 2022 15:59:20 +0000

At KDE we have multiple levels of quality assurance ranging from various degrees of a humans testing features to fully automated testing. Indeed automated testing is incredibly important for the continued quality of our software. A big corner stone of our testing strategy are so called unit tests, they test a specific piece of our software for its behavior in isolation. But for many aspects of our software we need a much higher level view, testing pieces of Plasma’s application launcher in isolation is all good and well but that won’t tell us if the entire UI can be easily navigated using the keyboard. For this type of test we require a different testing approach altogether. A couple months ago I’ve set set out to create a testing framework for this use case and I’m glad to say that it has matured enough to be used for writing tests. I’d like to walk you through the technical building blocks and a simple example.

Let us start of by looking at the architecture at large. So… there’s Selenium which is an incredibly popular, albeit web-oriented, testing framework. Its main advantages for us are its popularity and that it sports a server-client split. This means we can leverage the existing client tooling available for Selenium without having to write anything ourselves, we only need to grow a server. The server component, called a WebDriver, implements the actual interaction with UI elements and is generic enough to also apply to desktop applications. Indeed so thought others as well: there already exists Appium - it extends Selenium with more app-specific features and behaviors. Something for us to build upon. The clients meanwhile are completely separate and talk to the WebDriver over a well defined JSON REST protocol, meaning we can reuse the existing clients without having to write anything ourselves. They are available in a multitude of programming languages, and who knows maybe we’ll eventually get one for writing Selenium tests in QML ;)

That of course doesn’t explain how GUI testing can work with this on Linux. Enter: AT-SPI. AT-SPI is an accessibility API and pretty much the standard accessibility system for use on Linux. Obviously its primary use is assistive technologies, like the screen reader Orca, but to do its job it essentially offers a toolkit-independent way of introspecting and interacting with GUI applications. This then gives us a way to implement a WebDriver without caring about the toolkit or app specifics. As long as the app supports AT-SPI, which all Qt apps do implicitly, we can test it.

Since all the client tooling is independent of the server all we needed to get GUI testing going was a WebDriver that talks to AT-SPI.

That is what I set out to write and I’m happy to announce that we now have an AT-SPI based WebDriver, and the first tests are popping into existence already. There is also lovely documentation to hold onto.

So, without further ado. Let us write a simple test. Since the documentation already writes one in Python I’ll use Ruby this time around so we have some examples of different languages. A simple candidate is KInfoCenter. We can test its search functionality with a couple of lines of code.

First we need to install selenium-webdriver-at-spi, clone it, cmake build it, and cmake install it. You’ll also need to install the relevant client libraries. For ruby that’s simply running gem install appium_lib.

Then we can start with writing our test. We will need some boilerplate setup logic. This is more or less the same for every test. For more details on the driver setup you may also check the wiki page.

  def setup
    @appium\_driver = Appium::Driver.new(
      {
        'caps' => { app: 'org.kde.kinfocenter.desktop' },
        'appium\_lib' => {
          server\_url: 'http://127.0.0.1:4723',
          wait\_timeout: 10,
          wait\_interval: 0.5
        }
      }, true
    )
    @driver = @appium\_driver.start\_driver
  end

The driver will take care of starting the correct application and make sure that it is actually running correctly. Next we’ll write the actual test. Let’s test the search. The first order of business is using a tool called Accerciser to inspect the AT-SPI presentation of the application. For more information on how to use this tool please refer to the wiki. Using Accerciser I’ve located the search field and learned that it is called ‘Search’. So, let’s locate it and activate it, search for the CPU module:

  def test\_search
    search = driver.find\_element(:name, 'Search')
    search.click
    search.send\_keys('cpu')

Next let us find the CPU list item and activate it:

    cpu = driver.find\_element(:class\_name, '\[list item | CPU\]')
    assert(cpu.displayed?)
    cpu.click

And finally let’s assert that the page was actually activated:

    cpu\_tab = driver.find\_element(:class\_name, '\[page tab | CPU\]')
    assert(cpu\_tab.displayed?)

To run the complete test we can use the run wrapper: selenium-webdriver-at-spi-run ./kinfocentertest.rb (mind that it needs to be +x). If all has gone well we should get a successful test.

Finished in 1.345276s, 0.7433 runs/s, 1.4867 assertions/s.

1 runs, 2 assertions, 0 failures, 0 errors, 0 skips
I, \[2022-12-14T13:13:53.508516 #154338\]  INFO -- : tests done
I, \[2022-12-14T13:13:53.508583 #154338\]  INFO -- : run.rb exiting true

This should get you started with writing a test for your application! I’ll gladly help and review your forthcoming tests. For more detailed documentation check out the writing-tests wiki page as well as the appium command reference.

Of course the work is not done. selenium-webdriver-at-spi is very much still a work in progress and I’d be glad for others to help add features as they become needed. The gitlab project is the place for that. <3

The complete code of the example above:

#!/usr/bin/env ruby
# frozen\_string\_literal: true

# SPDX-License-Identifier: GPL-2.0-only OR GPL-3.0-only OR LicenseRef-KDE-Accepted-GPL
# SPDX-FileCopyrightText: 2022 Harald Sitter <sitter@kde.org>

require 'appium\_lib'
require 'minitest/autorun'

class TestKInfoCenter < Minitest::Test
  attr\_reader :driver

  def setup
    @appium\_driver = Appium::Driver.new(
      {
        'caps' => { app: 'org.kde.kinfocenter.desktop' },
        'appium\_lib' => {
          server\_url: 'http://127.0.0.1:4723',
          wait\_timeout: 10,
          wait\_interval: 0.5
        }
      }, true
    )
    @driver = @appium\_driver.start\_driver
  end

  def teardown
    driver.quit
  end

  def test\_search
    search = driver.find\_element(:name, 'Search')
    search.click
    search.send\_keys('cpu')

    cpu = driver.find\_element(:class\_name, '\[list item | CPU\]')
    assert(cpu.displayed?)
    cpu.click

    cpu\_tab = driver.find\_element(:class\_name, '\[page tab | CPU\]')
    assert(cpu\_tab.displayed?)
  end
end

Plasma Analyzer

Mon, 12 Dec 2022 15:26:00 +0000

It’s a Plasma widget that visualizes what’s going on on your system, music-wise that is. I’ve started this project years ago but only recently found the motivation to get it to a somewhat acceptable state. It’s pretty amazing to have bars flying across the screen to Daft Punk’s `Touch`.

https://store.kde.org/p/1953779

KDE Crash Tracking System 💣

Thu, 13 Oct 2022 10:31:32 +0000

KDE is now evaluating Sentry, a crash tracking system.

Who can get access? Everyone with a KDE developer account.

But what is it?

Since forever we have used Bugzilla to manage crash reports but this has numerous challenges that haven’t made any improvements in at least 10 years:

Finding duplicates crashes is hard and in our case involves a human finding them
When debug symbols are missing we need to ask the user to recreate the problem, which is not always possible
Users need to worry about debug symbols (this is in part improved by the rise of debuginfod - yay!)
We have no easily consumed graphs on how prevalent a specific crash is, and by extension we have a hard time judging the importance
The user needs to actually write a report for us to learn of the crash (spoiler: most crashes never get this far)
…

All in all it’s a fairly dissatisfactory situation we are in currently. Enter Sentry.

Sentry is a purpose-built crash tracking system. It receives crash reports via API ingestion points and traces missing frames with the help of debuginfod, can detect duplicates automatically and thus show us particularly aggressive crashes, and much more. Best yet, it supports many different programming languages which allows us to not only improve the quality of our software but also our infrastructure services.

The current evaluation instance is already amazing and helped fix numerous problems, and the current setup is not even using all features yet and we have hampered rollout a bit: only git builds currently submit data. If all goes well and we find it to be amazing I hope we’ll eventually be able to roll this out to production releases.

Let’s look at a crash I’ve fixed recently.

Here’s what Sentry received from the user:

Not terribly useful. So with the power of debuginfod it turned it into this:

I then applied some brain power to create a fix and consequently the crash has disappeared, as we can see in this neat graphic here:

Here’s a complete crash information page from a recent infrastructure problem in our bugzilla bot:

Also check out my Akademy talk:

https://youtu.be/A3LzemYMsIQ?t=15126

KIO Admin

Thu, 04 Aug 2022 19:57:58 +0000

I’ve gotten annoyed with the inability to manage system files so I’ve made a KIO worker that enables Dolphin to view and edit files as root.

This is dolphin viewing the system root with administrative access.

The way this works is actually fairly exciting. It’s pulling off worker chaining: The admin worker itself contains gloriously little logic, all it does is translate all worker calls to dbus calls, and those dbus calls go out to a privileged polkit helper. The polkit helper then translates the URIs from admin:///foo to file:///foo and uses the regular KIO API to recreate the request in root-scope. KIO then, behind the scenes, acts just like it would in dolphin proper, using the existing file worker code to execute the file operations.

The advantages are amazing! It’s fairly little actual code (albeit a lot of boilerplate). Since it’s an ordinary worker on the Dolphin side we can expect all file operations to just work™ because really admin:// is just like trash:// or desktop://. Because ultimately the file worker is actually in charge of doing the work, all things are generally expected to work (it’s the same code that powers regular file operations).

Disadvantageously it’s a fair large portal into root-scope, meaning the worker should really only be used in trusted environments (e.g. with only sandboxed applications on the system ;)). Even with polkit guarding the entrance, once you have given permissions you have to trust the application (e.g. dolphin) to not get exploited.

“But why a dedicated worker instead of integrated polkit support in the file worker?” Why, I’m glad you asked! Integrated polkit support sounds simple but is really rocket science. For example there is currently no good architectural way to “catch” operations that had gone wrong - you try to copy a file to `/srv` and that fails with 🤖PERMISSION DENIED🤖 but there is no consistent way to then go “well, let’s retry this entire operation with privileges then” so without huge code refactoring first, we’d end up tucking fallback logic onto every which error scenario… it’s messy and also easy to miss or mess up edge cases. There are also user experience problems. You’d not want to have every internal operation require dedicated permission, so you kind of have to bundle them up and then request permission for the bundle; but how do you know when a bundle is complete? It’s really frightfully complicated.

In conclusion admin:// is awesome today. Maybe one day integrated polkit will also be awesome.

Reviews and testing appreciated https://invent.kde.org/sitter/kio-admin (mind the readme - this currently wants some patching elsewhere in the stack).

DrKonqi ❤️ coredumpd

Wed, 25 May 2022 19:59:05 +0000

Get some popcorn and strap in for a long one! I shall delight you with some insights into crash handling and all that unicorn sparkle material.

Since Plasma 5.24 DrKonqi, Plasma’s infamous crash reporter, has gained support to route crashes through coredumpd and it is amazing - albeit a bit unused. That is why I’m telling you about it now because it’s matured a bit and is even more amazing - albeit still unused, I hope that will change.

To explain what any of this does I have to explain some basics first, so we are on the same page…

Most applications made by KDE will generally rely on KCrash, a KDE framework that implements crash handling, to, well, handle crashes. The way this works depends a bit on the operating system but one way or another when an application encounters a fault it first stops to think for a moment, about the meaning of life and whatever else, we call that “catching the crash”, during that time frame we can apply further diagnostics to help later figure out what went wrong. On POSIX systems specifically, we generate a backtrace and send that off to our bugzilla for handling by a developer - that is in essence the job of DrKonqi.

Currently DrKonqi operates in a mode of operation generally dubbed “just-in-time debugging”. When a crash occurs: KCrash immediately starts DrKonqi, DrKonqi attaches GDB to the still running process, GDB creates a backtrace, and then DrKonqi sends the trace along with metadata to bugzilla.

Just-in-time debugging is often useful on developer machines because you can easily switch to interactive debugging and also have a more complete picture of the environmental system state. For user systems it is a bit awkward though. You may not have time to deal with the report right now, you may have no internet connection, indeed the crash may be impossible to trace because of technical complications occurring during just-in-time debugging because of how POSIX signals work (threads continue running :O), etc.

In short: just-in-time really shouldn’t be the default.

Enter coredumpd.

Coredumpd is part of systemd and acts as kernel core handler. Ah, that’s a mouthful again. Let’s backtrace (pun intended)… earlier when I was talking about KCrash I only told part of the story. When fault occurs it doesn’t necessarily mean that the application has to crash, it could also neatly exit. It is only when the application takes no further action to alleviate the problem that the Linux kernel will jump in and do some rudimentary crash handling, forcefully. Very rudimentary indeed, it simply takes the memory state of the process and dumps it into a file. This is then aptly called a core dump. It’s kind of like a snapshot of the state of the process when the fault occurred and allows for debugging after the fact. Now things get interesting, don’t they? :)

So… KCrash can simply do nothing and let the Linux kernel do the work, and the Linux kernel can also be lazy and delegate the work to a so called core handler, an application that handles the core dumping. Well, here we are. That core handler can be coredumpd, making it the effective crash handler.

What’s the point you ask? – We get to be lazy!

Also, core dumping has one huge advantage that also is its disadvantage (depending on how you look at it): when a core dumps, the process is no longer running. When backtracing a core dump you are looking at a snapshot of the past, not a still running process. That means you can deal with crashes now or in 5 minutes or in 10 hours. So long as the core dump is available on disk you can trace the cause of the crash. This is further improved by coredumpd also storing a whole lot of metadata in journald. All put together it allows us to run drkonqi after-the-fact, instead of just-in-time. Amazing! I’m sure you will agree.

For the user everything looks the same, but under the hood we’ve gotten rid of various race conditions and gotten crash persistence across reboots for free!

Among other things this gives us the ability to look at past crashes. A GUI for which will be included in Plasma 5.25. Future plans also include the ability to file bug reports long after the fact.

Inner Workings

The way this works behind the scenes is somewhat complicated but should be easy enough to follow:

The application produces a fault
KCrash writes KCrash-specific metadata into a file on disk and doesn’t exit
The kernel issues a core dump via coredumpd
The systemd unit coredump@ starts
At the same time drkonqi-coredump-processor@ starts
The processor@ waits for coredump@ to finishes its task of dumping the core
The processor@ starts drkonqi-coredump-launcher@ in user scope
launcher@ starts DrKonqi with the same arguments as though it had been started just-in-time
DrKonqi assembles all the data to produce a crash report
the user is greeted by a crash notification just like just-in-time debugging
the entire crash reporting procedure is the same

Use It!

If you are using KDE neon unstable edition you are already using coredumpd based crash reporting for months! You haven’t even noticed, have you? ;)

If not, here’s your chance to join the after-the-fact club of cool kids.

KCRASH_DUMP_ONLY=1

in your `/etc/environment` and make sure your distribution has enabled the relevant systemd units accordingly.

kde-inotify-survey

Thu, 05 May 2022 11:27:09 +0000

I’ve finally gotten annoyed enough with inotify failing randomly, because of resource exhaustion, that I’ve built a tiny app to deal with it.

Introducing kde-inotify-survey.

It features a CLI to inspect the inotify state, as well as a kded to warn and help with bumping the maximums. Assuming it turns out amazing for others I hope to progress it to kdereview soon.

Plasma GameMode

Fri, 04 Mar 2022 10:08:06 +0000

For when you are using Feral Interactive’s GameMode and want some UI indication of its use.

https://invent.kde.org/sitter/plasma-gamemode

Reuse Licensing Helper

Tue, 06 Apr 2021 13:03:36 +0000

It’s boring but important! Stay with me! Please! 😘

For the past couple of years Andreas Cord-Landwehr has done excellent work on moving KDE in a more structured licensing direction. Free software licensing is an often overlooked topic, that is collectively understood to be important, but also incredibly annoying, bureaucratic, and complex. We all like to ignore it more than we should.

If you are working on KDE software you really should check out KDE’s licenses howto and maybe also glance over the comprehensive policy. In particular when you start a new repo!

I’d like to shine some light on a simple but incredibly useful tool: reuse. reuse helps you check licensing compliance with some incredibly easy commands.

Say you start a new project. You create your prototype source, maybe add a readme - after a while it’s good enough to make public and maybe propose for inclusion as mature KDE software by going through KDE Review. You submit it for review and if you are particularly unlucky you’ll have me come around the corner and lament how your beautiful piece of software isn’t completely free software because some files lack any sort of licensing information. Alas!

See, you had better use reuse…

pip3 install --user reuse

reuse lint: lints the source and tells you which files aren’t licensed

reuse download --all: downloads the complete license files needed for compliance based on the licenses used in your source (unfortunately you’ll still need to manually create the KDEeV variants)

If you are unsure how to license a given file, consult the licensing guide or the policy or send a mail to one of the devel mailing lists. There’s help a plenty.

Now that you know about the reuse tool there’s even less reason to start projects without 100% compliance so I can shut up about it :)

No SMB1 to Share Devices

Tue, 17 Mar 2020 10:13:00 +0000

As it recently came up I thought I should perhaps post this more publicly… As many of you will know SMB1, the super ancient protocol for windows shares, shouldn’t be used anymore. It’s been deprecated since like Windows Vista and was eventually also disabled by default in both Windows 10 and Samba. As a result you are not able to find servers that do not support either DNS-SD aka Bonjour aka Avahi, or WS-Discovery. But there’s an additional problem! Many devices (e.g. NAS) produced since the release of Vista could support newer versions of SMB but for not entirely obvious reasons do not have support for WS-Discovery-based … discovery. So, you could totally find and use a device without having to resort to SMB1 if you know its IP address. But who wants to remember IP addresses. Instead you can just have another device on the network play discovery proxy! One of the many ARM boards out there, like a rapsberrypi, would do the trick. To publish a device over DNS-SD (for Linux & OS X) you’ll first want to map its IP address to a local hostname and then publish a service on that hostname. avahi-publish -a blackbox.local 192.168.100.16 avahi-publish -s -H blackbox.local SMB_ON_BLACKBOX _smb._tcp 445 If you also want to publish for windows 10 systems you’ll additionally want to run wsdd wsdd.py -v -n BLACKBOX Do note that BLACKBOX in this case can be a netbios, or LLMNR, or DNS-SD name (Windows 10 does support name resolution via DNS-SD these days). Unfortunate caveat of wsdd is that if you want to publish multiple devices you’ll need to set up a bridge and put the different wsdd instances on different interfaces.