A few weeks back, I posted about my adventures in writing a kernel using the Nim programming language. Well, I’m still working at it, and I thought it would be fun to give a progress report. Fair warning: this is going to be one of the most technical posts I’ve written in a long time. If you’re not familiar with a lot of programming and OS terminology, you’re going to have a hard time following along.
The language
Let’s start by looking at my language of choice. Nim is an odd duck in the world of programming languages. In purpose, it sits in that mid tier between low-level languages such as C and “application” languages like Python. This middle space used to be the sole domain of C++, but recent years have seen a growing crop of contenders: Rust, Go, D, Vala, Swift, Zig, and so on.
Nim is definitely one of those. Syntactically, it shares a lot in common with Python, most notably its indentation-based structure. But it’s much closer to the metal. Since it compiles to (a very cryptic subset of) C rather than some kind of VM bytecode, you get a lot of optimizations for free, thanks to the GCC and Clang teams. Thus, you’ve got this great mix of high-level sugar and low-level power, which is really what I was looking for all along. And the Nim community, unlike Rust, does it without sacrificing basic scientific facts such as sexual dimorphism!
Still, being a good programming language—even a lower-level one—doesn’t make something good for writing an operating system. That’s the downside of D, for example; there, the language itself is solid, but its standard library relies on garbage collection, making it a no-go.
I bring up that specific example for a very good reason: Nim’s standard library just works. It’s almost all “pure” code, where the devs eat their own dogfood. The system module is hard-coded to use what amounts to a set of compiler intrinsics, but everything else is built off them. In an OS kernel, where you can’t expect to have a bulky runtime available, this is a dream come true. I only had to implement a dozen simple C functions (strlen, memcpy, etc.), hook in an allocator (liballoc is a good default for “hobby” OSes), and that was it. I don’t even have all the hardware initialized yet, but I already have access to dynamic arrays, hash tables, string formatting, and all those other goodies.
Of course, nothing’s ever perfect. Nim gets very verbose when you’re working so close to bare metal. The developers’ insistence on defaulting literal values to signed integers is a pain, because anyone who has ever worked at the assembly level knows that you have to use unsigned numbers for things like bitfields. Also, converting between integers and pointers (another thing absolutely necessary in OS programming, and absolutely antithetical to the “safe code” movement) is overly verbose. Yeah, I could use a template or macro or something, but…ugh.
The system
I’m going to continue with this project until I get bored or run out of ideas. Since building the bare-bones kernel in the previous post, I’ve expanded its scope. Now, I’m planning out a microkernel OS centered around a message passing interface. The catch is that it’s intended to be a single-user system; there will be “profiles” for multiple users to store their own programs, files, and so on, but only one user will be running it at a time. Other users’ data will be hidden away, though I do envision a kind of shared space.
Another design concept I’ve been toying with is doing away with processes. They’ll just be threads that don’t have a parent instead. So running a program will start a “main” thread, and that thread can then create children or siblings. Child threads inherit some state, and the parent has some direct control of their lifecycle. Siblings, on the other hand, are independent. This also affects IPC: parents and their children can use shared memory far more easily, and the design will reflect this.
The microkernel structure means that very little will run in kernel-space. The physical and virtual memory allocators are already in place, though I may redesign them as time goes on. Some hardware abstraction exists; I’ll need lots more before I can even consider a 0.1 release. I’m currently working out how I want to write the scheduler and mapping out system calls. Almost everything else will live in user-space. There’s no reason not to.
I’m calling this project Concerto. As with most of my works, that’s a name with multiple meanings. A concerto is, of course, a kind of musical composition where many instruments support a single lead—this is, to my eyes, essentially the musical equivalent of a microkernel. It also connotes many working together (i.e., in concert) to create something grand. And I can’t deny a bit of a political jab: concertos are a distinctly Western form of music that came from the era of Enlightenment. As our enemies insist on dragging us into a new Dark Age and the destruction of our heritage, every reminder of what we have built is welcome.
So that’s what I’ve been doing in the free time that is no longer as copious as it used to be. I’ll let you know how it turns out.