Tag: nim

A few weeks back, I posted about my adventures in writing a kernel using the Nim programming language. Well, I’m still working at it, and I thought it would be fun to give a progress report. Fair warning: this is going to be one of the most technical posts I’ve written in a long time. If you’re not familiar with a lot of programming and OS terminology, you’re going to have a hard time following along.

The language

Let’s start by looking at my language of choice. Nim is an odd duck in the world of programming languages. In purpose, it sits in that mid tier between low-level languages such as C and “application” languages like Python. This middle space used to be the sole domain of C++, but recent years have seen a growing crop of contenders: Rust, Go, D, Vala, Swift, Zig, and so on.

Nim is definitely one of those. Syntactically, it shares a lot in common with Python, most notably its indentation-based structure. But it’s much closer to the metal. Since it compiles to (a very cryptic subset of) C rather than some kind of VM bytecode, you get a lot of optimizations for free, thanks to the GCC and Clang teams. Thus, you’ve got this great mix of high-level sugar and low-level power, which is really what I was looking for all along. And the Nim community, unlike Rust, does it without sacrificing basic scientific facts such as sexual dimorphism!

Still, being a good programming language—even a lower-level one—doesn’t make something good for writing an operating system. That’s the downside of D, for example; there, the language itself is solid, but its standard library relies on garbage collection, making it a no-go.

I bring up that specific example for a very good reason: Nim’s standard library just works. It’s almost all “pure” code, where the devs eat their own dogfood. The system module is hard-coded to use what amounts to a set of compiler intrinsics, but everything else is built off them. In an OS kernel, where you can’t expect to have a bulky runtime available, this is a dream come true. I only had to implement a dozen simple C functions (strlen, memcpy, etc.), hook in an allocator (liballoc is a good default for “hobby” OSes), and that was it. I don’t even have all the hardware initialized yet, but I already have access to dynamic arrays, hash tables, string formatting, and all those other goodies.

Of course, nothing’s ever perfect. Nim gets very verbose when you’re working so close to bare metal. The developers’ insistence on defaulting literal values to signed integers is a pain, because anyone who has ever worked at the assembly level knows that you have to use unsigned numbers for things like bitfields. Also, converting between integers and pointers (another thing absolutely necessary in OS programming, and absolutely antithetical to the “safe code” movement) is overly verbose. Yeah, I could use a template or macro or something, but…ugh.

The system

I’m going to continue with this project until I get bored or run out of ideas. Since building the bare-bones kernel in the previous post, I’ve expanded its scope. Now, I’m planning out a microkernel OS centered around a message passing interface. The catch is that it’s intended to be a single-user system; there will be “profiles” for multiple users to store their own programs, files, and so on, but only one user will be running it at a time. Other users’ data will be hidden away, though I do envision a kind of shared space.

Another design concept I’ve been toying with is doing away with processes. They’ll just be threads that don’t have a parent instead. So running a program will start a “main” thread, and that thread can then create children or siblings. Child threads inherit some state, and the parent has some direct control of their lifecycle. Siblings, on the other hand, are independent. This also affects IPC: parents and their children can use shared memory far more easily, and the design will reflect this.

The microkernel structure means that very little will run in kernel-space. The physical and virtual memory allocators are already in place, though I may redesign them as time goes on. Some hardware abstraction exists; I’ll need lots more before I can even consider a 0.1 release. I’m currently working out how I want to write the scheduler and mapping out system calls. Almost everything else will live in user-space. There’s no reason not to.

I’m calling this project Concerto. As with most of my works, that’s a name with multiple meanings. A concerto is, of course, a kind of musical composition where many instruments support a single lead—this is, to my eyes, essentially the musical equivalent of a microkernel. It also connotes many working together (i.e., in concert) to create something grand. And I can’t deny a bit of a political jab: concertos are a distinctly Western form of music that came from the era of Enlightenment. As our enemies insist on dragging us into a new Dark Age and the destruction of our heritage, every reminder of what we have built is welcome.

So that’s what I’ve been doing in the free time that is no longer as copious as it used to be. I’ll let you know how it turns out.

I’ve been fascinated by operating systems for a very long time. For someone who genuinely loves low-level programming, they’re the lowest you can get in our modern age, barring a few microcontroller applications. So I’ve spent the occasional weeks over the past 20 or so years looking into the field, wondering if there’s a way to make my mark on it. At the same time, I’ve been looking at ways to write those low-level programs.

Combining those two threads of research has led me to create Nim Limine Barebones.

What is it?

It’s pretty simple. This is a port of the Limine Bare Bones tutorial kernel to the Nim programming language. It doesn’t do much; it’s literally the “Hello World” of OS development. But it can be used as the start of something much greater.

Why Nim?

I’ve looked at a lot of different languages that purport to be suitable for low-level “systems” programming. I settled on Nim because I wanted to learn something new, but also because every other option has a flaw.

• C is the gold standard for OS work, but it’s really a horrible language. Especially when you don’t have the luxury of an operating system to protect you from, say, filling every byte of memory with garbage.

• C++ is one of my favorite languages anyway, but using it on bare metal is harder than you might think. A freestanding implementation has to throw out most of the standard library (i.e., the bits that make C++ worth using), so you’re mostly left with C plus classes.

• Rust might be a decent language, but its syntax is as ugly as the politics of its developers. I’d never use it for an unpaid project.

• Go, from all I’ve read, requires a hefty runtime to get started, and Google has no inclination to change that. It’s also a language I find annoying for some irrational reason.

• D is pretty much the opposite of Rust in both politics and syntax. It would be a great choice if not for a bit of runtime you just can’t get rid of. Oh, and the fact that nobody can seem to agree on what, exactly, the language should be.

• Zig looks okay, and I found it extremely interesting when I delved into it a few months back. Alas, it’s just too immature for production use, and the latest revisions of the compiler have completely removed some necessary options for bare-metal development.

Nim isn’t perfect by any means, but what I’ve seen so far makes it look like a good “better C” that doesn’t require too many hoops to get the runtime out of the way. For application development, it wouldn’t beat out C++ for me—things like multiple inheritance are just too useful—but at the OS level? Sure beats trying to write my own std::vector. (Seriously. Where are the minimal STL implementations to go along with mlibc?)

Why Limine?

Most OS tutorials are centered on Multiboot. After all, it is kind of a standard. Here, though, I went with Limine. It’s a little more obscure, and much newer; the project is only a few months old here at the start of 2023, as it is intended to replace the older Stivale bootloader.

Limine has a lot of advantages, in my opinion. It’s entirely 64-bit. It sets up a call stack for you, which mostly cuts out the need for assembly in the boot phase. Framebuffers are sensible, there’s an integrated terminal that can work until your own is ready, and it’s just nice in general.

That said, it does have its annoyances. It requires a “higher half” kernel, and that makes paging a necessity sooner than it should be. But the page tables Limine gives you are intentionally sparse. And for this project in particular, dealing with an array of function pointers is just awful. Surely there’s a better way.

Conclusion

All told, I’m happy with what I wrote. It’s a good start, and it fills a niche that nobody else was really looking at. Yes, there’s another barebones Nim kernel out there, and I took inspiration from it. I like to think I’ve provided a better starting point for myself and anyone who would like to follow in my footsteps.