diff options
| -rw-r--r-- | content/zosimos/workflow.md | 295 |
1 files changed, 295 insertions, 0 deletions
diff --git a/content/zosimos/workflow.md b/content/zosimos/workflow.md new file mode 100644 index 0000000..2619fe7 --- /dev/null +++ b/content/zosimos/workflow.md @@ -0,0 +1,295 @@ ++++ +title = "Zosimos Part 1: Workflow Setup" +date = 2025-03-31T12:00:00-04:00 +tags = ["osdev", "zig", "raspberry pi"] ++++ + +Programming my own OS has been a dream of mine for years, but my attempts always +end up going the same way: writing some code that can run in a virtual machine, +learning some interesting stuff about x86_64, and then giving up before I even +reach userspace. Now I find myself called again by the siren song, but I'm +determined to not go down the same road again ... so I'm doing it on an arm +machine this time. I ended up settling on the Raspberry Pi 4B, since the price +was reasonable and the large community makes it easy to find answers to any +question I might have. But before I can set out to build the greatest OS ever, +I need to set up a good workflow for building and testing. + +## Building the Kernel + +For this go around, I decided to write my kernel in Zig. I've gotten too used to +the conveniences of a modern language to go back to C, and while I love Rust +dearly I've found myself frustrated by having to use shell scripts and make to +take care of the parts of getting to a bootable image that Cargo doesn't concern +itself with. Zig's build system is flexible enough to handle the whole process +itself, and while nothing in this post is particularly difficult, I have reason +to believe it'll keep up even as the complexity increases. + +I won't go into detail about the actual code since 1\. it's extremely trivial +and 2\. it's not the main focus of this post. All that matters is that it +consists of some Zig code, an assembly stub, and a linker script to put it all +together. In our `build.zig` file we therefore write: + +```zig +const std = @import("std"); + +pub fn build(b: *std.Build) void { + const target = b.standardTargetOption(.{}); + const optimize = b.standardOptimizeOption(.{}); + + const kernel = b.addExecutable(.{ + .name = "kernel", + .root_source_file = b.path("src/main.zig"), + .target = target, + .optimize = optimize, + }); + kernel.setLinkerScript(b.path("src/Link.ld")); + kernel.addAssemblyFile(b.path("src/startup.s")); + + b.installArtifact(kernel); +} +``` + +As it is, this will try to build our kernel for the host machine and OS. +To get a freestanding binary we need to change `target`: + +```zig +const target = b.resolveTargetQuery(.{ + .cpu_arch = .aarch64, + .os_tag = .freestanding, +}); +``` + +Now if we run `zig build` we'll see our compiled kernel in `zig-out/bin/kernel`. +But we're not done yet. This is an ELF file, and the Pi only knows how to boot +flat binaries. We'll still keep the ELF around for debugging, but we add the +following to create a flat binary:[^2] + +```zig +const kernel_bin = b.addObjCopy(kernel.getEmittedBin(), .{ + .format = .bin, + .basename = "kernel.img", +}); +const install_kernel = b.addInstallBinFile( +kernel_bin.getOutput(), +"kernel8.img", +); +b.getInstallStep().dependOn(&install_kernel.step); +``` + +Before we can boot this kernel image though, we need some other stuff. The Pi is +a little strange in that it reads most of its firmware from the boot drive +instead of from flash on the board. Normally we'd just download these firmware +files ourselves, but we can do better! + +[^2]: The filename `kernel8.img` is important since it tells the board to boot + up in 64 bit mode. This can be overridden in [`config.txt`][7] if you really + want a different filename. + +[1]: https://git.wires.systems/wires/zosimos/src/commit/8466e9b4d2fbca85d53d8dadc87914b4766c43de/src +[7]: https://www.raspberrypi.com/documentation/computers/config_txt.html + +## Fetching Firmware + +A great thing about the Zig package manager is that it can be used to fetch any +dependencies, not just Zig packages. In this case we're going to use it to grab +the firmware files for the Pi so that we can reference it in our build script. +If we run: + +```console +$ zig fetch --save=rpi_firmware 'https://github.com/raspberrypi/firmware/archive/refs/tags/1.20250305.tar.gz' +``` + +then the latest (at time of writing) release of the firmware will be added to +our project as a dependency. This also provides us a way to ensure the integrity +of the firmware (at least that it hasn't changed since we first fetched it) +since it's saved alongside a hash. Let's adjust the build script to place all +the files we need[^3] in a `boot` directory: + +[^3]: I determined this by trial and error, but it's probably documented + somewhere I missed. + +```zig +const kernel_bin = b.addObjCopy(kernel.getEmittedBin(), .{ + .format = .bin, + .basename = "kernel.img", +}); +const install_kernel = b.addInstallFile( +kernel_bin.getOutput(), +"boot/kernel8.img", +); +b.getInstallStep().dependOn(&install_kernel.step); + +const firmware = b.dependency("rpi_firmware", .{}); +b.installDirectory(.{ + .source_dir = firmware.path("boot"), + .install_dir = .prefix, + .install_subdir = "boot", + .include_extensions = &.{ + "start4.elf", + "start4db.elf", + "fixup4.dat", + "bcm2711-rpi-4-b.dtb", + }, +}); +``` + +## Network Booting + +Having to repeatedly remove the memory card, transfer a new kernel to it, and +put it back in sounds like a massive pain. Luckily, the Pi4B supports network +booting from a tftp server. The simplest way to set this up would be to install +the `tftp-hpa` package and then just run the server normally, having it serve +the `zig-out/boot` folder. But I don't like having daemons that I'm only going +to use for a single project installed on my system, so for no real reason other +than aesthetics I'm going to be running the tftp server inside a container. The +Dockerfile is reproduced below in its entirety: + +```dockerfile +# docker/tftp/Dockerfile + +FROM alpine:3 + +RUN apk add --no-cache tftp-hpa + +ENTRYPOINT ["in.tftpd"] +``` + +This is combined with the following `compose.yaml`: + +```yaml +services: + tftp: + build: ./docker/tftp + ports: + - 69:69/udp + volumes: + - ./zig-out/boot:/data:ro + command: --verbose --foreground --secure /data +``` + +running `docker compose up` should now start the tftp server properly.[^1] + +[^1]: Since tftpd only logs to syslog and the container won't normally have + a syslogd running in it, you'll have to modify the setup if you want to see + logs. I initially did this during testing by using busybox's syslogd in the + container, but found that it made shutdown times very long, which wasn't + ideal when frequently restarting the container. + +Lastly, we have to configure the Pi itself. Network boot config info is stored +in the board's EEPROM, so boot into Linux on the Pi and run + + +```console +# rpi-eeprom-config --edit +``` + +This will bring up a text editor where you can edit the EEPROM variables in +a `.ini`-like format. We'll write the following config: + +```ini +[all] +BOOT_UART=1 +BOOT_ORDER=0xf12 + +TFTP_PREFIX=1 +TFTP_IP=x.y.z.w +``` + +where `TFTP_IP` should be set to the IP of your tftp server. All of these +options are documented [here][2] if you want to know what specifically they do. + +[2]: https://www.raspberrypi.com/documentation/computers/raspberry-pi.html + +Now, assuming it has a connection, our Pi should successfully boot over the +network! But since all our so-called kernel does at this point is spin forever, +how can we tell that anything's even happening? + +## Remote Debugging + +The easiest choice would probably be to make the program actually do something, +like blink the activity LED, but I've decided to set up remote debugging +instead. I'm using the [FT2232H mini module][3] for this, since it lets me do +both UART and JTAG over a single USB connection on my dev machine. For an +explanation of how to hook it up to the Pi, see [this blog post][4]. Like the +author of that post I'll be using OpenOCD in a docker container to connect to +the board, but I went about it a little differently. Since 2021, OpenOCD has +included a config for the Pi4B by default, so I only had to write the following +short config for the FT2232H: + +```tcl +# docker/openocd/interface/ft2232h.cfg + +adapter driver ftdi + +ftdi device_desc "FT2232H MiniModule" +ftdi vid_pid 0x0403 0x6010 + +ftdi layout_init 0x0000 0x000b + +ftdi channel 0 +``` + +I'm extremely new to OpenOCD and JTAG in general, so for all I know this might +be terribly broken in some way I just haven't noticed yet, but it's been working +so far. The Dockerfile is then as follows: + +```dockerfile +# docker/openocd/Dockerfile + +FROM alpine:3 + +RUN apk add --no-cache openocd + +ADD interface/ft2232h.cfg /usr/share/openocd/scripts/interface/ + +ENTRYPOINT ["openocd"] +``` + +And we add the following entry under `services` in `compose.yaml`: + +```yaml +openocd: + build: ./docker/openocd + ports: + - 6666:6666 + - 4444:4444 + - 3333:3333 + devices: + - "/dev/bus/usb:/dev/bus/usb" + command: -f interface/ft2232h.cfg -f board/rpi4b.cfg -c "bindto 0.0.0.0" +``` + +Lastly, we need to add a file called `config.txt` with the following contents: + +```ini +[all] +gpio=22-27=np +enable_jtag_gpio=1 +enable_uart=1 +uart_2ndstage=1 +``` + +and make sure to install it in our `build.zig`: + +```zig +b.installFile("config.txt", "boot/config.txt"); +``` + +Now we should be able to attach to `/dev/ttyUSB0` to see the UART messages when +we reboot and connect to OpenOCD's GDB server to debug the running kernel. + +[3]: https://ftdichip.com/products/ft2232h-mini-module/ +[4]: https://vinnie.work/blog/2021-04-02-ft2232h-rpi4#wiring-up-the-hardware + +## What's Next? + +For the project in general, my next goal is to set up UART communication, and +then hopefully some physical memory management with ideas from [this paper][5]. +As far as the specific subject of this post goes, I want to work on making the +startup experience nicer. As it stands right now, OpenOCD is being started at +the same time as the tftp server, when the board is never ready, so the +container always has to be restarted. It would be good to have it wait until +after the board is booted to start trying to connect, but I don't have any great +ideas on how to pull that off right now. + +[5]: https://www.usenix.org/system/files/atc23-wrenger.pdf |