OpenBSD Soekris Read Only Root

Running OpenBSD on a Soekris net4801 / net5501

Introduction/Theory

A number of ways exist to do this, I've chosen the slightly more complicated but fairly rewarding method of
running with the root (only) filesystem mounted read-only. This means the compact-flash card will live a
little longer, not enduring the write cycles that are fabled to kill them, and if someone trips over the power
cord, you don't lose anything on the filesystems. You also never need to endure the delay of a lengthy fsck(8).

All the bonuses out of the way: it's kind of a bastard to setup.. I'll try my best to detail it, including a
replacement for '/etc/rc'. There are a number of "neat" tricks to play with symbolic links and MFS style
RAM-disks.

Caveats

Unfortunately, a few great features get disabled as part of this. KARL and library ASLR, for example.

KARL

KARL requires a lot of disk space for my target 512MB CF card, as well as needing a read-write disk.
If you have the space (say, a 1GB CF card!), you could easily run the relink process manually whenever
you feel like it to regain use of this feature to some degree:

 # /usr/libexec/reorder_kernel &

If you're not going to use KARL and are space-constrained on your CF card, you can remove the relink
directory like so:

 # rm -fr /usr/share/relink

library ASLR

ASLR is a bit more complicated, although it does happen at least one time (first boot after installation),
so not all is lost!

On-boot randomness seeding

The "random_seed()" function in /etc/rc gets largely commented out due to needing filesystem write privileges.
You can manually run these commands on occasion to regain partial use of this feature as well. Like ASLR, this
at least happens on first boot.

Installation

Perform a PXE-boot standard OpenBSD/i386 install, the following differences apply:
Power on your soekris with a serial console connected. They use a standard DB9M connector, and supply 19200 baud, 8n1.
If you don't know what that means, figure it out via google or wikipedia.

At some point, the soekris' BIOS will ask you to hit Ctrl+P to interrupt the boot and enter the monitor, do so, and at
the prompt, type "boot F0". "F0" is the name of the first ethernet port on the device.

You should see the soekris come up with an address via DHCP, and put you at an OpenBSD PXE boot loader prompt, you should
type the following to ensure a successful boot.. order matters:

 > stty com0 19200
 > set tty com0
 > boot bsd.rd

You should see the kernel boot, just wait until it comes to the Install, Upgrade, etc menu; it will take a minute or a few.

Choose 'Install', and proceed as any normal OpenBSD install would, with some exceptions:

• When installing, you should partition your CF card with 1 giant / partition, and *no swap*
• It'll ask you later on if you want to change the console to com0, tell it yes and choose 19200 to match the soekris BIOS settings.
  
• KARL will fail during install with "out of swap" style memory errors. The installation still completes, and we'll disable KARL post-install.
  
• Once installed, reboot into your installation and do basic post-install configuration, including setting up network interfaces, adding packages, configuring doas/sudo, etc.. the idea is to have a fully installed system that just needs configuration.

Post-Install Madness

Some general good-idea step are in order:

• Add yourself a user, and configure doas(8)/sudo(8) properly for that user.
• Setup any networking you'll need; this includes pf(4), any hostname.if(5) files, dhcpd(8), etc. I don't care what you do here.
• Install any packages you want.. GNU screen, zsh, multitail, openvpn, and ruby are some of the ones that I put on everything.
• Enable any services you'll want in '/etc/rc.conf.local', plus disable a few things that are unhelpful/unnecessary:
  

 library_aslr=NO               # takes ages and requires read-write disk
 sndiod_flags=NO
 watchdogd_flags="-i 3 -p 10"

Now to get to the real work.. modifying your config files and filesystem to work with the read-only root configuration.

Read-only Root Config

Login and 'doas -s' or 'sudo -s' if you haven't already: you'll need to perform a number of steps,
which I'll detail below..

 # cp /etc/rc /etc/rc-dist
 # ftp -o /etc/rc https://wtf.hijacked.us/random/OpenBSD-ro-rc/7.0-rc
 # cp /dev/MAKEDEV /etc/MAKEDEV
 # touch /var/run/dhcpd.leases
 # touch /var/run/dhclient.leases.sis{0,1,2}
 # touch /var/run/dhclient.leases.vr{0,1,2,3}
 # ln -s /var/run/dhclient.leases.* /var/db/
 # ln -s /var/run/dhcpd.leases /var/db/
 # mv /var/db/ntpd.drift /var/run/ntpd.drift
 # ln -s /var/run/ntpd.drift /var/db/
 # pkill -9 cron
 # mv /var/cron /var/cron-dist
 # rm -f /var/cron-dist/tabs/.sock
 # cp -Rp /var/cron-dist /var/log/cron-running
 # ln -s /var/log/cron-running /var/cron
 # sed -i '1,$s/\/ ffs rw/\/ ffs ro/g' /etc/fstab
 # mv /home /cfhome
 # mkdir /home
 # cat /etc/syslog.conf | grep -v -e xferlog -e lpd-errs >/etc/syslog.conf
 # cat /etc/newsyslog.conf | grep -v -e xferlog -e lpd-errs >/etc/newsyslog.conf
 # reboot

There.. if you did everything correctly and didn't do anything too funny with your system
before setting up read-only-root, things should come up with *no errors during boot*.. good luck.

I recommend leaving the serial console connected for a while. At least until you're used to
managing this setup, it'll be helpful.

Maintaining Your Soekris

From time to time, you'll obviously have to make changes to the thing; and it isn't very cool to have to reboot into
single-user and remount your filesystem read-write everytime this needs to be done. In fact, thats dumb.. BUT: "There's Another Way"..

Remounting the filesystem read-write on a live system

This should work in any case, and let you modify to your heart's content..

 # mount -uw /

Remounting the filesystem read-only on a live system

This will only work if you haven't started anything thats writing at the moment..

 # mount -ur /