OpenBSD Soekris Read Only Root

Running OpenBSD on a Soekris net4801

Introduction/Theory

A number of ways exist to do this, I've chosen the slightly more complicated but fairly rewarding method of
running with the root (only) filesystem mounted read-only. This means the compact-flash card will live a
little longer, not enduring the write cycles that are fabled to kill them, and if someone trips over the power
cord, you don't lose anything on the filesystems. You also never need to endure the delay of a length fsck(8).

All the bonuses out of the way: it's kind of a bastard to setup.. I'll try my best to detail it, including a
patch(1) file for '/etc/rc'. There are a number of "neat" tricks to play with symbolic links and MFS style
RAM-disks.

Installation

I usually go with the PXE boot method, its really the easiest on a soekris, since you need a small network with
internet connectivity (or a local FTP/HTTP mirror), a DHCP server and a TFTP server. OpenBSD already comes with
all of this capability, so a random crap PC on your network running it will do.

dhcpd.conf: You need to add 'next-server <IP of TFTP server>' and 'filename "pxeboot"' to your 'subnet' declaration.. thats it.

inetd.conf: Uncomment the IPv4 TFTP line and 'pkill -9 inetd && inetd'.

Once that's done, get 'pxeboot' and 'bsd.rd' from your desired version of OpenBSD (4.4, in my case) and put it in
'/tftpboot' on the TFTP server, make sure permissions are set world-readable.

Power on your soekris with a serial console connected. They use a standard DB9M connector, and supply 19200 baud, 8n1.
If you don't know what that means, figure it out via google or wikipedia.

At some point, the soekris' BIOS will ask you to hit Ctrl+P to interrupt the boot and enter the monitor, do so, and at
the prompt, type "boot F0". "F0" is the name of the first ethernet port on the device.

You should see the soekris come up with an address via DHCP, and put you at an OpenBSD PXE boot loader prompt, you should
type the following to ensure a successful boot.. order matters:

 > stty com0 19200
 > set tty com0
 > boot bsd.rd

You should see the kernel boot, just wait until it comes to the Install, Upgrade, etc menu; it will take a minute or a few.

Choose 'Install', and proceed as any normal OpenBSD install would; you'll make modifications when its booted multi-user.

Post-Install Madness

Some general good-idea step are in order:

• Add yourself a user, and configure sudo(8) properly for that user.
• Setup any networking you'll need; this includes pf(4), any hostname.if(5) files, dhcpd(8), etc. I don't care what you do here.
• Install any packages you want.. GNU screen, zsh, multitail, openvpn, and ruby are some of the ones that I put on everything.
• Enable any services you'll want in '/etc/rc.conf.local'.. I usually have something like this:
  

 rdate_flags="-n time.slic.com"
 watchdogd_flags="-i 3 -p 10"
 pf=YES
 ntpd_flags=""
 named_flags=""
 isakmpd_flags="-K"
 ipsec=""
 dhcpd_flags="sis1"

Now to get to the real work.. modifying your config files and filesystem to work with the read-only root configuration.

Read-only Root Config

• First: edit '/etc/fstab' and change "rw" to "ro" on your root filesystem line.
• Copy '/dev/MAKEDEV' to '/etc/MAKEDEV'; '/dev' will be MFS-mounted and won't have MAKEDEV. This step is IMPERATIVE.
• Edit 'sysctl.conf', enable 'net.inet.ip.forwarding' and disable 'vm.swapencrypt.enable' for speed reasons. Shouldn't matter, we don't use swap.
• Patch '/etc/rc', I provide my changes HERE because it isn't obvious what to do.

Explanation of 'rc' patch:

• I remove a number of things that don't apply to a soekris, to aid in speedy bootup. Everything helps.
• Remove the mixer stuff, CCD/RAID, swap
• Replace the read-write mount stuff with a bunch of properly sized MFS mounts: '/dev', '/var/run', '/var/empty', '/var/log', '/var/named', '/tmp', and '/home'.
• Create the RAM-disk version of '/dev', OpenBSD includes a neat target called 'ramdisk' that sets up the bulk of this for you; I add a few things for my own convenience (extra ptys, bpfs, and tun for VPNs)
• Create empty versions of a number of logfiles, and set their permissions correctly for syslogd(8).
• Create empty dhclient.leases files in a nonstandard (writable) location, fix their permissions.
• Copy a persistant version of '/home' from '/cfhome'
• Copy in BIND9 configs from '/var/named-dist'
• Setup a usable cron(8) environment from a persistant version
• Get rid of the resolv.conf mangling stuff, it won't work anyway
• Get rid of the '/var/spool' lockfile management, optionally move it to a writable location if you care.
• Get rid of 'amd', 'savecore', 'afs', quota checking, 'lpd', the kerberos master/slave stuff, 'apmd', 'wsmoused', and 'xdm'.

Filesystem Modifications

I present these details as commands, it's easier to express that way, and easier to understand.

 # pkill -9 syslogd
 # pkill -9 dhclient
 # pkill -9 pflogd
 # pkill -9 ntpd
 # pkill -9 named
 # pkill -9 dhcpd
 # pkill -9 sendmail
 # mv /var/named /var/named-dist &&  mkdir /var/named
 # rm -fr /var/run && mkdir /var/run
 # mv /home /cfhome && mkdir /home
 # mkdir -p /var/log/cron-running
 # ln -s /var/cron/tabs /var/log/cron-running/tabs
 # ln -s /var/cron/atjobs /var/log/cron-running/atjobs
 # mv /var/cron/tabs /var/cron/tabs-dist
 # mv /var/cron/atjobs /var/cron/atjobs-dist
 # rm /var/cron/log
 # touch /var/log/cron
 # ln -s /var/log/cron /var/cron/log
 # touch /var/run/dhclient.leases.sis0 && rm /var/db/dhclient.leases.sis0
 # ln -s /var/run/dhclient.leases.sis0 /var/db/dhclient.leases.sis0
 # touch /var/run/dhclient.leases.sis1 && rm /var/db/dhclient.leases.sis1
 # ln -s /var/run/dhclient.leases.sis1 /var/db/dhclient.leases.sis1
 # touch /var/run/dhclient.leases.sis2 && rm /var/db/dhclient.leases.sis2
 # ln -s /var/run/dhclient.leases.sis2 /var/db/dhclient.leases.sis2
 # touch /var/run/dhcpd.leases && rm /var/db/dhcpd.leases
 # ln -s /var/run/dhcpd.leases /var/db/dhcpd.leases
 # touch /var/run/ntpd.drift && rm /var/db/ntpd.drift
 # ln -s /var/run/ntpd.drift /var/db/ntpd.drift

Once you've got all this setup, reboot(8) and hope it comes back up. I recommend leaving the serial console connected for
a while. At least until you're used to managing this setup, it'll be helpful.

Maintaining Your Soekris

From time to time, you'll obviously have to make changes to the thing; and it isn't very cool to have to reboot into
single-user and remount your filesystem read-write everytime this needs to be done. In fact, thats dumb.. BUT: "There's Another Way"..

Remounting the filesystem read-write on a live system

This should work in any case, and let you modify to your heart's content..

 # mount -uw /

Remounting the filesystem read-only on a live system

This will only work if you haven't started anything thats writing at the moment..

 # mount -ur /