PXE Boot of OpenBSD on OpenBSD

The setup of PXE boot on OpenBSD is (as always) very well documented directly in the man pages. See pxeboot(8) for a most likely better explanation than I will give here.

I use PXE boot to install OpenBSD on a Alix 2d13.

Setup tftp

In order to boot via PXE the boot files must be available through a tftp server.

On OpenBSD the tftpd(8) command was originally a process run via inetd(8). It was however rewritten for OpenBSD 5.2 as a persistent non-blocking daemon which makes it even more trivial to run. Here I assume you have OpenBSD 5.2 or later.

So first take a look at tftpd(8). To setup the server is really trivial. Create a tftpd root directory, copy neccesary boot files and start the server. I happen to have the OpenBSD 5.2 release files already in my root directory (from a previous upgrade). Copy these files from CD release or official FTP if you don't.

# mkdir /tftpboot
# cp /5.2/i386/bsd.rd /tftpboot
# cp /5.2/i386/pxeboot /tftpboot
# tftpd /tftpboot

You may also use rcctl to setup tftpd ...

rcctl enable tftpd
rcctl set tftpd flags /tftpboot
rcctl start tftpd

You can ensure that the tftpd server runs correctly by accessing the file locally.

# tftp localhost
tftp> get bsd.rd
Received 6465608 bytes in 14.2 seconds
tftp> quit

Setup DHCP for PXE

Next step is to setup your DHCP server (I assume that you have one already setup) to point at this tftp server for PXE boot.

Add the following to the subnet section of your dhcpd.conf(5) which serves hosts in your local network. library is the name of the host with the tftp server installed (it may be a IP number as well).

# PXE Boot
filename "pxeboot";
next-server library;

Then restart the dhcpd.

/etc/rc.d/dhcpd restart

Now your network should be setup for PXE boot.

BIOS Setup

First you need to setup the BIOS on your Alix board. Mark's OpenBSD Router Guide is a really great general tutorial which descibes this and all other steps of the installation. I also used OpenBSD: Boot / Install Using PXE (Preboot Execution Environment) as a general guide.

First connect your serial console to the Alix board. I use a ATEN UC232A USB to Serial Converter due to lack of a real RS232 port.

cu -e -o -s 38400 -l /dev/ttyUSB0                  # OpenBSD OR
cu -e -o -s 38400 -l /dev/cuaU0                    # see ucom(4) OR
sudo cu -e -o -s 38400 -l /dev/tty.usbserial       # MacOS X (see appendix)

Switch on the power on the board and press s during memory check to enter BIOS setup.

PXE Boot

After exiting BIOS PXE should boot the OpenBSD pxeboot bootloader.

If you get the following, try to connect the ethernet cable to another port and do a power recycle.

...
PXE-E61: Media test failure, check cable                                       
PXE-M0F: Exiting Intel PXE ROM.

When connected to the correct ethernet port the following output should follow.

PC Engines ALIX.2 v0.99h
640 KB Base Memory
261120 KB Extended Memory

01F0 Master 848A SanDisk SDCFH-004G                      
Phys C/H/S 7751/16/63 Log C/H/S 968/128/63

Intel UNDI, PXE-2.0 (build 082)
Copyright (C) 1997,1998,1999  Intel Corporation
VIA Rhine III Management Adapter v2.43 (2005/12/15)

CLIENT MAC ADDR: 00 0D B9 2B C2 80  
CLIENT IP: 192.168.0.103  MASK: 255.255.255.0  DHCP IP: 192.168.0.1            
GATEWAY IP: 192.168.0.1 
probing: pc0 com0 com1 pci pxe![2.1] mem[640K 255M a20=on]                     
disk: hd0+*
net: mac 00:0d:b9:2b:c2:80, ip 192.168.0.103, server 192.168.0.4
>> OpenBSD/i386 PXEBOOT 3.17
boot> 
booting tftp:/bsd: open tftp:/bsd: No such file or directory
 failed(2). will try /bsd
boot> stty com0 38400 
boot> set tty com0
switching console to com0
>> OpenBSD/i386 PXEBOOT 3.17
boot> bsd.rd

Next you can install OpenBSD in the regular way.

Make sure you select com0 as the default console.

I made a custom disk layout of my 4GB flash card.

> p
OpenBSD area: 64-7805952; size: 7805888; free: 887040
#                size           offset  fstype [fsize bsize  cpg]
  a:          1056320               64  4.2BSD   2048 16384    1 # /     (512m)
  c:          7813120                0  unused                   
  d:          1056384          1056384  4.2BSD   2048 16384    1 # /tmp
  e:          1056384          2112768  4.2BSD   2048 16384    1 # /var
  f:          2104704          3169152  4.2BSD   2048 16384    1 # /usr
  g:          1645056          5273856  4.2BSD   2048 16384    1 # /home

I installed the sets (X sets excluded) from the default ftp site.

After install is ready. Reboot and disable PXE boot in the BIOS (press s during memory check to enter BIOS) and OpenBSD will boot from the flash disk.

# df -h
Filesystem     Size    Used   Avail Capacity  Mounted on
/dev/wd0a      506M   38.6M    442M     8%    /
/dev/wd0g      791M    2.0K    751M     0%    /home
/dev/wd0d      508M    4.0K    482M     0%    /tmp
/dev/wd0f     1009M    373M    585M    39%    /usr
/dev/wd0e      508M    3.6M    479M     1%    /var

Fujitsu Siemens Futro S400

Futro S400 is a thin client computer which can be bought very cheaply on e.g. eBay. It has a AMD Geode NX1500 @ 1GHz. It seems to consume about 25 W with little load which isn't super efficient but not bad either. There is one additional PCI port using a reiser card which would be very useful to e.g. extend the system with an additional ethernet port.

I made a similar setup as with the Alix board. 4GB flash card with the same setup. I installed through PXE, but it should be possible to boot through USB as well. There is a RealTec Boot Agent firmware which handles PXE boot but also a standard Award BIOS.

After the machine was booted through PXE and OpenBSD was installed I needed to disable PXE boot in the RealTec Boot Agent. Switch Boot Order from Int 18h to ROM Disabled.

I setup the default tty to stty through com0 at 38400 baud during install. It can be adapted later in /etc/boot.conf. It can be really useful to control boot through a serial port without needing to connect keyboard and display. Using this setting the boot messages are sent through serial port instead of on display, but it it still possible to login using keyboard after boot has finished. You have three ways to login to the machine.

  1. Through serial port
  2. Using a connected keyboard and display
  3. Using SSH

To allow the machine to boot headless I also needed to adapt Halt On setting in Award BIOS to All, but keyboard (found in Standard Features).

Setup PXE Boot Configuration (Optional)

You may setup a boot.conf(8) on your TFTP server under /tftproot/etc/boot.conf. This reflects the boot commands specified at first boot.

stty com0 38400
set tty com0
boot tftp:/bsd.rd

APU2

See Building a BSD home router for a thorough guide.

Also see OpenBSD on PC Engines APU2 for a shorter guide.

Connect to serial console. Use good old XTerm to avoid conflicts between special keys.

cu -s 115200 -l /dev/cuaU0

To update BIOS from OpenBSD use flashrom

pkg_add flashrom

I ended up updating using TinyCore/OSX to boot via USB because of the mistake below.

First I forgot to setup tty via serial which made me think something was wrong with the OpenBSD install ...

net0: 00:0d:b9:46:68:1c using i211 on PCI01:00.0 (open)
  [Link:up, TX:0 TXE:0 RX:0 RXE:0]
Configuring (net0 00:0d:b9:46:68:1c)...... ok
net0: 192.168.0.118/255.255.255.0 gw 192.168.0.1
Next server: 192.168.0.5
Filename: pxeboot
tftp://192.168.0.5/pxeboot... ok
pxeboot : 82300 bytes [PXE-NBP]
probing: pc0 com0 com1 pxe![2.1] mem[624K 1917M 372K a20=on]
disk: hd0+*
net: mac 00:0d:b9:46:68:1c, ip 192.168.0.118, server 192.168.0.5
>> OpenBSD/amd64 PXEBOOT 3.25
boot> 
booting tftp:/bsd: open tftp:/bsd: No such file or directory
 failed(2). will try /bsd
boot> boot bsd.rd
booting tftp:bsd.rd: 3329156+1408000+2409472+0+569344 [72+371712+242508]=0x7f3218
entry point at 0x1001000 [7205c766, 34000004, 24448b12, 3680a304]
PCEngines apu2
coreboot build 20170228
2032 MB DRAM

SeaBIOS (version rel-1.10.0.1)

iPXE (http://ipxe.org) 00:00.0 C100 PCI2.10 PnP PMM+77F528B0+77EB28B0 C100

Press F10 key now for boot menu, N for PXE boot
...

But when doing as supposed everything worked as expected ...

boot> stty com0 115200
boot> set tty com0
switching console to com0
>> OpenBSD/amd64 PXEBOOT 3.28
boot> boot bsd.rd
booting tftp:bsd.rd: 3402068+1430528+3876792+0+598016 [72+434040+282286]=0x9910f0
entry point at 0x1001000 [7205c766, 34000004, 24448b12, 4550a304]
Copyright (c) 1982, 1986, 1989, 1991, 1993
        The Regents of the University of California.  All rights reserved.
Copyright (c) 1995-2017 OpenBSD. All rights reserved.  https://www.OpenBSD.org

OpenBSD 6.1 (RAMDISK_CD) #19: Sat Apr  1 13:49:18 MDT 2017
    deraadt@amd64.openbsd.org:/usr/src/sys/arch/amd64/compile/RAMDISK_CD
real mem = 1995022336 (1902MB)
avail mem = 1930866688 (1841MB)
mainbus0 at root
...

Now install could be performed as as usual.

I think I will do something different with the cooling for next apu board.

Appendix: Connect via Serial port over MacOS Mountain Lion

I used a ATEN USB to serial adapter in a Mac Mini with Mountain Lion. First I followed the following guide to install another companies driver for the serial adapter.

Getting the ATEN USB Serial Adapter Working with Mac OSX Lion

Last step was to load the driver and use the UNIX call up utility cu to connect the ALIX 3d13 board. Quit cu by ~.. You need to reload the kernel driver on each boot of MacOS X. OpenBSD is easier to use!

sudo kextload /System/Library/Extensions/ProlificUsbSerial.kext
sudo cu -e -o -s 38400 -l /dev/tty.usbserial
Connected.
PC Engines ALIX.2 v0.99h
640 KB Base Memory
261120 KB Extended Memory

01F0 Master 848A SanDisk SDCFH-004G                      
Phys C/H/S 7751/16/63 Log C/H/S 968/128/63

BIOS setup:

(9) 9600 baud (2) 19200 baud (3) 38400 baud (5) 57600 baud *1* 115200 baud
(C) CHS mode *L* LBA mode (W) HDD wait (V) HDD slave *U* UDMA enable
(M) MFGPT workaround
(P) late PCI init
*R* Serial console enable 
*E* PXE boot enable 
(X) Xmodem upload 
(Q) Quit

Save changes Y/N ?
Writing setup to flash... d	f<@	PC Engines ALIX.2 v0.99h
640 KB Base Memory
261120 KB Extended Memory

01F0 Master 848A SanDisk SDCFH-004G                      
Phys C/H/S 7751/16/63 Log C/H/S 968/128/63

Intel UNDI, PXE-2.0 (build 082)
Copyright (C) 1997,1998,1999  Intel Corporation
VIA Rhine III Management Adapter v2.43 (2005/12/15)
PXE-E61: Media test failure, check cable                                       

PXE-M0F: Exiting Intel PXE ROM.
~[silver].
Disconnected.

Appendix: Static network configuration

Some important files for static network configuration.

# cat /etc/hostname.re0 
inet 192.168.0.12 255.255.255.0 192.168.0.255 description "Fuji's uplink"

# cat /etc/resolv.conf  
search lounge.se
nameserver 217.10.96.65
nameserver 217.10.96.44
lookup file bind

# cat /etc/myname       
fuji.lounge.se

# cat /etc/mygate       
192.168.0.1

References