This guide will tell you how-to put FreeBSD 9.x on an (ALIX) embedded device. For FreeBSD 8.x see the archived guide for hints Archive/FreeBSDOnEmbeddedDevice

TODO: This is NOT complete yet.

Tip: I set the prompt on various location such that the output reflects the terminal output. build# set prompt = "build# "

Initial disk configuration

Please do mind I create a 6G image to be used at a 8G CF disk as making it exactly fit, causes issues if sectors are wearing out. I create 2 x 2G partitions for the OS (and backup OS) and 1 x 1G partition for various configuration files and other shared stuff.

build# truncate -s 6GB embedded.img build# setenv MD mdconfig embedded.img

Master MBR code: build# gpart create -s MBR $MD

3 Partitions: build# gpart add -t freebsd -s 2G $MD build# gpart add -t freebsd -s 2G $MD build# gpart add -t freebsd-ufs -s 1G $MD

2 of them are BSD based: build# gpart create -s BSD -n 20 ${MD}s1 build# gpart add -t freebsd-ufs ${MD}s1 build# gpart create -s BSD -n 20 ${MD}s2 build# gpart add -t freebsd-ufs ${MD}s1

Make the two partitions bootable: build# gpart bootcode -b /boot/boot0 ${MD} build# gpart bootcode -b /boot/boot ${MD}s1 build# gpart bootcode -b /boot/boot ${MD}s2

Set the default boot to partition 1: build# gpart set -a active -i 1 $MD

Format the filesystems: build# newfs md0s1a md0s2a md0s3

Pepare mountpoint: build# setenv DESTDIR /mnt build# mount /dev/md0s1a $DESTDIR

Disk population

build# setenv RELEASE 9.0-RELEASE build# setenv BASEPKGDIR=/usr/local/data/$RELEASE build# mkdir -p $BASEPKGDIR

build# fetch -o $BASEPKGDIR ​ftp://ftp.nl.freebsd.org/pub/FreeBSD/releases/i386/$RELEASE/base.txz build# fetch -o $BASEPKGDIR ​ftp://ftp.nl.freebsd.org/pub/FreeBSD/releases/i386/$RELEASE/kernel.txz build# fetch -o $BASEPKGDIR ​ftp://ftp.nl.freebsd.org/pub/FreeBSD/releases/i386/$RELEASE/doc.txz

More sophisticated approach (rsync not in base): build# rsync -vaz ftp.nl.freebsd.org::FreeBSD/releases/i386/$RELEASE/ $BASEPKGDIR

Set path for OS extraction: build# setenv DESTDIR /mnt

build# xzcat $BASEPKGDIR/base.txz | tar --unlink -xpzf - -C $DESTDIR build# xzcat $BASEPKGDIR/kernel.txz | tar --unlink -xpzf - -C $DESTDIR build# xzcat $BASEPKGDIR/doc.txz | tar --unlink -xpzf - -C $DESTDIR

Tip: Soekris 2521 requires CPU_ELAN which is not included into the kernel by default. For optimal performance compile the kernel yourself

Image configuration Chroot

Need resolving power at later stage inside chroot: build# cp /etc/resolv.conf /mnt/etc/

Need ssh-keygen to work, so need access to /dev/urandom: build# mount -t devfs devfs /mnt/dev

build# chroot /mnt build# set prompt = "chroot# "

Image configuration

Important: if you FAIL to chroot and continue you will now start destroying your build system

Tip: Having your flash living in read-only gives you added security and increases the live-time of your (older) Flash cards. read-write is more convenient But you will need to do some tricks as some programs need rw on first boot for ssh keys and such.

chroot# echo "/dev/ada0s1a / ufs rw,noatime 1 1" > /etc/fstab

Make sure to continue to check the disks on reboot, instead of waiting for user input: chroot# echo "fsck_y_enable=YES" >> /etc/rc.conf

# chroot# cat <<EOF > /boot/loader.conf
console=comconsole
autoboot_delay=3

hw.ata.ata_dma=0   # Soekris boot failure
EOF

Serial console boot only: chroot# echo "-h" > /boot.config

Enable serial console: chroot# sed -I '/ttyu0/ s/off/on /' /etc/ttys

Cosmetic: disable all vidconsoles as a soekris embedded device does not have any: chroot# sed -I '/ttyv[0-7]/ s/on /off/g' /etc/ttys

Software package configuration

# chroot# cat <<EOF >> /etc/csh.cshrc
# Get your packages locally if possible:
setenv FTP_PASSIVE_MODE yes
setenv PACKAGEROOT ftp://ftp.nl.FreeBSD.org

# Use an alternate (persistent) pkgdb location.
setenv PKG_DBDIR /usr/local/var/db/pkg
setenv PKG_TMPDIR /usr/local/tmp

# Default (boring) prompt
set prompt = "`hostname -s`% "

EOF

chroot# source /etc/csh.cshrc chroot# set prompt = "chroot# " chroot# mkdir -p $PKG_DBDIR $PKG_TMPDIR

Accounting & access control

chroot# echo "sshd_enable=YES" >> /etc/rc.conf chroot# service sshd keygen

chroot# pw usermod root -w random

Administive user (admin for example): chroot# pkg_add -r sudo chroot# pw useradd admin -G wheel -c 'Admin User' -s /bin/csh -m -w random chroot# echo '%wheel ALL=(ALL) ALL' >> /usr/local/etc/sudoers

Warning: Only all root to login via SSH if you know what you are doing! chroot# echo 'PermitRootLogin yes' >> /etc/ssh/sshd_config

Networking

chroot# setenv HOSTNAME riff.vanderzwet.net chroot# echo hostname=$HOSTNAME >> /etc/rc.conf

Warning: IPv4 and IPv6 is provided example only, provide your own local IP configuration

# chroot# cat <<EOF > /etc/rtadvd.conf
bridge0:\
  :addrs#1:addr="2001:1af8:febe::":prefixlen#64:tc=ether:
EOF

# chroot# cat <<EOF >> /etc/rc.conf
cloned_interfaces="bridge0"
wlans_ath0="wlan0"

# Internal wired and wireless
ifconfig_vr0="up"
create_args_wlan0="wlanmode hostap country NL mode 11g ssid home.vanderzwet.net up"
ifconfig_bridge0="addm wlan0 addm vr0 up"
ipv4_addrs_bridge0="192.168.42.1/24 192.168.1.2/24"
ipv6_ifconfig_bridge0="fe80::2 prefixlen 64"
ipv6_ifconfig_bridge0_alias0="2001:1af8:febe::1 prefixlen 64"

# Wireless Leiden Connections
ifconfig_vr2="up" 
ipv4_addrs_vr2="172.20.145.130/26"

# Internet
ifconfig_vr1="DHCP"

static_routes="wleiden"
route_wleiden="-net 172.16.0.0/12 172.20.145.129"

ipv6_defaultrouter="2001:1af8:fe00:28e::1"
ipv6_network_interfaces="auto"
rtadvd_interfaces="bridge0"
rtadvd_enable="YES"

gateway_enable=YES
pf_enable=YES
EOF

# chroot# cat <<EOF > /etc/hostapd.conf 
interface=wlan0
driver=bsd

logger_syslog=-1
logger_syslog_level=0
logger_stdout=-1
logger_stdout_level=2
dump_file=/tmp/hostapd.dump
ctrl_interface=/var/run/hostapd
ctrl_interface_group=0

macaddr_acl=0
auth_algs=1

wpa=1
wpa_passphrase=ThisPasswordIsPublic!
wpa_key_mgmt=WPA-PSK
wpa_pairwise=CCMP TKIP
EOF

# chroot# cat <<'EOF' > /etc/pf.conf 
inet_if="vr1"
wleiden_if="vr2"
int_if="bridge0"
int_network="192.168.42.0/24"


# Do not do anything @ localhost
set skip on lo0

### all incoming traffic on external interface is normalized and fragmented
### packets are reassembled.
scrub in all fragment reassemble

nat on $inet_if from $int_network  -> ($inet_if:0)
nat on $wleiden_if from $int_network  -> ($wleiden_if:0)

# SSH Access for proxy13
rdr pass on $inet_if proto tcp from any to any port 2022 \
        -> 192.168.42.146 port ssh

# SSH Access for CNodeRick
nat on $wleiden_if from any to 172.20.145.129 port ssh -> ($wleiden_if:0)
rdr pass on $inet_if proto tcp from any to any port 1022 \
        -> 172.20.145.129 port ssh

# No firewall configured
pass quick on $int_if no state

pass out on $inet_if keep state
pass out on $wleiden_if keep state

pass in on $inet_if keep state
pass in on $wleiden_if keep state
'EOF'

Package install

Examples for addition packages.

dnsmasq

dnsmasq used for local network dns and dhcp:

chroot# pkg_add -r dnsmasq chroot# echo "dnsmasq_enable=YES" >> /etc/rc.conf chroot# setenv DOMAIN echo $HOSTNAME | sed 's/^[^.]*.//'

# chroot# cat <<EOF > /usr/local/etc/dnsmasq.conf
interface=bridge0
no-dhcp-interface=vr1
cache-size=10000

localise-queries
domain=${DOMAIN}.

dhcp-range=bridge0,192.168.42.100,192.168.42.200
dhcp-host=00:00:24:c1:1d:d0,192.168.42.130
dhcp-host=00:13:d3:a7:24:51,192.168.42.165
dhcp-option=option:router,192.168.42.1

server=/wleiden.net/172.20.145.129
`jot 16 16 | xargs -n 1 -I % echo server=/%.172.in-addr.arpa/172.20.145.129`
EOF

ntp

chroot# echo "ntpd_enable=YES" >> /etc/rc.conf

chroot# cat <<EOF > /etc/ntp.conf
server 0.nl.pool.ntp.org
server 1.nl.pool.ntp.org
server 2.nl.pool.ntp.org

driftfile /var/db/ntp.drift
EOF

motd

chroot# (sed '3,$ d' /etc/motd; echo "WWW: riff.vanderzwet.net - ​http://rickvanderzwet.nl" ) > /etc/motd.new; mv /etc/motd.new /etc/motd

Flash disk protection

I hate the details like package management details getting lost, when using the system as a development board. Flash cards these days are cheap anyways, mine lived for 1 year after extensive use. Of course with read-only this will be _much_ longer, so we use that as default.

chroot# echo "update_motd=NO" >> /etc/rc.conf chroot# echo "tmpmfs=YES" >> /etc/rc.conf chroot# echo "varmfs=YES" >> /etc/rc.conf

chroot# sed -I '/[12]a/ s/rw/ro/' /etc/fstab

various

Tip: Use your favourite package list over here chroot# pkg_add -r tinyproxy sudo vim-lite subversion sixxs-aiccu apache22 transmission-daemon

Last Minute Changes

Make any other optional changes before first boot inside chroot(8): chroot# vi /etc/rc.conf

Put into production

First make sure to exit your chroot: chroot# exit

Umount the active mountpounts: build# umount /mnt/dev /mnt

Stop your memory disk: build# mdconfig -d -u $MD

Tip: During the copy (dd) process you can press CTRL+T to view the current process count. Have a look at the 64k blocks/records to be copied: build# expr stat -f "%z" embedded.img \/ 64000

Hack to reload the usb card reader to reread partition tables: build# usbconfig -d usbconfig | awk -F : '/Mass/ {print $1}' reset

Have a look to which card (da1 in mine case) you like to install it: build# grep sectors /var/log/messages

Feb 2 12:55:12 brahm kernel: da1: 7629MB (15625216 512 byte sectors: 255H 63S/T 972C)

Important: The following action will DESTROY all the existing content on the configured device. USE AT OWN RISK.

Now clone your image onto your flashcard: build# dd if=embedded.img of=/dev/da1 bs=64k

Note: NOT READY YET: Insert card into your embedded device and get yourself a prompt, using serial of ssh login.

Production Maintenance

Adding packages

Tip: As space is limited unpacking packages can result in I/O errors due to (memory) disks which run out of space. Installing for example the package apache22 goes as follows: node# mkdir /usr/local/tmp node# setenv PKG_TMPDIR /usr/local/tmp node# pkg_add -r -t /usr/local/tmp/instmp.XXXXXX apache22