date: 06/23/2006

I've got a PC Engines WRAP.1E and a 32mb compact flash card, and I wanted Gentoo on it to be used and abused as a home router / firewall appliance. For this purpose some nice projects are available, like Gentoo Network Appliance (GNAP) for example. An other nice project is Soentoo although it's aiming for Soekris boards it works on WRAP since the hardware is similar. If you want the benefit of a nice webgui for your router check out M0n0wall or pfSense both FreeBSD based firewall distributions. First I started using the WRAP with M0n0wall but I don't need the webgui and the other solutions available didn't satisfy me.

Contents:

1. Introduction
2. Prepare the development root
3. Build the development root
4. Build the wrap root
5. Build and install non-system programs to the wrap root
6. Build and install a kernel to the wrap root
7. Packaging wrap root
8. Deploy wrap root to target
9. Todo
10. Links

1. Introduction

What is Gentoo for WRAP? This document is based on the Gentoo x86 Embedded Guide by Heath Holcomb with modifications to make it more suitable for the WRAP System Board.
Helpful links to other projects and pages are at the Links section.

Technical description of the WRAP.1C/D/E?
This embedded System is based on a National SC1100 266MHz 5x86 CPU with 16KB cache processor.
It has 128 Mbyte SDRAM main memory and three 10/100 Mbit ethernet ports.
It uses a CompactFlash module for user's operating system and application, can be expanded using a MiniPCI type III board, like 802.11 wireless cards and other expansions and features a built in Watchdog Timer and a LM77 thermal monitor.

2. Prepare the development root

You must be root, and cd to the working directory (/opt).

Code listing 2.1: need to root and set work directory

# su -
# cd /opt 

Create development root

Create the development root directory which will be the chroot for building the target system. Feel free to be more creative with the file and folder names.

Code listing 2.2: create the development root directory

# mkdir -p /opt/wrap/usr/portage

Download stage 1 tarball.

Choose a mirror http://www.gentoo.org/main/en/mirrors.xml, then download the stage1 tarball from experimental/x86/embedded/stages.

Code listing 2.3: download the latest stage 1 tarball

# wget http://linux.rz.ruhr-uni-bochum.de/download/gentoo-mirror/experimental/x86/embedded/stages/stage1-x86-uclibc-2005.0.tar.bz2

Untar stage.

Untar the stage to the development root.

Code listing 2.4: untar the stage to the development root

# tar -xvjpf stage1-x86-uclibc-2005.0.tar.bz2 -C /opt/wrap/

Mount needed directories.

Mount the proc and portage directories to your development root. This makes your system's proc and portage directory available from inside of your development root (after chrooting), so there is no need for downloading a portage snapshot.

Code listing 2.5: mount proc and portage

# mount --bind /proc /opt/wrap/proc
# mount --bind /usr/portage /opt/wrap/usr/portage

Copy DNS info.

Copy over DNS information to the development root.

Code listing 2.6: copy DNS info

# cp /etc/resolv.conf /opt/wrap/etc/resolv.conf

Chroot.

Chroot into the development root.

Code listing 2.7: chrooting into the new environment

# chroot /opt/wrap /bin/bash --login

3. Build the development root

Create new environment and load variables into memory.

Code listing 3.1: create environment variables and load into memory

# env-update
# source /etc/profile

Modify make.conf.

Modify make.conf file to your liking. This is for a WRAP system board, Geode x86 processor.

Code listing 3.2: modify make.conf

# nano -w /etc/make.conf

USE="minimal mmx -nls -unicode -perl -python -readline -ncurses"
CHOST="i586-gentoo-linux-uclibc"
CFLAGS="-march=i586 -Os -pipe -fomit-frame-pointer -mmmx"
CXXFLAGS="${CFLAGS}"
FEATURES="buildpkg"
PKGDIR="${PORTDIR}/packages/uclibc"
UCLIBC_CPU="586MMX"

Set profile to use 2.6 kernel.

Code listing 3.3: set profile to 2.6

# cd /etc/
# unlink make.profile
# ln -s ../usr/portage/profiles/uclibc/x86/2005.1 make.profile

Start the bootstrap script.

Start building the toolchain (the C library and compiler) for your development system after which you install all core system packages.

Code listing 3.4: start bootstrapping

# cd /usr/portage/scripts
# ./bootstrap.sh -p -v
# ./bootstrap.sh

Workaround - bootstraping
Failure compiling uclibc (gcc-config error: Could not run/locate "gcc")? If you get a failure while bootstrap is compiling uclibc here are the steps to work around the problem.

# gcc-config 1
# source /etc/profile
# ./bootstrap.sh

Emerge the system ebuild for the development root.

Next, build all core system packages with the newly built toolchain.

Code listing 3.5: emerge system

# emerge -e system

Workaround - emerge system
During emerge -e system, python-fchksum fails complaing about gcc-config error: Could not run/locate "i386-gentoo-linux-uclibc-gcc" The following commands work around this problem.

# emerge python
# emerge -e system

4. Build the wrap root

Emerge lines that do not have ROOT=/wrap_root will be installed into the chroot so that it will be able to compile the other packages without errors.

Create the wrap root directory.

Code listing 4.1: mkdir

# mkdir /wrap_root

Emerge baselayout-lite into wrap root.

Code listing 4.2: emerge baselayout-lite

# mkdir -p /etc/portage
# echo "sys-apps/baselayout-lite -*" >> /etc/portage/package.keywords
# ROOT=/wrap_root emerge -k baselayout-lite

Workaround - baselayout-lite
Baselayout-lite is still beta, so a few fixes are needed. There needs to be a directory "log" in /var. Inittab calls for /usr/bin/tail, but it needs to /bin.

# mkdir -p /wrap_root/var/log
# nano /wrap_root/etc/inittab

-tty3::respawn:/usr/bin/tail -f /var/log/messages
+tty3::respawn:/bin/tail -f /var/log/messages

Emerge uclibc into the wrap root.

Use the -k option because we don't get the extra files created by the build/emerge process into our embedded rootfs which needs to be as small as possible.

Code listing 4.3: emerge UClibc

# ROOT=/wrap_root emerge -k uclibc

Emerge busybox into the wrap root.

First you must emerge it into your development root. This does not create the symlinks in our development embedded rootfilesystem.

Code listing 4.4: emerge busybox

# ROOT=/wrap_root USE="make-symlinks" emerge busybox

Set time zone in your wrap_root.

Code listing 4.5: set the timezone

# nano /wrap_root/etc/TZ

GMT

Set the host- and domainname.

Code listing 4.6: set hostname

# nano /wrap_root/etc/hostname

wrap

Code listing 4.7: set domainname

# nano /wrap_root/etc/domainname

local

Install the boot loader.

Once you copy/deploy your wrap root to the compact flash card you will have to run grub on the command line to write to the master boot record (MBR). For some reason not all of /boot/grub is copied over to the wrap root, so a extra manual copy step is needed. The --nodeps gets rid of the run time need of ncurses.

Code listing 4.8: emerge grub

# ROOT=/wrap_root emerge -k --nodeps grub
# cp -R /boot/grub /wrap_root/boot/

Modify your boot configure file. The example below is for a boot partition on /dev/hda1.

Code listing 4.9: modify grub.conf

# nano -w /wrap_root/boot/grub/grub.conf

default 0
timeout  0
serial --unit=0 --speed=38400 --word=8 --parity=none --stop=12

title=Gentoo for WRAP
root (hd0,0)
kernel /vmlinuz reboot=bios  console=ttyS0,38400
initrd  /rootfs.igz

Set root password for the wrap root.

Code listing 4.10: set root password

# chroot /wrap_root /bin/sh
# passwd
# exit
# rm /wrap_root/etc/passwd-

Modify fstab.

The root filesystem will be mounted as ramfs by the kernel. The only downside to this is that unlike tmpfs it does not have a size limit, so writing a lot to / could fill up your ram. Anything that is on tmpfs uses nearly no ram when empty and grows when needed but wont use up all of the ram.

Feel free to be creative, just remember that flash doesnt like being written to lots and lots, so do not mount / or /var onto it. hda1 is the partiton on the compact flash that will contain the bootloader files, the kernel and the initrd. /dev/hda2 will be the partition that is used to store /etc containing any settings/files.

Code listing 4.11: modify fstab

# nano /wrap_root/etc/fstab

/dev/hda1   /boot       ext2    noauto,ro   0 0 
/dev/hda2   /mnt/wrap   ext2    noauto,rw   0 0
none        /proc       proc    defaults    0 0
none        /sys        sysfs   defaults    0 0

Writing init.

/init is the file that the kernel executes right after loading the content of the initramfs. In this example it is used to mount the filesystem and to restore the content of /etc.

Code listing 4.12: create init

# nano /wrap_root/init

#!/bin/sh
echo
echo
echo  "Welcome to Gentoo for Wrap"
echo
echo  " * Setting umask ..."
umask  022
echo  " * Mounting filesystems ..."
mount  -t proc none /proc
mount  -t ext2 /dev/hda2 /mnt/wrap
tar xpf /mnt/wrap/etc.tar -C /etc 
echo  " * Starting init ..."
exec  <dev/console >/dev/console 2>&1
exec  chroot . /sbin/init
echo  " *** Starting init failed! ***"
echo  " * Trying to start a shell..."
exec  /bin/sh

Remember to make it executable

Code listing 4.13:

# chmod +x /wrap_root/init

If you are using initrd instead of initramfs, rename this file to linuxrc or symlink them.

Code listing 4.15:

# cd /wrap_root
# ln -s init linuxrc

Startscript for watchdog.

If you are going to use the build in hardware watchdog of the SC1100 it will reboot your box if there is no activity on /dev/watchdog for more than 60 seconds. Busybox already provides a watchdog user-space program.

Code listing 4.15:

# nano /wrap_root/etc/init.d/watchdog

#!/bin/sh
echo " * Starting watchdog..."
watchdog -t 15 /dev/watchdog

Remember to make it executable

Code listing 4.16:

# chmod +x /wrap_root/etc/init.d/watchdog

and finally symlink it to S01watchdog to start it at system init.

Code listing 4.17:

# cd /wrap_root/etc/init.d
# ln -sf watchdog S01watchdog

Modify inittab.

Be sure to uncomment/add/edit following line for console on serial port support.

Code listing 4.18:

# nano /wrap_root/etc/inittab

# /etc/inittab
ttyS0::respawn:/sbin/getty -L ttyS0 38400 vt100

5. Build and install non-system programs to the wrap root

Emerge other software you need for you embedded target. This is very widly depending on your needs, if you want to have an IDS with snort, proxyservices with 3proxy or even bozohttpd and php.

The following is an example of ebuilds I'm using on my WRAP, snitch seems to work quiet well here for my needs but its also possible to use htbinit for your qos setup if needed. Remeber to have QoS and/or fair queueing supported by your kernel.

Code listing 5.1:

# ROOT=/wrap_root emerge -k bash nano rp-pppoe firehol dnsmasq dropbear openntpd ez-ipupdate snitch tcpdump iftop darkstat

My only need for firehol is for creating the iptables ruleset, because I'm not into iptables. Then using iptables-save to save it to a file and iptables-restore to restore the ruleset at system init which is much more faster.

6. Build and install a kernel to the wrap root

Install a kernel into wrap root.

First we will emerge it into our development root, then configure and build it. Alternatively you can fetch the kernelsources directly from kernel.org

Code listing 6.1: emerge kernel, enter menuconfig

# emerge vanilla-sources
# cd /usr/src/
# cd linux
# make menuconfig

Configure your kernel for the WRAP here. Not very hard really, and because we will be using an initramfs image most things can be compiled as modules if you wish to do so, although I wouldn't recommend it for this purpose. Disable everything that you don't want or need to keep the size of the kernel image down.

Code listing 6.2: kernel configuration

General setup  --->
 [*] Enable 16-bit UID system calls
 [*] Enable VM86 support
 [*] Optimize for size (Look out for broken compilers!)
 [*] Configure standard kernel features (for small systems)  --->
  [*]   Load all symbols for debugging/kksymoops
  [*]   Support for hot-pluggable devices
  [*]   Enable support for printk
  [*]   BUG() support
  [*]   Enable ELF core dumps
  [*]   Enable full-sized data structures for core
  [*]   Enable futex support
  [*]   Enable eventpoll support
  [*]   Use full shmem filesystem
  [*]   Use full SLAB allocator
Block layer  --->
 IO Schedulers  --->
  <*> Deadline I/O scheduler
Processor type and features  --->
 Subarchitecture Type (PC-compatible)  --->
 Processor family (GeodeGX1)  --->
 High Memory Support (off)  --->
Bus options (PCI, PCMCIA, EISA, MCA, ISA)  --->
 [*] PCI support
 [*]   Legacy /proc/pci interface
 [*] ISA support
 <*> NatSemi SCx200 support
Device Drivers  --->
 Block devices  --->
  <*> RAM disk support
  (8)   Default number of RAM disks
  (8192) Default RAM disk size (kbytes)
  [*] Initial RAM filesystem and RAM disk (initramfs/initrd) support
 ATA/ATAPI/MFM/RLL support  --->
  [*]     PCI IDE chipset support
  [*]       Generic PCI bus-master DMA support
  <*>         National SCx200 chipset support
 Network device support  --->
  [*] Network device support
      Ethernet (10 or 100Mbit)  --->
      [*] EISA, VLB, PCI and on board controllers
      <*>   National Semiconductor DP8381x series PCI Ethernet support
 Character devices  --->
  [*] Virtual terminal
  [*]   Support for console on virtual terminal
      Serial drivers  --->
      <*> 8250/16550 and compatible serial support
      [*]   Console on 8250/16550 and compatible serial port
      (1)   Maximum number of 8250/16550 serial ports
      (1)   Number of 8250/16550 serial ports to register at runtime
      Watchdog Cards  --->
      [*] Watchdog Timer Support
      <*>   National Semiconductor SCx200 Watchdog
  <*> Generic /dev/rtc emulation
  <*> NatSemi SCx200 GPIO Support
 I2C support  --->
  <M> I2C support
  <M>   I2C device interface
        I2C Hardware Bus support  --->
        <M> NatSemi SCx200 I2C using GPIO pins
        (12)  GPIO pin used for SCL
        (13)  GPIO pin used for SDA
        <M> Geode ACCESS.bus support
 Hardware Monitoring support  --->
  <M> Hardware Monitoring support
  <M> National Semiconductor LM77
File systems  --->
 <*> Second extended fs support

Code listing 6.3: build and install kernel

# make
# make INSTALL_MOD_PATH=/wrap_root modules_install
# cp /usr/src/linux/arch/i386/boot/bzImage /tmp/vmlinuz
# cp /usr/src/linux/System.map /tmp/System.map

7. Packaging

We want everything under /wrap_root to be turned into a single initramfs image.

Cleaning up.

Copy the files to some other place so you can easily restart if you mess up, then remove all the files you don't need or want.

Code listing 7.1:

# cp -r /wrap_root /tmp/
# rm -rf /tmp/wrap_root/tmp/*
# rm -rf /tmp/wrap_root/var/db
# rm -rf /tmp/wrap_root/var/lib/portage

If you want to see how much space its using:

Code listing 7.2:

# du -sh /tmp/wrap_root

Creating the initramfs image.

To create the initramfs image we will use cpio to create an archive of the files that the kernel can understand, and gzip it to make it smaller.

Code listing 7.3:

# cd /tmp/wrap_root
# find . | cpio -H newc -o | gzip -9 > /tmp/rootfs.igz

Creating the /etc archive.

Code listing 7.4:

# cd /tmp/wrap_root/etc
# tar -cpf /tmp/etc.tar ./

8. Deploy wrap root to target

Basically we just want one partiton for the boot files, and one partiton for saving configuration files in /etc that you want to keep even after a reboot.

*Partition*     *Mount Point*   *Size*
/dev/hda1       /boot           20MB
/dev/hda2       /mnt/wrap       4 MB

Prepare the compact flash card.

Code listing 8.1:

# fdisk /dev/sda
Command (m for help): /n/ 
Command action
  e extended
  p primary partition (1-4)
/p/
Partition number (1-4): /1/
First cylinder (1-1024, default 1): /<enter>/
Using default value 1  Last cylinder or +size or +sizeM or +sizeK (1-1024, default 1024):/+20M/
Command (m for help): /n/ 
Command action
  e   extended
  p   primary partition (1-4)
/p/
Partition number (1-4): /2/
First cylinder (684-1024, default 684): /<enter>/
Using default value 684 Last cylinder or +size or +sizeM or +sizeK (1-1024, default 1024):/+4M/
Command (m for help): /a/
Partition number (1-4): /1/
Command (m for help): /w/

Formatting the partitions.

There isn't really any good reason to use something other than ext2 for the boot partition.

Code listing 8.2:

# mke2fs /dev/sda1
# mke2fs /dev/sda2

Copying important files.

Code listing 8.3:

# mount /dev/sda1 /mnt/cfcard
# mount /dev/sda2 /mnt/cfcard/mnt/wrap
# cp /tmp/vmlinuz /tmp/System.map /tmp/rootfs.igz /mnt/cfcard/
# cp /tmp/etc.tar /mnt/cfcard/mnt/wrap/
# umount /mnt/cfcard

Put the compact flash card back in your WRAP, connect to the serial console and power it on.

Configure it for your needs, and save the changes to make them permanent.

Code listing 8.4:

# nano /etc/saveconfig.sh

#!/bin/sh
echo " * Mounting cfcard..."
mount -t ext2 /dev/hda2 /mnt/wrap
echo " * Compressing files and writing to cfcard..."
cd /etc
tar -cpf /mnt/wrap/etc.tar .
sleep 3
echo " * Syncing and unmounting cfcard..."
sync && sleep 3
umount /mnt/wrap
echo " * Finished!"

...have phun!

9. Todo

• better solution for saving configuration files (like the gnap overlay)
• Temperature readout
• Wireless support ( as soon as I get a minipci wireless device for the wrap board)
• scripts to make the build process easier
• keep it up to date

10. Links

• PCEngines embedded PC hardware and firmware design
• Gentoo Linux
• Booting Linux from Compact Flash
• Installing Linux on a CompactFlash Card Mini-HOWTO
• Embedded Linux Developer Forum
• Gentoo uCLibc howto
• MARC: Gentoo-embedded
• HOWTO build a LiveCD from scratch
• Gentoo Embedded x86 Guide