Installing Debian on the Hot-e HL100/200
by Paul Whittaker (whitpa at users dot sourceforge dot net)
Index
- Introduction
- What You'll Need
- Downloads
- Configuring uMon and TFS
- Starting the Debian Installer
- Installer Difficulties
- Finishing Up
Introduction
The Hot-e HL100 and HL200 are tiny single board computers based on the ARM920T CPU, created and sold by ThinLinx.
Hot-es can run the ARM port of Debian Linux (using a still-experimental 2.6 kernel) but installing the O/S is a little awkward because of the minimalistic nature of the hardware (e.g. no room for conventional internal hard disks or CD-ROMs), and its use of a relatively uncommon boot monitor, MicroMonitor (a.k.a. uMon), created by Ed Sutton. Hence this HowTo.
Note that there is an experimental ARMEL Debian port that Hot-es can run instead of the standard ARM port. The ARMEL port is superior in many ways to standard ARM (most notably vastly improved performance in floating point operations), and will no doubt eventually replace it, but right now the ARMEL port is unofficial and hence not supported by the Debian installer. It is possible to install Debian ARMEL on Hot-e by other means, but this HowTo only covers deployment of a standard ARM system.
What You'll Need
Unfortunately the only way to configure uMon is via the serial port, so you're going to need sufficient RS232 serial cables and adaptors to connect the Hot-e to your main computer (or whatever you're using as a terminal). You probably won't be able to source this exact cable, but with judicious use of gender changers and DB25-DB9 adaptors I'm sure you'll be able to construct a monstrosity that'll do the job.
This doesn't have to be anything fancy. Even the DHCP server built into a DSL modem or similar is sufficient. However, a more configurable host-based DHCP server to which you have administrative access will make life much easier for you.
This doesn't need to be anything fancy, either, but you will need administrative access to it. This server need not be co-hosted with the DHCP server, but if it isn't, your DHCP server must point at it (as "siaddr" or "next-server" or whatever your DHCP server calls this capability). If you can neither co-host TFTP nor modify the DHCP server (e.g. because your DHCP server is a modem or a router appliance), you will have to set a fixed IP address in the boot script to override this expectation.
If you don't already have a TFTP server and would rather install one on Windows, see my Etherboot W2K HowTo for options. On Debian, "apt-get install tftpd-hpa".
Perhaps a live connection all the way to your Hot-e isn't strictly necessary, but it will make things much easier. In particular, if you use NBD to access your install CD, the nbd-client binary must be hosted somewhere that the Hot-e can "wget" it.
-
The Hot-e has 8 MB of internal flash memory (accessible as an MTD, so you could install a JFFS2 filesystem on part of it), but there's no way that you can install anything resembling Debian in only 8 MB. We'll be using this memory to store the kernel only. Hence, you need an external mass storage device.
Ideally, you want an external USB HDD enclosure (a 2.5" fanless enclosure that runs off USB power only is perfect: small and quiet). You can also use various solid-state devices, including (in order of increasing obtrusiveness and undersirability) mini-SD cards, compact flash cards (but you need to be very careful, because the kernel we'll be using doesn't properly support some newer CF types), and USB sticks/keys/pens/thumbs/whatever-you-call-them.
HOWEVER, you need to understand that flash memory has a limited number of write cycles, that general purpose operating systems like Debian are not designed to minimise writes, and that you won't (easily) be able to use wear-leveling filesystems like JFFS2 on such devices because they're not MTDs. If you install Debian on flash using a conventional filesystem like ext2 (ext3 is a bad idea because the journal concentrates writes in one spot), you will eventually "wear holes" in the most commonly accessed parts of the flash disk, and it will fail. This could take a long time, though, if the O/S typically has a low level of activity (e.g. if you use it only occasionally for a very specific purpose). Needless to say, you should never, ever swap on a flash disk, so you will be running in only 64 MB of memory, which will limit the usefulness of the O/S.
Many embedded Linux-based operating systems exist that cater to flash memory storage devices much better than Debian does (hopefully among these my own project, DIET-PC, which hosts this web page). See the LWN.net Linux Distributions List for a very comprehensive list of Linux variants.
Although a truly minimal Debian system fits in about 200 megabytes, in practice you'll need at least 512 MB of mass storage, because the installer does dumb things like retain copies of installed DEBs in /var/cache/apt/archives. I have managed to install Debian on a 256 MB SD card, but only by using a 128 MB thumb drive as a /var filesystem for the duration of the install, and then merging it back into the root filesystem (minus the DEBs) afterwards.
You can get 5.25" external USB enclosures that you can put conventional CD/DVD-ROM units in, but they're rarer than omnipresent 3.5" ones, and you probably don't have one lying around. If you don't, don't worry, because we can use Linux's Network Block Device capability to access a remote CD image instead. The USB CD-ROM is easier, though!
These instruction assume that you'll be installing Debian Etch using the official ARM netinst CD available from http://cdimage.debian.org/debian-cd/4.0_r0/arm/iso-cd/. If you can mount this via loopback or some kind of virtualisation, you won't even need to burn the ISO to a hard medium.
Downloads
I've created a uMon boot script (based on earlier work by ThinLinx developers) that hopefully will save you the trouble of learning all the complexities of uMon. At your leisure, though, you may want to download and read the uMon manual available from http://www.microcross.com/html/micromonitor.html. You should also have a quick look at my boot script, as it is quite flexible and can do more than just boot a default disk-based O/S, and you're going to have to edit it later on.
I have also constructed an experimental Linux 2.6.18 kernel for Hot-e using source code provided by the ThinLinx developers that isn't in the mainline kernel.org tree and hasn't been made publicly available yet (I'll add a link to it when it is). My kernel has some peculiarities (such as a kernel panic when the system halts, which is alarming but benign), but has generally proven very reliable. You shouldn't expect too much from the SM501 framebuffer code, though - the code I possess (already out of date) has only been tested at certain resolutions and colour depths, and if you try to change it to something other than the default specified via the kernel command-line in my boot script (1024x768x16@70), results are far from guaranteed. Kernels later than 2.6.18 don't work well with Hot-e, because the AT91 and SM501 drivers that were added to the mainline kernel in 2.6.19 and later are (for Hot-e) less reliable than, and slightly incompatible with, the customised versions I've acquired from ThinLinx that were forked from earlier versions of the code.
I've provided two versions of my kernel, one with serial console support and one without. You can use either for the initial install, but these instructions assume that you will use the appropriate one for your final system. If you run the installer graphically, you will need a monitor and a USB keyboard, because nothing will appear on the serial console and after kernel initialisation. Conversely the serial console kernel will not display anything on the graphical console until init spawns a getty on the installed system. It isn't necessary to set "console=XXX" on the kernel command line to enable either behaviour, so I suggest that you leave this alone.
My kernels have compiled in drivers for the kinds of hardware and filesystem that are likely to be encountered early in the boot process, but once the O/S is up and running you're also going to need the loadable modules for this kernel. I haven't made a nice Debian package for the entire kernel because (a) it's too much work, (b) the active kernel can't reside in the root filesystem anyway (because uMon isn't ext3-aware), and (c) the source code isn't publicly available yet.
You'll also need the Debian 4.0 ARM netinst CD, which is about 140 MB. The businesscard ISO might also work, but probably not with the instructions in this HowTo. Once you've downloaded this, mount it by whatever means are at your disposal (eg. loopback mount under Linux, Daemon Tools under Windows, CD-ROM emulation under VmWare, etc), and copy the /install/netwinder/initrd.gz file off it.
Three items need to be placed in the top-level directory of your TFTP hierarchy (that's /var/lib/tftpboot on Debian) on your TFTP server:
- The uMon script (boot)
- The Hot-e kernel image (zImage/zImage.noserial)
- The initial ramdisk image from the /install/netwinder directory of the Debian netinst CD (initrd.gz)
Configuring uMon and TFS
Connect the serial port on the Hot-e to the serial port on whatever computer you're using as a terminal, power up the Hot-e, and then connect to uMon using whatever serial terminal software is available to you (on Windows, that's HyperTerminal; in Linux, you have a number of options - I use "cu", which is part of the uucp distribution (that's "apt-get install uucp" for you Debian folk)). The serial characteristics are 38400 bps, 8 data bits, 1 stop bit, no parity, no flow control.
Hopefully you got a ">uMON" prompt. Go ahead and plug in the ethernet cable if you haven't already done so, and then enter "dhcp -v" and see what happens. If it looks like the Hot-e just tried to download and execute x86 PXE code, then you have a problem that we'll have to sort out later, but in any case your IP address should have been set. Use "echo $IPADD" to make sure that you now have a sane-looking IP address. If you don't, you have unrelated DHCP server or network problems that I can't help you with.
Now enter "tftp -F boot -f Be $BOOTSRVR get boot" This will load the file "boot" (the rightmost "boot") from $BOOTSRVR, which should now be set to either the DHCP server or whichever "next-server" it points at (you'll need to substitute a known IP address for $BOOTSRVR if the DHCP probe didn't set this variable), and store it in nonvolatile flash memory using the local filename "boot" (the leftmost "boot") with flags to indicate that it is an executable script (e) and that it should be used at poweron for automatic bootstrap and is interruptable (B). Note that the lowercase "b" flag indicates non-interruptable autoboot, and you really, really, don't want to enter this by mistake, because you may never be able to get your uMON prompt back again!
If the tftp command timed out, your TFTP server isn't working or isn't reachable, or $BOOTSRVR is set to something it shouldn't be. If you just installed a TFTP server specfically for this task, make sure that you aren't blocking access to it via TCP wrappers or iptables or something.
If all went well, you should now be able to verify the existence of the "boot" script in flash using "tfs ls". TFS stands for "Tiny File System" which is the filesystem format that uMon uses for internal flash memory. Although uMon can also support FAT and JFFS2, TFS is the only filesystem format that the Hot-e build of uMon understands.
Now install the kernel image into flash using "tftp -F zImage $BOOTSRVR get zImage" (or "tftp -F zImage $BOOTSRVR get zImage.noserial", as appropriate). We're not actually going to execute this directly out of flash, although this is theoretically possible if the kernel is compiled with XIP (eXecute-In-Place) support - which mine aren't.
The PXE Problem
If you encountered an attempted PXE boot earlier, we'll have to deal with it now. If not, skip ahead to the next section.
The bad news is that there's no way to stop uMon from trying to download and execute the file named as a boot file in the DHCP response. The worse news is that taking this information out of the DHCP response is (presumably) not an option, because it'll stop whatever was using this from working (maybe other diskless clients, or maybe diskfull clients (re)imaging using a rapid-deployment system like RIS).
There are two ways around this. The first way is only viable if you have (or can substitute) DHCP software that can provide different DHCP responses to requesters with different Vendor Class Identifiers. The second way is to exploit a feature that uMon uses to obtain information about which TFS flags to use for downloaded files.
To do the first, you'll need to reconfigure ISC DHCP server (which is pretty much the only DHCP server you'll be able to use for the conditional-response method) so that it only provides DHCP option 67 (boot file) information in the response if the Vendor Class Identifier does not match what Hot-e uses, which is "ThinLinx HOTE1X":
if not option vendor-class-identifier = "ThinLinx HOTE1X" {
filename "/pxelinux.0";
}
Or conversely - and preferably - provide the boot file option only if the Vendor Class Identifier matches something that you know does need it:
if substring (option vendor-class-identifier, 0, 9) = "PXEClient" {
filename "/pxelinux.0";
}
The second technique involves changing the name of the boot file that you don't want the Hot-e to load such that it has a comma in it. uMon interprets a comma in a filename as a delimiter; everything before the comma is treated as the "actual" name of the file, and everything after it is considered to be a sequence of TFS flags to set on the file once it is downloaded. Moreover, because TFS doesn't have proper directories, uMon doesn't pay much attention to directory separator characters, which means that it applies comma delimiting to the entire pathname, not just the filename proper. This is fortunate, because PXELinux (a common PXE boot loader in non-Windows environments) gets very upset if its boot loader filename isn't "pxelinux.0", because it tries to derive other pathnames (pxelinux.cfg etc) from the boot loader name.
Anyway, to cut a long story short, the trick is to move the offending boot file into a subdirectory whose name ends in a comma, and to ensure that no object that matches the subdirectory name minus the comma exists at the top level. For example, if your boot loader is located at /path/to/tftproot/pxelinux.0 and referenced as "/pxelinux.0" (or just "pxelinux.0") by your DHCP server, create a directory "/path/to/tftproot/pxe," (note the comma), move pxelinux.0 and anything associated with it (pxelinux.cfg etc) into it, and change the boot file option in DHCP to "/pxe,/pxelinux.0". This will trick uMon into thinking that the real filename is "/pxe", which doesn't exist, so it will harmlessly fail to download it. Even if such an object did exist, uMon would still fail to store the object in TFS because "/" isn't a valid TFS flag.
Starting the Debian Installer
With any luck, you should now be able to enter "boot" (this being the name of my script, not a built-in command) and the Hot-e will boot to the Debian Installer using the zImage in flash and the initrd.gz on your TFTP server. If you want to add any extra kernel command line parameters (such as "debconf/priority=medium" or something) you can do this using "set CMDLINE_APPEND var=value..." first.
If you're fortunate enough to have an external USB CD-ROM unit, you can load the Debian CD into it, connect it to the Hot-e, and you're all set and you can skip ahead to the next section. If you don't, the installer will of course fail at the "Detect and mount CD-ROM" step, and will ask you if you want to load CD-ROM drivers from a driver floppy. Say no to this, and no again on the next screen when asked if you want to manually select a CD-ROM module and device. This will get you to the main menu, from which you should select "Execute a shell".
We'll use the Network Block Device (NBD) capability to access a CD (or CD image) mounted on a remote server. The kernel you're using has NBD support. It also has NFS support, and you're probably wondering why we don't just NFS mount the CD instead. We could, but the Busybox mount applet in the Debian Installer root filesystem doesn't support NFS mounting, so you'd have to download a replacement mount command, and that's just as awkward.
On the non-Hot-e host where the downloaded netinst ISO resides, you now need to install and run nbd-server software. This should be fairly straightforward for Linux hosts (eg. "apt-get install nbd-server" on Debian). For Windows hosts, use Folkert van Heusden's Windows NBD Server. I don't know of a prebuilt NBD server for MacOS X, but I'm sure you could compile one easily enough.
Although you can point nbd-server at a physical CD via a physical (or emulated) CD-ROM pathname (eg. /dev/hdc), it's much easier just to point it at the downloaded ISO file. You also need to specify an arbitrary port number, which for the purpose of this HowTo we'll assume is 3000. You also need to ensure that your host's firewall (if any) doesn't block inbound access to whichever port you choose.
Example:
nbd-server 3000 /home/paw/debian-40r0-arm-netinst.iso -r
Now back at the Hot-e shell prompt, use the following instructions to retrieve and execute nbd-client:
echo 'nameserver 192.168.1.1' > /etc/resolv.conf # or whatever your gateway address is cd /sbin wget http://diet-pc.sf.net/arm/hot-e/nbd-client chmod 755 nbd-client ./nbd-client bs=2048 192.168.1.100 3000 /dev/nbd0 -persist # Substitute your TFTP server address exit
Apologies for providing nbd-client as a huge static binary, but the mutant glibc 2.3.6 shared libraries used in the installer root (yes, they are glibc, not uClibC or anything else) aren't compatible with those on an installed system, such that you get a missing symbol (setvbuf) if you try to use a dynamic binary. Feel free to donate a replacement binary or tell me how to make a better one!
Now restart "Detect and mount CD-ROM" from the installer menu. It will still fail, but this time, after saying that you still don't want to install drivers from a floppy, say yes when asked if you want to manually select a CD-ROM module and device. Specify "none" for the module and "/dev/nbd0" for the device.
Installer Difficulties
Scary Warnings
The Debian Installer should now be able to proceed normally, but it will throw some scary-looking warnings as you. Say YES to each of the following:
- No kernel modules were found ... continue the install without loading kernel modules?
- This partitioner doesn't have information about the default type of the partition tables on your architecture ... continue with partitioning?
- Is the system clock set to UTC?
- No installable kernel was found in the defined APT sources ... continue without installing a kernel?
- No boot loader has been installed, either because you chose not to or because your specific architecture doesn't support a boot loader yet ... continue?
If you are installing to a flash disk, you should use Ext2 rather than Ext3 for the filesystem format, and you should not configure any swap partition. This means that you'll get another scary warning, to which you should answer NO:
- You have not selected any partitions for use as swap space ... do you want to return to the partitioning menu?
Freezes
If you're using NBD rather than a physical USB CD-ROM, don't be alarmed if the Hot-e appears to lock up when starting the disk partitioner (at around the 41% mark). NBD takes offense at something that the partitioner tries to do at this point (syslog reports warnings about use of deprecated SCSI ioctls), and will take a long time (15 minutes) to time out and reset. Just be patient.
If you get freezes at any other point, or have a freeze of more than 15 minutes, check the Hot-e's LEDs. The kernel you are using is set to show CPU activity on the red LED. If this LED is solid on, your system has most likely crashed. If it's off, wait a little longer and something may eventually time out.
Out of Memory
Remember how I said that you should never swap on flash memory devices? Well, I lied. If you are installing to a flash disk of some sort and (as per my advice) didn't set up any swap partitions, then the "Select and Install Software" step will probably fail generically (no specific error). What actually happened, as /var/log/syslog will reveal, is that you can out of virtual memory and the task got killed. The only way we can get around this is to set up a temporary swap file on your flash device.
In case you're wondering, the Debian Installer's "lowmem" feature is not going to help you here. This only reduces memory usage until the disk parititioner completes, not beyond (at which time it is assumed that you have a swap partition). You might also think of swapping over NBD, but let me disabuse you of that notion: swapping over NBD is, for all intents and purposes, an urban legend, because it requires a kernel patch that hasn't been updated for kernels later than the 2.2 series. There is also a patch for 2.4 series kernels that allows swapping over NFS, but since we are using a 2.6 kernel and can't easily do NFS mounts anyway that doesn't help either.
So, go back to the main menu, execute a shell, and try something like this:
dd if=/dev/zero of=/target/swap bs=1024 count=32768 # 32 MB should be enough mkswap /target/swap swapon /target/swap exit
This will prevent the installer from unmounting /target at the end of the installation, but your installed system will recover from this after an e2fsck.
Finishing Up
Don't be alarmed by the kernel panic when the installer reboots after a successful installation. This is just a bug in my kernel - it won't have any effect on your installed system. You'll have to unplug and replug the power cord to reset the Hot-e.
After doing this, you'll have a very limited window of opportunity (only a second or two) to interrupt the boot process before the Hot-e goes and boots the Debian Installer all over again. It won't boot the installed O/S until we reconfigure the boot script to do so. Press the spacebar to interrupt the autoboot while "boot?" is displayed.
To permanenly change the default boot device you'll have to hack the boot script. You can either use "tfs rm boot" do delete the existing script, and then use a tftp command to download a replacement (like you did before), or for simple changes you can use the built-in text editor (which bears a ghastly resemblance to ed (or vi's ex)).
My script can, with one very simple change, support booting from /dev/hda1 (CF card), /dev/sda1 (USB HDD or flash drive), and /dev/mmcblk0p1 (SD card) by default. Booting a different partition number requires three changes, and for anything else you'll have to add new blocks of code, which is not practical when using the internal editor, so delete the file and download an externally-edited one instead.
For the simplest case, you just need to enter "edit boot", then "e25" to begin editing line 25; use the cursor, delete and/or <ctrl>h keys to change this from "set ARG1 "ram"" to "set ARG1 "usb"" (or "cf" or "sd"), press enter and then "." on a line by itself to exit edit mode, and "q" to quit and save. For a different partition, you'll also have to change the values of CORE_ROOTDEV and ROOTFS_SPEC in the relevant code block somewhere between lines 120 and 180 (use "P120-180" to display these lines). If your DHCP server is not capable of hosting or nominating a TFTP server, you will also need to define TFTPSRVR appropriately by uncommenting and modifying line 31.
Having done this, you should now be able to enter "boot" or simply power cycle the Hot-e, and it should boot to your newly installed Debian O/S.
Real-Time Clock
Once you've booted and logged in, one final thing will need attending to. Although the kernel provides a working interface to the Hot-e's real time clock via /dev/rtc0, Debian doesn't create a /dev/rtc symlink, which the hwclock utility requires, so you will see errors related to reading and setting the clock. Your system will still have the right time, because the kernel actually sets the clock from /dev/rtc0 at boot (this is a compile-time settings), but without /dev/rtc modifications to the clock after boot (via ntp or whatever) won't get saved when the system shuts down.
The fix is quite simple: just add the following lines to /etc/udev/rules.d/udev.rules:
# Create /dev/rtc symlink for the first instance of the RTC class driver KERNEL=="rtc0", SYMLINK+="rtc"