Butter Mountain

Introduction

After the disk in my laptop recently died I decided that it would be a good time to try and install the latest version of Open Solaris onto my laptop. Unfortunately the only option I have for installing new operating systems onto this laptop is via PXE boot, this has not been an issue in the past as I’ve been installing Debian / Ubuntu.

Solaris is fully capable of being network installed, in fact there are a multitude of tools out there that assist you in doing so. However, this is not true for the latest release of Open Solaris, support is planned but its currently not available. As a learning exercise I decided that I’d attempt to get this working on my laptop, turns out its entirely possible with a little bit of fiddling.

Note: This does not create a hands-off installer, it merely boots the livecd from the network. You can then install the system as you would normally from the gui installer.

Note 2: Part of this guide contains an ugly hack, its awful and may not work for you at all - you have been warned.

Requirements

You will need to the following to PXE / Net boot Open Solaris

• Time
• One copy of the Open Solaris CD Image.
• One system already running Open Solaris (to manipulate the boot archive)
• One PXE Boot Server (one part DHCP server, one part TFTP server and one part NFS server)
• One PXE Boot Client (with sufficient hardware to run Open Solaris)
• One network which both the client and server are connected to.

Setting up PXE Boot.

I’m going to assume that you know how to setup a PXE boot server, there are many documents regarding this for other operating systems which can be used to fill in the blanks here. Personally I used ubuntu Gutsy Gibbon as my PXE boot server.

The following steps explain which files need to be taken from the cdrom image and placed on your PXE boot server.

• Mount the cdrom image somewhere on your system (I chose /mnt/cdromimage/)
• From the cdrom image copy out the following files into your tftp directory (/tftpboot/ on my system)

# cp /mnt/cdromimage/boot/grub/pxegrub /tftpboot/boot/grub/
# cp /mnt/cdromimage/boot/x86.microroot /tftpboot/solaris/
# cp -r /mnt/cdromimage/platform/* /tftpboot/solaris/platform/

• Note: you do not need all of the contents of of the /platform/ directory, however, its only 6mb in total its easier to include it all than remove what we don’t need.

• Create the file /tftpboot/boot/grub/menu.lst with the following contents:

default=0
timeout=60
timeout=3
title Open Solaris PXE Install
kernel$ /solaris/platform/i86pc/kernel/$ISADIR/unix -m verbose -B install_server=[ip of your nfs server]:/export/opensolaris/
module /solaris/x86.microroot

• When setting up your dhcpd boot entry for the Open Solaris client machine ensure that you include the following dhcp options (this is for the ubuntu dhcpd server, this is different under Solaris)

option grubmenu code 150 = text;

host laptop {
hardware ethernet 00:XX:00:XX:00:XX;
option grubmenu “/boot/grub/menu.lst”;
fixed-address 192.168.1.34;
option subnet-mask 255.255.255.0;
option routers 192.168.1.10;
next-server 192.168.1.20;
filename “/boot/grub/pxegrub”;
}

With your dhcpd and tftpd services up and running you should now be in a position to boot the client, this will get you to a point where the system has loaded the kernel + microroot. The boot will fail when we attempt to mount the cd rom.

Editing the miniroot

To complete the boot process we need to update the microroot and tell it to mount the cdrom from an nfs server instead of from a local cdrom drive. Now is the time to load up the Open Solaris machine you installed earlier, mount the open solaris iso and copy the x86.miniroot file somewhere handy (ready for editing).
The miniroot is a gziped ufs file system image, its relatively easy for us to open it up and manipulate as we please.

# mv x86.microroot x86.microroot.gz;gunzip x86.microroot.gz
# lofiadm -a /path/to/x86.microroot /dev/lofi/1
# mount /dev/lofi/1 /mnt/microroot

So now we have the microroot mounted under /mnt/microroot, lets make some changes. First of all there are three utilities I’ve added to the microroot to aid in net booting. These can be added in the following way:

# cp /usr/sbin/prtconf /mnt/microroot/usr/sbin/
# cp /usr/sbin/i86/prtconf /mnt/microroot/usr/sbin/i86/
# cp /usr/bin/cut /mnt/microroot/usr/bin/
# cp /usr/bin/sed /mnt/microroot/usr/bin

We now have all the utilities required on the microroot to mount the cd image over nfs, the next step is to update the script that mounts the cdrom image.

Note: Here is where the nasty hack is, as I couldn’t convince nwamd (network automagic daemon) or the default network service to actually bring up the network correctly I run perform a dhcp request in the script that mounts the cdrom image. Ideally we should bring networking online via the networking physical service (or nwamd if thats what its actually designed to be used for) and have the live-fs-root service depend on the networking. Anyway, as there is an official net boot utility in the pipeline I hope this will suffice until thats available.

On with the hacking, edit the /mnt/microroot/lib/svc/method/live-fs-root startup script. Find the following line:

echo “\rPreparing live image for use” >/dev/msglog

Just below this line add in our few lines to grab the install_server value, send out a dhcp request followed by mounting the network file system.

# grab the install_server setting from boot
install_server=`/usr/sbin/prtconf -v /devices|/usr/bin/sed -n ‘/install_server/{;n;p;}’|/usr/bin/cut -f 2 -d\’`
# Awful hack - configure dhcp for all the plumbed interfaces
/sbin/ifconfig -a dhcp
# now mount the install server onto /.cdrom
/sbin/mount $install_server /.cdrom

The above few lines do not perform any sanity checks, if any of the commands fail you’ll just get dumped to a single user login prompt.

Now that we are finished editing the micro root its time to unmount and re-compress it before dumping it back into the tftp directory on your PXE boot server.

# umount /mnt/microroot
# lofiadm -d /dev/lofi/1
# gzip x86.microroot;mv x86.microroot.gz x86.microroot

Finally..

The last thing to do is to export the contents of the cd image over nfs, I chose to export it from /export/opensolaris/. It may help you to know that Solaris has issues mounting nfs v4 from Linux servers (certainly from ubuntu gutsy release), I had to disable nfs v4 services under linux to mount the nfs file system from my Solaris system.

If everything has gone smoothly you should now be in a position to boot your client machine, which once the pxe boot has completed should run just as it would had it been booted from cdrom.

If you get dumped to a single user login prompt its safe to assume something has gone wrong, I’d recommend logging in (root pw: opensolaris) and verifying that each of the steps we take in the mount cdrom script works correctly.

Recently while attending a networking course which covered redundant multipathing for high availability systems I got to thinking - could this be applied to my wired / wireless network at home? The end result being that removing the wired network from my laptop by unplugging the cable would instantly fail over to my wireless connection. I’m sure many people are thinking “but gnome network manager already does that” well, not quite.

With redundant bonding or multi-pathing both interfaces are connected to the same network and put into a special group. These interfaces are constantly connected to the network and are ready to work, however only one of them is active at any one time. Should the active interface fail (such as a network cable being removed) the mac address, ip address and all other configuration is almost instantly assigned to one of the other available network interfaces. This allows you to continue working as if nothing had happened, songs playing from network shares will continue playing, instant messenger conversations will continue to work, downloads will not be interrupted… the list goes on.

It turns out that under Linux this works incredibly well, read on for details.

Requirements.

In terms of networking you must have your wired and wireless networks on the same physical network segment, in this post I am also assuming that your wireless network is setup and that you have knowledge of networking.
As for packages you will need to install the ifenslave package under ubuntu.

# apt-get install ifenslave-2.6

Step One.

Warning: if you follow the steps I’ve included here, reboot and anything goes wrong, I accept no responsibility - thanks.

Under Linux high availability and link aggregation is handled by the bonding module, this can be enabled by adding the following line to your /etc/modules configuration file.

bonding mode=1 miimon=100 downdelay=200 updelay=200

This will load the bonding module upon next boot, however, it can be loaded at any time using the modprobe command the options are as follows.

• mode=1 - This enables the active backup mode, this will provide link level redundancy but it does not allow for any kind of link aggregation.

• miimon=100 - This enables link level monitoring of the connection (the default value is 0 - which disables link monitoring). The value passed is the frequency in milliseconds that the link is checked. Link level monitoring only takes into account the physical connection, not if the network is correctly configured or not.

• downdelay=200 - This is the delay in milliseconds before the link is marked as failed, it must be a multiple of miimon

• updelay=200 - This is the delay in milliseconds before the link is marked as active, it must be a multiple of miimon

Step Two.

Next the bonding interface (bond0) must be configured, this can be done in the /etc/network/interfaces file an entry such as this needs to be added:

auto bond0
iface bond0 inet static
address 192.168.1.34
netmask 255.255.255.0
gateway 192.168.1.24
broadcast 192.168.1.255
post-up ifenslave bond0 eth0 eth1
post-up echo “eth1″ > /sys/class/net/bond0/bonding/primary
pre-down ifenslave -d bond0 eth0 eth1

The network address, netmask, gateway and broadcast should be configured for your network. This is the one single address that your machine will be known as, assuming that one of the connections to your laptop is available. You should be able to use dhcp to configure this interface, however I have chosen a static address.

• ‘post-up ifenslave bond0 eth0 eth1′ - This line assigns my wired (eth1) and wireless (eth0) interfaces into my failover group bond0. You should replace these with the interfaces you want to use.

• ‘post-up echo “eth1″ > /sys/class/net/bond0/bonding/primary’ - This line specifies a primary slave, that is if this interface is available it will always be used in preference to the others. In this case eth1 is my wired interface, this ensures that when I return to my desk and plug my network cable back in I will be using the faster ethernet network and not my slower wireless.

• ‘pre-down ifenslave -d bond0 eth0 eth1′ - This line removes the eth0 and eth1 interfaces from the bond group when networking is stopped.

Step Three.

Now reboot for the settings to take effect. It is of course possible to do all of this without rebooting, one would simply perform the following steps:

1. modprobe the bonding driver
2. ifconfig the bonding interface
3. add the default route for the bonding interface
4. ifenslave the network devices
5. set the primary bonding interface

To Test that everything is working simply ping a host on your local network then once its going unplug the wired interface it should fail over to the wireless interface without dropping any packets. You can then re-insert the network cable and instantly switch back to the wired interface.
Help, its not working.

I had a few problems getting this working, firstly ifenslave would assign the bonding interface the mac address of my wired interface. As part of my wireless network security I have mac filtering enabled, however, only the mac address of the wireless card was allowed. Adding the mac of the bonding interface to my mac filter list cleared up that problem.

My other problem involved gnome network applet, I believe that this process is called nm-monitor. This would attempt to reconfigure my wireless network or wired network whenever I removed the network cable to my laptop. This appeared to prevent the failover from working as quickly as it should have. killing off the nm-monitor fixed this little issue.

I’ve now been using this for roughly two weeks without any issues, being able to unplug my laptop and take it to another room of the house without having to even consider whats going to happen to my network has been a real step forward.