Mac Tips: NetBooting Across Subnets

How to NetBoot Across Subnets

Introduction

Due to the way that most routers are configured, NetBooting is typically constrained within a subnet. This means that if you wanted to NetBoot all the machines in your organization (and I don't mean simultaneously) you would have two choices:

Have your network administrator configure your routers to pass BSDP information along to your NetBoot server or
Maintain a NetBoot server on every subnet of your network.

While the first option is the "Right Way®" to do NetBooting across subnets, it is poorly documented and often includes political battles that many people would rather not fight. Option 2 is costly and probably not the most efficient use of resources, especially if you only NetBoot your machines once every few months for the purpose of reimaging them.

So what is left for those of us that simply want to occasionally NetBoot a bunch of machines scattered across a rather robust but fragmented network? Before I explain how to do this, I'd like to give a brief overview of how NetBoot works, and exactly what is happening during the process.

10.4.5 Update

The 10.4.5 update makes the procedure outlined below much easier. Simply use the bless command to set the appropriate firmware settings:

sudo bless --netboot --server bsdp://server.apple.edu

Or, with more granularity (--options only works on EFI-based Macs, continue to use the nvram settings below for OF-based Macs):

sudo bless --netboot --booter tftp://server.apple.edu/NetBoot/NetBootSP0/NetInstall.nbi/i386/booter --kernel tftp://server.apple.edu/NetBoot/NetBootSP0/NetInstall.nbi/i386/mach.macosx --options "rp=nfs:server.apple.edu:/private/tftpboot/NetBoot/NetBootSP0:NetInstall.nbi/NetInstall-Restore.dmg"

Consult the bless man page for more details. Also, you can use Apple Remote Desktop's "Send Unix Command" feature to deploy this command to your target machines.

The NetBoot process

BSDP and DHCP

A Network boot is initiated by the client broadcasting to the network for any computers that will respond to the Boot Service Discovery Protocol (BSDP). BSDP is a protocol similar to DHCP that is based on open standards and used by several OSes including Mac OS 9, Mac OS X, FreeBSD, and Sun Solaris. Generally, routers are configured to block broadcast traffic, but special exceptions are made for DHCP traffic, which allows the client to communicate with a DHCP server on another subnet. If the routers are not also configured to pass on BSDP traffic to the NetBoot server, then the client will only be able to see NetBoot servers on the same subnet.

Power on: network configuration acquisition

When the client machine is first turned on (and set to NetBoot), the first thing you see is a blinking globe icon on a square. During this phase the client first gets an IP address from a DHCP server, then probes the network for BSDP information. When the client gets a BSDP response from a NetBoot server, the blinking globe icon changes to an Apple logo and the progress indicator starts out as a small globe.

TFTP: loading a minimal system

Immediately after the client finds a NetBoot server, it starts to download three files using TFTP (trivial FTP): the booter, the kernel, and the kernel extension cache. The booter loads first, which then loads the kernel. The kernel initiates mach_init, which then loads the kernel extension cache and starts the /etc/rc.boot and /etc/rc scripts. These are standard Unix startup scripts that get the system up and running.

Fork in the road: NetBoot vs. NetInstall

At this point, the boot process for NetBoot and NetInstall become completely different. In the case of NetInstall, there is a folder located at /System/Installation on the NetInstall image that tricks the System into thinking that it is booting from a CDROM. At this point the system runs the /etc/rc.cdrom script. If the /System/Installation folder is not present, then the System continues on with a typical netboot running from the /etc/rc.netboot startup script. I won't go into too much detail on what this file does, basically it creates the shadowfile and virtual memory space and returns control to the /etc/rc script which proceeds with a normal startup that ultimately ends with the SystemStarter splash screen (the screen during startup that has the progress indicator and runs through the StartupItems).

That's really about all the detail you need to understand how you can be limited to a subnet while netbooting.

The Cross-subnet workaround: Subvert BSDP

On every Macintosh lies a low-level environment called Open Firmware (OF, PPC-based Macs), or Extensible Firmware Interface (EFI, Intel-based Macs). The firmware environment lives on the BootROM of the motherboard and is responsible for getting the machine through the initial stages of the boot process. Firmware settings are stored in a section of RAM that are kept "alive" by the motherboard battery. This section of RAM is called non-volatile RAM, or nvram. We can manipulate "nvram settings" using the Terminal application in Mac OS X. In short, you can modify the nvram settings such that the client avoids the NetBoot server discovery process and proceeds directly to a specific NetBoot image on a specific NetBoot server. By skipping the part of the process that relies on broadcasting, we avoid issues with routers blocking broadcast traffic.

Before you go too much further, you may wish to examine your current nvram settings. Mac OS X provides a very easy method of seeing these parameters, simply launch the Terminal and type nvram -p | grep boot-[adf].

[macosx:~] nvram -p | grep boot-[adf] boot-file boot-args boot-device mac-io/ata-4@1f000/@0:2,\\:tbxi

As you can see, in a typical scenario, only "boot-device" of the three important parameters is defined. If you choose NetBoot as the startup disk in the Startup Disk Preference Pane, your nvram settings will look like this:

[macosx:~] nvram -p | grep boot-[adf] boot-file boot-args boot-device enet:bootp

Basically, the boot-device is set to simply look to a bootp server over the ethernet interface. This, of course, relies entirely on getting a BSDP response from a NetBoot server.

To avoid this, you can set the paramters as follows:

[macosx:~] nvram -p | grep boot-[adf] boot-file boot-args boot-device enet:10.0.1.2

Where "10.0.1.2" is the IP address of your NetBoot server (more specifically, it is the address of the port on your server that NetBoot is configured to use). With this setting, your client will proceed past the BSDP portion of NetBoot and immediately start to boot from the default NetBoot image on that server.

Booting to more than just the default image set

Suppose you have a customized NetInstall-Restore image set that is setup with full automation -- thus when a client boots from the image it immediately formats the internal drive and images it with your custom lab image. Clearly this is not an image set that you would want to leave as the default image set, so how do you tell your clients to boot from this image specifically? Take a look at the following nvram settings:

[macosx:~] nvram -p | grep boot-[adf] boot-file enet:10.0.1.4,NetBoot\NetBootSP0\imagename.nbi\mach.macosx boot-args rp=nfs:10.0.1.4:/private/tftpboot/NetBoot/NetBootSP0:imagename.nbi/imagename.dmg boot-device enet:10.0.1.4,NetBoot\NetBootSP0\imagename.nbi\booter

Let's break this down so it makes a little more sense. First, lets focus on the two parameters that define how the client will download files via tftp.

boot-file

boot-file enet:10.0.1.4,NetBoot\NetBootSP0\imagename.nbi\mach.macosx

syntax:
boot-file [interface]:[server-ipaddress],[path to mach.macosx relative to tftp root]

The last bit is probably the most confusing part. Your Mac OS X Server is setup to provide a tftp service. For security purposes, the root of the tftp server is set to /private/tftproot. If you take a look in this directory, you will notice a NetBoot directory that has some links to your NetBoot sharepoints:

[xserve:/private/tftpboot/NetBoot] admin% ls -l lrwxr-xr-x root wheel Mar 6 20:44 NetBootSP0 -> /Library/NetBoot/NetBootSP0 lrwxr-xr-x root wheel Mar 6 20:44 NetBootSP1 -> /Volumes/Bay2/Library/NetBoot/NetBootSP1 lrwxr-xr-x root wheel Mar 6 20:44 NetBootSP2 -> /Volumes/Bay3/Library/NetBoot/NetBootSP2 lrwxr-xr-x root wheel Mar 6 20:44 NetBootSP3 -> /Volumes/Bay4/Library/NetBoot/NetBootSP3

This is a convenience that allows you to avoid path issues associated with some sharepoints being mounted at /Volumes instead of at /. Note that the slashes in this syntax are the BACKSLASH. This is very important, be sure you use the backslash here, not forward slashes.

boot-device

boot-device enet:10.0.1.4,NetBoot\NetBootSP0\imagename.nbi\booter

syntax:

boot-file [interface]:[server-ipaddress],[path to booter relative to tftp root]

Basically this is exactly the same as above, but referring to the booter file in the NetBoot image set.

boot-args

boot-args rp=nfs:10.0.1.4:/private/tftpboot/NetBoot/NetBootSP0:imagename.nbi/imagename.dmg