Previous Next Contents

3. initrd Cookbook for root mounted RAID

This is the procedure to make an 'initrd' ramdisk with rescue tools for raid.

Specifically, this document referrs to a RAID1 implementation, however it is generally applicable to any raid scheme with a root mounted raid device.

3.1 Security Reminder

The rescue file system may be used stand alone. Should your raid array fail to mount, you are left with the rescue system mounted and running. TAKE THE APPROPRIATE SECURITY PRECAUTIONS!!!

3.2 Build the Kernel and Raid Tools

The first thing that must be done is to patch and build your kernel and become familiar with the raid tools. Configure, mount and test your raid device(s). The details of how to do this are included in the raidtools package and briefly reviewed later in this document.

3.3 Build the initrd Rescue and Boot filesystem

I used the Slackware-3.4 distribution to build both the Rescue/Boot filesystem and the filesystem for the production machine. Any linux distribution should work fine. If you use a different distribution, review the Slackware specific portion of this procedure and modify it to suit your needs.

You can download the Slackware distribution from:

If you already have Slackware, you only need to download new 'a', 'ap', and 'n' disksets.

I use loadlin to boot the kernel image and ramdisk from a dos partition. I chose to create a minimum ramdisk system using the Slackware 'setup' script followed by installing the 'linuxthreads' package and 'raidtools' over the clean Slackware installation on my ramdisk. I used the identical procedure to build the production system. So the rescue and production systems are very similar.

This installation process gives me a 'bare' system (save a copy of the file) to which I overlay

        /etc .... with a modified fstab

from my current system to customize it for the particular kernel and machine that it is/will-be running on.

This makes the boot/rescue system the same system that is running on the root mounted raid device, just skinnyed down a bit, while allowing the library, etc... revisions to always be current.

3.4 Start the STEP by STEP instructions

From the root home directory (/root):

        cd /root
        mkdir raidboot
        cd raidboot

Create a mountpoints to work on

        mkdir mnt
        mkdir mnt2

Make a file large enough to do the file system install. This will be a lot larger than the final rescue file system. I chose 24 megs since 16 megs is not large enough

        dd if=/dev/zero of=build bs=1024k count=24
associate the file with a loop device and generate an ext2 file system on the file

        losetup /dev/loop0 build
        mke2fs -v -m0 -L initrd /dev/loop0
        mount /dev/loop0 mnt

3.5 Install the distribution - Slackware Specific

...skip Slackware Specific stuff and go to next section.

Now that an empty filesystem is created and mounted, run "setup".

Specify         /root/raidboot/mnt

as the 'target'. The source is whatever you normally install from. Select the packages you wish to install and proceed but DO NOT configure.

Choose 'EXPERT' prompting mode.

I chose 'A', 'AP, and 'N' installing only the minimum to run the system plus an editor I am familiar with (vi, jed, joe) that is reasonably compact.

x lqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqk x
x x   [X] aaa_base  Basic filesystem, shell, and utils - REQUIRED    x x
x x   [X] bash      GNU bash-1.14.7 shell - REQUIRED                 x x
x x   [X] devs      Device files found in /dev - REQUIRED            x x
x x   [X] etc       System config files & utilities - REQUIRED       x x
x x   [X] shadow    Shadow password suite - REQUIRED                 x x
x x   [ ] ide       Linux 2.0.30 no SCSI (YOU NEED 1 KERNEL)         x x
x x   [ ] scsi      Linux 2.0.30 with SCSI (YOU NEED 1 KERNEL)       x x
x x   [ ] modules   Modular Linux device drivers                     x x
x x   [ ] scsimods  Loadable SCSI device drivers                     x x
x x   [X] hdsetup   Slackware setup scripts - REQUIRED               x x
x x   [ ] lilo      Boots Linux (not UMSDOS), DOS, OS/2, etc.        x x
x x   [ ] bsdlpr    BSD lpr - printer spooling system                x x
x x   [ ] loadlin   Boots Linux (UMSDOS too!) from MS-DOS            x x
x x   [ ] pnp       Plug'n'Play configuration tool                   x x
x x   [ ] umsprogs  Utilities needed to use the UMSDOS filesystem    x x
x x   [X] sysvinit  System V-like INIT programs - REQUIRED           x x
x x   [X] bin       GNU fileutils 3.12, elvis, etc. - REQUIRED       x x
x x   [X] ldso      Dynamic linker/loader - REQUIRED                 x x
x x   [ ] ibcs2     Runs SCO/SysVr4 binaries                         x x
x x   [X] less      A text pager utility - REQUIRED                  x x
x x   [ ] pcmcia    PCMCIA card services support                     x x
x x   [ ] getty     Getty_ps 2.0.7e - OPTIONAL                       x x
x x   [X] gzip      The GNU zip compression - REQUIRED               x x
x x   [X] ps        Displays process info - REQUIRED                 x x
x x   [X] aoutlibs  a.out shared libs - RECOMMENDED                  x x
x x   [X] elflibs   The ELF shared C libraries - REQUIRED            x x
x x   [X] util      Util-linux utilities - REQUIRED                  x x
x x   [ ] minicom   Serial transfer and modem comm package           x x
x x   [ ] cpio      The GNU cpio backup/archiving utility            x x
x x   [X] e2fsbn    Utilities for the ext2 file system               x x
x x   [X] find      GNU findutils 4.1                                x x
x x   [X] grep      GNU grep 2.0                                     x x
x x   [ ] kbd       Change keyboard mappings                         x x
x x   [X] gpm       Cut and paste text with your mouse               x x
x x   [X] sh_utils  GNU sh-utils 1.16 - REQUIRED                     x x
x x   [X] sysklogd  Logs system and kernel messages                  x x
x x   [X] tar       GNU tar 1.12 - REQUIRED                          x x
x x   [ ] tcsh      Extended C shell version 6.07                    x x
x x   [X] txtutils  GNU textutils-1.22 - REQUIRED                    x x
x x   [ ] zoneinfo  Configures your time zone                        x x
x mqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqj x
From the 'AP series, I use only 'JOE', and editor I like, and 'MC' a small and useful file management tool. You choose the utilities you will need on your system.
x x     [ ] ispell    The International version of ispell          x x
x x     [ ] jove      Jonathan's Own Version of Emacs text editor  x x
x x     [ ] manpgs    More man pages (online documentation)        x x
x x     [ ] diff      GNU diffutils                                x x
x x     [ ] sudo      Allow special users limited root access      x x
x x     [ ] ghostscr  GNU Ghostscript version 3.33                 x x
x x     [ ] gsfonts1  Ghostscript fonts (part one)                 x x
x x     [ ] gsfonts2  Ghostscript fonts (part two)                 x x
x x     [ ] gsfonts3  Ghostscript fonts (part three)               x x
x x     [ ] jed       JED programmer's editor                      x x
x x     [X] joe       joe text editor, version 2.8                 x x
x x     [ ] jpeg      JPEG image compression utilities             x x
x x     [ ] bc        GNU bc - arbitrary precision math language   x x
x x     [ ] workbone  a text-based audio CD player                 x x
x x     [X] mc        The Midnight Commander file manager          x x
x x     [ ] mt_st     mt ported from BSD - controls tape drive     x x
x x     [ ] groff     GNU troff document formatting system         x x
x x     [ ] quota     User disk quota utilities                    x x
x x     [ ] sc        The 'sc' spreadsheet                         x x
x x     [ ] texinfo   GNU texinfo documentation system             x x
x x     [ ] vim       Improved vi clone                            x x
x x     [ ] ash       A small /bin/sh type shell - 62K             x x
x x     [ ] zsh       Zsh - a custom *nix shell                    x x
x mqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqj x
From the 'N' package I only loaded TCPIP. This isn't really necessary, but is very handy and allows access to the network while working on a repair or update with the root raid array dismounted. TCPIP also contains 'biff' which is used by some of the applications in 'A'. If you don't install 'N' you might want to install the biff package anyway.
x lqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqk x
x x    [ ] apache    Apache WWW (HTTP) server                      x x
x x    [ ] procmail  Mail delivery/filtering utility               x x
x x    [ ] dip       Handles SLIP/CSLIP connections                x x
x x    [ ] ppp       Point-to-point protocol                       x x
x x    [ ] mailx     The mailx mailer                              x x
x x    [X] tcpip     TCP/IP networking programs                    x x
x x    [ ] bind      Berkeley Internet Name Domain server          x x
x x    [ ] rdist     Remote file distribution utility              x x
x x    [ ] lynx      Text-based World Wide Web browser             x x
x x    [ ] uucp      Taylor UUCP 1.06.1 with HDB && Taylor configs x x
x x    [ ] elm       Menu-driven user mail program                 x x
x x    [ ] pine      Pine menu-driven mail program                 x x
x x    [ ] sendmail  The sendmail mail transport agent             x x
x x    [ ] metamail  Metamail multimedia mail extensions           x x
x x    [ ] smailcfg  Extra configuration files for sendmail        x x
x x    [ ] cnews     Spools and transmits Usenet news              x x
x x    [ ] inn       InterNetNews news transport system            x x
x x    [ ] tin       The 'tin' news reader (local or NNTP)         x x
x x    [ ] trn       'trn' for /var/spool/news                     x x
x x    [ ] trn-nntp  'trn' for NNTP (install 1 'trn' maximum)      x x
x x    [ ] nn-spool  'nn' for /var/spool/news                      x x
x x    [ ] nn-nntp   'nn' for NNTP (install 1 'nn' maximum)        x x
x x    [ ] netpipes  Network pipe utilities                        x x
x mqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqj x
With the installation complete, say no to everything else (no to all configuration requests) and exit the script.

3.6 Install linux pthreads

Now you must install the 'linuxthreads-0.71' library. I have included this diff for the linuxthreads Makefile rather than explain the details of the installation by hand. Save the original Makefile, apply the diff and then:

        cd /usr/src/linuxthreads-0.71
        make install
-------------------diff Makefile.old
> # If you are building "linuxthreads" for installation on a mount
> # point which is not the "root" partition, redefine 'BUILDIR' to
> # the mount point to use as the "root" directory
> # You may wish to do this if you are building an 'initial ram disk'
> # such as used with bootable root raid devices.
> # REQUIRES ldconfig version 1.9.5 or better
> # do ldconfig -v to check
> #
> BUILDIR=/root/raidboot/mnt
<       install pthread.h $(INCLUDEDIR)/pthread.h
<       install semaphore.h $(INCLUDEDIR)/semaphore.h
>       install pthread.h $(BUILDIR)$(INCLUDEDIR)/pthread.h
>       install semaphore.h $(BUILDIR)$(INCLUDEDIR)/semaphore.h
<       test -f /usr/include/sched.h || install sched.h $(INCLUDEDIR)/sched.h
>       test -f $(BUILDIR)/usr/include/sched.h || install sched.h $(BUILDIR)$(INCLUDEDIR)/sched.h
<       install $(LIB) $(LIBDIR)/$(LIB)
<       install $(SHLIB) $(SHAREDLIBDIR)/$(SHLIB)
<       rm -f $(LIBDIR)/$(SHLIB0)
<       ln -s $(SHAREDLIBDIR)/$(SHLIB) $(LIBDIR)/$(SHLIB0)
>       install $(LIB) $(BUILDIR)$(LIBDIR)/$(LIB)
>       install $(SHLIB) $(BUILDIR)$(SHAREDLIBDIR)/$(SHLIB)
>       rm -f $(BUILDIR)$(LIBDIR)/$(SHLIB0)
> ifneq ($(BUILDIR),)
>       ldconfig -r ${BUILDIR} -n $(SHAREDLIBDIR)
> else
<       cd man; $(MAKE) MANDIR=$(MANDIR) install
> endif
>       cd man; $(MAKE) MANDIR=$(BUILDIR)$(MANDIR) install

3.7 Install Raid Tools

The next step is the installation of the raid tools. raidtools-0.42

You must run the "configure" script to point the Makefile at the build directory for the ramdisk files

  cd /usr/src/raidtools-0.42
  configure --sbindir=/root/raidboot/mnt/sbin --prefix=/root/raidboot/mnt/usr
  make install
Now!! the Makefile for install is not quite right so do the following to clean up. This will be fixed in future releases so that the re-linking will not be necessary.

Fix the make install error

The file links specified in the Makefile at 'LINKS' must be removed and re-linked to operate properly.

        cd /root/raidboot/mnt/sbin
        ln -fs mdadd mdrun
        ln -fs mdadd mdstop

3.8 Remove un-needed directories and files from new filesystem.

Delete the following directories from filesystem (CAUTION DON'T DELETE FROM YOUR RUNNING SYSTEM) it's easy to do, guess how I found out!!!

        cd /root/raidboot/mnt
        rm -r home/ftp/*
        rm -r lost+found
        rm -r usr/doc
        rm -r usr/info
        rm -r usr/local/man
        rm -r usr/man
        rm -r usr/openwin
        rm -r usr/share/locale
        rm -r usr/X*
        rm -r var/man
        rm -r var/log/packages
        rm -r var/log/setup
        rm -r var/log/disk_contents

3.9 Create /dev/mdx

The last step simply copies the /dev/md* devices from the current file system onto the rescue file system. You could create these with mknode.

        cp -a /dev/md* /root/raidboot/mnt/dev

3.10 Create a bare filesystem suitable for initrd

Now you have a clean re-useable filesystem ready for customization. Once customized, this file system can be used for rescue should the raid device(s) become corrupted and the raid tools needed to fix them. It will also be used to boot and root-mount the raid device by adding the linuxrc file which will be discussed next.

Copy the file system to a smaller device for the initrd file, 16 megs should be large enough.

Create the smaller file system and mount it

        cd /root/raidboot
        dd if=/dev/zero of=bare.fs bs=1024k count=16
associate the file with a loop device and generate a ext2 file system on the file
        losetup /dev/loop1 bare.fs
        mke2fs -v -m0 -L initrd /dev/loop1
        mount /dev/loop1 mnt2
Copy the 'build' file system to 'bare.fs'
        cp -a mnt/* mnt2
Save the 'bare.fs' system before customization so later update is easy. The 'build' file system is no longer needed and may be deleted.
        cd /root/raidboot
        umount mnt
        umount mnt2
        losetup -d /dev/loop0
        losetup -d /dev/loop1
        rm build
        cp bare.fs rescue
        gzip -9 bare.fs

Create the BOOT/RESCUE initrd filesystem

Now copy the system dependent items that match the kernel from the development platform, or you can manually modify the files in the rescue file system to match your target system.

        losetup /dev/loop0 rescue
        mount /dev/loop0 mnt
Make sure your etc directory is clean of *~, core and log files. The next 2 commands creates some warning messages, ignore them.
        cp -dp /etc/* mnt/etc
        cp -dp /etc/rc.d/* mnt/etc/rc.d

        mkdir  mnt/lib/modules
        cp -a  /lib/modules/2.x.x mnt/lib/modules <--- your current 2.x.x
Edit the following files to correct them for your rescue system.
        cd mnt

        etc/fstab       comment out the mount of root and raid devices.
        etc/mdtab       should work OK
        etc/hosts.equiv         and related files
        etc/rc.d/rc.inet1       correct ip#, mask, gateway, etc...
        etc/rc.d/rc.S           remove entire section on file system status
                        # Test to see if the root partition isread-only
                to but not including:
                        # remove /etc/mtab* so that mount will .....
                                This avoids the annoying warning that
                                the ramdisk is mounted rw.
        etc/rc.d/rc.xxxxx       others as required, see later on in this doc
        root/.rhosts            if present
        home/xxxx/xxxx          others as required

    WARNING:    The above procedure moves your password and shadow
                files onto the rescue disk!!!!!

    WARNING:    You may not wish to do this for security reasons.
Create any directories for mounting /dev/disk... as may be required that are unique to your system. Mine need:
        cd /root/raidboot/mnt           <--- initrd root
        mkdir dosa                      dos partition mount point
        mkdir dosc                      dos mirror mount point
The rescue file system is complete!

You will note upon examination of the files in the rescue file system, that there are still many files that could be deleted. I have not done this since it would overly complicate this procedure and most raid systems have adequate disk and memory. If you wish to skinny down the file system, go to it!

Making initrd boot the RAID device - linuxrc

To make the rescue disk boot the raid device, you need only copy the executable script file:


to the root of the device.

---------------------- linuxrc --------------------
# ver 1.07 2-12-98
# mount the proc file system
/bin/mount /proc

# This may vary for your system.
# Mount the dos partitions, try both
# in case one disk is dead
/bin/mount /dosa
/bin/mount /dosc

# Set a flag in case the raid status file is not found
# then check both drives for the status file
RAIDOWN=" not found"
/bin/echo "Reading md0 shutdown status."
if [ -f /dosa/linux/ ]; then
  RAIDOWN=`/bin/cat /dosa/linux/`
  RAIDREF=`/bin/cat /dosc/linux/raidgood.ref`
  if [ -f /dosc/linux/ ]; then
    RAIDOWN=`/bin/cat /dosc/linux/`
    RAIDREF=`/bin/cat /dosc/linux/raidgood.ref`

# Test for a clean shutdown with all disks operational
if [ "${RAIDOWN} != ${RAIDREF}" ]; then
  echo "ERROR ${RAIDOWN}"
#  Use the next 2 lines to BAIL OUT and leave rescue running
   /bin/echo 0x100>/proc/sys/kernel/real-root-dev
   exit                 # leaving the error files in dosa/linux,etc...

# The raid array is clean, proceed by removing
# status file and writing a clean superblock
/bin/rm /dosa/linux/
/bin/rm /dosc/linux/
/sbin/mkraid /etc/raid1.conf -f --only-superblock

/bin/umount /dosa
/bin/umount /dosc

# Mount raid array
echo "Mounting md0, root filesystem"
/sbin/mdadd -ar

#  If there are errors - BAIL OUT and leave rescue running
if [ $? -ne 0 ]; then
   echo "RAID device has errors"
#  Use the next 3 lines to BAIL OUT
   /bin/rm /etc/mtab            # remove bad mtab
   /bin/echo 0x100>/proc/sys/kernel/real-root-dev

# else tell the kernel to switch to /dev/md0 as the /root device
# The 0x900 value the device number calculated by:
#  256*major_device_number + minor_device number
/bin/echo 0x900>/proc/sys/kernel/real-root-dev

# umount /proc to deallocate initrd device ram space
/bin/umount /proc
/bin/echo "/dev/md0 mounted as root"
#------------------ end linuxrc ----------------------
Add 'linuxrc' to initrd boot device
        cd /root/raidboot
        chmod 777 linuxrc
        cp -p linuxrc mnt

3.11 Modifying the rc-scripts for SHUTDOWN

To complete the installation, modify the rc scripts to save the md status to the real root device when shutdown occurs.

In slackware this is rc.0 -> rc.6

I have modified Bohumil Chalupa's raid stop work-around slightly. His original solution is presented in Appendix A.

Since there are no linux partitions left on the production system except md0, the dos partitions are used to store the raidOK readonly status. I chose to write a file to each dos partition containing the status of the md array at shutdown and signifying that the md device has been remounted RO. This allows the system to be fail safe when one of the hard drives dies.

I modified my rc.6 script to attempt dismount of the root raid1 array and any other raid device in mdtab. You may need slightly different scripts, but the basics should be the same. The complete rc.6 file is shown in Appendix B.

To capture the raid array shutdown status, just before the file systems are dismounted insert:

        RAIDSTATUS=`/bin/cat /proc/mdstat | /usr/bin/grep md0`
After all the file systems are dismounted (the root file system 'will not' dismount) add:
        # root device remains mounted RO
        # mount dos file systems RW
        mount -n -o remount,ro /
        echo "Writing RAID read-only boot FLAG(s)."
        mount -n /dosa
        mount -n /dosc
        # create raid mounted RO flag in duplicate
        # containing the shutdown status of the raid array
        echo ${RAIDSTATUS} > /dosa/linux/
        echo ${RAIDSTATUS} > /dosc/linux/

        umount -n /dosa
        umount -n /dosc

        # Stop all the raid arrays (except root)
        echo "Stopping raid"
        mdstop -a
This will cleanly stop all raid devices except root. Root status is passed to the next boot in

Copy the rc file to your new raid array, the rescue file system that is still mounted on /root/raidboot/mnt and the development system if it is on the same machine.

Modify rescue etc/fstab as needed and make sure rescue mdtab is correct.

Now copy the rescue disk to your dos partition and everything should be ready to boot the raid device as root.

        umount mnt
        losetup -d /dev/loop0
        gzip -9 rescue
Copy rescue.gz to your dos partition.

All that remains is to test the new file system by rebooting. See the loadlin parameters in my dos linux.bat file next.

3.12 Setting up loadlin boot for RESCUE and RAID

The disks I chose for my system are much larger than those manageable by lilo. Therefore, I used loadlin to boot the system from a small dos partition with a mirror (copy) on the companion disk.

My dos root system contains a small editor among the utilities so I can modify the boot parameters of loadlin if necessary, allowing me to reboot the linux system on my swap disk while testing.

The dos system contains this tree for linux"

        c:\linux\    (only at shutdown)
linux.bat contains:
---------------------- linux.bat ---------------------------
rem  Sample DOS batch file to boot Linux.
rem  Start the LOADLIN process:

rem c:\linux\loadlin c:\linux\zimage root=/dev/ram0 ro ramdisk_size=16384 initrd=c:\linux\rescue.gz mem=131072k
c:\linux\loadlin c:\linux\zimage root=/dev/md0 ro ramdisk_size=16384 initrd=c:\linux\rescue.gz mem=131072k

rem -- this is my development system -- it goes away later
rem c:\linux\loadlin c:\linux\zimage root=/dev/hda3 ro noinitrd mem=131072k

***** >> NOTE!! the only difference between forcing the rescue system to
            run and the raid device mounting, is the loadlin parameter

                root=/dev/ram0          for the rescue system
                root=/dev/md0           for RAID

                With root=/dev/ram0 the RAID device will not mount
                and the rescue system will run unconditionally.

If the RAID array fails, the rescue system is left mounted and running (this seems to fail sometimes, I don't know why, it works when the reset button is pushed but does not work with 'shutdown -r now').

Previous Next Contents