LUKS and Initramfs

Initramfs Introduction

Before we can continue with configuring the kernel when installing a new gentoo system, we need to create an initramfs.

Since the root partition is encrypted, it has to be decrypted during the boot process, which is not done by the linux kernel, so it has to be done in userspace - early userspace. Early userspace is a set of libraries and programs that provide various pieces of functionality that are important enough to be available while a linux kernel is booting up, but it doesn't need to be run inside the kernel itself.

An initramfs image is a gzipped cpio format archive, which in our case is responsible for decrypting the root partition.

Let's describe the ramfs, rootfs, and initramfs a little. A whole article about that can be read here [1,2].

  • ramfs: Normally all files are cached in memory. Pages of data read from hard drive are kept around in case they are needed again, but marked as clean in case the Virtual Memory system needs the memory for something else. Similarly, data written to files is marked as clean as soon as it has been written to a hard drive, but kept around for caching purposes, until the VM reallocates the memory. If there is no hard drive, files written into ramfs allocate dentries and page cache as usual, but there is nowhere to write them to. This means the pages are never marked clean, so they can't be freed by the VM when it's looking to recycle memory.
  • tmpfs: One downside of ramfs is we can keep writing data into it until we fill up all memory, and the VM can't free it because the VM thinks that files should get written to hard drive. However, ramfs don't have a backing store. Because of this, only root users should be allowed to write to a ramfs mount. A tmpfs has the ability to also write data to swap space, so normal users can be allowed to write to tmpfs mount.
  • rootfs: Rootfs is a special instance of ramfs, which is always present in 2.6 systems.
  • initramfs: Initramfs is a root filesystem which is embedded into the kernel and loaded at an early stage of the boot process. It is the successor of initrd. It provides early userspace which lets us do things that the kernel can't do by itself during the boot process. Using the initramfs is of course optional. By default, the kernel initializes hardware using built-in drivers, mounts the specified root partition, and loads the init system of the installed linux distribution. The init system then loads additional modules and starts services until finally allows us to log in. With initramfs we can do things even before the root partition is mounted. With initramfs we can do the following things:
  • Customize the boot process (print a welcome message, boot splash, etc).
  • Load modules that cannot be integrated into the kernel directly.
  • Mount the encrypted partition.
  • Provide a minimalistic rescue shell if something goes wrong.
  • We can do anything the kernel can't do as long as we do it in the user space by executing commands.

Creating Initramfs

Initramfs is a root filesystem which is embedded into the kernel and loaded at an early stage of the boot process. The initramfs must contain at least one file, the /init, which is executed by the kernel as the main init process (PID 1) and must do all the work. In addition there can be any number of additional files and directories required by /init. They are usually files we will also find on any other root filesystem, such as /dev for device nodes, /proc for kernel information, /bin for additional binaries, etc. When the kernel mounts the initramfs, our root partition is not yet mounted, so we can't access any of our files. This means that we only have files in initramfs available, which must contain everything we need. If we want a shell, we must include it in initramfs [3].

Installing Dependencies

Before doing anything, we must reinstall the following packages to use the static use flag, so they can be copied to the initramfs system without also needing to copy their dependencies:

# echo "sys-fs/lvm2 static" >> /etc/portage/package.use
# echo "sys-apps/busybox static" >> /etc/portage/package.use
# echo "sys-fs/cryptsetup static" >> /etc/portage/package.use

And reemerge them:

# emerge sys-fs/lvm2 sys-apps/busybox sys-fs/cryptsetup

We'll probably also need to add static-libs use flag to other dependency packages like libgpg-error, util-linux, etc.

Directory Structure

When creating the initrd image, we must first create a directory that will later become initramfs root as well as basic directory layout:

# mkdir /usr/src/initramfs
# cd /usr/src/initramfs/
# mkdir -p bin lib dev dev/mapper dev/vc etc mnt/root proc root sbin

Next, we need to copy required device nodes over to the initramfs:

# cp -a /dev/{null,console,tty,random,urandom /usr/src/initramfs/dev/

Character and Block Devices

We can also use the mknod command to create device nodes. The mknod command has the following syntax:

# mknod device-name device-type major-number minor-number

The device-name is the full name of the device, the device-type is a block or a character device, the major-number is a number referring to what group the device is in and a minor-number is the number of the device within a group. The minor and major numbers are chosen by the writers of the kernel, and are describes in /usr/src/linux/Documentation/devices.txt. Example from that file is presented below:

1 char Memory devices
1 = /dev/mem Physical memory access
2 = /dev/kmem Kernel virtual memory access
3 = /dev/null Null device
4 = /dev/port I/O port access
5 = /dev/zero Null byte source
6 = /dev/core OBSOLETE - replaced by /proc/kcore
7 = /dev/full Returns ENOSPC on write
8 = /dev/random Nondeterministic random number gen.
9 = /dev/urandom Faster, less secure random number gen.
10 = /dev/aio Asynchronous I/O notification interface
11 = /dev/kmsg Writes to this come out as printk's
12 = /dev/oldmem Used by crashdump kernels to access
the memory of the kernel that crashed.

1 block RAM disk
0 = /dev/ram0 First RAM disk
1 = /dev/ram1 Second RAM disk
250 = /dev/initrd Initial RAM disk

In the output above, the 1 is a major number, whereas the char is character device and block is a block device. The 1-12 and 0-250 are minor numbers and /dev/mem to /dev/initrd are names. The following command creates a character device /dev/null:

# mknod /dev/null c 1 3

Binary Programs

Any binary we want to execute at boot needs to be copied into initramfs; we also need to copy any libraries that the binary requires. We can use the ldd command to see what libraries are needed by certain binary.

Instead of creating countless utilities and libraries we can use busybox, which contains a set of utilities (such as ls, mkdir, mount, insmod, and more) in a single binary. To include busybox into initramfs, we need to enable the static USE flag and reemerge it, making it statically linked, so it doesn't require any additional libraries.

We can check with ldd to make sure that busybox is not dynamic executable:

# ldd /bin/busybox
not a dynamic executable

# cp /bin/busybox /usr/src/initramfs/bin/

The busybox is a nice utility to have, but not as important as cryptsetup and lvm, which we must have to be able to decrypt the contents of our encrypted partition. Of course, the file should not be dynamically linked, which can be verified with the following command:

# ldd /sbin/cryptsetup
not a dynamic executable

# ldd /sbin/cryptsetup
not a dynamic executable

# ldd /sbin/lvm.static
not a dynamic executable

# cp /sbin/cryptsetup /usr/src/initramfs/sbin/
# cp /sbin/lvm.static /usr/src/initramfs/sbin/
# mv /usr/src/initramfs/sbin/lvm.static /usr/src/initramfs/sbin/lvm

The Init Script

The /init script gets executed when the initramfs is loaded. Busybox includes a fully functional shell, which means we can write our /init in a bash scripting language instead of making a complicated application written in assembler or C/C++. The following example realizes the /init executable as a minimalistic shell script based on the busybox shell:

#!/bin/busybox sh

# init to execute after switching to real root

# Die function if something goes wrong
die() {
info "Dropping you into a minimal shell:"
exec /bin/sh

# Parse the arguments passed to the kernel option in grub.conf
parse_kernel_args() {
local x
CMDLINE=`cat /proc/cmdline`
for param in $CMDLINE; do
case "${param}" in
root_device="`echo "${param}" | cut -d'=' -f2`"
kmap="`echo "${param}" | cut -d'=' -f2 | cut -d':' -f1`"
swap_device="`echo "${param}" | cut -d'=' -f2 | cut -d':' -f1`"

# Main Function
# path to search for binaries
export PATH="/sbin:/bin:/usr/bin:/usr/sbin"
umask 0077

# create needed directories (for mountpoints)
#for dir in proc sys dev newroot; do mkdir -p /$dir; done

# mount needed filesystems
/bin/busybox mount -t proc proc /proc
/bin/busybox mount -t sysfs sysfs /sys
/bin/busybox mount -t devtmpfs udev /dev

# parse grub's kernel arguments

# load keymap if it exists
if [ -n "$kmap" ]; then
loadkmap < "/etc/${kmap}"
die "Error: keymap /etc/${kmap} does not exist."

# create /dev/sda encrypted partition
echo /bin/mdev > /proc/sys/kernel/hotplug
/bin/busybox mdev -s

# LUKS: decrypt the encrypted partition
/sbin/cryptsetup -T 5 luksOpen "${root_device}" system

# LVM: enable the LVM partitions
/sbin/lvm vgscan
/sbin/lvm vgchange -ay

# mount the root filesystem
#/bin/busybox mount /dev/mapper/system /newroot
/bin/busybox mount /dev/mapper/vg-root /newroot
if [ "$?" -ne 0 ]; then
/sbin/cryptsetup luksClose system 2>/dev/null || cryptsetup
remove system
die "Error: mount root failed, dm-crypt mapping closed."

# unmount unneeded filesystems
if mountpoint -q /dev/pts ; then umount /dev/pts; fi
/bin/busybox umount -l /proc
/bin/busybox umount -l /sys
/bin/busybox umount -l /dev

# move the mounted filesystems to newroot
#/bin/busybox mount --move /sys /newroot/sys
#/bin/busybox mount --move /proc /newroot/proc
#/bin/busybox mount --move /dev /newroot/dev

# resume from the hibernation file
echo 1 > /sys/power/tuxonice/do_resume

# switch to root of another filesystem and start the init process
exec switch_root /newroot "${init}"

The /init script is properly commented, so no additional comments are required. During the booting process, we will have to enter the password whenever the cryptsetup command will be executed. Once we enter the password, the partition will be available at /dev/mapper/system. In the example script we can also see that we used mdev, which is busybox version of udev. When we run mdev, the device nodes are created so that /dev/sda3 is available when we want to use it with cryptsetup and mount command. To make mdev available, we must build busybox with the mdev use flag as follows:

# echo "sys-apps/busybox static mdev" >>
# emerge busybox

We also need to make the /init script executable in order for the boot process to be able to run it. To do that we can issue the command below:

# chmod +x /usr/src/initramfs/init

If we want to be dropped to a rescue shell if an error occurs, we can add the following into /init and call it when something goes wrong:

rescue_shell() {
echo "Something went wrong. Dropping you to a shell."
busybox --install -s
exec /bin/sh

Then we need to call the rescue_shell script whenever the error occurs and we'll be dropped into the rescue shell where we can execute the commands and check what the error was.

If we have an internet connection to the network and we don't have a monitor present (which is quite common on servers), we can access the rescue shell remotely via the network, using telnetd. For this to work, we must first populate the /etc/passwd, /etc/shadow and /etc/group files in our initramfs. Then we can bring up the network and start a telnet server:

remote_rescue_shell() {
# Bring up network interface
ifconfig eth0 up

# telnetd requires devpts
mkdir -p /dev/pts
mount -t devpts none /dev/pts

# Start the telnet server

# Continue with the local rescue shell

Actually Creating the Initramfs Image

The initramfs can be made available to our kernel at boot time by packaging it as a compressed cpio archive. This archive is then embedded directly into our kernel image or stored as a separate file which can be loaded by grub during the boot process.

With either method, we need to enable support for Initial RAM filesystem and RAM disk (initramfs/initrd) support in the kernel:

General setup --->
[*] Initial RAM filesystem and RAM disk (initramfs/initrd) support

If we want the initramfs embedded into the kernel image, we need set the Initramfs source file(s) to the root of our initramfs, the /usr/src/initramfs.

General setup --->
(/usr/src/initramfs) Initramfs source file(s)

Now when we compile our kernel, it will automatically compress the files into a cpio archive and embed it into the bzImage.

# make && make modules && make modules_install

Note that we should also include the support for the ciphers we use and we should not build them as modules. To create a standalone archive image, issue the following command:

# cd /usr/src/initramfs
# find . | cpio --quiet -o -H newc | gzip -9 >

This will install the initramfs.cpio.gz in /boot directory. We need to instruct grub to load this file at boot time with the initrd line. The kernel line is the same as above, so the /boot/kernel is the actual kernel we copied from the arch/x86_64/boot/bzImage.

root (hd0,0)
kernel /boot/kernel root=/dev/sda1
initrd /boot/initramfs.cpio.gz

When compiling a new kernel, just replace the /boot/kernel with the newly compiled kernel and the initrd line can stay the same.

Apply the new grub:

# grub-install /dev/sda

We can now reboot our machine. On boot, the kernel will extract the files from our initramfs archive automatically and execute the /init script, which will take care of mounting the root partition and executing the init of our installed linux distribution.


[1]: ramfs, rootfs and initramfs,

[2]: initramfs,

[3]: DM-Crypt with LUKS,