more IBM historical archives ...
The vmlinuz and initrd files are in a /boot directory. This directory is special — it is reserved for boot files, and the bootloader knows to look for this directory. It is generally a separate partition of the primary disk where the operating system is installed.
The full names of the vmlinuz and initrd files includes the kernel version number and an optional final tag. This string must match between vmlinuz and initrd, and is also used as the name of a subdirectory of /lib/modules. This subdirectory is the root of the directory tree where the kernel objects are stored.
The boot directory can have multiple kernels. The bootloader grub is configured by a script to search through /boot and to prepare a boot screen listing all the kernels it sees in /boot. If the kernel you hack for the course projects crashes, or fails to boot, you can always select to boot the original kernel.
There are two kernels, with associated initrd's. In the lib/modules directory there are two subdirectories with the matching modules, identified by the tags "-generic" and "-cs421" in this image. Yours will have -generic and -pika, if you were pikachu.
While the Ubuntu version is 16.04 (more precisely, 16.04.6 in this install), the kernel is numbered separately. All the versions of Linux share the kernel sources. Use uname to get information on the currently running kernel (see the screen shot).
Rather than compile all the code that is every possibly needed into vmlinuz, Linux has a system of modules, which is code that can be drawn into the kernel dynamically, as needed, after boot. Modules are code files with .ko extensions.
For instance, a special driver for a USB device might require kernel code, but Linux cannot possible deliver all such special drivers to all installs. Instead, there is a system of device probing that will dynamically match up a USB device with the drivers it requires, and the module containing the driver will be drawn into the kernel on-demand.
Because there are drivers for almost every piece of hardware, and there is such a variety of hardware, there are a lot of modules. To compile all modules can take days. We will adjust the kernel configuration file to direct the build to the minimum of modules.
Besides, make employs strategies to minimize the rebuild, by recognizing that if a .o file exists, and it is younger than its .c files, it does not need to build the .o file — the existing .o file is current. This way all modules, and much of the kernel, is only built once (unless you clean your source tree, to force a complete rebuild).
sudo -s apt-get install build-essential ncurses-dev libssl-dev apt-get install linux-source apt-get --no-install-recommends install kernel-package cd /usr/src/linux-source-4.4.0 tar jvxf linux-source-4.4.0.tar.bz2
cd /usr/src/linux-source-4.4.0/linux-source-4.4.0 cp /boot/config-`uname -r` .config make menuconfig
make make modules_install make install
For the 221 edition of the course, students now have M1 chips and some things have changed even for the 32-bit i386 architecture.
CONFIG_SYSTEM_TRUSTED_KEYS="debian/canonical-certs.pem"to
CONFIG_SYSTEM_TRUSTED_KEYS=""
A single linux install can boot to any of several kernels. You will not be overwriting the old kernel with the new kernel. You will be provided an option at boot to boot any of the installed kernels. Since the modules are in directories with names that correspond to the kernel names, the proper modules will be used on any boot.
The BIOS (firmware) know how to find a small boot code on a disk. The purpose of this code is to find the Boot Loader on the disk, to load it into memory and transfer control to it. The Boot Loader then finds the operating system kernel on the disk, loads it, and transfer control to it. The operating system does startup tasks, first in single user mode, then in multi-processing mode, before starting the init process, which makes the computer available for logins.
Historically, there have been three linux boot loaders: lilo, grub and grub2. Most Linux distros are using grub2. A step in the makefile that installs the built kernel is to call an update script that includes all known kernels found in /boot into the grub boot menu.
The images show my grub menu after following these instructions for a kernel build. Once the first screen appears, use any key to stop the automatic boot to the default kernel. Navigate to advanced options and enter, then navigate to the kernel of choice and enter.
You can modify grub2's boot behavior. If your grub is doing an automatic, silent boot to the default kernel, without the option to select a kernel, edit /etc/default/grub as follows:
#GRUB_HIDDEN_TIMEOUT=0 # comment out this line
GRUB_CMDLINE_LINUX_DEFAULT="" # set this to the empty string
then run update-grub.
this will give a verbose boot with a pause to select among the alternative kernels.
Now you are ready to try your new kernel. Type reboot and your machine will reboot. At the boot menu, select your kernel. Hopefully it will boot properly. Once booted, use uname -r to see the revision number of the kernel you are running. If it is yours, congratulations!
Before building a kernel, you need to configure the build. This states the options that the kernel will include, will choose whether certain functionality is built-in or placed in a separable module; and what functionality and drivers will be omitted from the build altogether.
The master file for configuration is the hidden file .config which is placed in the root of the build tree. If unpacked your source into the directory /usr/src/linux-source-xyz, so the full pathname of the config file is /usr/src/linux-source-xyz/.config. To install the configuration, you must run a script, for example make oldconfig, to integrate into the Kconfig files that are scattered around the build tree those options specified in .config. (I think, I really don't understand this thoroughly.)
Other scripts are make defconfig, which creates a configuration based on the current platform, or make menuconfig that presents a curses (a ^5.2 Rogue) interface to make build selections. You will need ncurses-dev installed for menuconfig to work.
Make install does several things:
Run make help in the root of the build source for information on the build targets. Modules are each a separate file, with a .ko extension. They are installed by being copied into the special directory /lib/modules/[kernel revision name]. Kernel modules are parts of the kernel that are dynamically loaded as needed.
Linux uses a very intelligent buffer cache to shuttle data blocks to and from the disk. Each buffer in the buffer cache remembers what disk block it mirrors. A disk read can satisfied without going out to the disk if the block is sitting in the buffer cache, because the block has been recently read or written.
Given that the buffer cache exists, it was realized that a RAM disk can be created more efficiently by stopping the transfer between file system and disk at the buffer cache, and marking the buffer in the cache which carries data as "permanent".
This eliminates an essentially useless copy from one in-RAM buffer to another — from the buffer cache to the "RAM disk". Just sitting permanently in the buffer cache is sufficient to create an in-RAM file system.
This scheme was christened the ramfs.
After building and installing the kernel and the modules, you need to build the initial root directory. In case you google for additional help on this, beware that the initial root directory process has changed recently. It now uses a simple cpio into a ramfs, rather than imaging a RAM-disk. (See sidebar for discussion of Ramfs versus RAM-disk.)
At boot, the boot loader knows only enough about reading from disk so that it can copy the kernel from a file on disk into the memory, and then begin running the kernel (early boot). The kernel itself might rely on modules to provide the code necessary to access the file systems on disk. These modules are files that might reside on the file systems which the kernel cannot yet read.
Linux solves this problem by having a partition on disk in a file format that the boot loader and the initial kernel know how to read. The kernel is stored in this partition, as well as an archive of the initial file system required by the kernel during early boot. The kernel creates a ramfs and unpacks this archive into the ramfs and mounts this as the initial root directory.
This partition is typically ext2, and is mounted into the fully booted file system at the name /boot.
This root serves the kernel during early boot, but is later jettisoned when the real root directory is discovered on disk and mounted over the initial root, called re-rooting or pivoting on root.
The process of creating a initial root directory is automated by the update-initramfs script. This is generally called as part of the default build, and you can also see it run during certain kernel updates. If this file is not correct, the boot process will go well up until the first file access to a file system. The error messages will be about not finding a root file system, or not recognizing it. That means the initrd was not properly formed or placed.
Author: Burton Rosenberg
Created: September 5, 2010
Last Update: 10 Sept 2020