# Building installer images

This document only provides an introduction for building installer images using existing definitions. The README file under `installer/build/` in the GIT repository contains more detailed information about the build system and how to modify existing or define new images.

An image consists of:

- a kernel;
- an initrd, which is basically a collection of unpacked udebs;
- in some cases a boot loader and/or configuration files used for booting.

Most d-i images are ready for use. The exception are the cdrom images which form only the base (kernel and initrd) for creating the actual installer images. The package used for creating the installer images is `debian-cd`.

On some architectures there is one D-I image that is ready for use: the `mini.iso`. This image is produced as a by-product of the netboot target. A mini.iso image does not really support installing from local media as it retrieves all additional udebs and packages over the network.

It is important to distinguish between building images for release and building images for development/testing use.

A release build is done, as for other packages that are to be uploaded, from the `installer` directory using `debian/rules`. This will create a binary package (needed for uploading) containing some documentation, but the important bit is a tarball containing all installer images. After the upload this tarball needs BYHAND processing

This entails unpacking the tarball into the correct location on the master mirror server and creating/updating the correct symlinks. See for example .

by FTP-masters before the buildds will pick up the upload for other architectures. Nowadays the BYHAND processing for `debian-installer` (and also for `tasksel`) is automated.

Building images for development and testing is done from the `installer/build` directory

This includes the daily built images available from . These are generated and uploaded from machines run by d-i porters using the `daily-build` script.

using `fakeroot make <`target`>`.

## The D-I images

An important difference between release and development builds is that release builds will use udebs for the same suite as the target system being installed, while development builds will by default install testing, but use udebs from unstable. This allows to mostly avoid the occasional breakage of the base system and tasks in unstable while using the most recent udebs.

### Debian Installer images

- daily D-I builds are built in unstable and use udebs from unstable
- regular D-I uploads to the archive are built in unstable and use udebs from testing
- D-I uploads to stable are built in stable and use udebs from stable + proposed-updates

The two config variables that determine this are:

USE_UDEBS_FROM

- Defines the suite or codename from where udebs are taken. The default is set to `unstable` in `./config/common` for GIT/trunk. For official uploads the value of this variable is overruled in `debian/rules`.

  The value in `./config/common` should be changed to the correct codename for RC release uploads (after branching) to ensure correct builds from the branch post-release.

DEBIAN_RELEASE

- Defines the codename of the Debian release that should be installed by default. It gets included in /etc/default-release in most initrds and is read by various components during installation.

  Because this is set to a codename, the value is correct for the whole lifetime of a Debian release and helps ensure the correct suite gets installed even when a release becomes oldstable or gets archived.

  Some installation methods allow this value to be overruled by choosing a different suite during mirror selection or passing the `suite=` option boot parameter.

### CD/DVD/BD images

- daily sid_d-i installer images are built using daily D-I builds, udebs from unstable and regular packages from testing;
- daily testing_d-i installer images are built using the current D-I release in testing, udebs from testing and regular packages from testing;
- weekly builds are built using the current D-I release in testing, udebs from testing and regular packages from testing;
- official image builds use everything from stable and thus can only be built after the archive has already been updated to a new stable (point) release.

The sid_d-i images are what's normally linked as the daily built images from the D-I project page; when preparing for a release those will link to testing_d-i images to facilitate pre-release testing.

## Requirements for building

For both release and development builds the build dependencies as listed in `installer/debian/control` need to be satisfied.

To build installer images from GIT trunk, the build machine needs to be running unstable or you need to set up a sid chroot to build in. (To build images from one of the older release branches of the repository, the build machine needs to run or have a chroot for that release.)

During the build, the needed udebs will be retrieved from a mirror. By default this mirror is based on your `/etc/apt/sources.list` (see the generated file `build/sources.list.udeb`). To use a different source, create a file `sources.list.udeb.local`.

## Build targets

To see which targets are available, run `make`. This will result in a list of some 130 targets, most of which are not really relevant. A more useful list can be obtained with `make | grep ^build`. The table below has the most often used targets for x86.

`build_all` | Builds all images
`build_cdrom_isolinux` | Builds the cdrom images
`build_netboot` | Builds the netboot images and the mini.iso
`build_netboot_gtk` | Builds the netboot images and the graphical gtk/mini.iso
`reallyclean` | Completely cleans the build environment

The `reallyclean` target is often needed when changes are made between builds because otherwise udebs or information may be retrieved from temporary or cache directories and the changes will not take effect. The `rebuild_*` targets clean some of this, but not always enough.

## The build system explained

The easiest way to start is with the purpose of the subdirectories in the `installer/build` directory.

- `util`: contains helper scripts called from the Makefile

- `config`: defines the available targets (per architecture)

- `pkg-lists`: defines which udebs are included in an image (per image type)

- `boot`: contains configuration files and make targets used to make images bootable

- `localudebs`: allows using (versions of) udebs not available on the mirror you use

Two files containing important configuration info are `config/dir` and `config/common`. However, normally there should be no need to modify any of the variables defined in these files.

Both the `config` and `pkg-lists` directories have a tree structure with general configuration defined in the root and more specific configuration defined in branches and leaves. Branches are defined in directories that have the same name as a config file on the higher level. The `config` directory contains Makefile snippets.

### config

For example, the definition for amd64 images starts with `config/amd64.cfg` which, besides the current kernel versions, defines the media supported with the line:

    MEDIUM_SUPPORTED = cdrom cdrom-xen netboot netboot-xen hd-media

These media correspond to the main targets for amd64 and are further defined in `config/amd64`. The `hd-media.cfg` file in that directory contains, amongst others, the following lines:

    FLAVOUR_SUPPORTED = "" gtk
    EXTRATARGETS = build_hd-media_gtk

The first line defines that the netboot image has two flavors: the default one (for the newt frontend), and a second one for the gtk frontend. The latter is further defined in the `config/amd64/hd-media/gtk.cfg` file. The second line ensures that the gtk image will always be built together with the newt image.

The files in config are processed recursively to dynamically generate the build targets, so in this example you get a `hd-media`, a `hd-media_gtk` target and targets for the other media.

The structure of the config files can get quite complex and it can be hard to keep track of the exact role of the different variables set in them.

### pkg-lists

The list of udebs to be included in an image is built by the `util/pkg-list` script based on definitions in the `pkg-lists` directory. Again, processing can be quite complex. Let's take the netboot target for amd64 as an example to explain it.

First the file `pkg-lists/netboot/amd64.cfg` is considered and all udebs listed in it are added. Some example lines from that file:

    console-keymaps-at
    nic-modules-${kernel:Version}
    mmc-modules-${kernel:Version} ?

The variable `${kernel:Version}` is expanded to match the package name of the udeb based on the kernel version and flavor. If the name of a udeb is followed by a question mark it is skipped if the package is not available (without the question mark an error would be generated).

The `pkg-list` script will also look for the presence of files named `common` and `local` and thus `pkg-lists/netboot/common` is processed next. This file exists and lists a number of udebs that belong in any netboot image, independent of the architecture. This file contains two include directives which result in the specified files being processed next:

    #include "base"
    #include "kernel"

Thus, udebs listed in `pkg-lists/base` (containing udebs common to all images) and `pkg-lists/kernel` (included in all bootable images) are also processed.

The file `pkg-lists/netboot/local` does not normally exist as it is intended for the inclusion of non-standard udebs. It is also very useful for testing as it can be used to temporarily add udebs not normally included in an image without the need to modify the regular files.

Finally, the script will check for `pkg-lists/local` and `pkg-lists/exclude`. The latter exists and contains some udebs otherwise pulled in by dependencies, but that should not be included because of library reduction, which is covered in the next section. Note that the exclusion is not triggered by the file name, but rather by the dash after the name of the udebs.

All dependencies of udebs listed in `pkg-lists` will also be automatically included in the image.

To see how the package list is built for a particular image, set `my $debug=1;` in the `util/pkg-list` script.

## Result of the build

If the build is successful, the images that were built can be found under the `build/dest` directory. Depending on the type of build you will also find manifest and log files there.

Before the image is created, its contents are assembled in the directory `build/tmp/<`target`>`. The `tree` subdirectory there contains the full contents of the initrd; other subdirectories are used for different purposes.

## Library reduction

Library reduction (relinking a library leaving out unused symbols) is used as yet another method to minimize the size of initrds. The downside of library reduction is that this requires the `dev` and `pic` packages for the libraries to be reduced to be installed on the build system which also means that their version needs to match the version of the libraries in the udebs.

The size reduction is most significant for libc (40%) and libm (90%). Other libraries that are reduced include libresolv, libslang and libnewt. The reduction is done by calling `mklibs` from the main `Makefile`.

As only the executables that are included in an image are taken into account during the library reduction, we have to provide for executables in components that are installed later as they would fail if they use symbols that have been taken out.

This is the reason that the udebs containing reduced libraries are excluded in `pkg-lists/exclude` which results in the udeb not being listed in the `/var/lib/dpkg/status` file in the intrd. If no udebs that are installed later depend on the library, all is well. If a udeb that does depend on it is installed later, `anna` (or rather `udpkg`) will see that the dependency is not satisfied, and will install the udeb so the unreduced library replaces the reduced version.

Note that library reduction is only done after unpacking udebs for inclusion in an image; the libraries included in udebs are never reduced.

## Using localudebs

The `localudebs` directory allows to use a different version of udebs than is available from the mirror you use. This can be used to test a new version of a udeb or to run the installer with a debug version of a udeb. It can also be used to build an image with a custom udeb.

To use a local udeb, just copy it into the directory. A `Packages` file will be generated automatically. Your udeb should have a version equal to or greater than the udeb currently on the mirror you use.

Note that local udebs will only be included in the image if the udeb would be included in a normal build too. So it has to be selected by the `pkg-list` script. Create a `pkg-lists/local` or `pkg-lists/<`image`>/local` to add udebs to the image that would not normally be included.

Some things to keep in mind when using localudebs.

- If you add an extra udeb, its dependencies will be included too. If those dependencies include virtual packages, the result is not always what you'd expect.

- Adding extra udebs will increase the size of the initrd; some architectures have limits for initrd size.

- If you use a `sources.list.udeb.local`, make sure to add as the first line:

      deb copy:<path-from-root-to>/installer/build/ localudebs/

- Don't forget to clean up after you're finished.