Installing guix via kexec

I got one of those free oraclecloud vms and I want it to run guix as in guix system. The problem is, the attachable boot volumes only come preselected by oracle, how boring. That is why I wanted to find alternative ways to install guix on a computer. One such method is running the installer via a kexec. kexec is a syscall that runs a new kernel with new initrd and arguments. The basic idea is to take the installation image and put it all into a big initrd. The normal guix startup code does not expect this but we can change it to handle this. Here are the steps how to run a kexec installation for guix.

1. Installation image

The command for in guix to generate an installation image is guix system image -t iso9660 path/to/install-os.scm. Normally, one would use ./gnu/system/install.scm for the installation image. That is this relative path in your guix checkout. I want to suggest a slight alteration: Enable ssh at startup and add an ssh public key. This way we do not need to rely on flaky consoles during the installation. For this we write our own installation-os definition like this:

my-install.scm

(define-module (my system install)
           #:use-module (gnu)
           #:use-module (gnu services ssh)
           #:use-module (gnu system)
           #:use-module (gnu system install)
           #:export (my-installation-os))

(define my-installation-os
  (operating-system
    (inherit installation-os)
    (services (modify-services
        (operating-system-user-services installation-os)
        (openssh-service-type 
          config =>
          (openssh-configuration
            (port-number 22)
            (permit-root-login #t)
            ;; The root account is passwordless, so make sure
            ;; a password is set before allowing logins.
            (allow-empty-passwords? #f)
            (password-authentication? #f)

            ;; Do start it upfront.
            (%auto-start? #t)
            (authorized-keys
              `(("root" ,(plain-file "my_ssh_key.pub" "<ssh_pubkey_here>"))
             ))))))))

my-installation-os

This is just the installation os from gnu/system/install.scm from which we inherit plus a modified ssh-configuration. What has changed is %auto-start which is set to true and password-authentication to false. Also a key is added to authorized-keys so we can login via key authentication.

The previous guix system image command can now be run with the modified installation os and will put an .iso image file into the store.

2. Extract installation image

We want not the image itself but what is inside so we mount it via a loopback device. We can use losetup -f to find the first unused loopback device.

# losetup -f
dev/loop0
# losetup /dev/loop0 /gnu/store/longstorehash/guix-install.iso
# mount /dev/loop0 /mnt
# cp -r /mnt $HOME/guix-install-image

3. Extract initrd

Now we have the whole contents of the bootstick in a nice directory. The first thing we need is the initrd it uses. In /boot/grub/grub.cfg we find the argument initrd with a store path that points to an initrd. That is our candidate. We extract that initrd into another folder:

# mkdir $HOME/guix-install-initrd
# cd $HOME/guix-install-initrd
# gzip -cdk $HOME/guix-install-image/gnu/store/longstorehash/initrd.cpio.gz | cpio -i

4. Join bootstick and initrd

Now we pack everything that was on the bootstick into the initrd or the other way around. It should not matter since we join the files, that is what I hope at least. This way we end up with a honking big initrd that contains everything guix needs to boot.

# mkdir $HOME/my-install-initrd
# cp -r $HOME/guix-install-initrd/* $HOME/my-install-initrd
# cp -r $HOME/guix-install-image/* $HOME/my-install-initrd

Now we touched a bunch of stuff and modified their timestamp. Scheme will not like it if the source files are newer than compiled files so we touch them back to the beginning of time in 1970.

# find $HOME/guix/my-install-initrd/gnu/store -exec touch -m --date=@0 {} \;

5. Customize guix boot code

The normal guix boot code expects something to be mounted and to switch root to there. Since we plan to pass everything over via the initrd it will not be mounted. Instead it will be present as initramfs called rootfs living in memory with no backing device. You can read the full story here. For us this means we need to load a modified version of the guix boot code. Take a copy of ./gnu/build/linux-boot.scm and comment or delete these three parts so they do not get executed.

(unless (or root-fs* (and root-device rootfstype))
  (error "no root file system or 'root' and 'rootfstype' parameters"))

;; If present, ‘root’ on the kernel command line takes precedence over
;; the ‘device’ field of the root <file-system> record; likewise for
;; the 'rootfstype' and 'rootflags' arguments.
(define root-fs
  (if root-fs*
      (file-system
        (inherit root-fs*)
        (device (or root-device (file-system-device root-fs*)))
        (type (or rootfstype (file-system-type root-fs*)))
        (options (or rootflags (file-system-options root-fs*))))
      (file-system
        (device root-device)
        (mount-point "/")
        (type rootfstype)
        (options rootflags))))

;; Mount the root file system.
(mount-root-file-system (canonicalize-device-spec
                         (file-system-device root-fs))
                        (file-system-type root-fs)
                        #:volatile-root? volatile-root?
                        #:flags (mount-flags->bit-mask
                                 (file-system-flags root-fs))
                        #:options (file-system-options root-fs)
                        #:check? (check? root-fs)
                        #:skip-check-if-clean?
                        (skip-check-if-clean? root-fs)
                        #:repair (repair root-fs))

(switch-root "/root")

We do not have a (traditional) root-file-system we do not want to mount it and we do not want to switch-root to it. In a way we are already at the goal and have to do nothing additionally. The modified file can be placed at ./guix-initrd-test/guile-modules/share/guile/site/3.0. That reason for the long ~/share/guile/site/3.0/ is that this is the subpath that the initrd will expect.

6. Customize init file

The last thing that is to do is make some changes to the init file in ./guix-initrd-test. It will have a mount of the installation image preconfigured that we need to delete. Also we will need to tell it to use our modified boot code instead of the default one. I will post first the unmodified and then the modified version followed by an explanation of the edits

6.1. unmodified init file

#!/gnu/store/0g1a9v98jj0lffq4dysyjrg0vx7fjcpi-guile-static-stripped-3.0.9/bin/guile --no-auto-compile
!#
(eval-when
    (expand load eval)
  (let ((extensions '())
        (prepend (lambda (items lst)
                   (let
                       loop
                     ((items items) (lst lst))
                     (if (null? items)
                         lst
                         (loop
                          (cdr items)
                          (cons (car items)
                                (delete (car items) lst))))))))
    (set!
     %load-path
     (prepend
      (cons "/gnu/store/zb1m3h384in6f8ffqsmhvp8236jrvnqs-module-import"
            (map
             (lambda (extension)
               (string-append
                extension
                "/share/guile/site/"
                (effective-version)))
             extensions))
      %load-path))
    (set!
     %load-compiled-path
     (prepend
      (cons "/gnu/store/lamjscp9iqb4bcc6lq37b1k5pyz1chan-module-import-compiled"
            (map
             (lambda (extension)
               (string-append
                extension
                "/lib/guile/"
                (effective-version)
                "/site-ccache"))
             extensions))
      %load-compiled-path))))
(begin
  (use-modules
   (gnu build linux-boot)
   (gnu system file-systems)
   ((guix build utils)
    #:hide
    (delete))
   (guix build bournish)
   (srfi srfi-1)
   (srfi srfi-26)
   ((gnu build file-systems)
    #:select
    (find-partition-by-luks-uuid))
   (rnrs bytevectors))
  (with-output-to-port
      (%make-void-port "w")
    (lambda ()
      (set-path-environment-variable
       "PATH"
       '("bin" "sbin")
       '())))
  (parameterize
      ((current-warning-port
        (%make-void-port "w")))
    (boot-system
     #:mounts
     (map
      spec->file-system
      (quote
       (((uuid
          iso9660
          #vu8
          (49
           57
           55
           48
           48
           49
           48
           49
           49
           57
           51
           51
           51
           49
           56
           51))
         "/"
         "iso9660"
         ()
         #f
         #f
         #t
         #t
         preen))))
     #:pre-mount
     (lambda () (and #t))
     #:linux-modules
     (quote
      ("ahci"
       "usb-storage"
       "uas"
       "usbhid"
       "hid-generic"
       "hid-apple"
       "dm-crypt"
       "xts"
       "serpent_generic"
       "wp512"
       "nls_iso8859-1"
       "virtio_pci"
       "virtio_balloon"
       "virtio_blk"
       "virtio_net"
       "virtio_console"
       "virtio-rng"
       "isofs"
       "overlay"))
     #:linux-module-directory
     (quote
      "/gnu/store/hxkh82lzli9kflp82dygskpy2v8gw1dz-linux-modules")
     #:keymap-file
     #f
     #:qemu-guest-networking?
     #f
     #:volatile-root?
     '#t
     #:on-error
     'debug)))

6.2. modified init file

#!/gnu/store/0g1a9v98jj0lffq4dysyjrg0vx7fjcpi-guile-static-stripped-3.0.9/bin/guile --no-auto-compile
!#
(eval-when
    (expand load eval)
  (let ((extensions '("/guile-modules"))
        (prepend (lambda (items lst)
                   (let
                       loop
                     ((items items) (lst lst))
                     (if (null? items)
                         lst
                         (loop
                          (cdr items)
                          (cons (car items)
                                (delete (car items) lst))))))))
    (set!
     %load-path
     (prepend
      (cons "/gnu/store/zb1m3h384in6f8ffqsmhvp8236jrvnqs-module-import"
            (map
             (lambda (extension)
               (string-append
                extension
                "/share/guile/site/"
                (effective-version)))
             extensions))
      %load-path))
    (set!
     %load-compiled-path
     (prepend
      (cons "/gnu/store/lamjscp9iqb4bcc6lq37b1k5pyz1chan-module-import-compiled"
            (map
             (lambda (extension)
               (string-append
                extension
                "/lib/guile/"
                (effective-version)
                "/site-ccache"))
             extensions))
      %load-compiled-path))))
(begin
  (use-modules
   (my build linux-boot)
   (gnu system file-systems)
   ((guix build utils)
    #:hide
    (delete))
   (guix build bournish)
   (srfi srfi-1)
   (srfi srfi-26)
   ((gnu build file-systems)
    #:select
    (find-partition-by-luks-uuid))
   (rnrs bytevectors))
  (with-output-to-port
      (%make-void-port "w")
    (lambda ()
      (set-path-environment-variable
       "PATH"
       '("bin" "sbin")
       '())))
  (parameterize
      ((current-warning-port
        (%make-void-port "w")))
    (boot-system
     #:mounts
     '()
     #:pre-mount
     (lambda () (and #t))
     #:linux-modules
     (quote
      ("ahci"
       "usb-storage"
       "uas"
       "usbhid"
       "hid-generic"
       "hid-apple"
       "dm-crypt"
       "xts"
       "serpent_generic"
       "wp512"
       "nls_iso8859-1"
       "virtio_pci"
       "virtio_balloon"
       "virtio_blk"
       "virtio_net"
       "virtio_console"
       "virtio-rng"
       "isofs"
       "overlay"))
     #:linux-module-directory
     (quote
      "/gnu/store/hxkh82lzli9kflp82dygskpy2v8gw1dz-linux-modules")
     #:keymap-file
     #f
     #:qemu-guest-networking?
     #f
     #:volatile-root?
     '#t
     #:on-error
     'debug)))

The differences are:

add "/guile-modules" to the extensions variable in the beginning
in the (use-modules) part use (my build-linux-boot) instead of (gnu build linux-boot)
pass an empty list '() to the #:mounts keyword of boot-system

7. Repack the initrd

Now what is left is to repack the initrd. For that we can use a script from the kernel docs from earlier. Let us call this create_initramfs.sh

#!/bin/sh

# Copyright 2006 Rob Landley <rob@landley.net> and TimeSys Corporation.
# Licensed under GPL version 2

if [ $# -ne 2 ]
then
echo "usage: mkinitramfs directory imagename.cpio.gz"
exit 1
fi

if [ -d "$1" ]
then
echo "creating $2 from $1"
(cd "$1"; find . | cpio -o -H newc | gzip) > "$2"
else
echo "First argument must be a directory"
exit 1
fi

So we call create_initramfs.sh guix-initrd-test my-initrd.cpio.gz

8. Doing the kexec

Now we have the initrd. For the kexec we need a kernel and arguments still. We can copy them from the grub.cfg from earlier into a script

#!/usr/bin/bash
# load kernel and initramfs
kexec \
    -l "/gnu/store/storehash/Image" \
    --append='gnu.system=/gnu/store/storehash-system gnu.load=/gnu/store/storehash-system/boot quiet modprobe.blacklist=radeon,amdgpu' \
    --initrd='myinit.cpio.gz'
# execute
kexec -e

When we execute that script the guix system should successfully boot and let us in via ssh. In the end this is a cumbersome method but it works. With some work we could get guix to generate the kexec script for us in the first place but I have not gotten around to it yet.

9. Restarting the guix-deamon

Running a system on rootfs as of yet does not work with chroot building. That is why you will need to run herd stop guix-daemon to stop the guix daemon and start guix-daemon --disable-chroot to be able to build on the installation system. For security reasons it is adviced to also use the --build-users=guixbuild option but I have not tested that.