Xen Install HOWTO on IA64 with Debian GNU/Linux
This document provides a description of how Paul Davies at Gelato@UNSW performed a Xen installation on a hp zx6000 workstation. The installation described consists of the Xen hypervisor, Dom0, and an example guest, DomU.
Hardware used
- zx6000 workstation.
- 3x36G Hard disk
- No management processor
- 2G of RAM
Software used
- Xen unstable.
- Clone the xen unstable source from its mercurial repository using the following command.
hg clone http://xenbits.xensource.com/ext/xen-ia64-unstable.hg
DO NOT use xen-3.0.4_1-src.tgz from the xen-source download page. It does not build on debian unstable for IA64 (although it does build for x86 with unstable).
- Clone the xen unstable source from its mercurial repository using the following command.
- Debian unstable
Brief Summary of Approach
- Make two minimal standard Debian unstable installations.
- Installation 1 is to initially provide an environment to create and install xen components. It is then re-used to provide a root filesystem for Dom0.
- Installation 2 is used to provide the root file system for DomU.
- Download and build the xen source components using the first Linux installation.
- Use the first Linux installation to set up the efi partition to boot the Xen hypervisor and start it as Dom0 instead.
- Using the xen tools (in Dom0), create a virtual machine for DomU.
- Boot the virtual machine using the paravirtualized kernel built in step 2..
Detailed description of steps undertaken
1. Firmware upgrade
Upgraded firmware to version 2.31. The author found the Xen hypervisor hung on boot with 2.21, and that upgrading to 2.31 remedied the problem.
2. Plan out the hard disk partitioning
A spare 36G hard disk was used for the installation. For simplicity, I split it into three regions
- Region 1. 12G for Dom0
- Region 2. 12G for DomU.
- Region 3. 12G unused (available for an additional guest).
3. Make the first Debian unstable installation on Region 1
The first installation is initially used to build xen's components. A minimal Debian etch rc2 installation was made on the first 12 G region using a netinstall CD. The installation hostname was mocha. This was upgraded to unstable using the debian package manager.
sudo apt-get dist-upgrade
The following packages were then installed manually
- sudo
- parted
- ssh
- bzip2 (essential)
- patch (essential)
Partition Layout after the first Linux installation
Number Start End Size File system Name Flags 1 17.4kB 100MB 100MB fat16 boot 2 100MB 10.1GB 10.0GB ext3 5 10.1GB 12.1GB 2000MB linux-swap 3 12.1GB 24.1GB 12.0GB ext3 4 24.1GB 36.4GB 12.3GB ext3
4. Build the Xen components from source
The following (essential) packages were installed in order to be able to build the xen source.
- gcc
- make
- binutils
- zlib1g-dev
- python-dev
- libncurses5-dev
- libssl-dev
- libx11-dev
- bridge-utils
- iproute
- udev
Build the Xen hypervisor, and the paravirtualized kernels
Two paravirtualized kernels are built during the xen installation. One for Dom0, and one for DomU.
1. Boot into the first Linux installation. Clone the Xen source. Go to the root directory of the cloned xen source.
pauld@mocha:~/xen-ia64-unstable.hg$ sudo make KERNELS="linux-2.6-xen0 linux-2.6-xenU" world
This builds the xen hypervisor, downloads the 2.6.18 linux kernel, applies the paravirtualizing patch, and builds the Dom0 (linux-2.6-xen0) and DomU (linux-2.6-xenU) paravirtualized kernels. This takes some time. If the build breaks, it is most likely that one of the previously mentioned packages was not installed.
2. Install the xen hypervisor and paravirtualized kernels to the boot directory. Install kernel modules.
pauld@mocha:~/xen-ia64-unstable.hg$ sudo make KERNELS="linux-2.6-xen0 linux-2.6-xenU" install
After a successful install, the hypervisor and paravirtualized kernels are in the boot directory.
pauld@mocha:/boot$ ls -lah pauld@mocha:/boot$ ls -lah total 35M drwxr-xr-x 2 root root 4.0K 2007-04-05 11:34 . drwxr-xr-x 21 root root 4.0K 2007-04-04 07:15 .. lrwxrwxrwx 1 root root 18 2007-04-05 10:08 config -> config-2.6.18-xenU -rw-r--r-- 1 root root 56K 2007-03-26 20:09 config-2.6.18-4-mckinley -rw-r--r-- 1 root root 36K 2007-04-05 10:07 config-2.6.18-xen0 -rw-r--r-- 1 root root 32K 2007-04-05 10:08 config-2.6.18-xenU -rw-r--r-- 1 root root 8.2M 2007-04-04 07:56 initrd.img-2.6.18-4-mckinley -rw-r--r-- 1 root root 2.6M 2007-04-05 10:11 initrd.img-2.6.18-xen0 -rw-r--r-- 1 root root 2.6M 2007-04-05 10:28 initrd.img-2.6.18-xenU lrwxrwxrwx 1 root root 22 2007-04-05 10:08 System.map -> System.map-2.6.18-xenU -rw-r--r-- 1 root root 881K 2007-03-27 02:04 System.map-2.6.18-4-mckinley -rw-r--r-- 1 root root 1.4M 2007-04-05 10:07 System.map-2.6.18-xen0 -rw-r--r-- 1 root root 1.1M 2007-04-05 10:08 System.map-2.6.18-xenU lrwxrwxrwx 1 root root 19 2007-04-05 10:08 vmlinuz -> vmlinuz-2.6.18-xenU -rw-r--r-- 1 root root 2.7M 2007-03-27 02:04 vmlinuz-2.6.18-4-mckinley -rw-r--r-- 1 root root 6.0M 2007-04-05 10:07 vmlinuz-2.6.18-xen0 -rw-r--r-- 1 root root 4.6M 2007-04-05 10:08 vmlinuz-2.6.18-xenU lrwxrwxrwx 1 root root 19 2007-04-05 10:07 xen-3.0.gz -> xen-3.0-unstable.gz -rw-r--r-- 1 root root 386K 2007-04-05 10:07 xen-3.0-unstable.gz lrwxrwxrwx 1 root root 19 2007-04-05 10:07 xen-3.gz -> xen-3.0-unstable.gz lrwxrwxrwx 1 root root 19 2007-04-05 10:07 xen.gz -> xen-3.0-unstable.gz -rw-r--r-- 1 root root 4.5M 2007-04-05 10:07 xen-syms-3.0-unstable
5. Transform the first installation from build environment to Domain0
- Update elilo.conf to boot Dom0 via the xen hypervisor. elilo.efi supports xen in debian unstable.
- TODO: Write a discussion regarding elilo documentation.
## elilo configuration file generated by elilo 3.6-3.1 install=/usr/lib/elilo/elilo.efi boot=/dev/sda1 delay=20 default=xen relocatable image=/vmlinuz label=Linux root=/dev/sda2 read-only initrd=/initrd.img image=/boot/vmlinuz-2.6.18-xen0 label=xen vmm=/boot/xen.gz initrd=/boot/initrd.img-2.6.18-xen0 read-only append="nomca dom0_mem=96M root=/dev/sda2 console=vga" - Copy elilo.conf, the xen hypervisor, and the paravirtualized kernels to the efi partition
pauld@mocha:~$ sudo elilo
Boot Dom0 and set xend to start on boot. xend is a daemon that must run to use the xen tools. The xen tools are used for manipulating guests. A xend script (which invokes /usr/sbin/xend) was created and placed in /etc/init.d during make install. Do the following to start xend automatically next boot.
pauld@mocha:/etc/rc2.d$ sudo ln -s /etc/init.d/xend S20xen
Reboot the machine.
6. Make the second Debian unstable installation on Region 2
The second Linux installation is used as a simple way to create a root file system for a xen guest. Make the second Linux installation using the same procedure as the first (except on another region). My installation is called chocolate. Do not install the packages for building xen.
Partition Layout after the second Linux installation
Number Start End Size File system Name Flags 1 17.4kB 100MB 100MB fat16 boot 2 100MB 10.1GB 10.0GB ext3 5 10.1GB 12.1GB 2000MB linux-swap 3 12.1GB 22.1GB 10.0GB ext3 6 22.1GB 24.1GB 2000MB linux-swap 4 24.1GB 36.4GB 12.3GB ext3
The second installation is unaware of the first installation. To boot domain0 via the xen hypervisor again, do the following.
# mount the domain0 root filesystem pauld@chocolate:~$ sudo mount /dev/sda2 /mnt kjournald stating. Commit interval 5 seconds EXT3 FS on sda2, internal journal EXT3-fs: mounted filesystem with ordered data mode. # chroot to the domain0 root filesystem pauld@chocolate:~$ sudo chroot /mnt # reinstall the xen hypervisor and domain0 to the efi boot partition root@chocolate:/# sudo elilo EFI Variables Facility v0.08 2004-May-17 Loaded efivars kernel module to enable use of efibootmgr
Reboot to pauld@mocha.
7. Create example virtual machine and boot it
Set up the configuration file for the virtual machine. I used the following primitive VM confuration.
#paulsVM kernel = "/boot/vmlinuz-2.6.18-xen0" ramdisk = "/boot/initrd.img-2.6.18-xen0" memory = 256 name = "ExampleDomain" vif = [ '' ] disk = [ 'phy:sda3,sda3,w' ] root = "/dev/sda3 ro" extra = "4"
To boot the virtual machine.
pauld@mocha:/etc/xen$ sudo xm create -c paulsVM
Watch it boot, and log in (using the username and password from the second install)
Dom0
Debian GNU/Linux 4.0 mocha tty2 mocha login:
DomU
Debian GNU/Linux 4.0 chocolate tty1 chocolate login:
Problems encountered
- xen-3.0.4_1-src.tgz does not work on IA64
- Make sure you have a version of elilo that supports the booting of the virtual machine.
- Make sure you have firmware 2.31. Firmware 2.21 hung the machine
- Issues with udev and LDAP on tartufi
- Was unable to build xen successfully (following instructions correctly using xen-3.0.4_1-src.tgz)
- No VGA output on zx6000
- Make sure guest has enough memory. Couldn't boot the VM with the 64M default
