• InstallationNotes

Information on this page supplements the documentation included in the Xen distribution (also available at http://www.cl.cam.ac.uk/Research/SRG/netos/xen/documentation.html ).

Correct Kernel?

The standard Xen configuration has the guest domain kernels being supplied by dom0, typically from its own /boot/ directory. Once a guest starts up and loads modules, these will come from its own root filesystem. Note that this configuration does not use any kernel image which may exist within the guest's root filesystem.

If you rebuild the unprivileged kernel, the standard installation in domain 0 should be performed - this puts the new kernel in place for domain creation. Additionally, ensure that any modules built for the unprivileged kernel are placed with /lib/modules/ in all guest domain root filesystems.

brctl being found?

By examining /var/log/xend-debug.log, you may find that the various Xen scripts are not finding your installed brctl command. Under Red Hat Fedora Core 3, for instance, the command is found in /usr/local/sbin. You may need to edit the offending scripts (mentioned in the log above) to configure additional PATH entries. On CentOS 4 you can symlink /sbin/brctl to /usr/local/sbin/brctl. This fixed the network script without setting absolute paths in /etc/xen/scripts/network.

Guest filesystems

Distro specific tools are usually available for bootstrapping a guest filesystem of the same type as dom0. For third party prebuilt images and tarballs of many Linux distributions, see http://free.oszoo.org. These are QEMU images.

xenU services

The following services are not needed anymore:

* ntpd

the xenU uses the dom0 time

If you want to run ntp in the domU, try: echo 1 > /proc/sys/xen/independent_wallclock

Using a partitioned image

QEMU is very useful for creating guest images. These images will contain a full partition table though. To use this in Xen, simply expose the device as a disk (that is, /dev/hda, /dev/hdb, /dev/sda, etc.) instead of as a partition. For your root parameter, you want to use whatever you would normally use in grub for that image.

Using the -xen kernel rather than -xen0/U

This information applies only to Xen 3 and greater and has only been tested on Fedora/Redhat/Centos systems.

Many users will be better off using the "-xen" kernel instead of "-xen0" and "-xenU" kernels. The -xen0/U kernels are used to achieve faster compile times in the dev process. Each kernel can be compiled independently and since only a small subset of kernel components are compiled the overall process can save a great deal of developer time. The -xen kernel is more like the kernels that come with many distributions (Redhat/Fedora, SuSE). These kernels ship with a large number of components, like drivers for devices and file systems, compiled as modules. This allows these kernels to run on more hardware then the type of stripped down custom kernel you would find on an appliance. The -xen kernel will take longer to compile and will require a initrd but once built it will work on more hardware and "play well" with more distributions. Many of the recent problems reported on the user list would have been avoided by using the -xen kernel.

To build the -xen kernel edit the top level Makefile so that this line:

KERNELS ?= linux-2.6-xen0 linux-2.6-xenU

looks like this

KERNELS ?= linux-2.6-xen

Then build with:

# make world

You will get a single kernel and modules which can be used for both Domain0 and all DomainUs. Copy the modules directory /lib/modules/2.6.<version>-xen to the /lib/modules directory of your VM and make an initrd with mkinitrd.

# mkinitrd /boot/initrd-xen-3.0.img 2.6.<version>-xen

If running the mkinitrd command gives you problems e.g. something about no dep file found you need to run depmod like this:

# depmod /lib/modules/2.6.<version>-xen

Now you should be able to create the the image using mkinitrd.

You will need to add the initrd to your grub config under the -xen kernel line. Following the example above it should look something like the following.

module /vmlinuz-2.6-xen <your config here>
module /initrd-xen-3.0.img"

The same initrd can be used for the VM by adding the following to its config file.

ramdisk = "/boot/initrd-xen-3.0.img"

Virtual Technology VT/VMX

As of January 2006 the VMX capable processors are just now getting into systems. The Intel Xeon Dual Core + Intel Pentium D 9xx series are the first commonly available processors.

For now, read this message in the mailing list archive for tips: http://lists.xensource.com/archives/html/xen-devel/2006-01/msg00153.html

VmxQuickstartGuide

Serial port access from Dom0

The serial driver (8250) in the kernel is not available until you unset the option

XEN --> CONFIG_XEN_DISABLE_SERIAL

Now the serial drivers can be set to module or buildin and are compiled.

DEVICE DRIVES --> CHARACTER DEVICES --> SERIAL DRIVERS

To prevent Xen from taken the ttyS0 as a virtual console add in grub to kernel console=vga and add to module xencons=tty

This will give following waring in dmesg

WARNING: Failed to register Xen virtual console driver as 'tty1'

Conflict between Xen serial console port ttyS0 and kernel module 8250.ko

To load parport_serial kernel module (or some other depending on 8250.ko) we need to remap Xen serial console device out of conflicting range.

When you get:

FATAL: Error inserting 8250 (...): Device or resource busy

and kernel logs unresolved symbols in needed modules:

8250_pci: Unknown symbol serial8250_register_port

parport_serial: Unknown symbol pciserial_init_ports

try to add "xencons=ttyS16 console=ttyS16,<port settings>" to dom0 kernel boot options.

Serial port access from DomU

To have access to the serial port in DomU, we need to prevent Dom0 from taken hold of this piece of hardware

add to /etc/hotplug/blacklist the following modules to exclude from being loaded

8250

8250_pnp

serial_core

domU config:

irq = [ 4 ]

ioports = [ "3f8-3ff" ]

to load the modules in domU.

I had to install module-init-tools

I have copied /lib/modules/2.6.11.10-xenU/modules.dep and copied the serials modules to domU

I also had to change the permissions of /dev/ttyS0 to 666

I have added the modules to /etc/modules so they load on boot

Dell 2900 modules to include in mkinitrd

mkinitrd -v -f --with=sd_mod --with=scsi_mod --with=megaraid_mm --with=megaraid_sas --with=megaraid_mbox initrd-2.6.16.29-xen.img 2.6.16.29-xen

dom0 64 bit and domU 32 bit from source code

Build and install xen:

make xen
make install-xen

Build and install the dom0:

make linux-2.6-xen0-prep
cd build-linux-*-xen0-x86_64
make menuconfig
make
make install modules_install

And finally build and install 32 bit domU:

make XEN_TARGET_ARCH=x86_32 linux-2.6-xenU-prep
cd build-linux-*-xen0-x86_32
make ARCH=i386 menuconfig
make ARCH=i386
make ARCH=i386 install modules_install

IMPORTANT!!! You must set Processor type and features->High Memory Support to 64 GB in domU!!!!