Objectives
- Multiple Generic Guest Domains
- Each Guest Domain Has It's Own Logical Volume On The Host Domain
- Available Disk Space On Guest Domains Can Easily Be Expanded
- Guest Domains Can Easily Be Cloned
- Install Zimbra On One Or More Guest Domains
I currently have a single, non-virtualized server performing multiple functions, including firewall, web server, mail server, proxy server, and VPN. It is a nightmare to manage, as any changes or problems in one application invariably screws up the applications. My goal is to have a single physical server with multiple guest domains that can each be customized as necessary. In particular, I want to use Zimbra as my mail server. Zimbra doesn't play well with other applications, and thus needs a dedicated environment. While I haven't rolled my new server into production, I have successfully accomplished all of the above-noted objectives.
Prerequisites
A server that has plenty of memory and disk ( >= 1/50 GB). The disk will be wiped clean, so back up any existing data.
- A single DVD install disk as opposed to multiple CD install disks. You may be able to work around this prerequisite, but that's up to you. I'm using the "refreshed" version of Fedora Core 5.
- A DNS server (preferable) or DHCP server that can provide multiple IP addresses. The DNS server is mandatory if you want to use Zimbra.
- A working knowledge of Linux, including the vi editor, bash, and CLI tools
Yum Updates Disabled
This How-To purposely disables yum updates and blocks all access to online yum repositories.
Initially I was using yum to keep both the host and guest domains up-to-date. However, I ran into a significant problem with updated kernel 2.6.18-1.2200.fc5xenU. Because I had not experienced any problems with the packages on the install disc, I decided to make the install disk my sole yum repository. This provides a much more stable environment for testing and development. Obviously, this also leaves the system exposed to unpatched bugs and security threats.
Build Minimal FC5 Server
The first step is to build a minimal base server which will become the Xen host (Dom0). We want only the minimum amount of software for several reasons. First, second, and third is security. Other reasons include quicker installation time, less package dependency/conflict issues, less complexity, and less unnecessary services.
- Boot to FC5 Install disk and begin graphical installation process.
- Do not take the default disk partitioning scheme. Build a custom disk layout as shown below. I am actually using a software RAID 1 disc based on two identical 200 GB discs, but the layout would be the same if you only have one physical disc.
- Proceed with the installation up to the point of selecting software packages. We want to build a minimal server, so we will choose only the packages we need. Deselect everything. You can do this quickly by right-clicking over a package group and choosing the Deselect All option. The only packages we want to select are Yum, and the OpenSSH server and client packages.
- Finish the installation and log into the new server.
Disk Layout
- 100 MB "/boot" filesystem
- 20 GB for "/" (root) filesystem. This may be excessive, but if you have plenty of disk, it's always better to have too much than not enough.
- Swap as necessary based on physical RAM. Out of habit, I use 2xRAM. I'm not sure how much swap is utilized by Xen. Since a good portion of physical memory is given up for each guest, the host's physical memory is significantly reduced. This may or may not require additional swapping. The guests have their own swap areas.
- The remainder of the disk will be dedicated to a single Logical Volume Group called "xen". Don't create any logical volumes yet.
Install Xen on Host (Dom0)
We begin by disabling SELinux. Edit /etc/selinux/config and modify SELINUX=enabled To SELINUX=disabled followed by a reboot.
vi /etc/selinux/config >>> modify SELINUX=enabled To SELINUX=disabled <<< SELINUX=disabled
shutdown -r -y 0 # reboot to disable SELinux
Following the reboot, we want to make the install disk available as a filesystem. Obviously this requires that the FC5 install DVD is in the tray of the DVD drive. In the following example, my DVD drive is /dev/hda. Yours will probably be different. Run the command "ls -l /dev/cd*" to see what symbolic links have been created for your various CD/DVD devices. After mounting the install disc, run the command "ls -l /var/ftp/pub" to verify the install DVD's filesystem is available.
mkdir /var/ftp/pub mount -o loop /dev/hda /var/ftp/pub ls -l /var/ftp/pub # You should see the root directory of the install disk
Next, we are going to modify Yum to use the install disc as the sole repository. We do this by modifying three files in /etc/yum.repos.d.
cd /etc/yum.repos.d
vi fedora-core.repo >>> add the following line beneath #baseurl... <<< baseurl=file:///var/ftp/pub/ >>> comment out the mirrorlist line <<< #mirrorlist=http://fedora.redhat...
vi fedora-extras.repo >>> change enabled=1 to enabled=0 <<< enabled=0
vi fedora-updates.repo >>> change enabled=1 to enabled=0 <<< enabled=0
OK, now we are ready to install the Xen packages. First, we will also install the vsftpd ftp server package. The vsftpd server will be used when installing Xen guests.
yum clean all # just to be safe! yum -y install vsftpd yum -y install kernel-xen0
Next, we need to modify GRUB so that the new Xen kernel will boot by default. After modifying grub.conf, reboot.
vi /etc/grub.conf >>> change default=1 to default=0 <<< default=0
During your reboot, remember that the install DVD is in the tray. This would be a good time to modify the BIOS so that the hard disk is the first boot device instead of the DVD. Otherwise... Dooohhh!!!
Create And Boot The Initial Xen Guest
The initial Xen guest domain will be named xen0, and will be built on a logical volume. First, run a couple of checks to verify the Xen kernel booted and that xend is running OK.
uname -r 2.6.15-1.2054_FC5xen0
xm list Name ID Mem(MiB) VCPUs State Time(s) Domain-0 0 2487 2 r----- 467.5
Next, remount the install disc, start the vsftpd server, and create a 2 GB logical volume named xen0 (for the first guest).
mount -o loop /dev/hda /var/ftp/pub service vsftpd start lvcreate -L 2G -n xen0 xen
Use the xenguest-install.py tool to create the first Xen guest image. If all goes well, a text-based install script will start. The various options to xenguest-install.py are shown
-n xen0 Name Of Xen Guest
-f xen0 Pathname To Logical Volume To Use For Guest
-r 256 Memory To Allocate For Guest
-l ftp://192.168.151.6/pub/ Location Of FC5 Install Disk. The IP address is to the vsftpd server. Since the guest won't know the DNS domain, use the raw IP address. The guest is connecting over a virtual IP network, so 127.0.0.1 won't work either.
xenguest-install.py -n xen0 -f /dev/xen/xen0 -r 256 -l ftp://192.168.151.6/pub/
The xenguest-install.py script will start a text-based installation script almost identical to what you would see with a default FC5 installation. Early in the installation, the option to switch to a VNC-based GUI will be presented. Text-based is quicker, but VNC does a better job of package selection (and it's kind of cool), so take your pick.
- To use the VNC method, connect to the provided IP address from a VNC client from elsewhere on the network. A GUI install script will appear in the VNC client window. Proceed through install process, setting parameters as necessary. The script creates a virtual hard disk, /dev/xvda. Unless you have specific needs, you should probably let the install script automatically partition the virtual disk. Do a custom package selection, deselecting everything but Yum and OpenSSH server/client packages. Complete the installation.
- To use the text-based install process, everything will be functionally the same as the VNC method above, except for deselecting software packages. You won't be able to drill down into package groups, so just deselect everything. This will install a few unnecessary packages and services, which can be removed later if necessary.
After the guest installation script complete, control is returned to the xenguest-install.py script, which will hopefully indicate success and then exit. It is now (way past) time to start the first Xen guest. To start the guest, use the xm command. The xm command is used to manage all Xen clients. For a synopsis of availabe options, simply enter xm.
xm create -c xen0
The console window of the guest will appear, including a text-based GRUB screen, followed by normal boot/init messages, concluding with a login screen. To break from the console and return to the host, enter CTRL-] (control key plus the right-bracket key). To list the running Xen guests from the host (Dom0), use the "xm list" command as shown below.
xm list Name ID Mem(MiB) VCPUs State Time(s) Domain-0 0 2487 2 r----- 755.5 xen0 18 256 1 -b---- 14.4
To return to the Xen guest's console, enter the following command from the host.
xm console xen0
Configure xen0 As Template
The first guest, xen0, can be easily cloned. Therefore it makes sense to tweak xen0 and maintain it as a template for future guests. Log into the xen0 guest domain and make the following changes.
Disable SELinux as shown above when SELinux was disabled in the host. Don't forget to reboot xen0.
Modify Yum to use the sole repository on the host as shown above, excepting one difference. In the file /etc/yum.repos.d/fedora-core.repo, instead adding the line baseurl=file:///var/ftp/pub, add the line baseurl=ftp://192.168.151.6/pub/ but replacing the IP address with the IP address of the host (or wherever vsftpd is running).
Use Yum to add any packages that need to be available on all guests. For example, the rsync package is necessary on all of my systems to support my backup application, BackUpPC. I also install xterm and it's dependencies so that I can pop remote xterms back to my management system.
- Make any additional changes that should be available on all Xen guests.
Shut down xen0 by either issuing the shutdown command from the xen0 console, or breaking from the console and using the following command at the host. Do NOT forget to shut down xen0 prior to cloning.
xm shutdown xen0
Clone A Xen Guest From The xen0 Template
The hard work is behind us. Cloning is fairly simple. Make sure xen0 has been properly shut down and then proceed with the following steps.
lvcreate -L 2G -n xen1 xen dd if=/dev/mapper/xen-xen0 of=/dev/mapper/xen-xen1 bs=1M # this step will take several minutes cp /etc/xen/xen0 /etc/xen/xen1
vi /etc/xen/xen1 >>> modify name = "xen0" to name = "xen1" <<< name = "xen1" >>> modify disk to change logical volume xen0 to xen1 <<< disk = [ 'phy:/dev/xen/xen1,xvda,w' ] >>> modify MAC address to be 1 digit higher than it's current value. <<< >>> This is a quasi-randomly generated MAC, so your MAC won't match this example <<< vif = [ 'mac=00:16:3e:09:7d:56' ]
xm create -c xen1
At the xen1 console, log in and perform the following changes.
vi /etc/sysconfig/network >>> change HOSTNAME to new HOSTNAME value <<< HOSTNAME=xen1.freedomics vi /etc/sysconfig/network-scripts/ifcfg-eth0 >>> change IPADDR to new IPADDR value <<< IPADDR=192.168.151.161 >>> change HWADDR to new HWADDR value <<< HWADDR=00:16:3E:09:7D:56
shutdown -r -y 0
At this point, start xen0 to make sure the two guest domains are configured correct. If everything checks out, the clone is complete.
xm list Name ID Mem(MiB) VCPUs State Time(s) Domain-0 0 2487 2 r----- 755.5 xen0 18 256 1 -b---- 4.4 xen1 12 256 1 -b---- 1.2
Expand A Guest Domain's Disk Space And Filesystem
The xen0 template is only 2 GB. In fact, when swap and /boot are taken into account, the root filesystem is only 1.5 GB. This certainly isn't large enough for most servers or applications. Fortunately, it is quite easy to expand the size of the /root filesystem, or potentially add other filesystems. In this example, the guest domain xen1 will be expanded by 1 GB.
Begin by shutting down xen1 and expanding the underlying logical volume. After expanding the logical volume, restart xen1 and log in.
lvextend -L +1GB /dev/xen/xen1
xm create -c xen1
The next step is to use fdisk to create a new partition in xen1. The guest sees a special disk device, /dev/xvda, which it treats like a normal block device. By default, /dev/xvda is divided into two partions as seen below. The first partition is for "/boot". The second partition has a logical volume group containing two logical volumes. The first logical volume contains the root filesystem and the second logical volume is dedicated to swap.
fdisk -l Disk /dev/xvda: 2147 MB, 2147483648 bytes 255 heads, 63 sectors/track, 261 cylinders Units = cylinders of 16065 * 512 = 8225280 bytes Device Boot Start End Blocks Id System /dev/xvda1 * 1 13 104391 83 Linux /dev/xvda2 14 261 1992060 8e Linux LVM
Use fdisk to create the new partition as shown below. The partition will be the third primary partition. Accept the defaults for the first/last cylindars. This will use all available space.
fdisk /dev/xvda Command (m for help:) n >>> use "n" to create a new partition <<< Command action e extended p primary partition (1-4) p >>> use "p" to create a primary partition <<< Partition number (1-4): 3 >>> use "3" to create the third partition <<< First cylinder (262-391, default 262): >>> take the default <<< Using default value 262 Last cylinder or +size or +sizeM or +sizeK (262-391, default 391): >>> take the default <<< Command (m for help): w >>> use "w" to write the new partition table <<< The partition table has been altered! Calling ioctl() to re-read partition table. WARNING: Re-reading the partition table failed with error 16: Device or resource busy. The kernel still uses the old table. The new table will be used at the next reboot. Syncing disks. Command (m for help): q >>> enter "q" to quit <<<
To activate the new partition table, the guest domain must be rebooted. By now, you should know how to handle this task, so I won't show it here. After logging back into xen1, verify the new partition is available.
fdisk -l Disk /dev/xvda: 3221 MB, 3221225472 bytes 255 heads, 63 sectors/track, 391 cylinders Units = cylinders of 16065 * 512 = 8225280 bytes Device Boot Start End Blocks Id System /dev/xvda1 * 1 13 104391 83 Linux /dev/xvda2 14 261 1992060 8e Linux LVM /dev/xvda3 262 391 1044225 83 Linux
Now it's time to add the partition to the guest's volume group and expand the filesystem.
pvcreate /dev/xvda3 vgextend VolGroup00 /dev/xvda3 lvextend -l +32 /dev/VolGroup00/LogVol00 ext2online /dev/VolGroup00/LogVol00
Use "df -H" to verify the additional disk space is available.
Install Zimbra On Guest Domain
Download the latest version of Zimbra to /tmp on your guest. Make sure that your vsftpd server is running on your host so that you can install a few missing yum packages. Then, complete the following pre-installation steps.
yum install curl fetchmail gmp cd /tmp tar -xzvf zcs-4.5.0_GA_612.FC5.tgz cd zcs
Run the Zimbra installation script.
- For the most part, you can take the defaults.
- In a couple of places, before applying major changes, the default is [No], at which point you should change the default
- At one point, you will probably be prompted about a missing DNS MX record. You can select N to skip this step for development and testing, although at some point it will have to be resolved for Zimbra to function properly in a production environment.
- Towards the end of the script, you will be presented with a menu of configuration items. The admin password will need to be set. You can select option #6 to go into a sub-menu, where you can then select option #4 to set the admin password. This is important, as this is the password you will use when accessing the administrator's panel via https.
- Once all configuration options are per your desire, select "a" to apply the changes to your new config file.
- Complete the script to start Zimbra.
./install.sh
After running the script, you should be able to go to a web browser and enter https://<IP_address_of_zimbra_server:7071/ to reach the admin screen. User is admin and password is the the same as you set during the install script.
Party on, Garth.
xen.org,
