XEN under etch with LVM to save the DomU-Datafiles
Disclaimer
No liability for the contents of this document can be accepted. Use the concepts, examples and information at your own risk. There may be errors and inaccuracies, that could be damage your system. Proceed with caution, and although this is highly unlikely, the author do not take any responsibility. All copyrights are held by their respective owners, unless specifically noted otherwise. Use of a term in this document should not be regarded as affecting the validity of any trademark or service mark. Naming of particular products or brands should not be seen as endorsements.
There are many other howtos about the installation of XEN, but some of these doesn't work for me correctly. So, I have written this short howto. The XEN-configuration under Debian/GNU Etch is much easier as in Sarge. I use an LVM to store the DomU-Datafiles, so I can react when I need more disk space.
There are two types of xen-hypervisor, with and without PAE. With PAE you can use more than 4 GB RAM on a 32 Bit-system. So I use the PAE-version.
First install the necessary packages and its dependencies:
aptitude install xen-linux-system-2.6.18-5-xen-686 bridge-utils xen-tools lvm2 xen-hypervisor-3.0.3-1-i386-pae
Now, create the LVM, called my-lvm-vg.here. I use two RAID-Arrays (RAID-Controller) to build my Logical Volume Group:
pvcreate /dev/cciss/c0d2p1
pvcreate /dev/cciss/c0d1p1
vgcreate my-lvm-vg /dev/cciss/c0d1p1 /dev/cciss/c0d2p1
I've one NIC inside this server to connect the Dom0 and the DomUs. The interfaces needs no changes:
/etc/network/interfaces
# The loopback network interface
auto lo
iface lo inet loopback
# The primary network interface
allow-hotplug eth0
iface eth0 inet static
address 10.1.1.1
netmask 255.255.255.0
network 10.1.1.0
broadcast 10.1.1.255
gateway 10.1.1.250
dns-nameservers 10.161.11.200
dns-search <your domain>
There are many ways to create a DomU. I prefer xen-tools, it's easy. Edit /etc/xen-tools/xen-tools.conf and make your defaults:
# LVM volume group
#
lvm = my-lvm-vg
# Installation method.
#
debootstrap = 1
# Disk and Sizing options.
# Die Werte kann man aber per Kommandozeile ueberschreiben.
#
size = 10Gb # Disk image size.
memory = 128Mb # Memory size
swap = 256Mb # Swap size
# noswap = 1 # Don't use swap at all for the new system.
fs = ext3 # use the EXT3 filesystem for the disk image.
dist = etch # Default distribution to install.
image = sparse # Specify sparse vs. full disk images.
# Networking setup values.
#
gateway = 10.1.1.250
netmask = 255.255.255.0
# interactively setup a new root password for images.
#
passwd = 1
# Default kernel and ramdisk to use for the virtual servers
#
kernel = /boot/vmlinuz-2.6.18-5-xen-686
initrd = /boot/initrd.img-2.6.18-5-xen-686
# The default mirror for debootstrap which can be used to install
# Debian Sid, Sarge, and Etch.
#
mirror = http://ftp.de.debian.org/debian/
Normally Dom0 don't need much RAM. 32 MB is enough for me. Change this settings in /etc/xen/xend-config.sxp. Set your network model also there. I use bridging:
# loglevel
(loglevel INFO)
#
# named bridge network
(network-script 'network-bridge bridge=bruecke')
#
(vif-script vif-bridge)
#
# lowest memory level (in MB) for dom0
(dom0-min-mem 32)
#
# In SMP system, dom0 will use dom0-cpus # of CPUS
(dom0-cpus 0)
Now restart the XEN:
/etc/init.d/xend restart
If needed, edit your hostX.fqdn.cfg e. g. set a static MAC-address in the vif statement (Note: vif = [ "" , "" ] defines two interfaces).
For various reasons, I set the systemclock of some DomUs separately using ntp. This could be reached by setting the following parameter in the .cfg (it's a kernel boot parameter).
extra = 'independent_wallclock=1'
Check your system clock after first boot an check your timezone (dpkg-reconfigure tzdata).
It's time for creating DomU's (and set your proxy when needed). Values from xen-tools.conf can be overwritten by commandline parameters:
export http_proxy=http://<proxy-address>:<proxy-port>
xen-create-image --ip=10.1.1.2 --hostname=host1.fqdn
xen-create-image --ip=10.1.1.3 --hostname=host2.fqdn --size=150Gb --dist=sid
To enable DomU-start at boot time, just put a link from your .cfg-file into /etc/xen/auto:
cd /etc/xen/auto
ln -s ../host1.fqdn.cfg host1
ln -s ../host2.fqdn.cfg host2
Thats all. You can start your separate DomUs. To leave this console press CTRL+ALTGR+9. :-)
xm create -c /etc/xen/host1.fqdn.cfg
Now you have a minimal system at yout DomU and it's time to do some basic thinks like:
apt-get install locales console-data
dpkg-reconfigure locales
Bridging is my choice to connect the virtual machines to the rest of the world. When you are root of all the DomU's, then arp-spoofing is no problem. If not, you can patch the vif-bridge-file as shown below. The patch was published on de.comp.virtualisierung. I only add a rule to logging arp-anomalies.
diff -u vif-bridge.orig vif-bridge
--- vif-bridge.orig 2008-01-15 14:49:59.000000000 +0100
+++ vif-bridge 2008-01-17 14:04:38.000000000 +0100
@@ -50,15 +50,35 @@
online)
setup_bridge_port "$vif"
add_to_bridge "$bridge" "$vif"
- ;;
+ ebtables -N $vif
+ ebtables -P $vif DROP
+ ebtables -A INPUT -i $vif -j $vif
+ ebtables -A FORWARD -i $vif -j $vif
+ ebtables -A $vif -p ARP --arp-opcode 1 -j ACCEPT
+
+ if [ ! -z "$ip" ]
+ then
+ for oneip in $ip
+ do
+ ebtables -A $vif -p IPv4 --ip-src $oneip -j ACCEPT
+ ebtables -A $vif -p IPv4 --ip-dst $oneip -j ACCEPT
+ ebtables -A $vif -p ARP --arp-opcode 2 --arp-ip-src $oneip -j ACCEPT
+ ebtables -A $vif --log-prefix="arp-drop" --log-arp -j DROP
+ done
+ fi
+ ;;
offline)
do_without_error brctl delif "$bridge" "$vif"
do_without_error ifconfig "$vif" down
+ do_without_error ebtables -D INPUT -i $vif -j $vif
+ do_without_error ebtables -D FORWARD -i $vif -j $vif
+ do_without_error ebtables -F $vif
+ do_without_error ebtables -X $vif
;;
esac
-handle_iptable
+# handle_iptable
log debug "Successful vif-bridge $command for $vif, bridge $bridge."
if [ "$command" == "online" ]
Please also read /usr/share/doc/bridge-utils/README.Debian.gz and XenNetworking:
Sometimes you have to do a kernel-upgrade. First upgrade your Dom0 apt-get -uV dist-upgrade and reboot. Shutdown the DomX (xm shutdown <number>) and edit the appropriate /etc/xen/<name>.cfg:
kernel = '/boot/vmlinuz-2.6.18-6-xen-686'
ramdisk = '/boot/initrd.img-2.6.18-6-xen-686'
Mount the image (filesystem) and copy the required files:
mount -o loop /dev/my-lvm-vg/<your_DomX_image-disk> /mnt/
cp -vdpR /lib/modules/<your_new_kernel-xen-...> /mnt/lib/modules/
umount /mnt/
Now start the DomX. There are some tools to manage XEN e.g. xm help or xenstore-ls, xenstore-read, ....
In my system I've a dom0 under etch and one domU with lenny. The syslogd reports many 4gb seg fixup, process errors. I've fixed it with:
aptitude install libc6-xen
echo 'hwcap 0 nosegneg' > /etc/ld.so.conf.d/libc6-xen.conf
ldconfig
reboot
Here is my short howto to move DomU's from etch to lenny on a new machine.
Copyright and License
This document, XEN with LVM under Etch, is copyrighted (c) by Tom Geißler,.
last change: 18.06.2009
Permission is granted to copy, distribute and/or modify this document under the terms of the GNU Free Documentation License, Version 1.2 or any later version published by the Free Software Foundation; with no Invariant Sections, with no Front-Cover Texts, and with no Back-Cover Texts. A copy of the license is available at http://www.gnu.org/copyleft/fdl.html.
