For some reason citrix doesn’t like mdadm so they make everything possible to stop it from working on their xen server.
Here is a guide that would make it work there, but it may not survive system patching
Setup
Connect at least 2 disks to your box. Install a xen server without local storage on first disk.
Installing mdadm
The default install contains mdadm but it doesn’t load raid modules to kernel. In order to enable it, following needs to be done:
echo "modprobe raid1" > /etc/sysconfig/modules/raid.modules modprobe raid1 chmod a+x /etc/sysconfig/modules/raid.modules
Partitioning the disks
Now we create a final schema we want to use on our server on disk /dev/sdb, xen needs to have at least 3 partitions, 1 is for boot loader, second is for OS, I recommend 20gb or more, because this disk is pretty much impossible to extend, although citrix defaults it to 4GB, last partition is for local storage and it should take all remaining space on disk.
Note: Citrix by default creates 3 partitions, 1 for OS, second is empty, same sized as first one and probably used for system upgrade. Third is used for local storage LVM. You don’t have to create second partition for it to work, but system upgrades may not be available if you don’t create it. On other hand system upgrades will likely not work anyway as citrix doesn’t support mdadm installations.
In this guide I will use old MS-DOS partition table because although it’s old, it’s much better supported and it just works. You can also use GPT partitions if you want, but I had some issues getting them work with mdadm and syslinux.
We will have a separate /boot partition for boot loader, because syslinux shipped with xen is having troubles booting from raid device for some reason.
So this is how the layout of sdb should look after we finish the partitioning:
- /dev/sdb1 (2 GB) for bootloader
- /dev/sdb2 (20 GB) for OS
- /dev/sdb3 (rest) for LVM
# Configure disk sgdisk --zap-all /dev/sdb # Now run fdisk and create a new dos partition table # make 2 partitions, one is for boot (/dev/sdb1 2gb) and one for Dom0 (/dev/sdb2 20gb) #### after it's done create md device mdadm --create /dev/md0 --level=1 --raid-devices=2 missing /dev/sdb2 mkfs.ext3 /dev/md0 mkfs.ext3 /dev/sdb1 mount /dev/md0 /mnt cp -vxpR / /mnt cd /mnt mv boot /tmp/old_boot mkdir boot mount /dev/sdb1 /mnt/boot mv /tmp/old_boot/* boot # Fix /mnt/etc/fstab - replace LABEL with /dev/md0 and insert a record for /boot # EXAMPLE: head -2 /etc/fstab /dev/md0 / ext3 defaults 1 1 /dev/sda1 /boot ext3 defaults 1 1 # Update boot loader # You need to open /boot/extlinux.conf and replace all references to old disk with root=/dev/md0 mkdir /mnt/root/initrd-raid mkinitrd -v --fstab=/mnt/etc/fstab /mnt/root/initrd-raid/initrd-`uname -r`-raid.img `uname -r` cd /mnt/root/initrd-raid zcat initrd-`uname -r`-raid.img | cpio -i mdadm --detail --scan >> etc/mdadm.conf find . -print | cpio -o -Hnewc | gzip -c > /mnt/boot/initrd-`uname -r`-raid.img rm /mnt/boot/initrd-3.10-xen.img cd /mnt/boot ln -s initrd-`uname -r`-raid.img initrd-3.10-xen.img extlinux -i boot/ cat /usr/share/syslinux/mbr.bin > /dev/sdb # Open /mnt/boot/extlinux.conf # remove absolute path to xen.gz to relative, /boot will be root device for bootloader (/xen.gz), replace LABEL with /dev/md0 ###### example conf file for syslinux that works ###### # location mbr ui vesamenu.c32 serial 0 115200 default xe prompt 1 timeout 50 label xe menu label XenServer kernel mboot.c32 append xen.gz dom0_mem=752M,max:752M watchdog dom0_max_vcpus=2 crashkernel=128M@32M cpuid_mask_xsave_eax=0 console=vga vga=mode-0x0311 --- /boot/vmlinuz-3.10-xen root=/dev/md0 ro hpet=disable xencons=hvc console=hvc0 console=tty0 --- initrd-3.10-xen.img #######################EOF###################### reboot
Now you should be able to boot from /dev/sdb if you are not there is something wrong with the setup, you need to figure out if your problem is with
- MBR (No bootable device)
- Boot loader (Missing operating system.)
- /boot (Linux will start booting but die in progress – try removing quiet and splash from parameters)
Syncing the disks
Now if you were able to boot up you need to setup the sda disk
Create the same 3 partitions as you did on sdb on sda and then
dd if=/dev/sdb1 of=/dev/sda1 # mbr cat /usr/share/syslinux/mbr.bin > /dev/sda mdadm --add /dev/md0 /dev/sda2 Wait for disks to sync, meanwhile you can create a new local storage mdadm --create /dev/md1 --level=1 --raid-devices=2 /dev/sda3 /dev/sdb3 xe sr-create content-type=user device-config:device=/dev/md1 name-label="Local Storage" shared=false type=lvm