Overview
We were interested in getting XenServer 6.5 to boot via UEFI. Leaving servers in Legacy/BIOS boot was not an option in our target environment. We still have to do the initial install with the server in Legacy BIOS mode; however, I managed to compile Xen as an EFI bootable binary using the source and patches distributed by Citrix. With that I am able to change the servers boot mode back to UEFI and boot XenServer. Here are the steps I used to compile it.
Steps
- Prepare a DDK
- Prepare a build environment
- Build some prerequisites
- Unpack the SRPM
- Compile Xen
DDK Preparation
Development will be done inside a 6.5 DDK. This is a CentOS 5.4-based Linux that has the same kernel as Dom0 and some of the required development tools.
Import the VM template per Citrix DDK developer documentation.
After importing, set the following VM options:
- 2 vCPUs
- Increase memory to 2048MB
- Resize disk image to 10GB
- Add a network interface for SSH
Start the VM, set a root password, and then finalize resizing the disk by running:
# fdisk /dev/xvda cmd: d -del cmd: n -new cmd: p -primary cmd: 1 -num 1 use default size cmd: w -save
Preparing the Build Environment
Install rpmdevtools:
# yum --disablerepo citrix install rpmdevtools # rpmdev-setuptree
I also needed several packages many of which were provided on the binpkg ISO from Citrix. I made them available by inserting XenServer-6.5-binpkg.iso and running the following:
# mount /dev/xvdb /mnt # mkdir /opt/binpkg # cp -a /mnt/domain0/RPMS/* /opt/binpkg # cd /opt/binpkg # createrepo /opt/binpkg # cat >/etc/yum.repos.d/binpkg.repo [binpkg] name=binpkg baseurl=file:///opt/binpkg gpgcheck=0 ^D
I also added the epel repository:
rpm -Uvh http://dl.fedoraproject.org/pub/epel/5/i386/epel-release-5-4.noarch.rpm
and I added the following packages:
bzip2-devel
e4fsprogs-devel
gettext
glib-devel
glib2-devel
iasl
netpbm
netpbm-progs
psutils
tetex-dvips
tetex-latex
DejaGnu
tcl
expect
makeinfo
texinfo
pixman-devel
Building the Prerequisites
This page: http://xenbits.xen.org/docs/4.3-testing/misc/efi.html says that it is required to use gcc 4.5 or better and that that binutils must be compiled with –enable-targets=x86_64-pep. I could not satisfy this with packages in the repositories, so I compiled and installed some requirements for gcc:
ftp://ftp.gnu.org/gnu/gmp/gmp-4.3.2.tar.gz
http://www.mpfr.org/mpfr-2.4.2/mpfr-2.4.2.tar.gz
http://www.multiprecision.org/mpc/download/mpc-0.8.1.tar.gz
and made sure the required files could be found:
# export LD_LIBRARY_PATH=:/usr/lib:/usr/local/lib:/usr/local/lib64:/usr/lib64 # ln -s /usr/lib64/libcrypto.so.0.9.8e libcrypto.so
then I compiled binutils using: https://ftp.gnu.org/gnu/binutils/binutils-2.22.tar.gz:
# tar xzf binutils-2.22.tar.gz # mkdir binutils-build # cd binutils-build # ../binutils-2.22/configure --disable-werror --enable-targets=x86_64-pep # make # make install
then I compiled gcc I using: http://mirrors-usa.go-parts.com/gcc/releases/gcc-4.6.2/gcc-4.6.2.tar.gz. I adapted compilation instructions from: http://www.linuxfromscratch.org/lfs/view/7.1/chapter06/gcc.html:
# cd gcc-4.6.2 # sed -i 's/install_to_$(INSTALL_DEST) //' libiberty/Makefile.in # case `uname -m` in i?86) sed -i 's/^T_CFLAGS =$/& -fomit-frame-pointer/' gcc/Makefile.in ;; esac # sed -i 's@./fixinc.sh@-c true@' gcc/Makefile.in # mkdir -v ../gcc-build # cd ../gcc-build # ../gcc-4.6.2/configure --prefix=/usr --libexecdir=/usr/lib --enable-shared --enable-threads=posix --enable-__cxa_atexit --enable-clocale=gnu --enable-languages=c,c++ --disable-multilib --disable-bootstrap --with-system-zlib # make -j3 # ulimit -s 16384 # make -k check # ../gcc-4.6.2/contrib/test_summary # make install
Unpack the SRPM
At this point everything is ready to compile Xen as an EFI bootable binary. The source code for Xen with Citrix’s patches is available here: http://xenserver.org/open-source-virtualization-download.html so download XenServer-6.5.0-source-main-1.iso and mount it at /mnt/cdrom:
# mount -r /dev/xvdb /mnt/cdrom
Now install the SRPM:
# rpm -ivh /mnt/cdrom/xen/xen-4.4.1-1.9.0.459.28798.src.rpm
Compile Xen
The source code and required scripts are now all under /root/rpmbuild/, so just run:
# cd ~/rpmbuild # QA_RPATHS=$[ 0x0020 ] rpmbuild -bc SPECS/xen.spec
The -bc flag causes the process to follow the spec file and patch the source, but then stop just before running the make commands. The make commands would fail to compile with warnings about uninitialized variables being treated as errors. Fix this by changing line 45 of ~/rpmbuild/BUILD/xen-4.4.1/xen/Rules.mk to read:
CFLAGS += -Werror -Wno-error=uninitialized -Wredundant-decls
-Wno-pointer-arith
and line 39 of ~/rpmbuild/BUILD/xen-4.4.1/Config.mk to read:
HOSTCFLAGS = -Wall -Werror -Wno-error=uninitialized -Wstrict-prototypes -O2 -fomit-frame-pointer
after making those changes run:
# cd ~/rpmbuild/BUILD/xen-4.4.1/ # make clean # make max_phys_cpus=256 XEN_TARGET_ARCH=x86_64 -C xen XEN_VENDORVERSION=-xs90192 debug=n build
and the compile will finish successfully. I probably could have just made a quick patch to add the -Wno-error flag and allowed the rpmbuild to run the full spec file, but I didn’t actually need to compile xen-tools etc, those are already compiled and installed on the XenServer installation. The only file needed is ~/rpmbuild/BUILD/xen-4.4.1/xen/xen.efi. With that in hand I created a xen.cfg file like this:
[global] default=xen [xen] options=console=vga,com1,com2 com1=115200,8n1,0x3F8,4 com2=115200,8n1,0x2F8,3 loglvl=all noreboot kernel=vmlinuz-3.10-xen root=UUID=b4ee0ace-b587-41df-a66b-16f89731b2a8 rw ignore_loglevel acpi_rsdp=0x7B7FE014 ramdisk=initrd-3.10-xen.img
where the root UUID is the boot disk created during the XenServer install and the RSDP number came from running:
# dmesg | grep RSDP
I ran that in an EFI booted live Linux environment. I found that some
vendors’ UEFI implementations were able to provide the RSDP during boot
and some were not, so without specifying it in the xen.cfg I had trouble
with things like usb peripherals.
Boot XenServer
With the xen.efi and xen.cfg I was able to boot XenServer in UEFI boot mode using refind. We have done extensive testing on several different servers and found no problems. I was also able to repeat the process with the source code provided by the service packs up to and including Service Pack 1. I haven’t tried any further than that yet.
Editors Note
For those of you wishing to retain Citrix commercial support status, the above procedure will convert the XenServer 6.5 host into an „unsupported configuration”.
