Overview

We were interested in getting XenServer 6.5 to boot via UEFI.  Leaving servers in Legacy/BIOS boot was not an option in our target environment.  We still have to do the initial install with the server in Legacy BIOS mode; however, I managed to compile Xen as an EFI bootable binary using the source and patches distributed by Citrix.  With that I am able to change the servers boot mode back to UEFI and boot XenServer.  Here are the steps I used to compile it.

Steps

  1. Prepare a DDK
  2. Prepare a build environment
  3. Build some prerequisites
  4. Unpack the SRPM
  5. Compile Xen

DDK Preparation

Development will be done inside a 6.5 DDK. This is a CentOS 5.4-based Linux that has the same kernel as Dom0 and some of the required development tools. 

Import the VM template per Citrix DDK developer documentation.

After importing, set the following VM options:

  • 2 vCPUs
  • Increase memory to 2048MB
  • Resize disk image to 10GB
  • Add a network interface for SSH

Start the VM, set a root password, and then finalize resizing the disk by running:

# fdisk /dev/xvda 
cmd: d  -del
cmd:  n  -new
cmd:  p  -primary
cmd:  1  -num 1
	use default size
cmd:  w  -save

Preparing the Build Environment

Install rpmdevtools:

# yum --disablerepo citrix install rpmdevtools
# rpmdev-setuptree

I also needed several packages many of which were provided on the binpkg ISO from Citrix. I made them available by inserting XenServer-6.5-binpkg.iso and running the following:

# mount /dev/xvdb /mnt
# mkdir /opt/binpkg
# cp -a /mnt/domain0/RPMS/* /opt/binpkg
# cd /opt/binpkg
# createrepo /opt/binpkg
# cat >/etc/yum.repos.d/binpkg.repo
[binpkg]
name=binpkg
baseurl=file:///opt/binpkg
gpgcheck=0
^D

I also added the epel repository:

rpm -Uvh http://dl.fedoraproject.org/pub/epel/5/i386/epel-release-5-4.noarch.rpm

and I added the following packages:

bzip2-devel
e4fsprogs-devel
gettext
glib-devel
glib2-devel
iasl
netpbm
netpbm-progs
psutils
tetex-dvips
tetex-latex
DejaGnu
tcl
expect
makeinfo
texinfo
pixman-devel

Building the Prerequisites

This page: http://xenbits.xen.org/docs/4.3-testing/misc/efi.html says that it is required to use gcc 4.5 or better and that that binutils must be compiled with –enable-targets=x86_64-pep. I could not satisfy this with packages in the repositories, so I compiled and installed some requirements for gcc:

ftp://ftp.gnu.org/gnu/gmp/gmp-4.3.2.tar.gz
http://www.mpfr.org/mpfr-2.4.2/mpfr-2.4.2.tar.gz
http://www.multiprecision.org/mpc/download/mpc-0.8.1.tar.gz

and made sure the required files could be found:

# export LD_LIBRARY_PATH=:/usr/lib:/usr/local/lib:/usr/local/lib64:/usr/lib64
# ln -s /usr/lib64/libcrypto.so.0.9.8e libcrypto.so

then I compiled binutils using: https://ftp.gnu.org/gnu/binutils/binutils-2.22.tar.gz:

# tar xzf binutils-2.22.tar.gz
# mkdir binutils-build
# cd binutils-build
# ../binutils-2.22/configure --disable-werror --enable-targets=x86_64-pep
# make
# make install

then I compiled gcc I using: http://mirrors-usa.go-parts.com/gcc/releases/gcc-4.6.2/gcc-4.6.2.tar.gz. I adapted compilation instructions from: http://www.linuxfromscratch.org/lfs/view/7.1/chapter06/gcc.html:

# cd gcc-4.6.2
# sed -i 's/install_to_$(INSTALL_DEST) //' libiberty/Makefile.in
# case `uname -m` in
i?86) sed -i 's/^T_CFLAGS =$/& -fomit-frame-pointer/' 
gcc/Makefile.in ;;
esac
# sed -i 's@./fixinc.sh@-c true@' gcc/Makefile.in
# mkdir -v ../gcc-build
# cd ../gcc-build
# ../gcc-4.6.2/configure --prefix=/usr 
--libexecdir=/usr/lib --enable-shared 
--enable-threads=posix --enable-__cxa_atexit 
--enable-clocale=gnu --enable-languages=c,c++ 
--disable-multilib --disable-bootstrap --with-system-zlib
# make -j3
# ulimit -s 16384
# make -k check
# ../gcc-4.6.2/contrib/test_summary
# make install

Unpack the SRPM

At this point everything is ready to compile Xen as an EFI bootable binary.  The source code for Xen with Citrix’s patches is available here: http://xenserver.org/open-source-virtualization-download.html so download XenServer-6.5.0-source-main-1.iso and mount it at /mnt/cdrom:

# mount -r /dev/xvdb /mnt/cdrom

Now install the SRPM:

# rpm -ivh /mnt/cdrom/xen/xen-4.4.1-1.9.0.459.28798.src.rpm 

Compile Xen

The source code and required scripts are now all under /root/rpmbuild/, so just run:

# cd ~/rpmbuild
# QA_RPATHS=$[ 0x0020 ] rpmbuild -bc SPECS/xen.spec

The -bc flag causes the process to follow the spec file and patch the source, but then stop just before running the make commands. The make commands would fail to compile with warnings about uninitialized variables being treated as errors. Fix this by changing line 45 of ~/rpmbuild/BUILD/xen-4.4.1/xen/Rules.mk to read:

CFLAGS += -Werror -Wno-error=uninitialized -Wredundant-decls
-Wno-pointer-arith

and line 39 of ~/rpmbuild/BUILD/xen-4.4.1/Config.mk to read:

HOSTCFLAGS = -Wall -Werror -Wno-error=uninitialized -Wstrict-prototypes -O2 -fomit-frame-pointer

after making those changes run:

# cd ~/rpmbuild/BUILD/xen-4.4.1/
# make clean
# make max_phys_cpus=256 XEN_TARGET_ARCH=x86_64 -C xen
XEN_VENDORVERSION=-xs90192 debug=n build

and the compile will finish successfully.  I probably could have just made a quick patch to add the -Wno-error flag and allowed the rpmbuild to run the full spec file, but I didn’t actually need to compile xen-tools etc, those are already compiled and installed on the XenServer installation. The only file needed is ~/rpmbuild/BUILD/xen-4.4.1/xen/xen.efi. With that in hand I created a xen.cfg file like this:

[global]
default=xen

[xen]
options=console=vga,com1,com2 com1=115200,8n1,0x3F8,4
com2=115200,8n1,0x2F8,3 loglvl=all noreboot
kernel=vmlinuz-3.10-xen root=UUID=b4ee0ace-b587-41df-a66b-16f89731b2a8
rw ignore_loglevel acpi_rsdp=0x7B7FE014
ramdisk=initrd-3.10-xen.img

where the root UUID is the boot disk created during the XenServer install and the RSDP number came from running:

# dmesg | grep RSDP

I ran that in an EFI booted live Linux environment. I found that some
vendors’ UEFI implementations were able to provide the RSDP during boot
and some were not, so without specifying it in the xen.cfg I had trouble
with things like usb peripherals.

Boot XenServer

With the xen.efi and xen.cfg I was able to boot XenServer in UEFI boot mode using refind.  We have done extensive testing on several different servers and found no problems.  I was also able to repeat the process with the source code provided by the service packs up to and including Service Pack 1.  I haven’t tried any further than that yet.

Editors Note

For those of you wishing to retain Citrix commercial support status, the above procedure will convert the XenServer 6.5 host into an „unsupported configuration”.

 

Read More

Tagged with →  
Share →