Friday, November 28, 2014

x2apic on by default with qemu 2.0+, and some history

x2apic is a performance and scalability feature available in many modern Intel CPUs. Though regardless of whether your host CPU supports it, KVM can unconditionally emulate it for x86 guests, giving an easy performance win with no downside. This feature has existed since 2009 and been a regular recommendation for tuning a KVM VM.

As of qemu 2.0.0 x2apic is enabled automatically (more details at the end).
Priot to that, actually benefiting from x2apic required a tool like virt-manager to explicitly enable the flag, which has had a long bumpy road.

x2apic is exposed on the qemu command line as a CPU feature, like:

qemu -cpu $MODEL,+x2apic ...

And there isn't any way to specify a feature flag without specifying the CPU model. So enabling x2apic required hardcoding a CPU model where traditionally tools (and libvirt) deferred to qemu's default.

A Fedora 13 feature page was created to track the change, and we enabled it in python-virtinst for f13/rawhide. The implementation attempted to hardcode the CPU model name that libvirt detected for the host machine, which unfortunately has some problems as I explained in a previous post. This led to some issues installing 64bit guests, and after trying to hack around it, I gave up and reverted the change.

(In retrospect, we likely could have made it work by just trying to duplicate the default CPU model logic that qemu uses, however that might have hit issues if the CPU default ever changed, like on RHEL for example.)

Later on virt-manager and virt-install gained UI for enabling x2apic, but a user had to know what they were doing and hunt it down.

As mentioned above, as of qemu 2.0.0 any x86 KVM VM will have x2apic automatically enabled, so there's no explicit need to opt in. From qemu.git:
commit ef02ef5f4536dba090b12360a6c862ef0e57e3bc
Author: Eduardo Habkost
Date:   Wed Feb 19 11:58:12 2014 -0300

    target-i386: Enable x2apic by default on KVM

Sunday, November 23, 2014

Updated instructions for using QEMU, UEFI, and Secureboot

Last year I started a wiki page about testing Fedora's Secureboot support with KVM. Just now I've cleaned up the page and modernized it for the current state of virt packages in F21:

https://fedoraproject.org/wiki/Using_UEFI_with_QEMU

The Secureboot steps are now at:

https://fedoraproject.org/wiki/Using_UEFI_with_QEMU#Testing_Secureboot_in_a_VM

The main change is that nowadays the virt tools know how to create persistent configuration storage for UEFI, so you can setup Secureboot once. Previously you had to do all sorts of crazy things to turn on Secureboot for each restart of the VM.

Friday, November 21, 2014

Running F21 aarch64 with QEMU, libvirt, and UEFI

I just wrote up a wiki page describing how to run F21 aarch64 bits with QEMU, libvirt, and UEFI:

https://fedoraproject.org/wiki/Architectures/AArch64/Install_with_QEMU

This was tested on x86 but the same steps should work if running on real aarch64 HW.

Wednesday, November 19, 2014

Run linaro aarch64 images with f21 virt-install + libvirt

Linaro generates some minimal openembedded based aarch64 disk images, which are useful for virt testing. There's simple instructions over here for running them with qemu on an x86 host. But with Fedora 21 packages, you can also these images with virt-install + libvirt + qemu.

Output looks like:
 Starting install...
Creating domain...                                          |    0 B  00:00   
Connected to domain linaro-aarch64
Escape character is ^]
[    0.000000] Linux version 3.17.0-1-linaro-vexpress64 (buildslave@x86-64-07) (gcc version 4.8.3 20140401 (prerelease) (crosstool-NG linaro-1.13.1-4.8-2014.04 - Linaro GCC 4.8-2014.04) ) #1ubuntu1~ci+141022120835 SMP PREEMPT Wed Oct 22 12:09:19 UTC 20
[    0.000000] CPU: AArch64 Processor [411fd070] revision 0
[    0.000000] Detected PIPT I-cache on CPU0
[    0.000000] Memory limited to 1024MB
...
Last login: Wed Nov 19 17:16:22 UTC 2014 on tty1
root@genericarmv8:~#
(Maybe you're wondering, what about fedora images? They are a bit different, since they expect to run with UEFI. I'll blog about that soon once I finish some testing)

Tuesday, November 18, 2014

Booting Fedora 21 ARM with QEMU and U-Boot

Running Fedora ARM with qemu is a bit of a pain because you need to pull the kernel and initrd out of the disk image and manually pass them to qemu; you can't just point qemu at the disk image and expect it to boot. The latter is how x86 qemu handles it (via a bundles seabios build).

On physical arm hardware, the bit that typically handles fetching the kernel/initrd from disk is U-Boot. However there are no U-Boot builds shipped with qemu for us to take advantage of.

Well that's changed a bit now. I was talking to Gerd about this at KVM Forum last month, and after some tinkering he got a working U-Boot build for the Versatile Express board that qemu emulates.

Steps to use it:
  • Grab a Fedora 21 ARM image (I used the F21 beta 'Minimal' image from here)
  • Enable Gerd's upstream firmware repo
  • Install u-boot.git-arm (this just installs some binaries in /usr/share, doesn't mess with any host boot config)
To use it with libvirt, you can do:

sudo virt-install --name f21-arm-a9-uboot \
  --ram 512 \
  --arch armv7l --machine vexpress-a9 \
  --os-variant fedora21 \
  --boot kernel=/usr/share/u-boot.git/arm/vexpress-a9/u-boot \
  --disk Fedora-Minimal-armhfp-21_Beta-1-sda.raw

For straight QEMU, you can do:

qemu-system-arm -machine vexpress-a9
  -m 512 \
  -nographic \
  -kernel /usr/share/u-boot.git/arm/vexpress-a9/u-boot \
  -sd Fedora-Minimal-armhfp-21_Beta-1-sda.raw