Monday, July 7, 2014

Enabling Hyper-V enlightenments with KVM

Windows has support for several paravirt features that it will use when running on Hyper-V, Microsoft's hypervisor. These features are called enlightenments. Many of the features are similar to paravirt functionality that exists with Linux on KVM (virtio, kvmclock, PV EOI, etc.)

Nowadays QEMU/KVM can also enable support for several Hyper-V enlightenments. When enabled, Windows VMs running on KVM will use many of the same paravirt optimizations they would use when running on Hyper-V. For detailed info, see Vadim's presentation from KVM Forum 2012.

From the QEMU/KVM developers, the recommended configuration is:

 -cpu ...,hv_relaxed,hv_spinlocks=0x1fff,hv_vapic,hv_time  

Which maps to the libvirt XML:

   <relaxed state='on'/>  
   <vapic state='on'/>  
   <spinlocks state='on' retries='8191'/>  
 <clock ...>  
  <timer name='hypervclock' present='yes'/>  

Some details about the individual features:
  • relaxed/hv_relaxed. Available in libvirt 1.0.0+ (commit) and qemu 1.1+ (commit). This bit disables a Windows sanity check that commonly results in a BSOD when the VM is running on a heavily loaded host (example bugs here, here, and here). Sounds similar to the Linux kernel option no_timer_check, which is automatically enabled when Linux is running on KVM.
  • vapic/hv_vapic. Available in libvirt 1.1.0+ (commit) and qemu 1.1+ (commit).
  • spinlocks/hv_spinlocks. Available in libvirt 1.1.0+ (commit) and qemu 1.1+ (commit)
  • hypervclock/hv_time. Available in libvirt 1.2.2+ (commit) and qemu 2.0+ (commit). Sounds similar to kvmclock, a paravirt time source which is used when Linux is running on KVM.

It should be safe to enable these bits for all Windows VM, though only Vista/Server 2008 and later will actually make use of the features.

(In fact, Linux also has support for using these Hyper-V features, like the paravirt device drivers and hyperv_clocksource. Though these are really only for running Linux on top of Hyper-V. With Linux on KVM, the natively developed paravirt extensions are understandably preferred).

The next version of virt-manager will enable Hyper-V enlightenments when creating a Windows VM (git commit). virt-xml can also be used to enable these bits easily from the command line for an existing VM:

 sudo virt-xml $VMNAME --edit --features hyperv_relaxed=on,hyperv_vapic=on,hyperv_spinlocks=on,hyperv_spinlocks_retries=8191
 sudo virt-xml $VMNAME --edit --clock hypervclock_present=yes  

The first invocation will work with virt-manager 1.0.1, the second invocation requires virt-manager.git. In my testing this didn't upset my existing Windows VMs and they worked fine after a reboot.

Other tools aren't enabling these features yet, though there are bugs tracking this for the big ones:
(edit 2014-09-08: This change was released in virt-manager-1.1.0)


  1. Thanks for taking the time to post this. Incredibly helpful and helped us resolve an on-going BSOD issue.

  2. Correct link to the presentation:

  3. Thanks - this helped my W10 guest run a lot faster!

  4. hmmm, 3dmark crash windows8.1 (firestrike) in 64 program mode :-(
    [ 4491.294598] kvm [5592]: vcpu0, guest rIP: 0xfffff80160e01f14 ignored rdmsr: 0xe8