Tuesday, July 15, 2014

virt-manager: changing the default storage path and default virtual network

When creating a new virtual machine via virt-manager or virt-install, the tools make some assumptions about the default location for disk images, and the default network source.

For example, in the 'New VM' wizard, the storage page will offer to create a disk image in the default location:


The default location for most uses of virt-manager is /var/lib/libvirt/images, which is created by libvirt and has the expected selinux labelling and permission to run QEMU/KVM VMs.

Behind the scenes, virt-manager is using a libvirt storage pool for creating disk images. When the 'New VM' wizard is first run, virt-manager looks for a storage pool named 'default'; if it doesn't find that it will create a storage pool named 'default' pointing to /var/lib/libvirt/images. It then uses that 'default' pool for the disk provisioning page.

The default virtual network works similarly. The libvirt-daemon-config-network package will dynamically create a libvirt virtual network named 'default'. You can see the XML definition over here in libvirt.git.

When virt-manager reaches the last page of the 'New VM' wizard, if there's a virtual network named 'default', we automatically select it as the network source:


It's also the network source used when no explicit network configuration is passed to virt-install.

Every now and then someone asks how to make virt-manager/virt-install use a different storage pool or network as the default. As the above logic describes, just name the desired virtual network or storage pool 'default', and the tools will do the right thing.

You can rename storage pools and virtual networks using virt-manager's UI from Edit->Connection Details. It only works on a stopped object though. Here's an example renaming a virtual network 'default' to 'new-name':


Monday, July 7, 2014

Enabling Hyper-V enlightenments with KVM

Windows has support for several paravirt features that it will use when running on Hyper-V, Microsoft's hypervisor. These features are called enlightenments. Many of the features are similar to paravirt functionality that exists with Linux on KVM (virtio, kvmclock, PV EOI, etc.)

Nowadays QEMU/KVM can also enable support for several Hyper-V enlightenments. When enabled, Windows VMs running on KVM will use many of the same paravirt optimizations they would use when running on Hyper-V. For detailed info, see Vadim's presentation from KVM Forum 2012.

From the QEMU/KVM developers, the recommended configuration is:

 -cpu ...,hv_relaxed,hv_spinlocks=0x1fff,hv_vapic,hv_time  

Which maps to the libvirt XML:

 <features>  
  <hyperv>  
   <relaxed state='on'/>  
   <vapic state='on'/>  
   <spinlocks state='on' retries='8191'/>  
  </hyperv>  
 <features/>  
   
 <clock ...>  
  <timer name='hypervclock' present='yes'/>  
 </clock>  

Some details about the individual features:
  • relaxed/hv_relaxed. Available in libvirt 1.0.0+ (commit) and qemu 1.1+ (commit). This bit disables a Windows sanity check that commonly results in a BSOD when the VM is running on a heavily loaded host (example bugs here, here, and here). Sounds similar to the Linux kernel option no_timer_check, which is automatically enabled when Linux is running on KVM.
  • vapic/hv_vapic. Available in libvirt 1.1.0+ (commit) and qemu 1.1+ (commit).
  • spinlocks/hv_spinlocks. Available in libvirt 1.1.0+ (commit) and qemu 1.1+ (commit)
  • hypervclock/hv_time. Available in libvirt 1.2.2+ (commit) and qemu 2.0+ (commit). Sounds similar to kvmclock, a paravirt time source which is used when Linux is running on KVM.

It should be safe to enable these bits for all Windows VM, though only Vista/Server 2008 and later will actually make use of the features.

(In fact, Linux also has support for using these Hyper-V features, like the paravirt device drivers and hyperv_clocksource. Though these are really only for running Linux on top of Hyper-V. With Linux on KVM, the natively developed paravirt extensions are understandably preferred).

The next version of virt-manager will enable Hyper-V enlightenments when creating a Windows VM (git commit). virt-xml can also be used to enable these bits easily from the command line for an existing VM:

 sudo virt-xml $VMNAME --edit --features hyperv_relaxed=on,hyperv_vapic=on,hyperv_spinlocks=on,hyperv_spinlocks_retries=8191
 sudo virt-xml $VMNAME --edit --clock hypervclock_present=yes  

The first invocation will work with virt-manager 1.0.1, the second invocation requires virt-manager.git. In my testing this didn't upset my existing Windows VMs and they worked fine after a reboot.

Other tools aren't enabling these features yet, though there are bugs tracking this for the big ones:

Saturday, July 5, 2014

Fedora 21 Virt Test Day scheduled for August 20th 2014

Just a quick note that the Fedora 21 Virt Test Day is scheduled for Wednesday, August 20th 2014. The inprogress landing page is at:

https://fedoraproject.org/wiki/Test_Day:2014-08-20_Virtualization

So if you're interested in helping test new virt features, or you want to make sure that the stuff you care about isn't broken, please mark your calendars.

Sunday, June 1, 2014

python-bugzilla 1.1.0 released

I just released python-bugzilla 1.1.0, you can see the release announcement over here. Updates in progress for f19, f20, rawhide, and epel6.

This release includes:

- Support for bugzilla tokens (Arun Babu Nelicattu)
- bugzilla: Add query/modify --tags
- bugzilla --login: Allow to login and run a command in one shot
- bugzilla --no-cache-credentials: Don't use or save cached credentials
  when using the CLI
- Show bugzilla errors when login fails
- Don't pull down attachments in bug.refresh(), need to get
  bug.attachments manually
- Add Bugzilla bug_autorefresh parameter.

Some of these changes deserve a bit more explanation. This is just adapted from the release announcement, but I wanted to give these changes a bit more attention here on the blog.

Bugzilla tokens

 

Sometime later in the year, bugzilla.redhat.com will be updated to a new version of bugzilla that replaces API cookie authentication with a non-cookie token system. I don't fully understand the reasoning so don't ask :) Regardless, this release supports the new token infrastructure along side the existing cookie handling.

Users shouldn't notice any difference: we cache the token in ~/.bugzillatoken so things work the same as the current cookie auth.

If you use cookiefile=None in the API to tell bugzilla _not_ to cache any login credentials, you will now also want to specify tokenfile=None (hint hint fedora infrastructure).

bugzilla --login and bugzilla --no-cache-credentials

 

Right now the only way to perform an authenticated bugzilla command on a new machine requires running a one time 'bugzilla login' to cache  credentials before running the desired command.

Now you can just do 'bugzilla --login ' and the login process will be initiated before invoking the command.

Additionally, the --no-cache-credentials option will tell the bugzilla tool to _not_ save any credentials to ~/.bugzillacookies or ~/.bugzillatoken.


Bugzilla.bug_autorefresh

 

When interacting with a Bug object, if you attempt to access a property (say, bugobj.component) that hasn't already been fetched from the bugzilla instance, python-bugzilla will do an automatic 'refresh'/getbug to pull down every property for said bug in an attempt to satisfy the request.

This is convenient for a one off API invocation, but for recurring scripts this is a waste of time and bugzilla bandwidth. The autorefresh can be avoided by passing a properly formatted include_fields to your query request, where include_fields contains every Bug property you will access in your script.

However it's still quite easy to extend a script with a new property usage and forget to adjust include_fields. Things will continue to work due to the autorefresh feature but your script will be far slower and wasteful.

A new Bugzilla property has been added, bug_autorefresh. Set this to False to disable the autorefresh feature for newly fetched bugs. This will cause an explicit error to be raised if your code is depending on autorefresh.

Please consider setting this property for your recurring scripts. Example:

  bzapi = Bugzilla("bugzilla.redhat.com")
  bzapi.bug_autorefresh = False
  ...

autorefresh can be disabled for individual bugs with:

  bug.autorefresh = False

Tuesday, May 20, 2014

Invoking a bugzilla query URL from the command line

The /usr/bin/bugzilla tool provided by python-bugzilla is quite handy for managing batch actions on bugs or quickly performing simple queries. However when you want to perform a crazy query with all sorts of boolean rules that the web UI exposes, the command line gets pretty unwieldy.

However, there's a simple workaround for this. Generate a query URL using the bugzilla web UI, and pass it to /usr/bin/bugzilla like:

 bugzilla query --from-url "$url"  

That's it! Then you can tweak the output as you want using --outputformat or similar. This also works for savedsearch URLs as well.

So, say I go to the bugzilla web UI and say 'show me all open, upstream libvirt bugs that haven't received a comment in since 2013'. It generates a massive URL. Here's what the URL looks like:

https://bugzilla.redhat.com/buglist.cgi?bug_status=NEW&bug_status=ASSIGNED&bug_status=POST&bug_status=MODIFIED&bug_status=ON_DEV&bug_status=ON_QA&bug_status=VERIFIED&bug_status=RELEASE_PENDING&classification=Community&component=libvirt&f1=longdesc&list_id=2372821&n1=1&o1=changedafter&product=Virtualization%20Tools&query_format=advanced&v1=2013-12-31

Just pass that that thing as $url in the above command, and you should see the same results as the web search (if your results aren't the same, you might need to do 'bugzilla login' first to cache credentials).

This is also easy to do via the python API:

#!/usr/bin/python

import bugzilla
bzapi = bugzilla.Bugzilla("bugzilla.redhat.com")
buglist = bzapi.search(bzapi.url_to_query("URL"))

for bug in buglist:
    print bug.summary
    ....

The caveat: as of now this only works with bugzilla.redhat.com, which has an API extension that allows it to interpret the URL syntax as regular query parameters. My understanding is that this may be available upstream at some point, so other bugzilla instances will benefit as well.