Tuesday, May 20, 2014

Invoking a bugzilla query URL from the command line

The /usr/bin/bugzilla tool provided by python-bugzilla is quite handy for managing batch actions on bugs or quickly performing simple queries. However when you want to perform a crazy query with all sorts of boolean rules that the web UI exposes, the command line gets pretty unwieldy.

However, there's a simple workaround for this. Generate a query URL using the bugzilla web UI, and pass it to /usr/bin/bugzilla like:

 bugzilla query --from-url "$url"  

That's it! Then you can tweak the output as you want using --outputformat or similar. This also works for savedsearch URLs as well.

So, say I go to the bugzilla web UI and say 'show me all open, upstream libvirt bugs that haven't received a comment in since 2013'. It generates a massive URL. Here's what the URL looks like:


Just pass that that thing as $url in the above command, and you should see the same results as the web search (if your results aren't the same, you might need to do 'bugzilla login' first to cache credentials).

This is also easy to do via the python API:


import bugzilla
bzapi = bugzilla.Bugzilla("bugzilla.redhat.com")
buglist = bzapi.query(bzapi.url_to_query("URL"))

for bug in buglist:
    print bug.summary

The caveat: as of now this only works with bugzilla.redhat.com, which has an API extension that allows it to interpret the URL syntax as regular query parameters. My understanding is that this may be available upstream at some point, so other bugzilla instances will benefit as well.

Tuesday, May 13, 2014

virt-manager 1.0: reduced polling and CPU usage

A lot of work was done for virt-manager 1.0 to reduce the amount of libvirt polling and API calls we make for common operations. Up until this point, virt-manager had to poll libvirt at regular intervals to update the domain list, domain status, and domain XML. By default we would poll once a second (configurable in the Preferences dialog).

Although this burned far more CPU than necessary, things generally worked fine when talking to libvirtd on the local machine. However things really fell apart when connecting to a remote host with a lot of VMs, or over a high latency link: the polling would just saturate the connection and the app would be quite sluggish. Since the latter scenario is a pretty common setup for some of my remote colleagues at Red Hat, I heard about this quite a bit over the years :)

One of the major hurdles to reducing needless polling was that virt-manager and virtinst were separate code bases, as I explained in a previous post. For example, there is one routine in virtinst that will check if a new disk path is already in use by another VM: it does this by checking the path against the XML of every VM. Since virtinst was separate code, it had to do all this polling and XML fetching from scratch, despite the fact that we had this information cached in virt-manager. We could have taught virtinst about the virt-manager cache or some similar solution, but it was cumbersome to make changes like that while maintaining back compatibility with older virtinst users.

Well, with virt-manager 0.10.0 we deprecated the public virtinst API and merged the code into virt-manager git. This allowed us to do a ton of code cleanup and simplification during the virt-manager 1.0 cycle to remove much of the API spamming.

The other major piece we added in virt-manager 1.0 is use of asynchronous libvirt events. The initial events support in libvirt was added way back in October 2008 by a couple folks from VirtualIron. That's quite a while ago, so supporting this in virt-manager was long overdue. Though waiting a long time had the nice side effect of letting other projects like oVirt shake all the bugs out of libvirt's event implementations :)

Regardless, virt-manager 1.0 will use domain (and network) events now if connected to a sufficiently new version of libvirt and the driver supports it. We still maintain the old polling code for really old libvirt, and libvirt drivers that don't support the event APIs. Even on latest libvirt some polling is still needed since not all libvirt objects support event APIs, although now we poll on demand which reduces our CPU and network usage.

Tuesday, May 6, 2014

A brief-ish history of virtinst and virt-install

virt-install is a command line tool for creating new virtual machines via libvirt. It's an important piece of the libvirt ecosystem that has shipped in RHEL5.0 and up, and over a dozen Fedora versions.

It wasn't always called virt-install though: it started life as xenguest-install.py written by Jeremy Katz. I think it was just an internal Red Hat only thing for a brief period, until it first surfaced as part of the Fedora 'xen' package in January 2006:

 commit 02687b4e3f7fa0db5de34280a2cb7e1a8eb8ff18  
 Author: Stephen Tweedie <sct@fedoraproject.org>  
 Date:  Tue Jan 31 16:59:19 2006 +0000  
   Add xenguest-install.py in /usr/sbin  

(Strangely, the file isn't actually in git... I assume this is some accident of the CVS->git conversion. You can see the version shipped with Fedora Core 5 in the archived RPM).

Check out the original set of arguments

  -h, --help      show this help message and exit  
  -n NAME, --name=NAME Name of the guest instance  
  -f DISKFILE, --file=DISKFILE  
             File to use as the disk image  
  -s DISKSIZE, --file-size=DISKSIZE  
             Size of the disk image (if it doesn't exist) in  
  -r MEMORY, --ram=MEMORY  
             Memory to allocate for guest instance in megabytes  
  -m MAC, --mac=MAC   Fixed MAC address for the guest; if none is given a  
             random address will be used  
  -v, --hvm       This guest should be a fully virtualized guest  
  -c CDROM, --cdrom=CDROM  
             File to use a virtual CD-ROM device for fully  
             virtualized guests  
  -p, --paravirt    This guest should be a paravirtualized guest  
  -l LOCATION, --location=LOCATION  
             Installation source for paravirtualized guest (eg,  
             nfs:host:/path, http://host/path, ftp://host/path)  
  -x EXTRA, --extra-args=EXTRA  
             Additional arguments to pass to the installer with  
             paravirt guests  

All those bits are still working with virt-install today, although many are are deprecated and hidden from the --help output by default.

In early 2006, libvirt barely even existed, so the xenguest-install.py was generating xen xm config files (basically just raw python code) in /etc/xen.
Fedora CVS was the canonical home of the script.

In March 2006, Dan Berrangé started work on virt-manager. It was very briefly called gnome-vm-manager, then settled into gnome-virt-manager until July 2006 when it was renamed to virt-manager.

In August 2006, xenguest-install moved to its own repo, python-xeninst:

 commit 1e2e1aa0ca0b5ed8669be61aa4271a3e8c1d7333  
 Author: Jeremy Katz <katzj@redhat.com>  
 Date:  Tue Aug 8 21:37:49 2006 -0400  
   first pass at breaking up xenguest-install to have more of a usable API.  
   currently only works for paravirt and some of the bits after the install  
   gets started are still a little less than ideal  

Much of logic was moved to a 'xeninst' module. There were some initial bits for generating libvirt XML, but the primary usage was still generating native xen configuration.

(Both repositories were hosted in mercurial at hg.et.redhat.com for many years. We eventually transitioned to git in March 2011. Actually it's amazing it was only 3 years ago: I've pretty much entirely forgotten how to use mercurial despite using it for 4 years prior.)

In October 2006, the project was renamed python-virtinst and the tool renamed to virt-install. By this point virt-manager was using the xeninst module for guest creation and needed to handle the rename as well.

So now python-virtinst was its own standalone package, providing virt-install and a python library named virtinst. Over the next couple years the repo accumulated a few more tools: virt-clone in May 2007, virt-image in June 2007, and virt-convert (originally virt-unpack) in July 2008.

However over the next few years we had some growing pains with the virtinst module. It wasn't exactly a planned API, rather a collection of code that grew organically from a quick hack of a script. It never received too much thought for future compatibility. The fact that it ended up as a public API was more historic accident than anything. Once we accumulated external users (cobbler in March 2007 and koji in July 2010) we were stuck with the API in the name of back compatibility.

Then there was the general frustration of doing virt-manager development when it evolved in lockstep with virtinst: running upstream virt-manager always required running up to date python-virtinst, which was a barrier to upstream contribution.

So in February 2012 I layed out some reasons for dropping virtinst as a public API and merging the code into virt-manager.git, though it didn't fully happen until april 2013 during the virt-manager 0.10.0 cycle. In the intervening year, I sent patches to koan and koji to move off virtinst to calling the needed virt-* tool directly.

So nowadays virtinst, virt-install, etc. all live with virt-manager.git. If you're looking for a library that helps handle libvirt XML or create libvirt VMs, check out libvirt-designer and libvirt-gobject/libvirt-gconfig.