Tuesday, April 29, 2014

Github doesn't support pull-request notifications to mailing lists

Recently I played around with github a bit, with the intention of finding a useful setup for hosting python-bugzilla (and possibly virt-manager). However I want to preserve the traditional open source mailing list driven development model.

github's whole development model is built around pull requests. Personally I kinda like the setup, but for a pre-existing project built around patches on a mailing list it's quite a different workflow.

github doesn't allow disabling the pull-request option. This is understandable since it's pretty central to their entire model. However my main sticking point is that github doesn't provide a straightforward way to send pull request notifications to a mailing list. I don't want everyone on a mailing list to have to opt in to watching the repo on github to be notified of pull requests. I want non-github users to be able to contribute to pull request discussions on a mailing list. I want pull requests on the project mailing list since it's already a part of my workflow. I don't want my project to be one of those that accumulates ignored pull-requests because it isn't part of the project workflow and no one is watching the output.

Googling about this was quite frustrating, it was difficult to find a clear answer. I eventually found an abandoned pull request to github-services that made everything clear. But not before I tried quite a few things. Here's what I tried:
  • Opening a github account using the mailing list address, 'watch' the repository. It works, but yeah, not too safe since anyone can just trigger a 'forgot password' reset email.
  • Put the repo in an 'organization', add the mailing list as a secondary address to your account, have all notifications for the organization go to the mailing list. But even secondary accounts work for the password reset, so that's out.
  • 'email' webhook/service: At your github repo, go to settings->webhooks & services->configure services->email. Hey, this looks promising. The problem is it's quite limited in scope, only supporting email notifications for repo pushes and when  a public repo is added.
  • The actual webhook configuration is quite elaborate and allows notifying of pull-requests and everything else you would want to know, but that requires running an actual web service somewhere. But I have no interest in maintaining a public service just to proxy some email.
There's a public repo for the bit of github that the 'email' webhook lives under. I stuck some thoughts on an open issue that more or less tracks the RFE to extend the email capabilities.

Someone out there with some spare time and ruby-fu want to take a crack at this? I think many old school open source projects would be thankful for it :)

Tuesday, April 22, 2014

virt-convert command line has been reworked

One of the changes we made with virt-manager 1.0 was a large reworking of the virt-convert command line interface.

virt-convert started life as a tool for converting back and forth between different VM configuration formats. Originally it was just between vmx and virt-image(5), but it eventually grew ovf input support. However, for the common usage of trying to convert a vmx/ovf appliance into a libvirt guest, this involved an inconvenient two step process:

* Convert to virt-image(5) with virt-convert
* Convert to a running libvirt guest with virt-image(1)

Well, since virt-image didn't really have any users, it's planned for removal. So we took the opportunity to improve virt-convert in the process. Running virt-convert is now as simple as:

 virt-convert fedora18.ova  


 virt-convert centos6.tar.gz  

And a we convert directly to libvirt XML and launch the guest. Standard libvirt options are allowed, like --connect for specifying the libvirt driver.

The tool hasn't been heavily used and there's definitely still a lot of details we are missing, so if you hit any issues please file a bug report.

(Long term it sounds like gnome-boxes may grow a similar feature as mentioned over here, so maybe virt-convert isn't long for this world since there will likely be a command line interface for it as well)

Thursday, April 17, 2014

qemu 2.0.0 built for rawhide, but no qemu-system-aarch64 in fedora yet

qemu 2.0.0 was released today, and I've built it for rawhide now. Which means it will be in the fedora-virt-preview repo tomorrow when my scripts pick it up (need to convert to copr one of these days...).

The 2.0 release number is fairly arbitrary here and not related to any particular feature or development. Though each qemu release tends to have some newsworthy goodies in it :)

The top highlighted change in the announcement is about qemu-system-aarch64, though it isn't built in the Fedora package yet. Right now qemu-system-aarch64 _only_ works on aarch64 machines, but most people that will try and give it a spin now will be trying to do aarch64 emulation on x86. So I decided not to build it yet to save myself some bug reports :)

I suspect the next qemu release will have something usable for x86 usage, so in roughly 2 months time (if things stay on schedule) when qemu 2.1 rc0 is out, I'll add the qemu-system-aarch64 sub package. This is being tracked as a 'change' for Fedora 21: https://fedoraproject.org/wiki/Changes/Virt_64bit_ARM_on_x86

If anyone working on Fedora aarch64 actually cares about qemu-system-aarch64 at this stage and wants it packaged up, ping me and we can talk.

Tuesday, April 15, 2014

Deprecating little used tool virt-image(1)

In the recent virt-manager 1.0 release, we've taken a step towards deprecating one of the command line tools we ship, virt-image(1). This shouldn't have any real end user effect because I'm pretty sure near zero people are using it.

virt-image was created in June 2007 as an XML schema and command line tool for distributing VM images as appliances. The format would describe the fundamental needs of a VM image, like how many disk devices it wants, but leave the individual configuration details up to the user. The virt-image(1) tool would take the XML as input and kick off a libvirt VM.

While the idea was reasonable, the XML format would only be useful if people actually used it, which never happened. All desktop VM appliance usage nowadays is shipped with native VMWare config as .vmx files, or with .ovf configuration.

In the past, appliance-tools generated virt-image XML, but that was dropped. Same with boxgrinder.

But most users of it in the past few years have been way of the (also little used) virt-convert tool that we ship with virt-manager (which I'll cover in a later post). Historically virt-convert would output virt-image XML by default. Well, that too was changed in 1.0: virt-convert generates direct libvirt XML now. This makes virt-convert more convenient, and left us with no good reason to keep virt-image around anymore.

So the plan is to drop virt-image before the next major release of virt-manager. Likely sometime in the next 6 months.

(edit 2014-09-08: virt-image was removed in virt-manager-1.1.0)

Tuesday, April 8, 2014

pylint in Fedora 20 supports gobject introspection

GObject introspection is the magical plumbing that enables building multiple language bindings for a GObject-based library using not much more than API documentation. This is used by PyGObject to give us python access to gtk3, gdk, glib, etc. Pretty sweet.

Unfortunately the automagic nature of this confused pylint, claiming your 'from gi.repository import Gtk' import didn't exist, and losing many of its nice features when interacting with objects provided by introspection derived bindings.

I love pylint and it's a critical part of my python development workflow. So last year I decided to do my part and submit a patch to add some gobject introspection knowledge to pylint.

It works by actually importing the module (like 'Gtk' above), inspecting all its classes and functions, and building stub code that pylint can analyze. It's not perfect, but it will catch things like misspelled method names. (Apparently newer python-astroid has some infrastructure to inspect living objects, so likely the plugin will use that one day).

This support was released in python-astroid-1.0.1, which hit Fedora 20 at the beginning of March. Unfortunately a a bug was causing a bunch of false positives with gobject-introspection, but that should be fixed with python-astroid-1.0.1-3 heading to F20.

Tuesday, April 1, 2014

Spice USB redirection in virt-manager

A new feature we added in virt-manager 1.0 is out of the box support for Spice USB redirection.

When connected to a VM's graphical display, any USB device plugged in to your physical host will be automatically redirected to the VM. This is great for easily sharing a usb drive with your VM. Existing devices can also be manually attached via the VM window menu option 'Virtual Machine->Redirect USB Device'

The great thing about Spice USB redirection is that it doesn't require configuring the spice agent or any special drivers inside the VM, so for example it will 'just work' for your existing windows VMs. And since the streaming is done via the spice display widget, you can easily share a local USB device with a VM on a remote host.

This feature is only properly enabled for KVM VMs that are created with virt-manager 1.0 or later. Configuring an existing VM requires 3 changes:
  1. Set the graphics type to Spice
  2. Set the USB controller model to USB2
  3. Add a 'Redirection USB' device to the VM. Add multiple redirection devices to allow redirecting multiple host USB devices simultaneously.
All those bits should be fairly straight forward to do with the UI in virt-manager 1.0.

For more details, like how to do this using libvirt XML or the qemu command line, check the documentation over here: