Tuesday, April 9, 2019

Host 'Network Interfaces' panel removed from virt-manager

I released virt-manager 2.0.0 in October 2018. Since the release contained the full port to python3, it seemed like a good opportunity to drop some baggage from the app.

The biggest piece we removed was the UI for managing host network interfaces. This is the Connection Details->Network Interfaces panel, and the 'New Interface' wizard for defining host network definitions for things like bridges, bonds, and vlan devices. The main screen of the old UI looked like this:

Behind the scenes, this UI was using libvirt's Interface APIs, which also power the 'virsh iface-*' commands. These APIs are little more than a wrapper around the netcf library.

netcf aimed to be a linux distro independent API for network device configuration. On Red Hat distros this meant turning the API's XML format into an /etc/sysconfig/network script. There were even pie-in-the-sky ideas about NetworkManager one day using netcf.

In practice though the library never really took off. It was years before a debian backend showed up, contributed by a Red Hatter in the hope of increasing library uptake, though it didn't seem to help. netcf basically only existed to serve the libvirt Interface APIs, yet those APIs were never really used by any major libvirt consuming app, besides virt-manager. And in virt-manager's case it was largely just slapping some UI over the XML format and lifecycle operations.

For virt-manager's usecases we hoped that netcf would make it trivial to bridge the host's network interface, which when used with VMs would give them first class IP addresses on the host network setup, not NAT like the 'default' virtual network. Unfortunately though the UI would create the ifcfg files well enough, behind the scenes nothing played well with NetworkManager for years and years. The standard suggestion for was to disable NetworkManager if you wanted to bridge your host NIC. Not very user friendly. Some people did manage to use the UI to that effect but it was never a trivial process.

Nowadays NetworkManager can handle bridging natively and is much more powerful than what virt-manager/libvirt/netcf provide. The virt-manager UI was more likely to shoot you in the foot than make things simple. And it had become increasingly clear that virt-manager was not the place to maintain host network config UI.

So we made the decision to drop all this from virt-manager in 2.0.0. netcf and the libvirt interface APIs still exist. If you're interested in some more history on the interface API/netcf difficulties, check out Laine's email to virt-tools-list.

Wednesday, January 9, 2019

python-bugzilla + bugzilla 5.0 API keys

For many uses of /usr/bin/bugzilla and python-bugzilla, it's necessary to actually be logged in to a bugzilla server. Creating bugs, editing bugs, querying private data, etc.

Up until now anyone that's used the command line tool has periodically had to do a 'bugzilla login' to refresh their authentication cache. In older bugzilla versions this was an HTTP cookie, more recently it's a bugzilla API token. Generally 'login' calls were needed infrequently on a single machine as tokens would remain valid for a long time.

Recently, bugzilla.redhat.com received a big update to bugzilla 5.0. However with that update it seems like API tokens now expire after a week, which has necessitated lots more 'bugzilla login' calls than I'm used to.

Thankfully with bugzilla 5.0 and later there's a better option: API keys. Here's how to to use them transparently with /usr/bin/bugzilla and all python-bugzilla library usage. Here's steps for enabling API keys with bugzilla.redhat.com, but the same process should roughly apply to other bugzilla instances too.

Login to the bugzilla web UI, click your email, select Preferences, select API Keys. Generate an API key with an optional comment like 'python-bugzilla'. Afterwards the screen will look something like this:

MY-EXAMPLE-API-KEY is not my actual key, I just replaced it for demo purposes. The actual key is a long string of characters and numbers. Copy that string value and write a bugzillarc file like this:

$ cat ~/.config/python-bugzilla/bugzillarc

That's it, /usr/bin/bugzilla and python-bugzilla using tools should pick it up automagically. Note, API keys are as good as passwords in certain ways, so treat it with the same secrecy you would treat a password.

Thursday, October 4, 2018

Setting custom network names on Fedora

systemd predictable network names give us host interface names like enp3s0. On one of my hosts, I have two interfaces: one that is my regular hard wired connection, and another I only plug in occasionally for some virt network testing. I can never remember the systemd names, so I want to rename the interfaces to something more descriptive for my needs. in my case lan0main and lan1pcie

The page referenced says to use systemd links. However after struggling with that for a while I'm that's only relevant to systemd-networkd usage and doesn't apply to Fedora's default use of NetworkManager. So I needed another way.

Long story short I ended up with some custom udev rules that are patterned after the old 70-persistent-net.rules file:

$ cat /etc/udev/rules.d/99-cole-nic-names.rules 
SUBSYSTEM=="net", ACTION=="add", DRIVERS=="?*", ATTR{address}=="70:8b:cd:80:e5:5f", ATTR{type}=="1", NAME="lan0main"
SUBSYSTEM=="net", ACTION=="add", DRIVERS=="?*", ATTR{address}=="68:05:ca:1a:f5:da", ATTR{type}=="1", NAME="lan1pcie"

Thursday, May 18, 2017

Configuring offlineimap + dovecot + thunderbird

Recently some internal discussions at Red Hat motivated me to look into using offlineimap. I had thought about doing this for some time as a step towards giving mutt a try, but for now I decided to keep my original thunderbird setup. This turned out to be a bit more work than I anticipated, so I'm documenting it here.

The primary difficulty is that offlineimap stores mail locally in Maildir format, but thunderbird only reads mbox format. The common solution to this is to serve the offlineimap mail via a local mail server, and have thunderbird connect to that. For the mail server I'm using dovecot. Getting offlineimap output and dovecot to play nicely together in a format that thunderbird can consume was a bit tricky...

Here's the ~/.offlineimaprc I settled on:

accounts = work 
[Account work]
localrepository = local-work
remoterepository = remote-work

# Do a full check every 2 minutes
# autorefresh = 2
# Do 5 quick checks between every full check
# quick = 5

[Repository local-work]
type = Maildir
localfolders = ~/.maildir

# Translate your maildir folder names to the format the remote server expects
# So this reverses the change we make with the remote nametrans setting
nametrans = lambda name: re.sub('^\.', '', name)

[Repository remote-work]
type = IMAP
keepalive = 300
ssl = yes
sslcacertfile = /etc/ssl/certs/ca-bundle.crt
remoteuser = $YOUR-USERNAME
# You can specify remotepass= , but my work setup implicitly uses kerberos

# Turn this on if you are manually messing with your maildir at all
# I lost some mail in my experiments :(
#readonly = yes

# Need to exclude '' otherwise it complains about infinite naming loop?
folderfilter = lambda foldername: foldername not in ['']
# For Dovecot to see the folders right I want them starting with a dot,
# and dovecot set to look for .INBOX as the toplevel Maildir
nametrans = lambda name: '.' + name

A few notes here:
  • autorefresh/quick are commented out because I'm not using them: I'm invoking 'offlineimap -o' with cron ever 2 minutes, with a small wrapper that ensures offlineimap isn't already running (not sure if that will have nasty side effects), and also checks that I'm on my work VPN (checking for a /sys/net/class/ path). I went with this setup because running offlineimap persistently will exit if it can't resolve the remote server after a few attempts, which will trigger if I leave the VPN. Maybe there's a setting to keep it running persistently but I couldn't find it.
  • Enable the 'readonly' option and 'offlineimap --dry-run' when initially configuring things or messing with maildir layout: I lost a few hours of mail somehow during the setup process :/
  • My setup implicitly depends on having authenticated with my companies kerberos. Still haven't figured out a good way of keeping the kerberos ticket fresh on a machine that moves on and off the VPN regularly. I know SSSD can kinda handle it but it seems to tie local login to work infrastructure which I'm not sure I want.

For dovecot, I just needed to drop this into /etc/dovecot/local.conf and start/enable the service:

protocols = imap imaps
listen =
mail_location = maildir:~/.maildir:INBOX=~/.maildir/.INBOX

Then configure thunderbird to connect to User and password are the same as your local machine user account.

The tricky part seems to be formatting the maildir directory names in a way that dovecot will understand and properly advertise as folders/subfolders. I played with dovecot LAYOUT=fs, various sep/separator values and offlineimap renamings, but the above config is the only thing I found that gave expected results (and I can't take credit for that setup, I eventually found it on an internal wiki page :) )

Here's some (trimmed) directories in my ~/.maildir:

$ ls -1da .maildir/

So .Drafts, .INBOX, .Junk are all top level folders, and things like .INBOX.fedora is a 'fedora' subfolder of my inbox. That's the naming scheme the default dovecot config seems to expect.

Friday, March 24, 2017

Easy qemu commandline passthrough with virt-xml

Libvirt has supported qemu commandline option passthrough for qemu/kvm VMs for quite a while. The format for it is a bit of a pain though since it requires setting a magic xmlns value at the top of the domain XML. Basically doing it by hand kinda sucks.

In the recently released virt-manager 1.4.1, we added a virt-install/virt-xml option --qemu-commandline that tweaks option passthrough for new or existing VMs. So for example, if you wanted to add the qemu option string '-device FOO' to an existing VM named f25, you can do:

  ./virt-xml f25 --edit --confirm --qemu-commandline="-device FOO"

The output will look like:

--- Original XML
+++ Altered XML
@@ -1,4 +1,4 @@
-<domain type="kvm">
+<domain xmlns:qemu="http://libvirt.org/schemas/domain/qemu/1.0" type="kvm">
   <memory unit="KiB">4194304</memory>
@@ -104,4 +104,8 @@
       <address type="pci" domain="0x0000" bus="0x00" slot="0x0a" function="0x0"/>
+  <qemu:commandline>
+    <qemu:arg value="-device"/>
+    <qemu:arg value="foo"/>
+  </qemu:commandline>

Define 'f25' with the changed XML? (y/n):