Thursday, May 18, 2017

Configuring offlineimap + dovecot + thunderbird

Recently some internal discussions at Red Hat motivated me to look into using offlineimap. I had thought about doing this for some time as a step towards giving mutt a try, but for now I decided to keep my original thunderbird setup. This turned out to be a bit more work than I anticipated, so I'm documenting it here.

The primary difficulty is that offlineimap stores mail locally in Maildir format, but thunderbird only reads mbox format. The common solution to this is to serve the offlineimap mail via a local mail server, and have thunderbird connect to that. For the mail server I'm using dovecot. Getting offlineimap output and dovecot to play nicely together in a format that thunderbird can consume was a bit tricky...

Here's the ~/.offlineimaprc I settled on:

[general]
accounts = work 
 
 
[Account work]
localrepository = local-work
remoterepository = remote-work

# Do a full check every 2 minutes
# autorefresh = 2
# Do 5 quick checks between every full check
# quick = 5


[Repository local-work]
type = Maildir
localfolders = ~/.maildir

# Translate your maildir folder names to the format the remote server expects
# So this reverses the change we make with the remote nametrans setting
nametrans = lambda name: re.sub('^\.', '', name)


[Repository remote-work]
type = IMAP
keepalive = 300
ssl = yes
sslcacertfile = /etc/ssl/certs/ca-bundle.crt
remotehost = $YOUR-WORK-MAIL-SERVER
remoteuser = $YOUR-USERNAME
# You can specify remotepass= , but my work setup implicitly uses kerberos

# Turn this on if you are manually messing with your maildir at all
# I lost some mail in my experiments :(
#readonly = yes

# Need to exclude '' otherwise it complains about infinite naming loop?
folderfilter = lambda foldername: foldername not in ['']
# For Dovecot to see the folders right I want them starting with a dot,
# and dovecot set to look for .INBOX as the toplevel Maildir
nametrans = lambda name: '.' + name

A few notes here:
  • autorefresh/quick are commented out because I'm not using them: I'm invoking 'offlineimap -o' with cron ever 2 minutes, with a small wrapper that ensures offlineimap isn't already running (not sure if that will have nasty side effects), and also checks that I'm on my work VPN (checking for a /sys/net/class/ path). I went with this setup because running offlineimap persistently will exit if it can't resolve the remote server after a few attempts, which will trigger if I leave the VPN. Maybe there's a setting to keep it running persistently but I couldn't find it.
  • Enable the 'readonly' option and 'offlineimap --dry-run' when initially configuring things or messing with maildir layout: I lost a few hours of mail somehow during the setup process :/
  • My setup implicitly depends on having authenticated with my companies kerberos. Still haven't figured out a good way of keeping the kerberos ticket fresh on a machine that moves on and off the VPN regularly. I know SSSD can kinda handle it but it seems to tie local login to work infrastructure which I'm not sure I want.

For dovecot, I just needed to drop this into /etc/dovecot/local.conf and start/enable the service:

protocols = imap imaps
listen = 127.0.0.1
mail_location = maildir:~/.maildir:INBOX=~/.maildir/.INBOX

Then configure thunderbird to connect to 127.0.0.1. User and password are the same as your local machine user account.

The tricky part seems to be formatting the maildir directory names in a way that dovecot will understand and properly advertise as folders/subfolders. I played with dovecot LAYOUT=fs, various sep/separator values and offlineimap renamings, but the above config is the only thing I found that gave expected results (and I can't take credit for that setup, I eventually found it on an internal wiki page :) )

Here's some (trimmed) directories in my ~/.maildir:

$ ls -1da .maildir/
.Drafts
.INBOX
.INBOX.fedora
.INBOX.libvirt
.INBOX.qemu
.INBOX.virt-tools
.Junk

So .Drafts, .INBOX, .Junk are all top level folders, and things like .INBOX.fedora is a 'fedora' subfolder of my inbox. That's the naming scheme the default dovecot config seems to expect.

Friday, March 24, 2017

Easy qemu commandline passthrough with virt-xml

Libvirt has supported qemu commandline option passthrough for qemu/kvm VMs for quite a while. The format for it is a bit of a pain though since it requires setting a magic xmlns value at the top of the domain XML. Basically doing it by hand kinda sucks.

In the recently released virt-manager 1.4.1, we added a virt-install/virt-xml option --qemu-commandline that tweaks option passthrough for new or existing VMs. So for example, if you wanted to add the qemu option string '-device FOO' to an existing VM named f25, you can do:

  ./virt-xml f25 --edit --confirm --qemu-commandline="-device FOO"

The output will look like:

--- Original XML
+++ Altered XML
@@ -1,4 +1,4 @@
-<domain type="kvm">
+<domain xmlns:qemu="http://libvirt.org/schemas/domain/qemu/1.0" type="kvm">
   <name>f25</name>
   <uuid>9b6f1795-c88b-452a-a54c-f8579ddc18dd</uuid>
   <memory unit="KiB">4194304</memory>
@@ -104,4 +104,8 @@
       <address type="pci" domain="0x0000" bus="0x00" slot="0x0a" function="0x0"/>
     </rng>
   </devices>
+  <qemu:commandline>
+    <qemu:arg value="-device"/>
+    <qemu:arg value="foo"/>
+  </qemu:commandline>
 </domain>

Define 'f25' with the changed XML? (y/n):

Wednesday, March 8, 2017

virt-manager 1.4.1 released!

I've just released virt-manager 1.4.1. The highlights are:
  • storage/nodedev event API support (Jovanka Gulicoska)
  • UI options for enabling spice GL (Marc-AndrĂ© Lureau)
  • Add default virtio-rng /dev/urandom for supported guest OS
  • Cloning and rename support for UEFI VMs (Pavel Hrdina)
  • libguestfs inspection UI improvements (Pino Toscano)
  • virt-install: Add --qemu-commandline
  • virt-install: Add --network vhostuser (Chen Hanxiao)
  • virt-install: Add --sysinfo (Charles Arnold)
Plus the usual slew of bug fixes and small improvements.

Wednesday, February 8, 2017

python-bugzilla 2.0.0 released!

I'm happy to announce a new release of python-bugzilla, version 2.0.0.

This release contains several small to medium API breaks as previously mentioned on the blog. If you hit any issues, check that page first to see if it's an expected change.
 
The major changes in the release are:
  • Several fixes for use with bugzilla 5
  • This release contains several smallish API breaks:
  • Bugzilla.bug_autorefresh now defaults to False
  • Credentials are now cached in ~/.cache/python-bugzilla/
  • bin/bugzilla was converted to argparse
  • bugzilla query --boolean_chart option is removed
  • Unify command line flags across sub commands
  • More details at: http://blog.wikichoon.com/2016/06/python-bugzilla-api-changes-in-git.html

Wednesday, June 29, 2016

UEFI virt roms now in official Fedora repos

Kamil got to it first, but just a note that UEFI roms for x86 and aarch64 virt are now shipped in the standard Fedora repos, where previously the recommended place to grab them was an external nightly repo. Kamil has updated the UEFI+QEMU wiki page to reflect this change.

On up to date Fedora 23+ these roms will be installed automatically with the relevant qemu packages, and libvirt is properly configured to advertise the rom files to applications, so enabling this with tools like virt-manager is available out of the box.

For the curious, the reason we can now ship these binaries in Fedora is because the problematic EDK2 'FatPkg' code, which had a Fedora incompatible license, was replaced with an implementation with a less restrictive (and more Fedora friendly) license.