Make sure virtualization is enabled in the BIOS

I just wasted a fair amount of time on a RedHat 6.1 box being setup to be a hypervisor with KVM, trying to figure how why when I ran virsh version it was telling me among other things

internal error Cannot find suitable emulator for x86_64

All the appropriate packages such as qemu-kvm were installed, but it just didn't seem to want to work. Finally as I was about to try reinstalling RHEL, I remoted into the actual console and saw:

kvm: disabled by bios

Doh!, and looking back in /var/log/messages the same thing was buried deep within all the boot noise. While trying to figure this out I managed to just be looking for virt or qemu in the logs and somehow didn't search for kvm. Enabled virtualization in the BIOS and everything's gravy now.

So there you go, if you're Googling that first error message and get lots of other nonsense, look for the message about the BIOS.

amqplib 1.0.0

I attended OSCON for the first time this year, and to celebrate I thought I'd wrap up the Python amqplib library a bit and consider it more-or-less finished for what it is (a simple blocking 0-8 client), and call it 1.0.0 You can find it on the in PyPi and Google Project Hosting

It's definitely a worthwhile upgrade in that it's significantly faster than amqplib 0.6.1, and has a fair number of bug fixes. Also noteworthy are support for Python 3.x (via 2to3) and IPv6

smbpasswd 1.0.2 submitted to PyPi

smbpasswd is a really old piece of software (9 years!) for generating NT/LM password hashes, suitable for use with Samba. It's in Debian/Ubuntu/Redhat repositories, and FreeBSD ports, and who knows where else.

Somehow it never got submitted to PyPi, but I took care of that today at the request of someone working on another Python module that wanted to use this as a dependency. Look for smbpasswd-1.0.2, or just easy_install smbpasswd if you're setup for that.

I changed the packaging slightly, so that the tarball extracts to smbpasswd-x.x.x instead of py-smbpasswd-x.x.x, and so bumped the version number to 1.0.2 just for the packaging changes. The library itself is unchanged.

However, I think you'd want to be very careful generating and storing LM hashes of user's passwords, they seem to be wildly insecure.

If your app can get by with just NT hashes, and you have a Python >= 2.5, you may be able to generate those using the standard Python library, and don't need this package at all. See the notes on my py-md4 page.

Rebuilding Hyper-V Linux Integration Components for a kernel upgrade

At work I've been running RedHat Enterprise Linux (RHEL) 5.6 on top of a Windows Server 2008 R2 Hyper-V host, with Linux Integration Components (LinuxIC) installed.

That all worked fine until I did a yum update on RHEL to pick up a new kernel and tried to reboot. The new kernel panicked, saying it couldn't find my LVM volume groups. Fortunately, the old kernel was still on the menu and booted OK. Turns out the new kernel didn't for whatever reason wouldn't see the virtual disks for that machine.

OK, so I needed to rebuild LinuxIC for the new kernel while running the old kernel, how to do that? The Makefile and various scripts that come with LinuxIC basically builds for and installs on the currently running kernel. Fortunately I came across this post showing a trick of replacing /bin/uname with a fake version that shows the kernel version number you want to build for. Tried that and was back in business.

I think this would work too, without messing with the original /bin/uname: create a directory somewhere, say '/tmp/fake_uname' and stick this file in it with the name uname (changing the "echo" line with the installed kernel version number you want to build for.

case $1 in
        echo "2.6.18-238.9.1.el5"
        exec /bin/uname $1

Then build and install your Linux IC with the /tmp/fake_uname prepended to your PATH as in

PATH=/tmp/fake_uname:$PATH make
PATH=/tmp/fake_uname:$PATH make install

When the LinuxIC build calls uname it finds the fake version first, and if the argument is -r shows you desired version number, otherwise falls back to the real uname.

Simple debugging output in C

I don't do a whole lot of C programming, but when I do it tends to be in difficult environments like Apache modules or Samba VFS modules, where you can't just do simple printfs to get some output from your program.

I've come up with this small chunk of code I can plop in a C file to allow for optionally writing out useful information to a file somewhere on the disk.

    #include <stdarg.h>
    #include <stdio.h>
    #include <time.h>
    #define QUOTE(name) #name
    #define STR(macro) QUOTE(macro)

    static void debug_log(const char *msg, ...) {
        char timestamp[32]; // really only need 21 bytes here
        time_t now;
        va_list ap;
        FILE *f;

        now = time(NULL);
        strftime(timestamp, sizeof(timestamp), "%Y-%m-%d %H:%M:%S ", localtime(&now));

        f = fopen(STR(DEBUG_FILENAME), "a");

        fputs(timestamp, f);

        va_start(ap, msg);
        vfprintf(f, msg, ap);

        fputc('\n', f);
    #define debug_log

Within your program, you'd just sprinkle in calls to debug_log() with a format string and optional arguments, such as:

x = 5;
y = 10;
debug_log("Currently, x=%d, y=%d", x, y);

The code can then be enabled and configured to output to /tmp/foo.log (for example), by adding either

#define DEBUG_FILENAME /tmp/foo.log

to the top of your source file, or even more slickly for some things, from the commandline with

cc -DDEBUG_FILENAME=/tmp/foo.log myprogram.c

When the program is run, in your /tmp/foo.log file you'd find something like:

2011-04-03 20:30:05 Currently, x=5, y=10

If you don't define DEBUG_FILENAME, the code basically goes away, shouldn't take up any space in your binary at all.

Flash playback on MacOSX Firefox

For a long time I've been annoyed by really jerky playback on Flash videos under Firefox on MacOSX.
This YouTube video for example, was just awful to watch, stuttering very frequently.

Turns out the fix is pretty simple: just go into about:config and increase the browser.sessionstore.interval setting in Firefox from the default of 10000 (10 seconds) to something larger like 120000 (120 seconds).
Got it from this page, even though it's talking about Ubuntu Firefox, it still applies to MacOSX and seems to have made a world of difference.

amqplib and bpgsql on Google Code

I've created Google Code projects for my amqplib and bpgsql packages, to take advantage of their nice infrastructure including issue tracking.

amqplib 0.6

Wrapped up another release of py-amqplib, version 0.6 - which features a major reorganization of the codebase to make the library more maintainable and lays the groundwork for an optional thread-assisted mode that allows for flow control and timeouts (being worked on in a development repository).

Building additional PHP modules on OSX

Normally I try to avoid dealing with PHP if at all possible, but there is now a PHP port of py-amqplib called php-amqplib, and I offered to help out with it a bit. Maybe partially out of guilt for having written the mess of Python code it was based on :)

I thought it would be handy to work on it using my MacBook. OS X 10.5 (Leopard) has PHP 5.2.6 built in standard, but unfortunately it doesn't have the bcmath extension included, which php-amqplib makes use of. Turns out building the module wasn't that difficult. This page got me going - although building bcmath was much simpler. Since I had the Apple Developer Tools for 10.5 installed, it was just a matter of ...

  • mkdir /SourceCache
  • cd /SourceCache
  • fetch
  • tar xzvf php-5.2.6.tar.gz
  • cd php-5.2.6/ext/bcmath
  • phpize
  • ./configure
  • make
  • sudo make install
  • cd /etc
  • cp php.ini.default php.ini

And then edit /etc/php.ini to make these two changes:

--- php.ini.default     2008-07-15 14:19:15.000000000 -0500
+++ php.ini     2008-12-08 21:44:52.000000000 -0600
@@ -483,7 +483,7 @@
 user_dir =

 ; Directory in which the loadable extensions (modules) reside.
-extension_dir = "./"
+;extension_dir = "./"

 ; Whether or not to enable the dl() function.  The dl() function does NOT work
 ; properly in multithreaded servers, such as IIS or Zeus, and is automatically
@@ -595,6 +595,7 @@
 ; needs to go here.  Specify the location of the extension with the
 ; extension_dir directive above.

 ; Windows Extensions
 ; Note that ODBC support is built in, so no dll is needed for it.

After that, I was able to run the amqp_test.php file for the first time, sending a message and receiving it in py-amqplib's demo/

KVM Networking

Still playing with KVM (Kernel-based Virtual Machine), this time checking out some networking features. I've been running Ubuntu 8.04 LTS Server (Hardy Heron), both as the host and as a VM on that host. Networking is setup to use a bridge.

KVM offers different emulated NICs, I took a quick look at running iperf between the VM and the host, and got these speeds for a few select NIC models:

  • RTL-8139C+ (the default): ~210 Mb/sec
  • Intel e1000 (somewhat recommended here): ~ 330Mb/sec
  • virtio: ~ 700Mb/sec

The thing about virtio though is that it doesn't work when the VMs RAM is set to 4GB. So I guess you can have fast networking, or lots of memory, but not both.