I attended OSCON for the first time this year, and to celebrate I thought I'd wrap up the Python amqplib library a bit and consider it more-or-less finished for what it is (a simple blocking 0-8 client), and call it 1.0.0 You can find it on the in PyPi and Google Project Hosting
It's definitely a worthwhile upgrade in that it's significantly faster than amqplib 0.6.1, and has a fair number of bug fixes. Also noteworthy are support for Python 3.x (via 2to3) and IPv6
smbpasswd 1.0.2 submitted to PyPi
smbpasswd is a really old piece of software (9 years!) for generating NT/LM password hashes, suitable for use with Samba. It's in Debian/Ubuntu/Redhat repositories, and FreeBSD ports, and who knows where else.
Somehow it never got submitted to PyPi, but I took care of that today at the request of someone working on another Python module that wanted to use this as a dependency. Look for smbpasswd-1.0.2, or just easy_install smbpasswd if you're setup for that.
I changed the packaging slightly, so that the tarball extracts to smbpasswd-x.x.x instead of py-smbpasswd-x.x.x, and so bumped the version number to 1.0.2 just for the packaging changes. The library itself is unchanged.
However, I think you'd want to be very careful generating and storing LM hashes of user's passwords, they seem to be wildly insecure.
If your app can get by with just NT hashes, and you have a Python >= 2.5, you may be able to generate those using the standard Python library, and don't need this package at all. See the notes on my py-md4 page.
Rebuilding Hyper-V Linux Integration Components for a kernel upgrade
At work I've been running RedHat Enterprise Linux (RHEL) 5.6 on top of a Windows Server 2008 R2 Hyper-V host, with Linux Integration Components (LinuxIC) installed.
That all worked fine until I did a yum update on RHEL to pick up a new kernel and tried to reboot. The new kernel panicked, saying it couldn't find my LVM volume groups. Fortunately, the old kernel was still on the menu and booted OK. Turns out the new kernel didn't for whatever reason wouldn't see the virtual disks for that machine.
OK, so I needed to rebuild LinuxIC for the new kernel while running the old kernel, how to do that? The Makefile and various scripts that come with LinuxIC basically builds for and installs on the currently running kernel. Fortunately I came across this post showing a trick of replacing /bin/uname with a fake version that shows the kernel version number you want to build for. Tried that and was back in business.
I think this would work too, without messing with the original /bin/uname: create a directory somewhere, say '/tmp/fake_uname' and stick this file in it with the name uname (changing the "echo" line with the installed kernel version number you want to build for.
#/bin/sh case$1 in
*)exec /bin/uname $1
Then build and install your Linux IC with the /tmp/fake_uname prepended to your PATH as in
PATH=/tmp/fake_uname:$PATH make install
When the LinuxIC build calls uname it finds the fake version first, and if the argument is -r shows you desired version number, otherwise falls back to the real uname.
Simple debugging output in C
I don't do a whole lot of C programming, but when I do it tends to be in difficult environments like Apache modules or Samba VFS modules, where you can't just do simple printfs to get some output from your program.
I've come up with this small chunk of code I can plop in a C file to allow for optionally writing out useful information to a file somewhere on the disk.
Within your program, you'd just sprinkle in calls to debug_log() with a format string and optional arguments, such as:
x=5;y=10;debug_log("Currently, x=%d, y=%d",x,y);
The code can then be enabled and configured to output to /tmp/foo.log (for example), by adding either
#define DEBUG_FILENAME /tmp/foo.log
to the top of your source file, or even more slickly for some things, from the commandline with
cc -DDEBUG_FILENAME=/tmp/foo.log myprogram.c
When the program is run, in your /tmp/foo.log file you'd find something like:
2011-04-03 20:30:05 Currently, x=5, y=10
If you don't define DEBUG_FILENAME, the code basically goes away, shouldn't take up any space in your binary at all.
Flash playback on MacOSX Firefox
For a long time I've been annoyed by really jerky playback on Flash videos under Firefox on MacOSX. This YouTube video for example, was just awful to watch, stuttering very frequently.
Turns out the fix is pretty simple: just go into about:config and increase the browser.sessionstore.interval setting in Firefox from the default of 10000 (10 seconds) to something larger like 120000 (120 seconds).
Got it from this page, even though it's talking about Ubuntu Firefox, it still applies to MacOSX and seems to have made a world of difference.
amqplib and bpgsql on Google Code
I've created Google Code projects for my amqplib and bpgsql packages, to take advantage of their nice infrastructure including issue tracking.
Wrapped up another release of py-amqplib, version 0.6 - which features a major reorganization of the codebase to make the library more maintainable and lays the groundwork for an optional thread-assisted mode that allows for flow control and timeouts (being worked on in a development repository).
Building additional PHP modules on OSX
Normally I try to avoid dealing with PHP if at all possible, but there is now a PHP port of py-amqplib called php-amqplib, and I offered to help out with it a bit. Maybe partially out of guilt for having written the mess of Python code it was based on :)
I thought it would be handy to work on it using my MacBook. OS X 10.5 (Leopard) has PHP 5.2.6 built in standard, but unfortunately it doesn't have the bcmath extension included, which php-amqplib makes use of. Turns out building the module wasn't that difficult. This page got me going - although building bcmath was much simpler. Since I had the Apple Developer Tools for 10.5 installed, it was just a matter of ...
And then edit /etc/php.ini to make these two changes:
--- php.ini.default 2008-07-15 14:19:15.000000000 -0500+++ php.ini 2008-12-08 21:44:52.000000000 -0600@@ -483,7 +483,7 @@
; Directory in which the loadable extensions (modules) reside.
-extension_dir = "./"+;extension_dir = "./"
; Whether or not to enable the dl() function. The dl() function does NOT work
; properly in multithreaded servers, such as IIS or Zeus, and is automatically
@@ -595,6 +595,7 @@
; needs to go here. Specify the location of the extension with the
; extension_dir directive above.
; Windows Extensions
; Note that ODBC support is built in, so no dll is needed for it.
After that, I was able to run the amqp_test.php file for the first time, sending a message and receiving it in py-amqplib's demo/demo_receive.py
Still playing with KVM (Kernel-based Virtual Machine), this time checking out some networking features. I've been running Ubuntu 8.04 LTS Server (Hardy Heron), both as the host and as a VM on that host. Networking is setup to use a bridge.
KVM offers different emulated NICs, I took a quick look at running iperf between the VM and the host, and got these speeds for a few select NIC models:
The thing about virtio though is that it doesn't work when the VMs RAM is set to 4GB. So I guess you can have fast networking, or lots of memory, but not both.
Playing with KVM and LVM on Linux
I'm still experimenting with Ubuntu 8.04 Server (Hardy Heron), and have switched from Xen to KVM (Kernel-based Virtual Machine). Xen worked well on a little test machine I had, but when I tried it on a brand-new Supermicro server, it turned out to have a problem with the Intel NIC. Since it seems Ubuntu is recommending KVM over Xen, and the server supports hardware virtualization, I figured I'd give it a try.
One big difference is that KVM does full emulation, which means any disk space you give it from LVM (Logical Volume Manager), will be a full virtual disk, with a partition table. It's a little more complicated to access filesystems within the virtual disk that it was with Xen, I wanted to jot some notes down here mostly for myself on how to do that.
If I've created a logical volume named /dev/myvg/test_vm and installed another linux on it with a single ext3 filesystem (/dev/sda1 from the point of view of the VM) and some swap space (/dev/sda5), it can be accessed when the VM isn't running with the help of the kpartx utility...
kpartx -av /dev/myvg/test_vm
would read the partition table on the virtual disk and create: