Normally I try to avoid dealing with PHP if at all possible, but there is now a PHP port of py-amqplib called php-amqplib, and I offered to help out with it a bit. Maybe partially out of guilt for having written the mess of Python code it was based on :)

I thought it would be handy to work on it using my MacBook. OS X 10.5 (Leopard) has PHP 5.2.6 built in standard, but unfortunately it doesn't have the bcmath extension included, which php-amqplib makes use of. Turns out building the module wasn't that difficult. This page got me going - although building bcmath was much simpler. Since I had the Apple Developer Tools for 10.5 installed, it was just a matter of ...

• mkdir /SourceCache
• cd /SourceCache
• fetch https://www.php.net/get/php-5.2.6.tar.gz/from/this/mirror
• tar xzvf php-5.2.6.tar.gz
• cd php-5.2.6/ext/bcmath
• phpize
• ./configure
• make
• sudo make install
• cd /etc
• cp php.ini.default php.ini

And then edit /etc/php.ini to make these two changes:

--- php.ini.default     2008-07-15 14:19:15.000000000 -0500
+++ php.ini     2008-12-08 21:44:52.000000000 -0600
@@ -483,7 +483,7 @@
 user_dir =

 ; Directory in which the loadable extensions (modules) reside.
-extension_dir = "./"
+;extension_dir = "./"

 ; Whether or not to enable the dl() function.  The dl() function does NOT work
 ; properly in multithreaded servers, such as IIS or Zeus, and is automatically
@@ -595,6 +595,7 @@
 ; needs to go here.  Specify the location of the extension with the
 ; extension_dir directive above.

+extension=bcmath.so

 ; Windows Extensions
 ; Note that ODBC support is built in, so no dll is needed for it.

After that, I was able to run the amqp_test.php file for the first time, sending a message and receiving it in py-amqplib's demo/demo_receive.py

Still playing with KVM (Kernel-based Virtual Machine), this time checking out some networking features. I've been running Ubuntu 8.04 LTS Server (Hardy Heron), both as the host and as a VM on that host. Networking is setup to use a bridge.

KVM offers different emulated NICs, I took a quick look at running iperf between the VM and the host, and got these speeds for a few select NIC models:

• RTL-8139C+ (the default): ~210 Mb/sec
• Intel e1000 (somewhat recommended here): ~ 330Mb/sec
• virtio: ~ 700Mb/sec

The thing about virtio though is that it doesn't work when the VMs RAM is set to 4GB. So I guess you can have fast networking, or lots of memory, but not both.

I'm still experimenting with Ubuntu 8.04 Server (Hardy Heron), and have switched from Xen to KVM (Kernel-based Virtual Machine). Xen worked well on a little test machine I had, but when I tried it on a brand-new Supermicro server, it turned out to have a problem with the Intel NIC. Since it seems Ubuntu is recommending KVM over Xen, and the server supports hardware virtualization, I figured I'd give it a try.

One big difference is that KVM does full emulation, which means any disk space you give it from LVM (Logical Volume Manager), will be a full virtual disk, with a partition table. It's a little more complicated to access filesystems within the virtual disk that it was with Xen, I wanted to jot some notes down here mostly for myself on how to do that.

If I've created a logical volume named /dev/myvg/test_vm and installed another linux on it with a single ext3 filesystem (/dev/sda1 from the point of view of the VM) and some swap space (/dev/sda5), it can be accessed when the VM isn't running with the help of the kpartx utility...

kpartx -av /dev/myvg/test_vm

would read the partition table on the virtual disk and create:

/dev/mapper/myvg-test_vm1 
/dev/mapper/myvg-test_vm2 
/dev/mapper/myvg-test_vm5

Then you can

mount /dev/mapper/myvg-test_vm1 /mnt

to mess with the VMs /dev/sda1. To clean things up when finished, run:

umount /mnt
kpartx -d /dev/myvg/test_vm

Snapshots

If you want to look at the contents of a running VM's disks (perhaps for backing it up) you can use LVM snapshots. For example:

lvcreate --snapshot --size 1G --name test_snap /dev/myvg/test_vm
kpartx -av /dev/myvg/test_snap
mount /dev/mapper/myvg-test_snap1 /mnt
   .
   (play with VM's /dev/sda1 in /mnt)
   .
umount /mnt
kpartx -dv /dev/myvg/test_snap
lvremove /dev/myvg/test_snap

I've been using a Macbook for a couple years now, and really love the thing - but it's had an annoying sleep disorder that I finally found a workaround for.

I rarely shut it off, usually just closing the lid when I'm not using it, so it goes to sleep. The problem has been waking up back up - often I'd open the lid and start to enter the password to unlock the system, and the screen would go black and the machine would act sort of half-asleep. It almost seemed like it was confused as to whether the lid was open or not - maybe it was something with the lid sensor (which some other discussions online hinted at).

It turns out there's a command-line utility to change some power settings, and using it to set the machine not to wake up when the lid opens seems to help. The command is:

sudo pmset -a lidwake 0

The machine still goes to sleep when the lid closes, but now after opening the lid it waits until I hit a key on the keyboard before it rouses itself. After 3 or 4 days of this, it hasn't acted up yet, so I think this is a good fix.

This is mostly a note to myself... After setting up a minimal Ubuntu server install (in Xen), following these instructions using debootstrap I saw lots of errors like this:

perl: warning: Setting locale failed.
perl: warning: Please check that your locale settings:
        LANGUAGE = (unset),
        LC_ALL = (unset),
        LANG = "en_US.UTF-8"
    are supported and installed on your system.
perl: warning: Falling back to the standard locale ("C").

Checking with locale -a would show

C
POSIX

While a full Ubuntu server install (off a CD) would show:

C
en_US.utf8
POSIX

This command seems to have generated the missing locale and made everybody happy.

localedef --no-archive -i en_US -c -f UTF-8 en_US.UTF-8

I've been experimenting with setting up Ubuntu Server 8.04 (Hardy Heron) to run Xen, and had a minor problem with UFW (Uncomplicated Firewall) running in the dom0 blocking network access to a domU running in bridged mode. It seems the fix is just to edit /etc/defaults/ufw and make this change to enable forwarding:

--- a/default/ufw       Thu Oct 23 10:00:33 2008 -0500
+++ b/default/ufw       Thu Oct 23 10:34:36 2008 -0500
@@ -16,7 +16,7 @@ DEFAULT_OUTPUT_POLICY="ACCEPT"

 # set the default forward policy to ACCEPT or DROP.  Please note that if you
 # change this you will most likely want to adjust your rules
-DEFAULT_FORWARD_POLICY="DROP"
+DEFAULT_FORWARD_POLICY="ACCEPT"

 #
 # IPT backend

and then run ufw disable; ufw enable.

I believe dom0 is now protected, and it'll be up the the domU to protect itself. I can't say I'm entirely comfortable with Linux IPTables, sure wish PF was available as an alternative.

For many years I've been using bpgsql, my own pure-Python PostgreSQL client, and I've finally sat down and got things somewhat polished up enough to put together as a real package.

One thing that motivated the work was the desire to use in with Django - after seeing psycopg2 do some funny things when used under mod_wsgi. There's no doubt it's slower, but it's much easier to hack on, and might be of interest to people running Djano under other Pythons such as PyPy or Jython. Getting it to pass all the Django unittests really ironed out a lot of bugs, so I think it's in fairly decent shape now.

I was happy to see a FreeBSD port added for RabbitMQ, net/rabbitmq, although I found a couple problems with it: it doesn't start automatically when your machine or jail boots, and when building the rabbitmq-erlang-client, it errors out with:

src/amqp_channel.erl:28: can't find include lib "rabbitmq_server/include/rabbit.hrl"
src/amqp_channel.erl:29: can't find include lib "rabbitmq_server/include/rabbit_framing.hrl"

I worked on the port a bit, and submitted a bug report and patch, ports/127033, that fixes these problems.

Put out a new release of py-amqplib, labeled 0.5, featuring the reworking mentioned earlier of how frames from the server are handled, and a big speed-improvement in receiving messages that was prompted by doing some profiling after reading Initial Queuing Experiments on the Second p0st blog.

I've been reading Joe Armstrong's Programming Erlang book, and it's been a real eye-opener. I've been intrigued by functional programming languages like Haskell and OCaml for some time, but it's been hard seeing how they'd be used for real programs, instead of just silly little factorial or quicksort examples. Joe's book is awfully well written and gives lot of very clear examples, so now when I look at bits of code like RabbitMQ, it doesn't seem like a bunch of gibberish :)

Haven't finished the book yet, but one thing I picked up from it already was the interesting way Erlang handles messages sent to processes, in how they're pattern-matched and saved for later if they're not what the process is looking for right at that moment.

At the heart of amqplib versions 0.3 or lower is a terrible mess that tries to deal with waiting for particular AMQP frames. Previously it would raise exceptions in some situations when it really shouldn't have. Specifically: if you had called basic_consume on a channel, and then called some other synchronous method like another basic_consume call - while it was expecting a basic_consume_ok response a basic_deliver could arrive and the library would raise an Exception because that wasn't expected.

A lame workaround is to use the nowait option most calls have, but I've now reworked things, cleaning up a lot of ugly code, and saving unexpected messages for later, similar to how Erlang does it. So now I believe the client library behaves in the way you'd generally expect. The improved code is currently in the Mercurial repository, and will be put out as a new release after it's had a chance to settle.