Fun with ones and zeros

In experimenting with jails, I've had a need to put them on machines in which I didn't have extra public IP addresses to assign to the NIC. Turns out you can easily assign private addresses to an interface, and setup NAT (Network Address Translation) to allow the jails access to the rest of the world.

The loopback interface lo0 seems to work pretty well for this. On one machine I put ezjail on, I just picked the IP block 10.51.50.x out of my hat, and added an alias address on-the-fly with this command:

ifconfig lo0 alias 10.51.50.1 netmask 255.255.255.255

To make it happen at boot time, add this to /etc/rc.conf:

ifconfig_lo0_alias0="inet 10.51.50.1 netmast 255.255.255.255"

To setup FreeBSD's PF to NAT to the 10.51.50.x block, this went into /etc/pf.conf, after any scrub directives but before any block/pass type rules:

nat on $ext_if from 10.51.50.0/24 to any -> $ext_if

Reload the PF configuration with:

pfctl -f /etc/pf.conf

On another machine, I did mostly the same setup, except for using 127.x.x.x numbers. Not sure if there's any advantage one way or the other, both machines seemed to work pretty much the same.

posted at: 16:52 | tags: freebsd | Comments | permanent link to this entry

Virtualization is something I've been interested in for some time, dabbling with VMWare on Windows, and eagerly awaiting Xen+BSD and AMD's Pacifica-enabled chips. FreeBSD's jail feature gives many of the same benefits but with relatively little overhead, as long as you're interested in working with the same version of FreeBSD in your "virtual" system as your "host" is running. Jails are a great way to isolate software - for security reasons, to run different versions of the same package, or just to allow yourself a sandbox to mess with that you can easily wipe out and recreate in a few seconds.

The man page for jail describes how to setup a jail by hand, which seems a bit involved. Luckily I stumbled across ezjail, which makes creating jails a breeze. Once it's setup, you can create and "boot" a fully functioning jail with just three commands. ezjail arranges things so most of the FreeBSD userland is shared between the jails, and the files unique to each jail take up as little as 2mb.

The initial setup is basically:

1. install the port in sysutils/ezjail
2. Add ezjail_enable="YES" to /etc/rc.conf
3. edit /usr/local/etc/ezjail.conf to set where you want your jails created. (In my case I used /data/jails)
4. make sure your /usr/src tree is complete
5. run ezjail-admin update

That last command can take a lot of time (maybe hours), since it does a full make buildworld, make installworld. If you've already built your world, there's a -i parameter for skipping that step and just doing the make installworld.

Once that's all done, in your jail directory there is a basejail which contains about 130+mb of files that will be shared between jails, newjail which is a skeleton containing about 2mb of files that gets copied to any new jails you create, and flavours which is basically another set of skeleton directories that get copied over the newjail skeleton when your jail is created.

At this point, you can create and boot a jail with:

1. ifconfig lo0 alias 127.66.0.1 netmask 255.255.255.255 or similar to give one of your network interfaces an IP the jail can use.
2. ezjail create myjail 127.66.0.1 creates a new directory (/data/jails/myjail in my case) that's a copy of newjail and sets a few other things up.
3. /usr/local/etc/rc.d/ezjail.sh start myjail

At this point the jail is up and running. You can "log into" it by first finding out the integer id of the jail with jls, and then running jexec <jail-id> /bin/sh

There are a few things that are missing in this barebones install, mainly no /etc/resolv.conf so domain name lookups don't work, no /etc/localtime so time in the jail shows as UTC. You can fix these problems and add your own customizations easily by using a flavour (don't mess with the newjail template directory).

You can stop and wipe out your jail with

/usr/local/etc/rc.d/ezjail.sh stop myjail
ezjail-admin delete -w myjail

Then, to make a new flavour and make a jail using that flavour, something like

cd /data/jails/flavours
cp -pr default myflavour
cd myflavour/etc
cp -p /etc/resolv.conf .
cp -p /etc/localtime .
ezjail-admin create -f myflavour myjail 127.66.0.1
/usr/local/etc/rc.d/ezjail.sh start myjail

At this point, you've created a new jail with your customizations, and would use jls again to find the jail-id, and jexec to start a shell inside the running jail.

A flavour may also contain packages you wish to install upon jail creation, and commands to execute when the jail is created. Check out the ezjail.flavour file in your flavour directory. I've used it to install common useful things like bash, vim, gmake, and libiconv and gettext which take a long time to build that you don't want to repeat for every jail.

posted at: 21:30 | tags: freebsd | Comments | permanent link to this entry

This is one of those little things that I just want to jot down for myself so I have it written down until I learn it for good. To view a man file, that's not installed in the regular man file locations, just run

nroff -man filename | more

Stupidly simple, but unfortunately not mentioned in the manpage for man.

posted at: 23:49 | tags: unix | Comments | permanent link to this entry

Got back from PyCon 2006, in mostly one piece. Picked up a terrible cold at the conference, I suppose scrounging food off the same buffet tables as 400 other people wasn't the most hygenic thing in the world.

Attended the mainly web-oriented sessions, came away very impressed with Django. I had sort of blown it off before because I didn't like the look of the templating language, and the ORM seemed weird. But after seeing what's coming in the Removing the Magic branch, I think it will be much much nicer. Was even inspired to spend the little time I had there Monday morning and afternoon sprinting with the Django guys, but I don't see how one can sprint effectively in such a short time with the limited knowledge of the codebase I had. Maybe if I go next year ... and I know more Django ... and can spend more than a day there, then I could accomplish something useful during the time.

The TurboGears guys demonstrated some nice things with AJAX widgets, but the SQLObject part of TG has given me trouble in the past when working with an existing DB, and seems to get in the way more than it helps. Even so, the TG guys, and Ian Bicking seemed pretty cool, so I hope they polish things up a bit more. Maybe SQLObject 2 will be the answer, or maybe a switch to SQLAlchemy (which wasn't represented at the conference), would make TG a nicer environment to work in.

I've been struggling with Zope for some years now. From a user standpoint I guess it's OK, from a programmer standpoint it's a nightmare, both 2.x and 3.x. The documentation and community attitude have rubbed me wrong for a long time. I attended a couple Zope sessions at the conference, but didn't hear anything to inspire me to keep up with it. I'll probably switch what little Zope things I have going to Django/TurboGears/CherryPy/whatever.

The PyParsing presentation on writing an adventure game was interesting, wish I could have attended the more in-depth one but it conflicted with a Django session. PyParsing looks to make a hard job pretty easy, and I'd love to play with it somewhere.

The party at NerdBooks had some decent food, they had a pretty deep selection of books, and the prices on some of the things I looked up were much better than Amazon. Will definitely look there next time I need something.

Lastly, I hope Django or someone who was at the sprint uses the codename "Vacuum Assassin" somewhere. That would just be too cool.

posted at: 14:06 | tags: django pycon python | Comments | permanent link to this entry

Quite often I find myself needing to generate self-signed certificates for use with OpenSSL. There are only three steps required...

Generate a key file, named ssl.key for example:

openssl genrsa -out ssl.key 1024

Generate a Certificate Signing Request for the key, named ssl.csr in this example. You'll be asked a bunch of questions, when asked for Common Name (eg, YOUR name) be sure to enter the domain-name you're making the certificate for (such as www.foobar.edu).

openssl req -new -key ssl.key -out ssl.csr

Generate a signed certificate given the request and key, valid for 10 years (3650 days) and named ssl.crt in this example. When you're done, the ssl.key and ssl.crt files are what you usually need to install in your server.

openssl x509 -req -days 3650 -in ssl.csr -signkey ssl.key -out ssl.crt

As a bonus, here's how to view the contents of a certificate file named ssl.crt

openssl x509 -in ssl.crt -text

posted at: 17:07 | tags: openssl | Comments | permanent link to this entry

Just as a followup, it seems the segfault in mod_python on FreeBSD I mentioned before was found and fixed. Turns out to not be any kind of pointer/memory corruption like I thought, but rather a mishandled return code from an APR (Apache Portable Runtime) function. Oh well, I got to play with gdb, ddd, and valgrind a bit, which is good stuff to be familiar with.

posted at: 21:40 | tags: apache bugs freebsd mod_python | Comments | permanent link to this entry

At work the other day, we had a long power outage, and afterwards one of our FreeBSD 5.2.1 boxes refused to come back up. It'd power up, go through the BIOS stuff, show the FreeBSD boot manager that lets you select which slice to boot, but when you hit F1, the screen would go black and the machine would reset.

Booted off the 5.2.1 install CD, and after entering fixit mode, was able to mount the disk and see that the files seemed to be intact. Couldn't run fsck though, the 5.2.1 CD seemed to be missing fsck_4.2bsd.

FreeSBIE 1.1 on the other hand, was able to fsck the disk, but that didn't solve the problem. Next guess was that something in the /boot directory was hosed. I'd setup the machine to do weekly dumps of the root partition to another machine, and was able to extract /boot from a few days before and pull it back onto this machine over the network using FreeSBIE, but it still wouldn't boot.

Next theory was that something in the boot sectors was bad. First tried restoring the MBR (Master Boot Record) from copy that's kept in /boot - even though it was working well enough to show the F1 prompt to select the slice. Wanted to keep what 5.2.1 had been using, so mounted the non-booting disk readonly and made sure to have boot0cfg use the copy there instead of anything that might have been on the FreeSBIE disc.

mkdir /foo
mount -r /dev/twed0s1a /foo
boot0cfg -B -b /foo/boot/boot0 /dev/twed0
reboot

Unfortunately, that didn't help. Each slice (partition in non-BSD terminology) also has boot sectors, and to restore them, turns out you use the bsdlabel (a.k.a. disklabel) utility. Again from FreeSBIE:

mkdir /foo
mount -r /dev/twed0s1a /foo
bsdlabel -B -b /foo/boot/boot /dev/twed0s1
reboot

That did it. Apparently something in the slice's boot sectors was messed up.

posted at: 18:34 | tags: freebsd unix | Comments | permanent link to this entry

Lately I've been using Firefox on DragonFlyBSD with xorg installed from pkgsrc, and one thing that bugged me was that when reading Advogato, the fonts on that page looked like crap. The CSS stylesheet shows "lucida" as the preferred font, and my machine evidently was using a bitmap font for that.

At first I thought, just get rid of the bitmapped fonts from the FontPaths listed in /etc/X11/xorg.conf, but surprisingly that didn't seem to have any effect, at least on Firefox.

Secondly, I tried just removing those bitmap font directories completely, such as /usr/pkg/xorg/lib/X11/fonts/75dpi/ and that did work, but seemed a little clumsy in that an update to xorg would probably replace them.

Finally, stumbled across Fontconfig's files, and saw that there is a whole separate configuration of font paths and such, starting in /usr/pkg/etc/fontconfig/, which explains why changing the xorg.conf FontPath didn't work. Turns out there are even some optional configs in /usr/pkg/etc/fontconfig/conf.d/ including a no-bitmaps.conf which will cause fontconfig to "blacklist" the bitmap fonts.

The Fontconfig user manual mentions that things in conf.d/ are processed if they begin with decimal digits. So to enable that no-bitmaps.conf, I just made a symlink.

cd /usr/pkg/etc/fontconfig/conf.d
ln -s no-bitmaps.conf 10barryp-no-bitmaps.conf

Then, just had to stop/restart Firefox to see the results.

It would be nice to be a bit more selective about what gets blacklisted, so that non-Roman characters not supported in the scalable fonts on my machine would have some chance of displaying. I'll have to work on that.

posted at: 13:57 | tags: unix | Comments | permanent link to this entry

Other people have reported the same problem with mod_python on FreeBSD I had seen before, so I'm happy that I'm not losing my mind.

I took a stab at using Valgrind to find the problem. Didn't actually find anything, but I thought I'd jot down notes on how I went about this.

First, the Valgrind port didn't seem to work on FreeBSD 6.0. When I tried running it against the sample code in the Valgrind Quick Start guide, it didn't find anything wrong with it. Ended up finding a FreeBSD 5.4 machine, which did see the expected problem.

Next, I built the Apache 2.0.x port with: make WITH_THREADS=1 WITH_DEBUG=1, and then built mod_python which uses APXS and picks up the debug compile option from that.

Then, in the mod_python distribution, went into the test directory, and downloaded a Valgrind suppression file for Python, valgrind-python.supp, and in it uncommented the suppressions for PyObject_Free and PyObject_Realloc (otherwise the Valgrind output is full of stuff that is really OK). Then tweaked test/test.py around line 307 where it starts Apache, to insert

valgrind --tool=memcheck --logfile=/tmp/valgrind_httpd --suppressions=valgrind-python.supp

At the front of the cmd variable that's being composed to execute httpd.

Finally, ran python test.py, and then looked at /tmp/valgrind_httpd.pid#### to see the results.

posted at: 22:17 | tags: bugs freebsd mod_python | Comments | permanent link to this entry

A while ago I threw together this script to automatically create package files for all installed ports on a FreeBSD box. That way, if a portupgrade doesn't work out, you can delete the broken package, and pkg_add the backup.

Stick this in /usr/local/etc/periodic/daily, and the system will automatically bundle up copies of the installed software and stick them in /usr/local/packages if they don't already exist in there.

#!/bin/sh
#
# Make sure backups exist of all installed FreeBSD packages
#
# 2005-03-20 Barry Pederson <bp@barryp.org>
#

ARCHIVE="/usr/local/packages"

#
# Figure out which pkg_tools binaries to use
#
if [ -f /usr/local/sbin/pkg_info ]
then
    PKG_TOOLS="/usr/local/sbin"
else
    PKG_TOOLS="/usr/sbin"
fi

#
# Make sure backup directory exists
#
if [ ! -d $ARCHIVE ]
then
    mkdir $ARCHIVE
fi

cd $ARCHIVE

for p in `${PKG_TOOLS}/pkg_info -E "*"`
do
    if [ ! -f ${p}.tgz ]
    then
        ${PKG_TOOLS}/pkg_create -b ${p}
    fi
done

posted at: 21:04 | tags: freebsd unix | Comments | permanent link to this entry