Fun with ones and zeros - daemontools

Daemontools mishap

2008-02-11T14:01:00-08:00

While working on my DiskCompare.com website I had a weird problem with daemontools. The Django process I had running under daemontools became unresponsive, it wouldn't shutdown normally with svc -d, and after kill -9 and restarting, it still wouldn't respond - it just seemed hung up or frozen. No messages in /var/log/messages, but running the service manually outside of daemontools worked fine. What the heck?

I had the service also setup to run multilog, and it turns out that I had mistakenly changed the ownership of the log directory it was supposed to be writing to. My guess is that it was the multilog process that was really hung up, since it couldn't write to the files it wanted to, and that my Django processes were then blocked because the pipe between them and multilog was full.

Simply correcting the permissions of the log directory cleared the jam and things took off again. So if you're seeing strange behavior with daemontools, that's one area to check out.

Doing things the DJB way

2006-01-16T12:11:00-08:00

While doing a bit more searching for daemontools info, found the djb way website, which has some nice writeups on daemontools and djbdns (which I also use a fair amount).

Running a SCGI server under daemontools

2006-01-16T09:55:00-08:00

Yesterday, I was working on Running PyBlosxom through SCGI, but during that time, I was running the SCGI server by hand in a console window. Once it was working I needed to arrange a way to run this in a more permanent fashion. Daemontools seems like an easy way to set this up, I already had it running on my server.

Daemontools runs a process called svscan that looks for directories in /var/service (the default when installed through the FreeBSD port) that contain an executable named run. If svscan also finds a log/run executable in that directory, it starts that too and ties the two together with a pipe. Daemontools includes a multilog program that reads from the pipe (stdin), and writes out and rotates log file for you automatically.

To get PyBlosxom/SCGI running under this, started by making a temporary directory, and copying in the three files needed to run PyBlosxom through SCGI

mkdir /tmp/pyblosxom
cd /tmp/pyblosxom

cp ~/config.py .
cp ~/wsgi_app.py .
cp ~/scgi_server.py .

(The first two files come from the PyBlosxom distribtution (the first one is customized). The third file is the one I came up with yesterday)

Next, I came up with a run script to execute the SCGI server under the www userid, with stderr tied to stdout. Daemontools has a setuidgid program that makes this pretty easy

#!/bin/sh
exec 2>&1
exec setuidgid www ./scgi_server.py

Next, made a log subdirectory, a log/main subdirectory to hold the actual log files (owned by www).

mkdir log
mkdir log/main
chown www:www log/main

And in the log directory put another tiny run script.

#!/bin/sh
exec setuidgid www multilog t ./main

Finally, made both run scripts executable, and moved the whole thing into /var/service

chmod +x run
chmod +x log/run
cd ..
mv pyblosxom /var/service

svscan sees the new directories within a few seconds, starts up both run scripts automatically, and you're in business. See the current contents of the log with:

cat /var/service/pyblosxom/log/main/current

Stop and restart the server with the Daemontools svc utility:

cd /var/service
svc -d pyblosxom ; svc -u pyblosxom