« Fighting Hotlinkers with htaccess | Main | How to Customize Default Styles in MT4 »

Running Publish Queue under daemontools

In Movable Type 4.0 users had the option of offloading the task of publishing to a separate program or application called Publish Queue. This had the advantage of dramatically increasing Movable Type's performance and reliability.

Most users run Pubish Queue via a scheduled task that wakes up at a fixed internal, executes and then quits. However, some users use Publish Queue's "daemon mode" to make it so that it is always running. But its own daemon mode does not have the capability to monitor and restart itself should it unexpectedly quit, exit or die.

Should Publish Queue crash the consequence would be that publishing would simply stop without you ever really knowing about it. Plus you couldn't start it up again without you manually going into the server and starting the publish queue up again. This rarely happens in actuality, but in a business critical application, you never want this to happen. Period.

Of course this problem is not unique to Movable Type. Many programs in Linux should be running at all times to ensure that the operating system functions properly. That is why there exists a suite of utilities in Linux called "damon tools." Together these tools can be configured to monitor any script and ensure that it is always running. If the script/application dies, then daemon tools will restart it. Handy.

This article discussed the process for those unfamiliar with daemon tools with how to get started and how to configure it so that you can relax knowing that Movable Type's background publishing engine will never stop running.

Ok, before we get started I have to admit something: this article is a little technical. cough. It requires an above average knowledge of Linux and requires people to interact with a command line that many are not comfortable doing. If you are one of those people then your key take away from this article is the knowledge that a solution exists to this problem. Your job will then be to send a link to this article to your system administrator or a more technical friend or yours and tell them, "help me do this."

For all the technical folks who are sticking around, this article will show you how to start up Publish Queue and have it monitored by daemon tools so that it will automatically be restarted (or "respawned" to use the proper term) if the process for whatever reason dies.

Prerequisites

You will need each of the following:

  • Movable Type 4.0 or greater
  • daemon tools, a Linux package

The simplest way to install daemon tools is via yum or apt. For example, running the following command as root should do the trick:

yum install daemontools

If this doesn't work or is not appropriate for you Linux distribution, then I must refer you to the daemon tools website to follow their more specific instructions.

Setting up the Publish Queue service

Once daemon tools is installed you need to create a script that will be run anytime the system wants to start or respawn Publish Queue. This is done by creating a few directories and files on your system. Execute the following command:

mkdir /etc/publishqueue

Then create a file called /etc/publishqueue/run and enter the following contents into the file:

#!/bin/sh
MT_HOME=/var/www/cgi-bin/mt
cd $MT_HOME
exec ./tools/run-periodic-tasks -daemon

Change the value of MT_HOME to be the location of the directory that contains mt.cgi.

Finally, execute the following command:

ln -s /etc/publishqueue /service/publishqueue

That will setup daemon tools with the correct scripts and handlers to run Publish Queue.

Setting up svscan

A program comes with daemon tools called svscan. It is responsible for monitoring a list of services and if any of them stop running to restart them. In lieu of my documenting the process, let me refer you to a more complete and definitive article that can guide you through this process. There are lots of ways to setup svscan, but I chose to use inittab as the means by which my system will monitor and respawn processes.

Once you have successfully edited /etc/inittab you can execute the following command to complete the entire daemonization process:

telinit q

This command will instruct your system to reprocess /etc/inittab and your publish queue daemon should then start momentarily. Run the following command and see if you can see publish queue running:

ps -aef

You should see something like:

root 30663     1  0 Jan21 ? /bin/sh /usr/local/bin/svscan-start
root 30666 30663  0 Jan21 ? /bin/sh /usr/local/bin/svscan-start
root 30667 30666  0 Jan21 ? svscan /service
root 30669 30667  0 Jan21 ? supervise publishqueue
root 26564 30669  0 04:00 ? /usr/bin/perl -w run-periodic-tasks -daemon

Summary

And that as they say, is that. I know these instructions can be an intimidating, and hopefully in the future we can find ways to automate this process. In the meantime, I would really like to hear how others have solved this problem.

Also, if you have any questions or problems following the steps above, please leave a comment and I will see if I can't help you resolve your problem.

Comments (1)

At least for RHEL5 there's no rpm available from RedHat or RPMForge, so you do have to get the source and compile it.

The link given for Daemon Tools (The package for monitoring daemons on Unix like systems) is more appropriately: http://cr.yp.to/daemontools.html and for installation instructions: http://cr.yp.to/daemontools/install.html

Post a comment

(If you haven't left a comment here before, your comment may need to be approved before will appear on the entry. Thanks for waiting.)