Sep 22

One of the traditional problems with mod_perl development is getting the server to recognize module changes, without restarting after every change. I’m thinking about an improved solution for this.

These are the solutions I know about:

1. Apache2::Reload / Module::Reload

These check whether modules have changed on each request, and if so, clear their symbols and reload them inside the process.

Problem: some modules fail to reload properly. Sometimes the failure is intermittent, depending on the order of module loading and other esoteric details. Moose and ORM modules seem particularly prone to reload failures. For me, this level of unpredictability makes *::Reload too frustrating to use.

2. Catalyst Auto-restart

Catalyst has an engine (Catalyst::Engine::HTTP::Prefork::Restarter) which forks off a “watcher” process that waits for your modules to change. When they change, it restarts the server. The usual effect is that, between the time you hit “save” in your editor and reload your page, the server has restarted or at least begun restarting.

Problems: Doesn’t work well if you make a few changes in a row; the restart only captures your first change. (EDIT: Dave Rolsky points out that this is fixed now under some OS’s.) Bad user experience if there’s an error in your module; you have to realize the server has died, find the error message in some shell or log, and manually start up the server again. Finally, you are still ultimately waiting for a full restart.

3. MaxRequestsPerChild=1

Perrin Harkins recently alerted me to the MaxRequestsPerChild=1 technique. That is, set MaxRequestsPerChild to 1, then load any potentially-changing modules in the child, not the parent (obviously only for development environments). Each request will hit a fresh child server, which will load all of your potentially-changing modules anew.

This is the nicest solution I’ve seen so far. The only problem I can see is its performance – each potentially-changing module has to be loaded on each request. **

4. My idea: The Pied Piper

As in 3, load any potentially-changing modules in the child. Leave MaxRequestsPerChild alone. As in 2, fork off a “watcher” process that waits for your modules to change. When they change, kill all the server’s children explicitly.

The end result is that you get reasonable performance when your modules don’t change (e.g. when you are only futzing with templates), but when modules do change, you should see the effects immediately.

This should be able to work with mod_perl, fastcgi, Net::Server, etc., as long as the parent server responds appropriately to the killing of all its children (by launching new ones). Apache, at least, seems to be ok with this.

Going to try to implement this as a plugin in Server::Control. Will see how it goes.


** – You can try to load things only on demand, but often mod_perl code is written without ‘use’ statements as it assumes everything is loaded in the parent. You can also try to minimize the number of potentially-changing modules, but then you run the risk of leaving something off and having to adjust it and restart.

6 Responses to “How not to restart mod_perl servers”

  1. Dave Rolsky Says:

    You’re somewhat wrong about #2. If you are using a smart events-based notification Watcher with Catalyst, it sees every change you make. If you make a change before the server has restarted, it just kills it and starts over.

    Right now this only works on Linux with Linux::Inotify2 installed. However, I know there’s work being done on a Mac fs events module for File::ChangeNotify. Once that’s on CPAN, you can just install it and get the same effect.

    Other platforms like BSD and Solaris have their own FS events systems that could be hooked into as well.

  2. Jonathan Swartz Says:

    Dave Rolsky: Ok, thanks for the update. Does the built-in Catalyst watcher use File::ChangeNotify, or are you talking about a separate third-party watcher?

  3. originalgeek Says:

    #3 isn’t really a good idea, will cause your server to die from forkdeath unless if it receives more than moderate use.

  4. Jonathan Swartz Says:

    originalgeek: You realize this is strictly for personal development use? None of this is meant for live production traffic.

  5. Gracie Says:

    You’re somewhat wrong about #2. If you are using a smart events-based notification Watcher with Catalyst, it sees every change you make. If you make a change before the server has restarted, it just kills it and starts over.

    Right now this only works on Linux with Linux::Inotify2 installed. However, I know there’s work being done on a Mac fs events module for File::ChangeNotify. Once that’s on CPAN, you can just install it and get the same effect.

    Other platforms like BSD and Solaris have their own FS events systems that could be hooked into as well.

  6. Randolf Richardson Says:

    I read about another solution once, but I don’t recall if it’s part of the standard set of modules. It involved having a “touch file” that would trigger a “Reload” operation whenever the timestamp on the “touch file” changed (any timestamp compares as different from a non-existing file as I recall).

    This seems, to me, a reasonable solution even for a production environment.

    Thanks for writing this interesting article.

Leave a Reply

preload preload preload