Aug 21

I’m an avid user of code tidiers and validators to enforce quality and consistent style, both in personal and team projects.

In the Perl world you are probably familiar with perltidy, perlcritic and podtidy, but every language has its own tools: htmltidy for HTML, jslint and jshint for Javascript, csstidy for CSS, etc.

In a web site project I might work with half a dozen of these tools, each with their own syntax and applicable only to certain files. I want to apply some of them while editing, some when I commit, and some only when I run tests. There must be a better way!

Enter tidyall

tidyall is a unifier for code tidiers and validators. You can run it on a single file or an entire project hierarchy, and configure which tidiers/validators are applied to which files. Features include:

  • A cache to only process files that have changed

  • A standard backup mechanism with auto-pruning

  • A plugin API that makes it trivial to add new tidiers, validators, and pre/post processors

  • Support for multiple modes (e.g. editor, commit, test, dzil), with different plugins running in each mode

Configuration

To use tidyall in a project, simply put a tidyall.ini file at the top of it. Here’s the tidyall.ini that I’m using for CHI:

[PerlTidy]
argv = -noll -blbp=0
select = {bin,lib,t}/**/*.{pl,pm,t}

[PodTidy]
select = {bin,lib}/**/*.{pl,pm,pod}

[PerlCritic]
select = {bin,lib}/**/*.{pl,pm}
argv = --profile $ROOT/perlcriticrc
except_modes = editor

[Perl::IgnoreMethodSignaturesSimple]
select = {bin,lib,t}/**/*.{pl,pm,t}

[Perl::AlignMooseAttributes]
select = {bin,lib,t}/**/*.{pl,pm,t}

These sections, in order, do the following:

  • Apply perltidy with settings “-noll -blbp=0″ to *.pl, *.pm, and *.t files.

  • Apply podtidy with default settings to *.pl, *.pm and *.pod files.

  • Apply perlcritic using the perlcriticrc in the same directory to *.pl and *.pm files; but skip this when invoking tidyall from an editor.

  • Use a preprocessor/postprocessor to hide Method::Signatures::Simple keywords (method and function) from perltidy and perlcritic.

  • Use a preprocessor/postprocessor to hide Moose attributes from perltidy, then sort and align attributes in a way I prefer.

Ways of using tidyall

Here are a variety of modes you might use tidyall in.

In your code editor

I like having a single keystroke (ctrl-t) to process the file I’m working on. The distribution contains an Emacs implementation of this command. Its effects are fully undoable and it reports any errors in a separate window.

This is the only editor I know how to program, so others will have to be contributed. :)

From the command line

Of course tidyall can be run manually, against a specific file:

% tidyall file [file...]

or against all the files in the project (skipping those that haven’t changed):

% tidyall -a

or against all the files you’ve added or modified according to svn:

% tidyall --svn

In svn and git commit hooks

The distribution includes an SVN precommit hook that checks if all files are tidied and valid according to tidyall, and rejects the commit if not. e.g.

% svn commit -m "fixups" CHI.pm CHI/Driver.pm 
Sending        CHI/Driver.pm
Sending        CHI.pm
Transmitting file data ..svn: Commit failed (details follow):
svn: Commit blocked by pre-commit hook (exit code 255) with output:
2 files did not pass tidyall check
lib/CHI.pm: *** 'PerlTidy': needs tidying
lib/CHI/Driver.pm: *** 'PerlCritic': Code before strictures are enabled
  at /tmp/Code-TidyAll-0e6K/Driver.pm line 2
  [TestingAndDebugging::RequireUseStrict]

This replaces myriad scripts out there that perform perltidy or perlcritic in a precommit hook.

Git support is coming next.

In unit tests

Test::Code::TidyAll checks that all the files in your distribution are in a tidied and valid state. This replaces a bunch of separate testing modules (Test::Perl::Tidy, Test::Perl::Critic, Test::Pod, etc.), each with their own syntax and rules about which files to select.

In Dist::Zilla

Dist::Zilla::Plugin::TidyAll is a Dist::Zilla plugin that runs tidyall on files when building a release.

Next steps

Git support, then more plugins for Perl and other languages!

Any other neat-freaks find this useful? Feedback welcome.

Aug 16

Lately I’ve become a big fan of the Nginx + Starman combination for Perl-based web development. mod_perl has served me well for fifteen years, but with Plack/PSGI replacing the mod_perl API, it makes less sense to configure and install Apache just to invoke a PSGI handler. Starman has near-zero configuration, and Nginx provides a perfect complement for HTTP acceleration and serving static files.

To facilitate using these servers I added Nginx and Starman plugins for Server::Control, which previously existed mainly for Apache.

So what’s Server::Control?

Server::Control is a set of libraries for controlling servers, where a server is any background process which listens to a port and has a pid file. Think apachectl on steroids
(and not for just Apache).

In the happy case, controlling a pid-file server is simple – just run the command to start it, and run kill `cat /path/to/pidfile` to stop it. Where Server::Control comes in is handling all the little unhappy cases.

For example, accidentally starting a server that’s already running or stopping a server that isn’t:

    % bin/server.pl -k start
    server 'mhq' is already running (pid 5912) and listening to port 5000

    % bin/server.pl -k stop
    server 'mhq' is not running

or trying to start a server whose port being blocked by another process:

    % bin/server.pl -k start
    cannot start server 'mhq' - pid file
    '/Users/swartz/git/mason-site.git/data/starman.pid' does not exist,
    but something (possibly pid 5943 - "/usr/local/bin/plackup") is listening
    to localhost:5000

or a corrupt pid file, left over from a reboot or a kill -9:

    % bin/server.pl -k start
    pid file '/Users/swartz/git/mason-site.git/data/starman.pid' contains 
    a non-existing process id '5985'!
    deleting bogus pid file '/Users/swartz/git/mason-site.git/data/starman.pid'
    ...

or a server that starts but isn’t listening to the expected port:

    % bin/server.pl -k start
    waiting for server start
    after 10 secs, server 'mhq' appears to be running (pid 6167), but not
    listening to port 5000

or a server that starts but isn’t serving content correctly (using the validate_url and validate_regex parameters):

    % bin/server.pl -k start
    waiting for server start
    server 'mhq' is now running (pid 6080) and listening to port 5000
    validating url 'http://localhost:5000/'
    content of 'http://localhost:5000/' (12798 bytes) did not match
    regex 'qr/(?-xism:Welcome to Mason and Poet)/'

So I always take the extra time to set up Server::Control, and it usually pays off in reduced frustration in the end. For convenience I have aliases like this set up to start, stop, restart and ping (check the status of) each server on a machine:

    alias ctlmhq='/home/swartz/servers/mhq/bin/server.pl -k'
    alias stamhq='ctlmhq start'
    alias stomhq='ctlmhq stop'
    alias remhq='ctlmhq restart'
    alias pingmhq='ctlmhq ping'
preload preload preload