Sep 21

Mason finally has a tidier: masontidy. It runs perltidy on the various sections of Perl code embedded in Mason components. It works on both Mason 1 and Mason 2 syntax.

One nice trick is that it indents %-lines relative to each other, regardless of intervening content. e.g.

    <ul>
    % foreach my $article (@articles) {
    <li>
    %     my $title = $article->title;
    <% $title %>
    %     if ( $article->has_related_links ) {
    %         foreach my $link ( $article->related_links ) {
    ...

Given my devotion to code tidying in general, this ought to have been written a long time ago, but the perfect became the enemy of the good. I always imagined a tidier that would reformat the HTML content simultaneously, so that for example the <li> would be indented inside the <ul> above.

This turns out to be difficult and fraught with edge cases. What if an HTML tag is generated inside Perl code and can’t be seen by the tidier? What about embedded javascript and CSS? Every time I encountered these problems I’d shelve the project.

In the end I decided half a solution is better than none. The current masontidy doesn’t attempt to tidy the HTML or non-Perl content. Perhaps someday I’ll figure out how to do it, but the current tool is still a big improvement.

Apr 21

I’m pleased to announce Poet, a modern Perl web framework designed especially for Mason developers.

Features include:

Poet was designed and developed over the past six years at Hearst Digital Media. Today it is used to generate Hearst’s magazine websites (including Cosmopolitan, Esquire, and Good Housekeeping) as well as associated content management, subscription management and ad rotation systems. I’m very grateful to Hearst for agreeing to this open source release (though they bear no responsibility for its support or maintenance).

Why another Perl web framework?

To answer this requires a bit of history.

HTML::Mason was one of the early Perl “web frameworks”. Like its JSP/ASP/PHP contemporaries, its main trick was embedding code in HTML, but it contained enough web-specific goodies to serve as a one-stop solution. It relied heavily on mod_perl and had mailing lists filled with web-related discussions having nothing to do with templating.

Over time, a new breed of web framework emerged – Catalyst and Jifty and Mojolicious and Dancer in the Perl world, Rails and Sinatra and Django elsewhere. In these frameworks the templates moved from center stage to become just one piece of a large system.

HTML::Mason faced an identity dilemma; should it be a pure templating framework, or try to expand and better serve its traditional web development audience? In the end, and with coaxing from co-author Dave Rolsky, Mason 2 shifted decisively towards the former. It shed most of its web-specific code, thanks in large part to Plack/PSGI, and became more of a generic templating system (albeit still destined to spend much of its time generating HTML).

There’s two ways to use Mason

So one legitimate way to use Mason is as a dedicated View component in a larger MVC framework like Catalyst or Dancer. Hence Catalyst::View::Mason2 and Dancer::Template::Mason2. (This is how Dave prefers to roll.)

But for me, and for some others, Mason remains a great way to handle the whole web request – to dispatch URLs to components and process HTTP arguments and implement common behaviors for sets of pages. I prefer my page logic right next to my page view, rather than flipping between a controller and view that are often annoyingly coupled.

Moreover, fifteen+ years had left me with a pile of useful ideas, techniques, and conventions for web development. Mason wasn’t the appropriate place for them any more (if it ever was) but I need to collect them somewhere.

This is where Poet comes in. Poet doesn’t need a controller layer; it turns web requests into Mason requests, and happily lets Mason handle the rest of the work. Poet doesn’t have Mason’s identity crisis; it is proudly web-centric, the place to put all the web-related goodness that Mason developers want nearby.

There’s much more to come than I could put in this initial release, and I’m looking forward to pressing on with it! I hope it makes at least a few of your lives’ easier, and as always I welcome the feedback.

Mar 04

At work we have over 200 modules and Mason components that use
CHI to cache some data or HTML. Each has its own
distinct namespace to prevent collisions.

Each namespace uses one of several different storage types — memcached, local file, NFS
file – depending on its usage characteristics. Each storage type has a set of default
parameters (e.g. root_dir for file) that rarely change. Finally, there are some defaults
we want to use across all of our caches.

To maintain a coherent cache strategy — and our sanity — we need a single place to
see and adjust all this configuration.

So instead of repeatedly embedding parameters like this:

my $cache = CHI->new
   (namespace => 'Foo', driver => 'File', root_dir => '/path/to/root',
    depth => 3, expires_in => '15m', expires_variance => 0.2);

...

my $cache = CHI->new
   (namespace => 'Bar', driver => 'Memcached',
    servers => [ "10.0.0.15:11211", "10.0.0.17:11211" ],
    compress_threshold => 10_000, expires_in => '1h', expires_variance => 0.2);

we can do this:

my $cache = CHI->new(namespace => 'Foo');

...

my $cache = CHI->new(namespace => 'Bar');

then in a YAML configuration file:

defaults:
  expires_variance: 0.2

storage:
  local_file:
    driver: File
    root_dir: /path/to/local/root
  nfs_file:
    driver: File
    root_dir: /path/to/nfs/root
  memcached:
    driver: Memcached
    servers: [ ... ]
    compress_threshold: 10_000

namespace:
  Foo: { storage: local_file, expires_in: 15m }
  Bar: { storage: memcached,  expires_in: 1h }
  Baz: { storage: memcached,  expires_in: 2h }
  ...

In the first paragraph we define overall defaults. In the second we define a set of
storage types, each with their own defaults. In the third we assign each namespace to a
storage type and an expiration time. Each level can override the defaults of previous
levels, and arguments passed in CHI->new override anything in configuration.

Support for this kind of configuration is available as of CHI 0.52. You should first
create a CHI subclass for your application, so as not to interfere with other CHI users in
the same process:

package My::CHI;
use base qw(CHI);

Then specify configuration with a hash or file:

My::CHI->config({ storage => ..., namespace => ..., defaults => ... });

My::CHI->config(YAML::XS::LoadFile("/path/to/cache.yml"));

Even if you don’t have 200 namespaces, it’s nice to have a single place where you can
fiddle with cache controls.

Feb 25

For those using Mason 2′s request dispatching features, I just released Mason::Plugin::RouterSimple, which adds basic route support courtesy of Router::Simple.

Let’s say we want to support URLs like

    /archives/2010/05

where the second and third part of the path are year and month parameters respectively.

Using the new plugin:

    %% route "{year:[0-9]{4}}/{month:[0-9]{2}}";

    Archives for the month of <% $.month %>/<% $.year %>:
    ...

    <%init>
    # Use $.month and $.year to fetch the archives
    </%init>

The route command adds a route indicating how the path_info (the remainder of the path beyond /archives) should be parsed.

Any named captured arguments are placed in component attributes, which are declared automatically if you don’t do so manually. So above, we get auto-declared attributes $.year and $.month that are then set to ’2010′ and ’05′ for that URL.

You can add multiple routes and they’ll each be tried in turn. If the URL does not match any route, the component will decline, which generally results in a 404.

Other examples of Router::Simple routes:

    route "wiki/:page";     # sets $.page
    route "download/*.*";   # sets $.splat to a 2-element arrayref

I chose Router::Simple because I liked the compact declaration syntax, but one could create similar plugins (with some shared common code) to support other routers like Path::Dispatcher and Path::Router.

Feb 21

I’m pleased to announce Mason 2, the first major version of Mason in ten years.

For those not familiar with it, Mason is a templating framework for generating web pages and other dynamic content. Mason 2 has been rearchitected and reimplemented from the ground up, to take advantage of modern Perl techniques (Moose, Plack/PSGI) and to correct long-standing feature and syntax inadequacies. Its new foundations should allow its performance and flexibility to far exceed Mason 1.

Though little original code or documentation remains, Mason’s core philosophy is intact; it should still “feel like Mason” to existing users.

I’ve talked about plans for Mason 2 here before, but as things have changed in the past year and a half, here’s an updated summary:

  • Name. The name is now Mason, instead of HTML::Mason.

  • Component classes. Each component is represented by its own (Moose) class, rather than just an instance of a common class. This means that components have their own namespaces, subroutines, methods, and attributes, and can truly inherit from one other. See Mason::Manual::Components.

  • Filters. A single powerful filter syntax and mechanism consolidates three separate filter mechanisms from Mason 1 (filter blocks, components with content, and escape flags). See Mason::Manual::Filters.

  • Plugins. Moose roles are utilized to create a flexible plugin system that can modify nearly every aspect of Mason’s operation. Previously core features such as caching can now be implemented in plugins. See Mason::Manual::Plugins.

  • Web integration. Mason 1′s bulky custom web handling code (ApacheHandler, CGIHandler) has been replaced with a simple PSGI handler and with plugins for web frameworks like Catalyst and Dancer. The core Mason distribution is now completely web-agnostic. See Mason::Plugin::PSGIHandler.

  • File naming. Mason now facilitates and enforces (in a customizable way) standard file extensions for components: .m (top-level components), .mi (internal components), and .pm (pure-perl components).

See Mason::Manual::UpgradingFromMason1 for a more detailed list of changes.

Mason 2 is obviously still in alpha status, but it has a fair sized test suite and I’m eager to start building web projects with it. I hope you’ll give it a try too! Post feedback here or on the Mason user’s list.

Jan 14

After converting Mason 2 to use Dist::Zilla and Pod::Weaver, I am starting to think in plugins and see opportunities for them everywhere.

As just one example, I initially added a cache feature to Mason 2, just because it was there in Mason 1 and seemed generally useful. But now I realize it can be cleanly separated into its own plugin.

There’s no real performance or footprint advantage for those that omit the plugin. What it does do is put all the cache related code, documentation, and tests in a single place. As small as it is, the cache feature touched many of the main Mason modules in some way; now it is all in its own directory. It also makes me more amenable to expanding the feature (e.g. with new parameters or methods), because I’m not worried about disproportionately “polluting” the main code and API.

Of course, deciding what is a plugin versus a built-in feature (plus that third category, a plugin-in-name that really everyone expects to install) is the age-old question. Too many plugins will of course hurt performance, because each one adds a layer of method modifiers, and will also hurt useability and readability. I’m still quite confused about where and how Dist::Zilla performs many of its tasks, and part of that is the sheer number of plugins one needs for a typical installation. It’s a virtual certainty that in my new enthusiasm I’ll over-plugin somewhere.

Dec 27

In Mason 1 there are several ways to filter content: the <%filter> section, escape flags, and Component Calls with Content. Each of these has their own disparate implementation and API.

In Mason 2 we aim to unify these with a single implementation and a more consistent syntax.

A set of standard filters are automatically available in components, and other filter packages can be loaded via plugins.

Invoking filters (block syntax)

Here’s the typical way of invoking a filter:

<% $.Trim { %>
This string will be trimmed
</%>

Things to note here:

  • An { at the end of a <% %> tag denotes a filter call.
  • The </%> tag marks the end of the filtered content.
  • The expression $.Trim, aka $self->Trim, is a method call on the component object which returns a filter. In general everything before the brace is evaluated and is expected to return a filter or list of filters.
  • Filter names use CamelCase to distinguish themselves from other methods in the Mason::Component namespace. Another option was to use a standard prefix or suffix, e.g. trim_filter or filter_trim, but I thought this was the least ugly option and made filters look a little more like built-in tags.

Filters can take arguments:

<% $.Repeat(3) { %>
  There's no place like home.
</%>

  ==>  There's no place like home.
 There's no place like home.
 There's no place like home.

Again, the expression $.Repeat(3) returns a filter, meaning that it can
be curried:

% my $repeat_three = $.Repeat(3);
<% $repeat_three { %>
  There's no place like home.
</%>

A simple filter is just a subroutine that takes text as input and return the
new text. Thus you can create one-off filters with anonymous subroutines:

<% sub { reverse($_[0]) } { %>Hello</%>

     ==> olleH

Filters can be nested, with separate tags:

<% $.Trim { %>
  <% sub { uc($_[0]) } { %>
    This string will be trimmed and uppercased
  </%>
</%>

or within a single tag:

<% $.Trim, sub { uc($_[0]) } { %>
  This will be trimmed and uppercased
</%>

Multiple filters within the same tag are applied, intuitively, in reverse order with the last one being innermost. e.g. in this block

% my $i = 1;
<% $.Repeat(3), $.Cache($key, '1 hour') { %> <% $i++ %> </%>

  => 1 1 1

the output of <% $i++ %> is cached, and then repeated three times, whereas in this block

% my $i = 1;
<% $.Cache($key, '1 hour'), $.Repeat(3) { %> <% $i++ %> </%>

  => 1 2 3

<% $i++ %> is executed and output three times, and then the whole thing cached.

Invoking filters (pipe syntax)

Filters can also appear in a limited way inside a regular <% %> tag; this replaces Mason 1′s escape flags.

<% $content | NoBlankLines,Trim %>

The filter list contains one or more comma-separated names, which are treated as methods on the current component class. With this syntax you cannot use anonymous subroutines or variables as filters, or pass arguments to filters (but you can define local filter methods to get around this).

One common use of this form is to escape HTML strings:

<% $message_body | H %>

The H filter is provided along with other web-related filters in Mason::Plugin::HTMLFilters.

Applying filter to current component

Mason 1′s <%filter> section was good for filtering the content of the current component. e.g. to make it uppercase:

<%filter>
$_ = uc($_);
</%filter>

In Mason 2 the most succinct replacement is currently an around modifier:

<%around main>
<% sub { uc($_[0]) } { %>
% $self->$orig();
</%>
</%around>        

Creating filters

Here’s a filter package that implements two filters, Upper and Lower:

package MyApp::Filters;
use Method::Signatures::Simple;
use Moose::Role;
 
method Upper () {
    return sub { uc($_[0]) }
}
 
method Lower () {
    return sub { lc($_[0]) }
}
 
1;

To use these in a component:

<%class>
with 'MyApp::Filters';
</%class>

<% $.Upper { %>
...
</%>

Or if you want them available to all components, put them in Base.pm at the top of your component hierarchy, or in your application’s Mason::Component subclass.

Simple vs. dynamic filters

A simple filter is a code ref which takes a string and returns the output, like Upper and Lower above.

A dynamic filter (a Mason::DynamicFilter object) contains a code ref which takes a yield block and returns the output. A yield block is a zero-argument code ref that returns a content string.

The power of dynamic filters is that they can choose if and when to execute the yield block. For example, here’s the standard Cache filter in action:

<% $.Cache($key, '1 hour') { %>
... some computed content ...
</%>

This caches the inner content for one hour based on cache key $key. Here’s the implementation:

method Cache ( $key, $set_options ) {
    Mason::DynamicFilter->new(
        filter => sub {
            my $yield = shift;
            $self->cmeta->cache->compute( $key, $yield, $set_options );
        }
    );
}

Using CHI’s compute method, we check the cache first and return the output immediately if it is available. Only on a cache miss do we actually execute the (presumably expensive) yield block. This could not be implemented with a simple filter since the content would be computed every time.

Dec 15

(These posts are starting to follow a familiar pattern: “In Mason 1 we did X, but in Mason 2, Moose makes it so much easier…)”

Anyway.

In Mason 1, content wrapping (the practice of wrapping HTML in one or more surrounding templates) was achieved via autohandlers. Each autohandler would print its content and then call $m->call_next, which would pass control to the next component down the subclass chain. This was all special-purpose code written against Mason’s already hacky inheritance scheme.

But in Mason 2, Moose gives us a natural solution for this pattern: inner and augment.

At the start of a Mason 2 request, after determining the top-level component (aka “page component”), Mason will call that component’s render() method. By default, render() will just call main(), which outputs the main body of the component. However, each ancestor of the page component has an opportunity to wrap render() with its own template. For example:

# /Base.m - top-most template
#
<%augment render>
  <html><body>
      <% inner() %>
      <div class="footer">Copyright 2010 McHuffy Inc.</div>
    </body></html>
</%augment>

# /product/Base.m - product-specific template
#
<%augment render>
  <h2>Products</h2>
  <& product_nav.mi &>
  <div class="products_body">
    <% inner() %>
  </div>
</%augment>

# /product/sales/Base.pm - utility methods, no additional template
#
<%method some_utility_method>
   ...
</%method>

# /product/sales/display.m:
#
<h3><% $product->title %></h3>
...

The page /product/sales/display will be wrapped by two templates: the outer one
in /Base.pm, and the inner one in /product/Base.pm.

A couple of new things here for Mason 1 users:

  • ‘autohandler’ has changed to ‘Base.m’. A component of this name becomes the automatic superclass of every component in its directory and subdirectories.
  • There are sections for declaring Moose methods and method modifiers with embedded-Perl syntax: <%after>, <%around>, <%augment>, <%before>, and <%method>. (Why we “need” all these explicit sections is a subject for another post.)

Now here’s an implementation snag. Notice that neither /product/sales/display nor /product/sales/Base.pm have a render method. For the former, we want to automatically create a render that just calls main, which will display the main body of the component. For the latter, we want render to just fall through to the next level. But there’s no way to know in advance whether a component will be used as the page component or as one of its superclasses, so we have to create a default render for every component that does the right thing at runtime. In pseudo-code:

if (we are a superclass of the page component) {
    # Go to the next level.
    inner();
} else {
    # We are the page component. Draw the main content.
    $self->main();
}

Unfortunately there’s no official way to ask Moose whether we’re at the bottom of the inner() chain. So I’ve got a couple of choices here:

1. Check ref($self):

if ( ref($self) ne __PACKAGE__ ) {
    inner();
} else {
    $self->main();
}

As long as we provide a default render for every component, I think this should correctly distinguish the bottom of the chain. But I’m not entirely sure – Moose does a lot of with auto-generated subclasses.

2. Steal the code out of Moose::inner:

if ( $Moose::INNER_BODY ) {
    inner();
} else {
    $self->main();
}

Of course this is awful, as it relies on implementation details that are subject to change.

For now I’ll be going with #1.

Dec 14

In Mason 1, extending Mason’s behavior means either using the limited plugin API (allowing you to take action before and after a request or component call), or else writing subclasses, which is problematic when you try to use more than one together.

In Mason 2, plugins are based on Moose roles, allowing you to modify behavior in each of Mason’s main subclasses — Interp, Request, Compiler, Compilation, and Component — in a composable way.

Here’s my first Mason 2 plugin, to perltidy object files after components are compiled – useful in development. First, a nominal base class:

package Mason::Plugin::TidyObjectFiles;
use Moose;
extends 'Mason::Plugin';

1;

(This has to exist for Mason to recognize the plugin, but there isn’t much to do there right now.)

Then, a role for each of Mason’s subclasses that we want to modify. In this case we only need to modify Mason::Compiler:

package Mason::Plugin::TidyObjectFiles::Compiler;
use Moose::Role;
use Perl::Tidy;
use strict;
use warnings;

has 'tidy_options' => ( is => 'ro' );

around 'write_object_file' => sub {
    my ( $orig, $self, $object_file, $object_contents ) = @_;

    my $argv = $self->tidy_options || '';
    my $source = $object_contents;
    Perl::Tidy::perltidy(
        'perltidyrc' => '/dev/null',
        source       => $source,
        destination  => $object_contents,
        argv         => $argv
    );
    $self->$orig( $object_file, $object_contents );
};

1;

To use this, I simply list ‘TidyObjectFiles’ in my plugins:

my $mason = Mason->new(..., plugins=>[ 'TidyObjectFiles' ]);

or, if I want to specify perltidy options:

my $mason = Mason->new(..., plugins=>[ 'TidyObjectFiles' ], tidy_options => '-noll -l=100');

The Mason::Plugin:: prefix is automatically added to plugin names. You can also specify a full path with a leading ‘+’, e.g.

plugins => ['+MyApp::Plugin::MyPlugin']

Now my object files go from looking like this:

no warnings 'redefine';
sub _comp_info { return {comp_dir_path => '/',comp_is_external => 1,comp_path => '/hi.m'} }
sub main {
my $self = shift;
my $m = $self->m;

my $_buffer = $m->current_buffer;
 #line 1 "/Users/swartz/git/mason.git/tmp/comps/hi.m"
$$_buffer .= 'Hi there! The time is ';
 #line 1 "/Users/swartz/git/mason.git/tmp/comps/hi.m"
{ $$_buffer .=  scalar(localtime)  if defined( scalar(localtime) ) }
 #line 1 "/Users/swartz/git/mason.git/tmp/comps/hi.m"
$$_buffer .= '.
';


;return;
}

to this:

no warnings 'redefine';

sub _comp_info {
    return {
        comp_dir_path    => '/',
        comp_is_external => 1,
        comp_path        => '/hi.m'
    };
}

sub main {
    my $self = shift;
    my $m    = $self->m;

    my $_buffer = $m->current_buffer;

    #line 1 "/Users/swartz/git/mason.git/tmp/comps/hi.m"
    $$_buffer .= 'Hi there! The time is ';

    #line 1 "/Users/swartz/git/mason.git/tmp/comps/hi.m"
    { $$_buffer .= scalar(localtime) if defined( scalar(localtime) ) }

    #line 1 "/Users/swartz/git/mason.git/tmp/comps/hi.m"
    $$_buffer .= '.
';

    return;
}
Dec 12

In Mason 1 you can declare component arguments and then refer to them as normal lexically scoped variables:

<%args>
$address
$city
$name
</%args>

Dear <% $name %>: We will come to your house at <% $address %> in the
fair city of <% $city> to deliver your prize!

In Mason 2, components are classes rather than subroutines, so we have attributes rather than arguments. When you call a component, you create an instance of the component class and pass attributes to the constructor.

Attributes have several advantages over arguments: you can refer to them in any of the component’s methods (unlike in Mason 1, where arguments have to be awkwardly passed to each method) and you can take advantage of Moose’s powerful attribute declarations.

Here’s how this looks without any special syntax:

<%class>
has 'address' => (is => 'ro');
has 'city'    => (is => 'ro');
has 'name'    => (is => 'ro');
</%class>

Dear <% $self->name %>: We will come to your house at <% $self->address %> in the fair city of <% $self->city> to deliver your prize!

Clearly, we’ll need to pour some sugar on this.

Declaring attributes

Mason 2 will provide an <%attr> section, analagous to Mason 1′s <%args>:

<%attr>
a
b => 'foo'
c => (isa => 'Str', default => 'something')
</%attr>

This illustrates the three kinds of <%attr> declarations:

  • A name by itself: required attribute.
  • A name followed by => and a non-paren: optional attribute with a default.
  • A name followed by => a paren: specifies Moose attribute options.

You can mix and match these attributes with standard has declarations. The above is equivalent to:

<%class>
has 'a' => (required => 1);
has 'b' => (default => 'foo');
has 'c' => (isa => 'Str', default => 'something');
</%class>

All attributes will be read-write by default, via MooseX::HasDefaults::RW. Although read-write attributes are sometimes discouraged, I believe they’re appropriate and convenient for these short-lived component objects that are not generally expected to be accessed outside of their class.

Accessing attributes

To make the frequent access of attributes more convenient, we resort to some limited source filtering evil and implement Perl 6ish $. notation. Specifically, all occurrances of

$.foo

within Perl sections will be replaced with

$self->foo

where foo is any valid attribute name ([A-Za-z_]w*).

Note that with read-write lvalue attributes, this could even be used for assignment:

$.foo = 5;

Very convenient for attributes which exist chiefly for sharing data between methods.

The $. substitution feature will be made easy to disable, for those who are against it.

Putting it all together

The first example above with attribute declaration and access sugar:

<%attr>
address
city
name
</%attr>

Dear <% $.name %>: We will come to your house at <% $.address %> in the
fair city of <% $.city> to deliver your prize!
preload preload preload