Mar 02

I’ve released CHI 0.41 with a RawMemory driver. It’s like the regular Memory driver except that data structure references are stored directly instead of serializing / deserializing. This makes the cache faster at getting and setting complex data structures, but unlike most drivers, modifications to the original data structure will affect the data structure stored in the cache, and vica versa. e.g.

   my $cache = CHI->new( driver => 'Memory', global => 1 );
   my $lst = ['foo'];
   $cache->set('key' => $lst);   # serializes $lst before storing
   $cache->get('key');   # returns ['foo']
   $lst->[0] = 'bar';
   $cache->get('key');   # returns ['foo']

   my $cache = CHI->new( driver => 'RawMemory', global => 1 );
   my $lst = ['foo'];
   $cache->set('key' => $lst);   # stores $lst directly
   $cache->get('key');   # returns ['foo']
   $lst->[0] = 'bar';
   $cache->get('key');   # returns ['bar']!

It should work well as a short-lived L1 cache in front of memcached, for example.

I was motivated to create this by Yuval Kogman’s Cache::Ref, which is still a little faster (not having some of the overhead of CHI’s metadata and features). See the CHI benchmarks, also new with this release.

Feb 25

For those using Mason 2′s request dispatching features, I just released Mason::Plugin::RouterSimple, which adds basic route support courtesy of Router::Simple.

Let’s say we want to support URLs like

    /archives/2010/05

where the second and third part of the path are year and month parameters respectively.

Using the new plugin:

    %% route "{year:[0-9]{4}}/{month:[0-9]{2}}";

    Archives for the month of <% $.month %>/<% $.year %>:
    ...

    <%init>
    # Use $.month and $.year to fetch the archives
    </%init>

The route command adds a route indicating how the path_info (the remainder of the path beyond /archives) should be parsed.

Any named captured arguments are placed in component attributes, which are declared automatically if you don’t do so manually. So above, we get auto-declared attributes $.year and $.month that are then set to ’2010′ and ’05′ for that URL.

You can add multiple routes and they’ll each be tried in turn. If the URL does not match any route, the component will decline, which generally results in a 404.

Other examples of Router::Simple routes:

    route "wiki/:page";     # sets $.page
    route "download/*.*";   # sets $.splat to a 2-element arrayref

I chose Router::Simple because I liked the compact declaration syntax, but one could create similar plugins (with some shared common code) to support other routers like Path::Dispatcher and Path::Router.

Feb 21

I’m pleased to announce Mason 2, the first major version of Mason in ten years.

For those not familiar with it, Mason is a templating framework for generating web pages and other dynamic content. Mason 2 has been rearchitected and reimplemented from the ground up, to take advantage of modern Perl techniques (Moose, Plack/PSGI) and to correct long-standing feature and syntax inadequacies. Its new foundations should allow its performance and flexibility to far exceed Mason 1.

Though little original code or documentation remains, Mason’s core philosophy is intact; it should still “feel like Mason” to existing users.

I’ve talked about plans for Mason 2 here before, but as things have changed in the past year and a half, here’s an updated summary:

  • Name. The name is now Mason, instead of HTML::Mason.

  • Component classes. Each component is represented by its own (Moose) class, rather than just an instance of a common class. This means that components have their own namespaces, subroutines, methods, and attributes, and can truly inherit from one other. See Mason::Manual::Components.

  • Filters. A single powerful filter syntax and mechanism consolidates three separate filter mechanisms from Mason 1 (filter blocks, components with content, and escape flags). See Mason::Manual::Filters.

  • Plugins. Moose roles are utilized to create a flexible plugin system that can modify nearly every aspect of Mason’s operation. Previously core features such as caching can now be implemented in plugins. See Mason::Manual::Plugins.

  • Web integration. Mason 1′s bulky custom web handling code (ApacheHandler, CGIHandler) has been replaced with a simple PSGI handler and with plugins for web frameworks like Catalyst and Dancer. The core Mason distribution is now completely web-agnostic. See Mason::Plugin::PSGIHandler.

  • File naming. Mason now facilitates and enforces (in a customizable way) standard file extensions for components: .m (top-level components), .mi (internal components), and .pm (pure-perl components).

See Mason::Manual::UpgradingFromMason1 for a more detailed list of changes.

Mason 2 is obviously still in alpha status, but it has a fair sized test suite and I’m eager to start building web projects with it. I hope you’ll give it a try too! Post feedback here or on the Mason user’s list.

Jan 14

After converting Mason 2 to use Dist::Zilla and Pod::Weaver, I am starting to think in plugins and see opportunities for them everywhere.

As just one example, I initially added a cache feature to Mason 2, just because it was there in Mason 1 and seemed generally useful. But now I realize it can be cleanly separated into its own plugin.

There’s no real performance or footprint advantage for those that omit the plugin. What it does do is put all the cache related code, documentation, and tests in a single place. As small as it is, the cache feature touched many of the main Mason modules in some way; now it is all in its own directory. It also makes me more amenable to expanding the feature (e.g. with new parameters or methods), because I’m not worried about disproportionately “polluting” the main code and API.

Of course, deciding what is a plugin versus a built-in feature (plus that third category, a plugin-in-name that really everyone expects to install) is the age-old question. Too many plugins will of course hurt performance, because each one adds a layer of method modifiers, and will also hurt useability and readability. I’m still quite confused about where and how Dist::Zilla performs many of its tasks, and part of that is the sheer number of plugins one needs for a typical installation. It’s a virtual certainty that in my new enthusiasm I’ll over-plugin somewhere.

Dec 27

In Mason 1 there are several ways to filter content: the <%filter> section, escape flags, and Component Calls with Content. Each of these has their own disparate implementation and API.

In Mason 2 we aim to unify these with a single implementation and a more consistent syntax.

A set of standard filters are automatically available in components, and other filter packages can be loaded via plugins.

Invoking filters (block syntax)

Here’s the typical way of invoking a filter:

<% $.Trim { %>
This string will be trimmed
</%>

Things to note here:

  • An { at the end of a <% %> tag denotes a filter call.
  • The </%> tag marks the end of the filtered content.
  • The expression $.Trim, aka $self->Trim, is a method call on the component object which returns a filter. In general everything before the brace is evaluated and is expected to return a filter or list of filters.
  • Filter names use CamelCase to distinguish themselves from other methods in the Mason::Component namespace. Another option was to use a standard prefix or suffix, e.g. trim_filter or filter_trim, but I thought this was the least ugly option and made filters look a little more like built-in tags.

Filters can take arguments:

<% $.Repeat(3) { %>
  There's no place like home.
</%>

  ==>  There's no place like home.
 There's no place like home.
 There's no place like home.

Again, the expression $.Repeat(3) returns a filter, meaning that it can
be curried:

% my $repeat_three = $.Repeat(3);
<% $repeat_three { %>
  There's no place like home.
</%>

A simple filter is just a subroutine that takes text as input and return the
new text. Thus you can create one-off filters with anonymous subroutines:

<% sub { reverse($_[0]) } { %>Hello</%>

     ==> olleH

Filters can be nested, with separate tags:

<% $.Trim { %>
  <% sub { uc($_[0]) } { %>
    This string will be trimmed and uppercased
  </%>
</%>

or within a single tag:

<% $.Trim, sub { uc($_[0]) } { %>
  This will be trimmed and uppercased
</%>

Multiple filters within the same tag are applied, intuitively, in reverse order with the last one being innermost. e.g. in this block

% my $i = 1;
<% $.Repeat(3), $.Cache($key, '1 hour') { %> <% $i++ %> </%>

  => 1 1 1

the output of <% $i++ %> is cached, and then repeated three times, whereas in this block

% my $i = 1;
<% $.Cache($key, '1 hour'), $.Repeat(3) { %> <% $i++ %> </%>

  => 1 2 3

<% $i++ %> is executed and output three times, and then the whole thing cached.

Invoking filters (pipe syntax)

Filters can also appear in a limited way inside a regular <% %> tag; this replaces Mason 1′s escape flags.

<% $content | NoBlankLines,Trim %>

The filter list contains one or more comma-separated names, which are treated as methods on the current component class. With this syntax you cannot use anonymous subroutines or variables as filters, or pass arguments to filters (but you can define local filter methods to get around this).

One common use of this form is to escape HTML strings:

<% $message_body | H %>

The H filter is provided along with other web-related filters in Mason::Plugin::HTMLFilters.

Applying filter to current component

Mason 1′s <%filter> section was good for filtering the content of the current component. e.g. to make it uppercase:

<%filter>
$_ = uc($_);
</%filter>

In Mason 2 the most succinct replacement is currently an around modifier:

<%around main>
<% sub { uc($_[0]) } { %>
% $self->$orig();
</%>
</%around>        

Creating filters

Here’s a filter package that implements two filters, Upper and Lower:

package MyApp::Filters;
use Method::Signatures::Simple;
use Moose::Role;
 
method Upper () {
    return sub { uc($_[0]) }
}
 
method Lower () {
    return sub { lc($_[0]) }
}
 
1;

To use these in a component:

<%class>
with 'MyApp::Filters';
</%class>

<% $.Upper { %>
...
</%>

Or if you want them available to all components, put them in Base.pm at the top of your component hierarchy, or in your application’s Mason::Component subclass.

Simple vs. dynamic filters

A simple filter is a code ref which takes a string and returns the output, like Upper and Lower above.

A dynamic filter (a Mason::DynamicFilter object) contains a code ref which takes a yield block and returns the output. A yield block is a zero-argument code ref that returns a content string.

The power of dynamic filters is that they can choose if and when to execute the yield block. For example, here’s the standard Cache filter in action:

<% $.Cache($key, '1 hour') { %>
... some computed content ...
</%>

This caches the inner content for one hour based on cache key $key. Here’s the implementation:

method Cache ( $key, $set_options ) {
    Mason::DynamicFilter->new(
        filter => sub {
            my $yield = shift;
            $self->cmeta->cache->compute( $key, $yield, $set_options );
        }
    );
}

Using CHI’s compute method, we check the cache first and return the output immediately if it is available. Only on a cache miss do we actually execute the (presumably expensive) yield block. This could not be implemented with a simple filter since the content would be computed every time.

Dec 19

I wrote previously about the inability of perltidy to handle the method keyword of Method::Signatures::Simple. Now, Steve Hancock has graciously accepted my patches in the latest Perl::Tidy, including prefilter and postfilter options. This allows me to easily add method support:

Perl::Tidy::perltidy(
    ...,
    prefilter => sub {
        $_ = $_[0];
        s/^method (.*)/sub $1 #__METHOD/gm;
        return $_;
    },
    postfilter => sub {
        $_ = $_[0];
        s/^sub (.*?)s* #__METHOD/method $1/gm;
        return $_;
    }
);
  • The prefilter code substitutes method for sub, and adds a comment
    so we’ll be able to find and convert these back afterwards.
  • When perltidy operates on the code, it will just see regular subs and treat them accordingly.
  • The postfilter code converts the sub back to method wherever the special comment appears.

I put this in my private Perl::Tidy subclass, along with other tweaks, such as telling perltidy to leave my Moose ‘has’ lines alone (I prefer them to always be on one line).

Then I create my own perltidy script which uses this subclass:

#!/usr/local/bin/perl
package main;
use JS::Perl::Tidy;
JS::Perl::Tidy::perltidy();

A little messy, but it works. I’m a devoted perltidy user, with enforced perltidy-on-commit policy on every project, so I’m glad I don’t have to choose between perltidy and my favorite Perl tweaks.

Dec 15

(These posts are starting to follow a familiar pattern: “In Mason 1 we did X, but in Mason 2, Moose makes it so much easier…)”

Anyway.

In Mason 1, content wrapping (the practice of wrapping HTML in one or more surrounding templates) was achieved via autohandlers. Each autohandler would print its content and then call $m->call_next, which would pass control to the next component down the subclass chain. This was all special-purpose code written against Mason’s already hacky inheritance scheme.

But in Mason 2, Moose gives us a natural solution for this pattern: inner and augment.

At the start of a Mason 2 request, after determining the top-level component (aka “page component”), Mason will call that component’s render() method. By default, render() will just call main(), which outputs the main body of the component. However, each ancestor of the page component has an opportunity to wrap render() with its own template. For example:

# /Base.m - top-most template
#
<%augment render>
  <html><body>
      <% inner() %>
      <div class="footer">Copyright 2010 McHuffy Inc.</div>
    </body></html>
</%augment>

# /product/Base.m - product-specific template
#
<%augment render>
  <h2>Products</h2>
  <& product_nav.mi &>
  <div class="products_body">
    <% inner() %>
  </div>
</%augment>

# /product/sales/Base.pm - utility methods, no additional template
#
<%method some_utility_method>
   ...
</%method>

# /product/sales/display.m:
#
<h3><% $product->title %></h3>
...

The page /product/sales/display will be wrapped by two templates: the outer one
in /Base.pm, and the inner one in /product/Base.pm.

A couple of new things here for Mason 1 users:

  • ‘autohandler’ has changed to ‘Base.m’. A component of this name becomes the automatic superclass of every component in its directory and subdirectories.
  • There are sections for declaring Moose methods and method modifiers with embedded-Perl syntax: <%after>, <%around>, <%augment>, <%before>, and <%method>. (Why we “need” all these explicit sections is a subject for another post.)

Now here’s an implementation snag. Notice that neither /product/sales/display nor /product/sales/Base.pm have a render method. For the former, we want to automatically create a render that just calls main, which will display the main body of the component. For the latter, we want render to just fall through to the next level. But there’s no way to know in advance whether a component will be used as the page component or as one of its superclasses, so we have to create a default render for every component that does the right thing at runtime. In pseudo-code:

if (we are a superclass of the page component) {
    # Go to the next level.
    inner();
} else {
    # We are the page component. Draw the main content.
    $self->main();
}

Unfortunately there’s no official way to ask Moose whether we’re at the bottom of the inner() chain. So I’ve got a couple of choices here:

1. Check ref($self):

if ( ref($self) ne __PACKAGE__ ) {
    inner();
} else {
    $self->main();
}

As long as we provide a default render for every component, I think this should correctly distinguish the bottom of the chain. But I’m not entirely sure – Moose does a lot of with auto-generated subclasses.

2. Steal the code out of Moose::inner:

if ( $Moose::INNER_BODY ) {
    inner();
} else {
    $self->main();
}

Of course this is awful, as it relies on implementation details that are subject to change.

For now I’ll be going with #1.

Dec 14

In Mason 1, extending Mason’s behavior means either using the limited plugin API (allowing you to take action before and after a request or component call), or else writing subclasses, which is problematic when you try to use more than one together.

In Mason 2, plugins are based on Moose roles, allowing you to modify behavior in each of Mason’s main subclasses — Interp, Request, Compiler, Compilation, and Component — in a composable way.

Here’s my first Mason 2 plugin, to perltidy object files after components are compiled – useful in development. First, a nominal base class:

package Mason::Plugin::TidyObjectFiles;
use Moose;
extends 'Mason::Plugin';

1;

(This has to exist for Mason to recognize the plugin, but there isn’t much to do there right now.)

Then, a role for each of Mason’s subclasses that we want to modify. In this case we only need to modify Mason::Compiler:

package Mason::Plugin::TidyObjectFiles::Compiler;
use Moose::Role;
use Perl::Tidy;
use strict;
use warnings;

has 'tidy_options' => ( is => 'ro' );

around 'write_object_file' => sub {
    my ( $orig, $self, $object_file, $object_contents ) = @_;

    my $argv = $self->tidy_options || '';
    my $source = $object_contents;
    Perl::Tidy::perltidy(
        'perltidyrc' => '/dev/null',
        source       => $source,
        destination  => $object_contents,
        argv         => $argv
    );
    $self->$orig( $object_file, $object_contents );
};

1;

To use this, I simply list ‘TidyObjectFiles’ in my plugins:

my $mason = Mason->new(..., plugins=>[ 'TidyObjectFiles' ]);

or, if I want to specify perltidy options:

my $mason = Mason->new(..., plugins=>[ 'TidyObjectFiles' ], tidy_options => '-noll -l=100');

The Mason::Plugin:: prefix is automatically added to plugin names. You can also specify a full path with a leading ‘+’, e.g.

plugins => ['+MyApp::Plugin::MyPlugin']

Now my object files go from looking like this:

no warnings 'redefine';
sub _comp_info { return {comp_dir_path => '/',comp_is_external => 1,comp_path => '/hi.m'} }
sub main {
my $self = shift;
my $m = $self->m;

my $_buffer = $m->current_buffer;
 #line 1 "/Users/swartz/git/mason.git/tmp/comps/hi.m"
$$_buffer .= 'Hi there! The time is ';
 #line 1 "/Users/swartz/git/mason.git/tmp/comps/hi.m"
{ $$_buffer .=  scalar(localtime)  if defined( scalar(localtime) ) }
 #line 1 "/Users/swartz/git/mason.git/tmp/comps/hi.m"
$$_buffer .= '.
';


;return;
}

to this:

no warnings 'redefine';

sub _comp_info {
    return {
        comp_dir_path    => '/',
        comp_is_external => 1,
        comp_path        => '/hi.m'
    };
}

sub main {
    my $self = shift;
    my $m    = $self->m;

    my $_buffer = $m->current_buffer;

    #line 1 "/Users/swartz/git/mason.git/tmp/comps/hi.m"
    $$_buffer .= 'Hi there! The time is ';

    #line 1 "/Users/swartz/git/mason.git/tmp/comps/hi.m"
    { $$_buffer .= scalar(localtime) if defined( scalar(localtime) ) }

    #line 1 "/Users/swartz/git/mason.git/tmp/comps/hi.m"
    $$_buffer .= '.
';

    return;
}
Dec 12

In Mason 1 you can declare component arguments and then refer to them as normal lexically scoped variables:

<%args>
$address
$city
$name
</%args>

Dear <% $name %>: We will come to your house at <% $address %> in the
fair city of <% $city> to deliver your prize!

In Mason 2, components are classes rather than subroutines, so we have attributes rather than arguments. When you call a component, you create an instance of the component class and pass attributes to the constructor.

Attributes have several advantages over arguments: you can refer to them in any of the component’s methods (unlike in Mason 1, where arguments have to be awkwardly passed to each method) and you can take advantage of Moose’s powerful attribute declarations.

Here’s how this looks without any special syntax:

<%class>
has 'address' => (is => 'ro');
has 'city'    => (is => 'ro');
has 'name'    => (is => 'ro');
</%class>

Dear <% $self->name %>: We will come to your house at <% $self->address %> in the fair city of <% $self->city> to deliver your prize!

Clearly, we’ll need to pour some sugar on this.

Declaring attributes

Mason 2 will provide an <%attr> section, analagous to Mason 1′s <%args>:

<%attr>
a
b => 'foo'
c => (isa => 'Str', default => 'something')
</%attr>

This illustrates the three kinds of <%attr> declarations:

  • A name by itself: required attribute.
  • A name followed by => and a non-paren: optional attribute with a default.
  • A name followed by => a paren: specifies Moose attribute options.

You can mix and match these attributes with standard has declarations. The above is equivalent to:

<%class>
has 'a' => (required => 1);
has 'b' => (default => 'foo');
has 'c' => (isa => 'Str', default => 'something');
</%class>

All attributes will be read-write by default, via MooseX::HasDefaults::RW. Although read-write attributes are sometimes discouraged, I believe they’re appropriate and convenient for these short-lived component objects that are not generally expected to be accessed outside of their class.

Accessing attributes

To make the frequent access of attributes more convenient, we resort to some limited source filtering evil and implement Perl 6ish $. notation. Specifically, all occurrances of

$.foo

within Perl sections will be replaced with

$self->foo

where foo is any valid attribute name ([A-Za-z_]w*).

Note that with read-write lvalue attributes, this could even be used for assignment:

$.foo = 5;

Very convenient for attributes which exist chiefly for sharing data between methods.

The $. substitution feature will be made easy to disable, for those who are against it.

Putting it all together

The first example above with attribute declaration and access sugar:

<%attr>
address
city
name
</%attr>

Dear <% $.name %>: We will come to your house at <% $.address %> in the
fair city of <% $.city> to deliver your prize!
Nov 26

I’ve been meaning to try Method::Signatures::Simple ever since the last YAPC and just finally found the time. Something about typing “my ($self) = @_” for the millionth time pushing me over the edge.

Converted a random module in Mason 2. Reduced the number of lines by 65 (10%) and I feel like I’m developing in a modern language again. I’m hooked! Hats off to Rhesa Rozendaal and those responsible for the underlying Devel::Declare.

Now I’m eager to convert Mason 2 and possibly CHI to this across the board. The question is, is it reasonable for me to use this in published CPAN modules? It seems like very few CPAN modules do, and given how clearly awesome it is :) , I’m wondering if there is a reason, other than not wanting the extra dependency or not wanting to depend on “black magic”.

One snag is that perltidy (which I use rigorously on commit) doesn’t yet recognize the ‘method’ keyword. I managed to fix by patching Perl::Tidy to allow arbitrary transformations before and after tidying – so I can replace ‘method’ with ‘sub’ beforehand and convert back afterwards. Will attempt to contribute this back.

preload preload preload