Rendering with blocks in HTML::FormHandler

When rendering, FormHandler loops through the sorted fields in the form and executes the 'render' method on each field. Fields in FormHandler forms, particularly those that interface with a database, are usually structured in a way that matches the data structure. This doesn't always fit with the way that you want to display the form.

'Blocks' provide an alternative way of structuring the display. A 'block' is a fairly basic object that contains a 'render' method. The standard block class, HTML::FormHandler::Widget::Block, has Moose attributes to set the HTML tag, the label, the classes, etc, plus a 'render_list' which contains the names of a list of fields or other blocks to render.

You create a block with 'has_block':

has_block 'comment' => ( tag => 'p', content => 'This is a comment',
        class => ['comment'] );

Blocks can contain lists of fields:

has_block 'fset1' => ( tag => 'fieldset', render_list => ['foo', 'bar'] );

In order to use blocks when rendering instead of fields, you need to provide the form with a 'render_list' too (in addition to any render_lists that might exist in the blocks).

sub build_render_list { ['fset1', 'comment', 'a_field' 'submit_btn' ] }

A complete form that renders using blocks:

package MyApp::Test::Form;
    use HTML::FormHandler::Moose;
    extends 'HTML::FormHandler';

    sub build_render_list {['abc', 'def', 'ghi', 'submit_btn']}
    has_field 'foo';
    has_field 'bar';
    has_field 'flim';
    has_field 'flam';
    has_field 'fee';
    has_field 'fie';
    has_field 'fo';
    has_field 'fum';
    has_field 'submit_btn';
  

    has_block 'abc' => ( tag => 'fieldset', render_list => ['foo', 'fo'] );
    has_block 'def' => ( render_list => ['bar', 'fum'], label => "My DEF Block",
        label_class => ['block', 'label'] );
    has_block 'ghi' => ( render_list => ['flim', 'flam'], wrapper => 0 );

More flexible form rendering in Perl with FormHandler....

HTML::FormHandler result object

Sometime last year I re-structured HTML::FormHandler so that the input and values that are produced by the form validation process are stored in separate result objects. There is an alternate method 'run' (as opposed to the usual 'process' method) to get these objects, or you can get them from a form after 'process'.

I expected that those who were interested in using this would materialize and provide input into what additional facilities/hooks were needed, but as far as I can tell nobody has actually used the feature. I think that it's a fair assumption that nobody actually knows about it, or can't figure out the point.

The standard way to use FormHandler is (simplified) something like:

    my $form = MyApp::Form->new;
    $form->process( params => $params, item => $item, ...);
    if( $form->validated ) {
     ....
    }

When using the the result object, the form acts like a kind of factory, and it looks like this:

    my $form = MyApp::Form->new;
    my $result = $form->run( params => $parsms, item => $item, ... );
    if( $result->validated ) {
       ...
    }

...and all the state is stored in the result objects, ideally leaving none left in the form

Currently result objects refer back to some attributes of the form object for rendering, since a number of rendering related definitions are stored there. It would be possible to decouple that, but then you'd end up having to define the validation and rendering aspects of the fields in two places.

In practice, the separation between static (form definition) information and ephemeral data is not nearly as clearcut as you might think. I find that people often want to change attributes that would probably be initially defined as static based on values that are passed in. It is possible to do this in a clean way, of course. But since the previous results are always cleared out at the beginning of each 'process' call, in practice the difference between using the form with 'process' and using it with 'run' aren't huge.

Nonetheless, I am sure there are people who have good use cases for this alternative architecture, and would love to have people pound on it and use it.

creating a FormHandler form from a DBIx::Class result source

Someone regularly pops up in #formhandler looking for a way to automatically create a form from a DBIx::Class result source. After explaining how to create a fairly simple role that would do that, the people have always gone away and nothing ever came of it.

I have mixed feelings about the idea, myself. Sure, there's a theoretical benefit to not defining things in multiple places. But in practice, there's almost always differences in selection, presentation, etc, that mean that an automatically derived form is of surprisingly little use. At least in my experience. YMMV. It's awfully easy to take the list of column names and type "has_field 'column';"

But then the eleventeenth person asked about automatic form generation - and the yaks must have been particularly hairy this weekend...because I made a good start at shaving this particular one.

There is now a new trait: HTML::FormHandler::TraitFor::DBICFields, and a package to do the mapping from source info to field definitions: HTML::FormHandler::Model::DBIC::TypeMap. I'm pretty sure that it's not quite right yet, but at least a start has been made, and it should be easier for those that want this kind of functionality to polish, add, and tweak. For one thing, it doesn't handle relationships yet. FormHandler can create form fields out of a lot of the standard DBIx::Class relationships, yet it is often not what you would want to have happen automatically. So I think skipping relationships as a default is probably OK.

Anyway, here is how to create a form by looking at the source columns:

   my $book = $schema->resultset('Book')->find(1);
   my $form = HTML::FormHandler::Model::DBIC->new_with_traits( 
      traits => ['HTML::FormHandler::TraitFor::DBICFields'],
      field_list => [ 'submit' => 
          { type => 'Submit', value => 'Save', order => 99 } ],
      item => $book );

The field_list attribute is because I thought it was better to explicitly specify the submit field, rather than automatically create it. I could be argued out of it...

The interface is only a few hours old, so if you have thoughts about how this should work, this is the time to speak up, before somebody starts using it and it becomes harder to change. :-)

Speaking of dynamic forms... HTML::FormHandler is known for it's Moose-y sugar interface, but there are other ways to create a FormHandler form. Using a 'field_list' as a parameter to 'new' works fine to create a dynamic form (though there are things that are possible inside a class that just can't be done with that kind of interface):

my @select_options = ( {value => 1, label => 'One'}, 
       {value => 2, label => 'Two'}, {value => 3, label => 'Three'} );
my $args =  {
    name       => 'test',
    field_list => [
        'username' => {
            type  => 'Text',
            apply => [ { check => qr/^[0-9a-z]*/, message => 
                   'Contains invalid characters' } ],
        },
        'password' => {
            type => 'Password',
        },
        'a_number' => {
            type      => 'IntRange',
            range_min => 12,
            range_max => 291,
        },
        'a_select' => {
            type    => 'Select',
            options => \@select_options,
        },
        'sub' => {
            type => 'Compound',
        },
        'sub.user' => {
            type  => 'Text',
            apply => [ { check => qr/^[0-9a-z]*/, 
               message => 'Not a valid user' } ],
        },
        'sub.name' => {
            type  => 'Text',
            apply => [ { check => qr/^[0-9a-z]*/, 
                message => 'Not a valid name' } ],
        },
        'submit' => {
            type => 'Submit',
        },
    ]
};
my $form = HTML::FormHandler->new( %$args );

And of course you can use one of the rendering roles, HTML::FormHandler::Render::Simple, or the rendering widgets, or write your own renderer... Too Many Ways To Do Things (tm).

Where the cleaver falls...

Way back in the dark ages, I worked on one of the first projects at IBM to use that fancy new methodology, object oriented programming. They sent the whole team to a month of OO training, and then that wasn't quite enough to bring us up to speed, so they sent us to another month with a different traning company. I remember that one month used a well-regarded (at that time) book by Grady Booch. The whole object thing wasn't that hard to understand, but the thing that I couldn't quite wrap my mind around was how you figured out what to make the objects, how to split up the problem domain into pieces. So I eagerly flipped through the Booch book, thinking that they must have some words of wisdom on this point, and finally I found it. Buried somewhere in the middle they had one page with three paragraphs which basically said: and then a miracle occurs.

Pretty much the equivalent of the humorous saying somebody I know was fond of: Where the cleaver falls, there the chicken parts. (Maybe you had to be there...)

So I was working on a comment screen that needed a captcha. I had never used one before, but I figured it couldn't be that hard. HTML::FormHandler is a form processor, and a captcha is a form element, so it seemed obvious that a captcha field for HFH was in order. That part was easy... And then I realized that in order to check the value typed in by the user, the captcha had to be stored in the session. The session (in this case) is managed by a Catalyst plugin, and for the form field to be aware of the session just seems wrong. Too much mixing of areas of concern.

So a Moose role comes to the rescue. (Perl!) I created a Moose role that contained the captcha field, plus provided 'get_captcha' and 'set_captcha' methods. This seems relatively clean - or at least better than having the field mess with the session. Of course in order to access the session, the Catalyst context or session needs to be passed in to the form object. Not ideal, but not all that bad either. But now these methods are form methods... They could be turned into callbacks, or coderefs that are set on the field, but I've already had to deal with not-entirely-clean relationships between form and field before, so for now I decide to just call them as form methods. I don't really like this, because I'd prefer that the fields not know about the object that contains them, but it's easy and it works.

Then I realize that I'll need a URL for the image in the field. Damn. This is getting messy. I suppose that I can add yet another attribute 'captcha_url' and set THAT too. But then there's another required piece of code that's neither form nor field, but a controller action. Sigh. It's too many different pieces of code for one simple thing.

Oject oriented programming is no longer the new, shiny thing that it was so long ago, but the problem of splitting up the problem into nice little pieces is still with us. That OO training was for was an early attempt at an object oriented interface to a SQL database. It failed utterly. Looking back, I have to say that the problem wasn't fully understood - or at least we didn't understand it. It was a brave attempt, but a fair amount of knowledge about the problems associated with doing ORMs has grown up, and they're better these days.

So I guess progress is being made. In some places by some people..

Flexible extensions with Moose method modifiers

When constructing an API for a library intended to be subclassed by users, you run into the classic tension between power and simplicity. Too many ways to call your code, too many different methods and the user is overwhelmed. Yet if you don't have enough power, the user may by stymied when he reaches some level of complexity.

Moose method modifiers provide a lot of flexibility that doesn't necessarily have to be provided by an explicit API. In some cases they may provide stopgap ways of extending, in other cases they may actually be better than creating more complexity by providing more explicit hooks.

The main processing method for HTML::FormHandler is pretty straightforward:

sub process
{
   my $self = shift;

   $self->clear if $self->processed;
   $self->_setup_form(@_);
   $self->validate_form if $self->has_params;
   $self->update_model if $self->validated;
   $self->processed(1);
   return $self->validated;
}

The above method is called when a form is processed:

   my $form = MyApp::Form::User->new;
   $form->process( item => $user, params => $c->req->params );

The form is something like:

   package MyApp::Form::User;
   use HTML::FormHandler::Moose;
   extends 'HTML::FormHandler::Model::DBIC';

   has_field 'username' ( required => 1 );
   has_field 'some_attr';

A common need is to have particular fields be required in some circumstances and not in others. So maybe it would be nice to have some kind of callback to determine whether the field is required or not... But a simple Moose method modifier on one of the methods called in the 'process' routine will do the trick just fine:

   before 'validate_form' => sub {
      my $self = shift;
      $self->field('some_attr')->required(1)
         if( ...some condition... );
   };

Now maybe there's some magical API syntax that could achieve the same thing, but really this is pretty straightforward and easy. Maybe there's some additional database update that you want to do that's not directly related to the fields, such as recording that the user updated his record. Another Moose method modifier comes to the rescue:

   before 'update_model' => sub {
      shift->item->user_updated;
   };

...where 'user_updated' is a method on your DBIx::Class result source that sets a flag or an updated time or whatever you want. The same result could be achieved by subclassing the methods of course, but the method modifiers can also be used in Moose roles, making it possible to split up your form pieces into nice chunks that can be reused in multiple forms.

   package MyApp::Form::Options;
   use HTML::FormHandler::Moose::Role;

   has_field 'opt_in';
   has_field 'something_else';
   after 'validate' => sub {
      ...do some specific cross validation...
   };
   before 'update_model' => sub {
      ...some db processing...
   };

And then a form class could just be a collection of roles:

   package MyApp::Form::User;
   use HTML::FormHandler::Moose;
   extends 'HTML::FormHandler::Model::DBIC';
   with 'MyApp::Form::Options';
   with 'MyApp::Form::Login';
   1;

So the power and flexibility of HTML::FormHandler's API are increased without extra code simply by using Moose and its method modifiers. When I was originally hired to program in Perl I wasn't that happy about it, partly because the native Perl OO features are weak and kludgy. But with Moose that's a thing of the past. Perl + Moose are as good as or better than any other object oriented language I've worked with.

Validating structured data

In an earlier post I discussed how HTML form processing, when sufficiently generalized, leads to processing structured data (by which I mean data in Perl-ish hashes and lists). Here is an example of some structured data:

my $user_record = {
   username => 'Joe Blow',
   occupation => 'Programmer',
   tags => ['Perl', 'programming', 'Moose' ],
   employer => {
      name => 'TechTronix',
      country => 'Utopia',
   },
   options => {
      flags => {
         opt_in => 1,
         email => 0,
      },
      cc_cards => [
         {
            type => 'Visa',
            number => '4248999900001010',
         },
         {
            type => 'MasterCard',
            number => '4335992034971010',
         },
      ],
   },
   addresses => [
      {
         street => 'First Street',
         city => 'Prime City',
         country => 'Utopia',
         id => 0,
      },
      {
         street => 'Second Street',
         city => 'Secondary City',
         country => 'Graustark',
         id => 1,
      },
      {
         street => 'Third Street',
         city => 'Tertiary City',
         country => 'Atlantis',
         id => 2,
      }
   ]
};

Here is the HTML::FormHandler form that defines field validators to process that structure:

{
   package Structured::Form;
   use HTML::FormHandler::Moose;
   extends 'HTML::FormHandler';

   has_field 'username';
   has_field 'occupation';
   has_field 'tags' => ( type => 'Repeatable' );
   has_field 'tags.contains' => ( type => 'Text' );
   has_field 'employer' => ( type => 'Compound' );
   has_field 'employer.name';
   has_field 'employer.country';
   has_field 'options' => ( type => 'Compound' );
   has_field 'options.flags' => ( type => 'Compound' );
   has_field 'options.flags.opt_in' => ( type => 'Boolean' );
   has_field 'options.flags.email' => ( type => 'Boolean' );
   has_field 'options.cc_cards' => ( type => 'Repeatable' );
   has_field 'options.cc_cards.type';
   has_field 'options.cc_cards.number';
   has_field 'addresses' => ( type => 'Repeatable' );
   has_field 'addresses.street';
   has_field 'addresses.city';
   has_field 'addresses.country';
   has_field 'addresses.id';

}

The names of the fields are flattened references to the elements of the structure, with special field types for Repeatable and Compound elements. These types of structures can be used to update a database with DBIx::Class (although there are limits, of course).

I've left off the actual validators, but they can be defined pretty easily using Moose types or other constraints.

  has_field 'cc_type' => ( apply => [ CCType ] );

It would probably be better to define some of the fields in a role or field, to keep some of the related validation in the same place. The validation of the credit card numbers depend on the type of the credit card, for example. Error messages can be retrieved from an array of error fields or plain error messages, but there need to be more flexible ways of getting those messages. I'm not exactly sure what people will want yet.

Real Soon Now (tm) I'm going to work on a KiokuDB model... So many programming tasks, so little time.

Defining the form processing problem

This week I've been discussing the goals of a form processor and working on adding support to HTML::FormHandler for multiple rows. When I first started doing web programming and learning Perl 18 months ago, I looked for a module that did what I wanted, and thought "it can't be that hard". Once I got into it, the problem started to look more and more complex. You need to map different representations of data onto each other, each level having different kinds of relationships to each other, and yet maintain the accessibility of the information from multiple representations.

Let's start with a Perl-ish structure of hashes and lists:

{
   addresses => [
      {
         city => 'Middle City',
         country => 'Graustark',
         address_id => 1,
         street => '101 Main St',
      },
      {
         city => 'DownTown',
         country => 'Utopia',
         address_id => 2,
         street => '99 Elm St',
      },
      {
         city => 'Santa Lola',
         country => 'Grand Fenwick',
         address_id => 3,
         street => '1023 Side Ave',
      },
   ],
   'occupation' => 'management',
   'user_name' => 'jdoe',
}

This structure represents a user with a user name, an occupation, and an array of addresses. This example includes an array of hashrefs, because that's what I'm working on this week... The first problem is that this structure can't be directly represented in CGI/HTTP parameters, since they don't do nested hashrefs. So in order to get this structure into and out of an HTML form, we flatten it into a hashref with names that can be munged by something like CGI::Expand:

my $params = {
   'addresses.0.city' => 'Middle City',
   'addresses.0.country' => 'Graustark',
   'addresses.0.address_id' => 1,
   'addresses.0.street' => '101 Main St',
   'addresses.1.city' => 'DownTown',
   'addresses.1.country' => 'Utopia',
   'addresses.1.address_id' => 2,
   'addresses.1.street' => '99 Elm St',
   'addresses.2.city' => 'Santa Lola',
   'addresses.2.country' => 'Grand Fenwick',
   'addresses.2.address_id' => 3,
   'addresses.2.street' => '1023 Side Ave',
   'occupation' => 'management',
   'user_name' => 'jdoe',
};

A corollary of this is that the form processing program should be able to take in structures of either type, and output at least the flattened structure so that it can be used to fill in the form with current data. Then we consider where the initial data is going to come from. Often the data is in a database, so now we have the problem of taking a database object, like a 'user' row with a relationship pointing to a number of addresses, and convert it to the flat CGI hash. And we also want to go in the opposite direction, converting a flat CGI hash into a structure suitable for putting back into the database. The data in a database (or other data soruce) isn't necessarily in a form suitable for displaying as strings in an HTML form. So there are inflation and deflation steps. The database structure and data must be mapped to a CGI structure.

Then there's the question of how to define the validators which are the main purpose of this exercise. The data that's input from the parameters passed in must be validated (and/or inflated). If there are errors, the program has to present that information in such a way that an HTML form can be constructed with the errors presented to the user for correction.

There are a number of choices to be made about how to define the fields to allow these validations and conversions to happen in a simple and regular fashion. A common solution is to treat the nested elements as subforms, but they are not actually separate forms, they are simply ... nested elements.

The way that feels best to me is to allow the definition to be done in one 'form' class, which represents one HTML form.

   package HasMany::Form::User;
   use HTML::FormHandler::Moose;
   extends 'HTML::FormHandler::Model::DBIC';

   has_field 'user_name';
   has_field 'occupation';

   has_field 'addresses' => ( type => 'Repeatable' );
   has_field 'addresses.address_id' => ( type => 'PrimaryKey' );
   has_field 'addresses.street';
   has_field 'addresses.city';
   has_field 'addresses.country';

This flat representation matches the flatness of the HTML form. The field names with dots give information to allow the creation of nested elements. The field names are also related to the database object, where 'addresses' is the DBIC relationship accessor, and street/city/country are columns in the address table. In practice there would be more to these field definitions, since there would be validators associated with them, but I'll leave them out for now to simplify the problem.

Constructing the arrays is tricky. The form object doesn't know how many elements are in the array until it is handed the information from the database or the parameters from the form. So the arrays of address fields must be cloned from the fields that have been defined and put into some structure to hold the definitions and the data. There is a choice of structures here. We can either match the

 'addresses.1.country' => 'Utopia' 

format, or match the

 { addresses => []}

structure. These structures have different numbers of levels, since we have to add the ".1." level to indicate the array. It could be set up either way and mapped to the other. For the purposes of constructing HTML, however, you want to have some place to act as a container for an individual address so that it can be wrapped in a div, so the structure with the numbered level seems more useful. So now the 'HasMany' field container will create an array of field container objects (instances) that contain an address record.

Once constructed and filled, the nested fields can be accessed with

 $form->field('addresses')->field('1')->field('city')->value

or using the shortcut method

 $form->field('addresses.1.city')->value 

. There's something awkward about this, because it's oddly modal. The field structures are different depending on whether the form has been filled out with data or not. The implementation which I have working right now has an array of fields (the same as other non-has_many compound fields) which is cloned into subfields which are created on the fly. It would be possible, I suppose, to have a dummy subfield to contain the field definitions. I'll have to think about that one...

In order to interface with the database object in a regular, MVC-ish way, the form program should output structured data that can be saved by the database model. Inflations may be associated with this.

So in the end, it seems like what you end up with is program which will take structured data, process it and validate it, and return structured data. This is a much more general problem than it first appears when "all" you want to do is process an HTML form.

Compound fields in forms

Many Perl form packages require nested forms of some sort for compound fields, but there are problems with that model. A form class--whether constructed from config settings in a file or declared in a Perl class--has lots of other attributes besides the fields. It made more sense to me to have a form with nested fields, where the fields can form a tree. That way attributes of the form do not have to be repeated in sub-forms. It also allows declaring subfields in the containing form, if that's what makes sense for this particular case. Here's an example of how to do a form with a nested field in HTML::FormHandler:

   package MyApp::Form;
   use HTML::FormHandler::Moose;
   extends 'HTML::FormHandler';

   has_field 'address' => ( type => 'Compound' );
   has_field 'address.street';
   has_field 'address.city';
   has_field 'address.zip' => ( type => '+Zip' );
   1;

The subfields are indicated by prefacing the field name with the name of the compound field and a dot. This form class will create a form with an 'address' field that contains an array of fields.
You can achieve the same thing by creating an 'Address' field:

   package MyApp::Field::Address;
   use HTML::FormHandler::Moose;
   extends 'HTML::FormHandler::Field::Compound';

   has_field => 'street';
   has_field => 'city';
   has_field => 'zip' => ( type => '+Zip' );
   1;

(Where '+Zip' is a field class that you have written...) Then this field can be used in a form by simply doing:

   has_field 'addresss' => ( type => '+Address' );

This feels simple and straightforward and, for me at least, matches my conceptual model of a form better than nested forms.

HTML::FormHandler

When I started on a Catalyst project a year and a half ago I picked Form::Processor for form handling. I liked the architecture and I liked having the form in a Perl class so that I could do more-or-less anything that I wanted to with it. But then there was Moose, and I didn't want to have to switch back and forth between object systems. Plus you can do lots of shiny things with Moose. It's like a big Christmas present with lots of little parts that you can play with for ages...

So I converted Form::Processor to Moose. But it was tough to maintain back compatibility and still use all that Moose-y goodness. In the end it seemed rational to start fresh with a new project: HTML::FormHandler. It has many excellent features, such as being able to define the fields declaratively, and there are plans for many more. Moose continues to be a pleasure to work with. It makes features easy that would have been horrendously difficult without it. Of course, Reaction does wonderful things with Moose too, and is in many ways a more complete solution. But I think that HTML::FormHandler can still fill a niche. This is Perl after all--there has to be more than one way to do things.