WWW::Mechanize

A good target for declarative coverage.

Data::Match

This might be a decent place to start with some kind of declarative pattern-matching ... thing.

Lisp in Perl

Makes you wonder why you couldn't just write a Lisp that would "compile" to Perl. It would be instructive and fun.

Edit: it's been done in Ruby.

Edit 2: a serendipitous link to the same in Python. Freaky.

What makes Lisp great?

Of all the languages out there, Lisp is the one that seems to have the most proponents who sound believable. Occasionally I think I should learn some of it - then I bounce off what to me are always the stopping points: I have a hard time with the radically different vocabulary, I don't like capital letters, I miss CPAN like childhood innocence, and all this - pathetically - is enough to stop me.

So I decided to check what other people think makes Lisp great, and put that into Class::Declarative if possible. But when you get down to it, Perl already does a lot of what makes Lisp great, it turns out. (The closure epiphany I had in January is what got me started on this path in the first place, after all.)

Paul Graham lists nine new ideas that Lisp embodied: conditionals, first-class functions, recursion, dynamic variables, garbage collection, programs as expressions (i.e. functional programming), a symbol type, a code notation that is a tree of symbols, and the whole language available at all times. Of those, the first three or four are now universal, and the first five unambiguously part of Perl. The sixth is mostly covered by Perl and can be simulated with anonymous subroutines in cases where it's not covered.

That leaves the symbol type, a code notation that is accessible to the program, and the whole language available at all times. I'm not terribly interested in the symbol type, because it's a performance issue (testing for equality using the symbol handles instead of checking string contents); I'm really interested in what makes Lisp more expressive than other languages, at least for those versed in it.

Moving on to several sources, "Lisp is a programmable language", meaning that Lisp can easily write Lisp code in order to provide higher-level semantics for a given domain. I think this is getting closer to the crux of the matter. Due to Lisp's minimal syntax, its control structures are all functions, and so you can easily extend the language to suit your domain. That, plus Lisp's interactive nature - the way a Lisper effectively enters into a dialog with the language while evolving new semantics - make Lisp unique, or at least partly unique. (Here is another good presentation of this notion.)

Python has considerable interactivity, of course, and the introspection that Lispers find so useful. But I've never had much luck with that mode. I find it far more instructive to break things down into unit tests in the CPAN module paradigm, and work out semantics that way - although I can certainly see how an interactive data inspection facility would really speed the process in many cases (actually, it would make some things possible that aren't any other way.)

Perl is supposed to make easy things easy, and hard things possible. An interactive data facility would be a great addition to the language. Well - I suppose the Perl debugger already does this, to a certain extent, but the Perl debugger has always made my brain hurt.

The Lisp macro system and quasiquotation are powerful facilities making it easy to extend the structure of the language; a Lisp macro is a program that writes other programs that perform the ultimate tasks. Class::Declarative is most definitely moving in that direction; my macro system will be equivalent to Lisp's (I think) and nearly as terse as quasiquotation. More on that when I've got something working.

I'm left with the following chief advantages of Lisp: introspection at all levels, and interactivity allowing multiple approaches to new semantics. And I'm forced to admit that at the moment, Class::Declarative is not doing this - but could, if it grows more in the direction of semantic programming.

On paper, I've been exploring some possible approaches to realistic semantic programming; it appears that the key insight is recognition. That is, matching. If I posit a given structure for the world, then allow a matching engine to tell me how the world can be made to match that structure, then I've parsed the world, or recognized some structure in it. That's really what semantics is.

Here's an example. I want to write an invoice. I create an "invoice" structure. The Lexicon looks up what an invoice is, and fills in some structure as follows:

invoice
 customer {{index customer}}
 data items (description, price, unit, subtotal)
 {{.assert count(items) > 0}}
 total "{{select sum(price) from items}}"
 currency "{{index currency default USD}}"
 {{.section comments}}
 comments
    {{@}}
 {{.end}}

This is still pretty crude, but effectively we can scan the template to see whether all the information is available that we need for a full invoice. Some sort of interactive process will allow me to fill in what's missing, and some other specification not shown here will allow the invoice to be stored. That part's probably some sort of workflow environment. Once defined, the invoice can be expressed using a template to produce PDF or a Word file, and attached to an email. These actions are certainly workflow.

The point is that this is semantic programming - I define an invoice, the environment knows what an invoice is, and we interact to define the case. It's fuzzy in my mind, but I'm getting there.

Note that some of the fields in this example (the index fields) are expectations that prevent the object from being fully specified until they are fulfilled. The same applies to the assertion. However, the select field is simply a calculated field that doesn't require - in fact, doesn't support - an interactive assignment of value.

That calculated field gives us something like the capabilities of Excel, by the way. It would be instructive to be able to build a full spreadsheet in a system like this.

Why Perl?

Occasionally I'm beset by the worry that I've chosen Perl as the basis for all this development. Wouldn't some other language be a better choice?

All I can say in my defense is that my brain seems to work better with Perl for some reason. It's not that I've done a great deal of work in Perl - I've written more lines of C and Tcl. It's not that I have no exposure to other languages - I've worked in Python and Scheme. And while I enjoy the joke that Perl is a write-only language as well as the next programmer, still - when I look at my own code written in Perl, I can read it more easily than my own code, let alone that of others, in other languages.

Then there's CPAN. Oh, sure, half the stuff on CPAN is alpha-quality just-off-the-ground stuff, but oddly enough, it's usually enough to get me off the ground on a given project. Then, after I've got something halfway done, I can rewrite the CPAN modules that weren't what I wanted, and get the rest of the way.

At the end of the day, all programming is about semantics. You are taking the inchoate space of possible meaning, and narrowing it down to the meaning that gets you where you want to go. I find it easy to do that in Perl, for whatever reason.

Rebuild complete

I've been doing some kind of neat stuff with unit testing in the Wx::Declarative domain, although timing makes the tests kind of brittle - sometimes you're testing a condition before everything in the tested process has set things up, so you always have to remember to wait a little bit for applications to start, stop, etc. But Wx is working again! So there's that.

And I've been working a little more with it. It occurs to me that you could set up a runtime modification thing using the Wx facilities pretty easily - that and a command line and runtime object editor, and ... you got a lot of what makes Lisp so cool, but in Perl.

Food for thought.

Score one for unit testing

I'll tell you, having extensive unit tests written that the system already passed before this major overhaul makes me ... let's put it this way. It makes it possible to do refactoring like this. I would definitely have lost hope without my unit tests, and this project would be another dead one. Instead, I'm taking the old unit tests, dropping them into the directory, and letting the unit tests quickly and easily find the places in the code that need attention.

I've almost got everything working again. Incroyable!

Parsing works!

OK, so my parsing unit test passes 139 individual tests having to do with all aspects of tokenization, parsing, and building of nodes and nodal structure. I'm independent of Parse::RecDescent and Parse::Indented now, and I'm entirely using Class::Declarative::Parser objects to parse everything.

I still have a little work to adapt the existing classes to their new non-XML::xmlapi existence, but that seems to be going well. So in a couple of days, I guess I'll be starting templates, with a much more satisfactory parse-and-build regime going into the exercise.

Progress!