Dada Engine

OK, so back in the 90's, Andrew Bulhak came up with a snazzy little engine that interprets a grammar to generate random text. Well, we've all done that, of course, (here's a Tcl translation of one grammar) but his is implemented as an interpreter that takes the grammar as a specification language, and it can use troff to generate some pretty nice output. Like this, famously. And more recently, spam, apparently, which explains a lot. Here's a page that can run it on any grammar you like. (Incidentally, a Google search on "Dada Engine" turns up a paid ad for a code generator.)

So the engine itself is kind of boring, although he's put some really nice features into it. What really gets me going is thinking about that grammar specification language. I know this is Not a New Idea, but his sentence or phrase patterns are templates - syntactic units expressing semantic units, getting back to my Langacker days - that bear research. What he's missing is the semantic pole, although the context his grammar carries along is something in that direction.

What would be interesting would be something that could mine the Web for such patterns. And a statistical analysis of their use and interrelatedness to produce some kind of indication of voice/register for the text. That kind of thing. Not to mention a compact set of "typical error message" patterns, etc. for practical text generation in software. The point being that I think the pattern-based approach could work for both text analysis and text generation (not that this is a new idea).

This is back to my notion of the Lexicon, last looked at seriously in 2005. I really have to take a sabbatical from this damned having-to-earn-money thing.

Hassle maps

Here's a PDF article about "hassle maps". This is essentially a way to think about user experiences in a typical task in the day, describing a narrative and identifying hassles in it - pain points - so that a solution can be evolved to remove them. It's worth a read.

Text generation porn

Oh man, the NYT has a thinly-veiled PR piece today about Narrative Science, a company that spins out stories automatically based on whatever data feed you've got. It makes me drool because I want to do that. Obligatory HNN reaction. (With mention of a competitor.)

Ifttt.com: Neat automation service

Ifttt.com: lets you build little web robots that update every 15 minutes and can take API actions. Pretty neat!

Video talk: You're stealing it wrong

I am really not normally one for video (it's inherently tl;dr to me), but this presentation by tech historian Jason Scott at this year's Defcon was quite entertaining. And at the very end, there is a list of URLs of sites in which Scott has had influence, to wit:

• Textfiles.com - a vast archive of text files from yesteryear. There's some great stuff in there!
• Archiveteam.org - a group into archiving stuff before it evaporates. They archived Geocities.
• And some other stuff: two documentaries, Cow.net, and Welcometointernet.org.

Scott seems like a guy after my own tastes. We shall watch his career with great interest.

Nephtali: functional PHP Web framework

Interesting. I took some time to read through things and get my head around it - I think its abstraction is a little too low-level for my taste, but it's a fascinating project.

Free Rails 3.0 tutorial

Here, on Github. I like this trend towards githubbing of documentation like this.

NLP parsing tutorial

A short tutorial on the use of some tools for NLP. [pdf] I really want to start developing a toolbox for NLP but need some direct motivation to do so. (Although I suspect the motivation would crop up as soon as I had a toolbox I could trust.)

ClueWeb09 - NLP database

The ClueWeb09 database is a set of a billion Web pages harvested in 10 languages in 2009 for research in natural language programming. You can self-host if you buy a 2T hard drive for $600, or you can use their APIs to do your research.

The license essentially requires you to not to republish content, and to delete anything anybody asks them to delete from the repository, both of which make eminent sense.

Self-education in Web design

Good, if short, article on where to start with learning Web design. It's always nice to find this kind of summary article.

Patterns For Large-Scale JavaScript Application Architecture

Well-thought-out article about architectural choices in large JS applications. Easily extended to any app, of course - all-around good writing.

Dist::Zilla

Wow. I never heard of this before - it's good for me to work on Padre!

ORMs don't kill databases - people do

A cogent article about how wrapping your SQL in an ORM can get you quick returns - until it does something boneheaded. Better to isolate your SQL in a transparent manner like I'm planning, right? Right.

SURBL

Interesting: a database of URLs/domains that have been spammed.

D3 (data-driven documents) tutorial

Coooool. [here]

NPSML: Naval Postgraduate School Machine Learning Library

Worth looking at some slow day. [docs]

Qt

Another GUI framework I've never taken the time to look at closely: Qt. What drew my attention was this blog post. I'm not the only one who wants to streamline semantics for UI descriptions.

Padre integration

Well, Padre is really incredibly extensible - except for one thing (I'm going to have to patch my current copy); you can't add a MIME type/file extension mapping from a plugin, which is very disappointing. So here's basically what I'm doing:

• Add .dpl => application/x-decl to EXT_MIME in Padre::MimeTypes.pm (patch)
• Create a Padre::Plugin::Decl.pm
• There, registered_documents can assign x-decl to Padre::Document::Decl.pm
• Create Padre::Document::Decl.pm

The Plugin module handles general UI stuff (menus, About, special tools, etc.) while the Document module customizes the editor's behavior to suit you. I'm particularly interested in folding - but so far the documentation is really chaotic.

I've got about ten windows open on this task:

• Gábor's overview of wxPerl and wxWidgets
• Somebody's cookbooks for plugins. I find these cookbooks essentially impenetrable.
• Padre::Plugin, Padre::Document, Padre::Document::Perl, Perl::MimeTypes
• Another how-to on plugins

Also, I had some initial success with creating a new document class and doing some rudimentary folding in it - it was harder than hell to find documentation, though. It's scattered all around the Web. Here's some links that came in handy when dealing with folding in Scintilla/wxStyledTextControl:

• Somewhat random access, but here is some POD for the wxScintilla control.
• Here's some guy doing folding against Scintilla in a language I haven't even identified.
• Otherwise, there's loads of stuff written for/about wxPython that works pretty well.

Decl integration with Padre; musings about actions

Something I haven't spent much actual development time on yet, but ... it's starting to seem that the natural working mode of Decl will be to state that an action will be necessary, then to invoke various interactive services to specify that action more specifically.

So for example, if I have an invoice to issue (yes, yes, yawn) I'd start by just stating, "invoice" and passing that to ... some kind of action-consuming agent. The agent looks at this, sees that it is underspecified, and starts a conversation with me to determine just what it is I expect to do.

Once the action is fully specified (a customer has been selected, one or more outstanding jobs selected, and so on), then I can say, "Do it", and it is done, logged, whatever.

So what interfaces are available to me? Well, let's assume I want to be able to see the structure as it is built: that structure should be in Padre. With Padre::Plugin and Padre::Document, I can do that. Really easily, actually. The documentation sucks incredibly because it's not exactly a mature environment yet - but it's a great design.

So I could easily imagine right-clicking on anything in a Decl tree and getting immediate expert tools to deal with it: set up a database, select boilerplate, etc. It should end up going really fast. I'm looking forward to exploring it.

Python Web microframework BATTLE!

Very nice comparative lineup of about ten different Python Web microframeworks.