Kohonen maps in Perl

Just so I don't forget it, there's a Kohonen map implementation on CPAN.

I once wrote a rather nifty implementation of Kohonen maps in Visual Basic (it was another country, and besides, the wench is dead), but lost it years ago. Nice graphical demo, too. Always a shame when software gets lost.

Web development in Perl these days

As you know, Bob, it's been a while since I did active Web development work. I've been interviewing, though, and man, Webdev in Perl has moved on a lot since I last looked at it! Which was maybe, you know, over a decade ago - so I'm not sure why I'm surprised.

Anyway, the CGI::Application ecosystem is a pretty darned declarative framework for Web application development, and I'm totally hooked. I did a little programming test for one company, and here's the list of modules touched, just for later reference:

• CGI, of course. But it's largely invisible, being wrapped in
• CGI::Application.
• I used CGI::Application::Server for testing.
• HTML::Template was used for formatting.
• Data::FormValidator was the core of the application in this case (just a little contact form)
• Finally, MIME::Lite did the mail handling.
• And there was a database insert, of course. A nice boilerplate page, because I can never remember the details.

This little application took me a total of maybe 3 hours, with preparation (and preparation failure on a couple of platforms) and learning the new tools. The actual coding could probably be done in less than an hour if you knew the modules.

Why should it even take that long? Why don't we already have a central data dictionary with validation rules and so forth? That's another to-do for Decl.

Abhijit Mahabal strikes again

Abhijit has been working on a Python FARG framework - something I once wanted to do (OK, granted: something I still want to do, but now he's done it, or started one). It's here on Github.

Twitter open-sources MySQL work

Twitter has done a lot of work on MySQL scalability, now open-sourced.

Casual programming

Here's a very thought-provoking post, generally about the notion of "casual programming", i.e. a language/IDE/whatever that would (as I phrase it) work with you at a semantic level to co-write the software you're working on. It would do this by having context-sensitive assistance for API inclusion. This is a pretty good approach.

WebPutty open-sourced

WebPutty is a gallery site for CSS that runs on GAE. FogCreek just open-sourced it.

Programming is hard - let's go scripting

Some musings by famed linguist Larry Wall about the dimensions of variability of programming languages, including a couple of obscure ones he had a hand in influencing. Very interesting article, naturally.

Making easy things easy with node.js

Here's a nice little article noting that true popularity for a platform comes from how easy it is to build form-based database applications: consider Visual Basic, then PHP, then Rails. The idea is that making easy things easy lets a large community with varying skill levels use a technology, and that creates a dynamic environment that lets you get things done.

The author kind of ends on the note that he hopes this will happen for node.js.

Frustration with statistical methods

The "custom language model" section of HW2 is impossible. I implemented a hacked Kneser-Ney smoothing (I think perhaps the corpus is too small, but the problem was lots of zeroes), a trigram-to-bigram backoff, and linear interpolation, and all failed. Linear interpolation of trigrams, bigrams, and unigrams failed miserably.

I dunno. Clearly people are making these things work, but I get the impression that mostly it's throwing a bucket of tacks at the problem and hoping something will stick.

Update: some perusing of the forum led me to the realization that I wasn't testing trigrams correctly. Looking at only the first trigram in a sentence gave me "[s] One two", so that any spelling error in "One" would be lost. Once that was fixed, trigram double-backoff worked as well as bigram backoff. A little twiddling with the backoff coefficients got me slightly better performance than my original bigram backoff with a 0.4.

Moral of the story: the choice of backoff coefficient makes a difference. Which is why I hate statistical approaches.

NLP homework II

You'd think spelling correction would be old hat, but you know what? I'm starting to get kind of excited about the idea of a spelling corrector that's trained on my actual typing, knows what language I'm translating from, and knows what customer I'm translating for. I think it will catch a whole lot more errors...

In other news, I took the plunge and signed up for Caltech's ML class. I hope I'm going to have time for it.

That's what she really said

Geek chick Jessamyn of MeFi fame wrote a Python bot to respond to "That's what SHE said" on her work IRC server. (Given that somebody had already written a TWSS bot to say that whenever even vaguely a propos.) Did a nice writeup, too.

This is nice, by the way - it's a great example of social engineering through symbolic computing. We'll see more and more of this kind of thing as we get better with handling NLP.

Domain: networking

So lately I've been delving into programming my router. I lend bandwidth to a neighbor, but they tend to watch Netflix when I want to watch Netflix on the Wii, and the Wii is not a happy camper when it comes to sharing bandwidth.

I run DD-WRT on my router, one of several open-source router firmware products out there these days. That means my router is a Unix box. That means that - given that it runs iptables - technically, I could make it do nearly anything, from logging bandwidth to choking it by specific MAC or even by MAC-and-port combination. Or at least I could do that if I truly understood iptables and networking, which, when you get down to brass tacks, I don't.

For example, there is a dandy little embryonic shell script wrtbwmon.sh that munges your iptables to log bandwidth by MAC address. (Here is a forum post by its author as he was trying to get it working, and here's his announcement on the DD-WRT forum.) It's slick, but as I have a much more capable Unix machine sitting right next to my router, what I'd really like to do is have that machine grab stats on a regular basis, more or less keeping the router running as a ganglion.

OK, so link dump out of the way, here's why I find this relevant to semantics. Note that I said up there that if I understood networking I could make iptables do some cool stuff. What does it really mean to understand networking? Clearly, it means to have a mental model of the network and the router and all the utilities used to work with the network, how packets get handled by iptables, and so on. I have a vague idea of all these, and given patience I can usually manage to get networks running or fix them, but I can hardly call that understanding (although, when you think about it, the notion of a "vague idea" is already pretty intriguing from the standpoint of modeling - after all, if you could give a computer system a "vague idea" you'd already be 90% of the way to intelligence; arguably, vagueness of ideas is the most human ability there is...)

So the domain of networking would have these concepts in some kind of Lexicon (borrowing from my Hoftstadter days), along with utility calls for diagnostics, and a library of C code snippets for writing your own special-case utilities. That sort of thing. I really do have some notion of where I'm going; I wish I could characterize it better.

Inline and pexports

So I have Inline actually compiling my C wrapper for ImageMagick - but since pexports 0.43 crashes under Windows 7 with the 64-bit ImageMagick DLLs, I was stymied for a while when it came time to link. Turns out it's been fixed in 0.44, but finding the fixed version took me a bit of research.

• The R people figured this bit out for me
• Turns out PerlMagick should work as well, now that I've got pexports fixed, but now I'm having too much fun to stop. Inline rocks.
• Wasn't Inline on track to be a standard module? It isn't.

As long as I'm posting on Image::Magick::Wand, I should mention two links about dealing with hashes from inside C. [1] [perlguts]

Devel::Declare

Here's a little module Dave Rolsky turned me onto - I'm going to have to read it a couple of times, but essentially it takes control of the Perl parser by arcane means and lets you define your own syntax.

I love the idea of getting away from rickety code generation and into rickety on-the-fly code munging, but the real value would be letting the Perl parser do what it does best; trying to fake it with line-by-line regexps is just asking for trouble.

Porter stemming algorithm

Just for completeness, here's a link to the Porter stemming algorithm for English. It's another hash of regexps and special cases, but it's standard and does a fair-to-middling job.

NLP class assignment 1

The first assignment for the Coursera/Stanford NLP class was a regexp approach to finding email addresses and phone numbers in a set of faculty pages from the Stanford CS department.

I ended up doing a first pass with a list of regexps, then doing some post-processing afterwards. As is always the case, to hit all their test cases, a lot of fiddly special-casing had to be done, the most irritating being Jurafsky's own email address, which consists of a JavaScript call. As I had no particular intention to embed Spidermonkey in the Python code, this had to be entirely special-cased, although a sane system design would have put this into the page retrieval spider, not in a regexp-based recognizer.

But that's neither here nor there (just low-level irritation) - the real point of the assignment was of course both to provide a little exposure to regular expressions while making it clear that natural language is utterly rife with special cases, and it succeeded on both counts.

It led me to note that splitting the logic between regular expressions and a separate post-processing step was irritating from the point of view of code understandability and maintainability. Probably it would be better to have a more powerful text processing language, perhaps explicitly based on a parser. And it should work from a real tokenizer to eliminate all possible HTML obfuscation - for example, the simple trick of having DIVs break up your email address would have made this code useless. (Which is why it irritated me to have the JavaScript instance in there.)

Anyway, it was fun.

A little Word scripting

As you may know, I'm a technical translator by trade these days, so I work under Windows with Word documents a lot. Word has a scripting mechanism that I've often considered to be Microsoft's way of making it technically possible to program whatever you need in Word, without actually making it sensibly easy (making hard things possible and making easy things hard, thus ensuring grist for a robust consulting mill), but back in a previous life I spent a lot of paid time writing Word macros to do text processing. Profiting from that grist, I suppose.

Anyway, this week I have a huge pile of process definitions to be placed into Word, each consisting of a table on each page with a list of steps in a process to be carried out manually (many with barcodes for acknowledgement and so on). Each step has a unique identifier - by which I mean that each type of step has a unique identifier and there can be any number of tokens of each type of step.

And it's all in German and my job is to translate it to English. And the originals are all scanned PDFs, so I can't actually just use a regular translation tool to find repetitive text, or even cut and paste into Word; it all has to be typed. I do have properly formatted blank tables to start with for each document.

What I've ended up doing is putting each type of text into a table in a dictionary document, then writing a macro that looks up the unique ID I've typed into the given target document's first column and, if it's found, copies the already-translated row into the target document. If that type hasn't been encountered yet, then I translate it, and copy the row into the dictionary by hand.

I'm writing about it here, because at a semantic level that Word dictionary document is a database (a NoSQL database, if you want to be all cool-kids about it). If this macro were expressed in Decl, it would be much easier to get a lot of implications of that construal and work them into new ramifications.

In other words, construing one thing as another (a semantic mapping) is a form of recognition that could explicitly underlie the programming process in a semantic language. Also, I think I like the word "construal" as a technical term for this kind of mapping.

Caltech ML

So I didn't manage to keep up with Stanford's ML class in November/December - but Caltech will be giving me another shot in April. Maybe I'll participate! (Although if I have to decide between NLP and ML, you know NLP will win.)

Incidentally, the lectures from Ng's full CS229 treatment of machine learning, with all the math left in, are all online as well. I wish his homework material was also online.

Plucene

I didn't know this, but there's a pretty well-developed Lucene port on CPAN, Plucene. Sadly, development seems to have run aground in 2006; there are bugs reported (even including posted patches!) but nobody seems to be minding the store. I don't know to what extent it would be useful to revitalize it.

NLP class

The NLP class started today on Coursera. So far, it's easy (but so far, I already essentially know what they're telling me). It's still been valuable to go over it in a coherent manner. This whole "take derivatives of Ivy league classes for free" fad is fantastic. I hope people keep doing it.