Autoassociative memory

Back in, oh, the 90's sometime, I had the entire summer off and we were in Budapest. During that time I implemented a Kohonen network in Visual Basic (and invented Gray codes while I was doing it, because I wanted neighboring numbers to differ by only one bit - only much later did I discover they had a name before I was born), and successfully saw it retrieve geometrical shapes given a noisy input.

My ultimate idea was to implement some kind of "cognitive stub" that would be a (long) vector that somehow encoded a "semantic flavor" of a given semantic structure. Put those into an autoassociative memory and you've got something that kind of feels like human memory.

I still think it's a good idea, but there have been a lot of higher-priority things, like getting the kids through school and making sure I can retire before I'm 98. Also, sometime between then and now I seem to have entirely lost that code, which sucks, actually, because I really hate data loss.

But you know, I just ran across a reference to autoassociative memories (a password reminder, odd application but there you go). That author uses a discrete Hopfield network library of his own devising, but I wanted a better overview of the field, like this here.

Maybe I'll dabble with these things soon. Like this winter.

LISP macros on JavaScript ASTs

Oh, another project that's all kinds of cool: eslisp is an S-syntax for JavaScript parse trees, with a macro expander that treats any JavaScript function as a perfectly valid macro. That is just brilliant!

Spidermonkey exposes the AST as an API, incidentally. This project makes use of that.

PlantUML

Here's a UML diagrammer driven by DSLs. Ran across it linked from the description of another project by its author that used it to document his class diagram. Slick!

Compressing word lists into useful DAWGs

Cool data structure article.

NYT extracts recipe data from natural language

Very, very cool. They used CRF.

Selenium

Remember how I said my desktop machine is now running Linux? Yeah, I don't have to worry about getting Selenium to work under Windows any more if I want to automate browser tasks. And I do, yes I do, because the browser is one basic way business tasks are presented.

Selenium: http://www.seleniumhq.org/download/
Selenium client in Perl: https://metacpan.org/release/Test-WWW-Selenium

Adventures in Wubuntu architecture

So I bought a new desktop machine, a MiniITX setup in a very small case that fits in carry-on luggage. But because it's a desktop, and new, it's got a much faster processor and more memory than my three-year-old laptop, and I put in a large HDD and a good-sized SDD for quick booting.

Given that I had all that power at my disposal now, I decided to install Ubuntu as the main OS, with VirtualBox running a VM with Windows 7 for my work tools (the translation industry is entirely Windows-based). With the guest additions and seamless mode, you are literally running both operating systems at once, and it rocks.

I call this unholy chimera "Wubuntu".

Anyway, there are obviously a few little weirdnesses. I use Linux for all browsing, email, and coding, and Windows for editing documents under Office and running all the different tools of the translation trade. But honestly, the Windows Explorer is a good way to get around a project directory. I'm used to it. Wouldn't it be nice to have a right-click that could open a Linux bash shell in the directory? And open a Linux text editor on a given file?

Well, running commands on the guest OS from the host is already supported under VirtualBox, but doing things the other way around is obviously ... weird, under most circumstances, so that's not really supported.

My current notion for a solution is to run lighttpd on the Linux side, and hit it with tinyget from Windows. The latter is a component in the IIS 6 Resource Kit, so not terribly easy to find, but it does work great. From Windows, the Linux localhost is 10.0.2.2 (it's the gateway from Windows' point of view), and I've confirmed that works perfectly.

So put some useful commands on lighttpd, build command files on the Windows side to install on the context menu, and it should all work just fine!

tinywm: Linux windows manager in 50 lines of C

A great example to get off the ground in Linux/X programming!

Relational programming

An article on implementing simple relational programming (a declarative style) in Ruby. Features unification, among other things. More later, when I've actually read it.

Forth on RISC architectures

Here's an article about writing a Forth compiler for RISC architectures - which turns out to be harder than you'd think, given the simplicity of Forth.

Random walk in Python

An interesting article about the random walk hypothesis (quantitative finance) and an implementation in Python that passes the NIST tests for random algorithms. Very cool.

SCID

Source Code In Database. This was actually a passing reference at the very end of a humorous essay listing ways to write Java code that ensure your continued employment (i.e. writing easily misunderstood code). The essay is copied to a place not its original home, because frankly its original home is oddly formatted and broken up into multiple pages, and the copy was all one readable text extent.

But following that link brings you to this oddly-formatted list of excellent things you could do if you were dealing with a system that actually understood your code. Refactoring on the fly, as it were, because you've stored at least part of the code at the conceptual level instead of at the pure syntax level.

Very thought-provoking indeed! Some of them are sheer brilliance, obviously written by a guy who has done a lot of maintenance programming.

Stan, a probabilistic programming language

Wow. Stan is a "probabilistic programming language", apparently meaning that it is an engine for executing statistical models directly. My level of statistical naïveté being what it is, I can't actually even understand the manual very well, but it looks fascinating. Writing a set of tutorials for this would be a meaningful endeavor. As always, though, the HNN discussion is valuable, especially in regards to alternatives and books.

Donald Knuth's list of "readable programs"

Literate programming by the master.

Flowchart.js

Well, this is all kinds of attractive. A declarative language that is converted into flowcharts. You just can't beat that.

Evolving analog circuitry, and some not-very-related thoughts about compilation

So this thing here is all kinds of cool: given a problem, run a bunch of circuits through SPICE until a solution is found. That is just ... wow.

Weirdly, it kind of dovetails with some notions I was sorting through on my walk today. Modern GUI programs are heavy and use plugins because they tend towards a solution-for-all-problems approach. If I want to reuse the concept of "email", it's easier to write plugins for Thunderbird than start a new email client from scratch - or even a new quasi-email client. Like a workflow client - which is why Thunderbird, and other email clients, tends to spread towards task management, calendars, etc.

But this is because our software technology is primitive. Really, we should be thinking in terms of compiling special-purpose tools and then recompiling when our needs change, which was of course the original approach of Unix back when the world was young and resources scarce.

Resource consumption is serious business. This may be another motivation for semantic-level programming.

Well...

I know I said I was only going to write if I had something to say, but I just bought a new desktop machine to replace/augment my aging laptop. And since this new machine has ample capacity, I put Linux on it and I'm running Windows in a virtual machine (a real break for me - I've always run Windows on the desktop and Linux on the server, so this Wubuntu mashup is heady stuff!)

Anyway, so I'm using Firefox on the new machine, Chrome on the laptop. I like Chrome synching because I can cue things up to peruse later on my Android tablet, but synching between Chrome and Firefox? That would require third-party stuff I don't want to get into.

So now I've got two sets of queued links. I know, I know. More first-world problems. But my solution is just going to be to resume posting link posts, at least sometimes for things I find on Firefox that I also want to peruse on my tablet.

Spitbol

Spitbol is the fast version of SNOBOL, and it's being maintained by Dave Shields. Single-handedly. This came up on HNN a couple of weeks ago, and now Vice seems to have picked up on the story for a personal angle [hnn].

I've considered using Spitbol for some kind of NLP stuff, but after all the work I've already put into some pretty sophisticated tokenization I'm not sure it's worth the effort for my main language-handling tasks. We'll see. It's certainly tempting.

Why the semantic level?

For some reason I seem to have gotten off onto a philosophical tangent while walking the dog this evening, with the resulting epiphany: programming is a process of comprehension. And then documentation of that comprehension.

Back in the day, I spent several years working on two large systems (and here and there some other stuff, but the bulk of my programming work was on two systems). The first was a pharmaceutical document management system, and the second was a searchable online machine tool database. The first was for a corporate gig, obviously - pharmaceuticals are not something done in the garage by a startup - and so it was relatively well-managed and the technical debt was relatively well controlled.

The second was a startup, I was the technical lead and entire staff, and over the course of some dozen years I managed to run up a significant technical debt. Due to that debt, working on the system became an increasingly painful chore (every attempt to address one issue simply reminded me of the dozen related issues that were not going to be fixed because the customer had a very limited budget).

The process of leaving that situation and moving to technical translation took many years, and something in me fought it every step of the way. It was the last paid programming I ever did. Since that time, I've been a hanger-on in the startup programming community, but nothing ever gets off the ground, essentially because I have a fear of technical debt.

But why do we have technical debt? I'll tell you: because we commit to specific platforms and solutions during the programming process, and it is very difficult to undo those early decisions later. By programming at the syntactic level (some of which is of course unavoidable) we lock ourselves into low-level structure we can't easily back out of.

Addressing things at the semantic level - were it possible with existing tools - would avoid at least some of that technical debt. If we have semantic structure - if we are defining not software but the concepts behind the software - then the programming itself starts to look more like a compilation process. And just as we can recompile most code onto a new platform (maybe after fiddling with some flags and libraries), we could back out of syntactic-level, stack-level decisions by "recompiling" a set of concepts on a new platform.

Indeed, in a sense a new set of requirements would be a sort of recompilation.

It's a vague ideal, but this is essentially what I see as the promise of semantic-level programming. I earnestly hope I'll be able to make some forward progress on this over the next year.

Literate programming: contemplative versus exploratory programming styles

A couple of days ago, HN was asked, "Why did literate programming not catch on?" Predictably, answers ranged from "Because it's useless for real requirements" to "What do you mean? We use it all the time by policy." But a rough synthesis of the overall sense of the meeting, as it were, led me to consider that literate programming requires contemplation. And sometimes you just don't have the time, or sufficient knowledge, to contemplate.

In the typical startup environment, code is written quickly to address specific needs, and as the business pivots and refines, it mutates quickly. So many of the responses addressed that: literate styles don't react well to code churn, and you end up with a literate explanation of code that no longer matches the code (which is of course always the objection to documentation of any kind).

Reading actual code to determine its purpose is effectively reverse engineering. Sure, well-written code should be readable and so it feels odd to call that reverse engineering - but so much real-world code is unreadable that I think it's a good default attitude.

Ultimately, the exegetical stance of integrating literate programming with reverse engineering should support a pretty good overall software development style: quick prototypes to sound out a new task, followed by contemplation of lessons learned and a literate presentation of useful tooling. That's the goal.