Selenium

Remember how I said my desktop machine is now running Linux? Yeah, I don't have to worry about getting Selenium to work under Windows any more if I want to automate browser tasks. And I do, yes I do, because the browser is one basic way business tasks are presented.

Selenium: http://www.seleniumhq.org/download/
Selenium client in Perl: https://metacpan.org/release/Test-WWW-Selenium

Adventures in Wubuntu architecture

So I bought a new desktop machine, a MiniITX setup in a very small case that fits in carry-on luggage. But because it's a desktop, and new, it's got a much faster processor and more memory than my three-year-old laptop, and I put in a large HDD and a good-sized SDD for quick booting.

Given that I had all that power at my disposal now, I decided to install Ubuntu as the main OS, with VirtualBox running a VM with Windows 7 for my work tools (the translation industry is entirely Windows-based). With the guest additions and seamless mode, you are literally running both operating systems at once, and it rocks.

I call this unholy chimera "Wubuntu".

Anyway, there are obviously a few little weirdnesses. I use Linux for all browsing, email, and coding, and Windows for editing documents under Office and running all the different tools of the translation trade. But honestly, the Windows Explorer is a good way to get around a project directory. I'm used to it. Wouldn't it be nice to have a right-click that could open a Linux bash shell in the directory? And open a Linux text editor on a given file?

Well, running commands on the guest OS from the host is already supported under VirtualBox, but doing things the other way around is obviously ... weird, under most circumstances, so that's not really supported.

My current notion for a solution is to run lighttpd on the Linux side, and hit it with tinyget from Windows. The latter is a component in the IIS 6 Resource Kit, so not terribly easy to find, but it does work great. From Windows, the Linux localhost is 10.0.2.2 (it's the gateway from Windows' point of view), and I've confirmed that works perfectly.

So put some useful commands on lighttpd, build command files on the Windows side to install on the context menu, and it should all work just fine!

tinywm: Linux windows manager in 50 lines of C

A great example to get off the ground in Linux/X programming!

Relational programming

An article on implementing simple relational programming (a declarative style) in Ruby. Features unification, among other things. More later, when I've actually read it.

Forth on RISC architectures

Here's an article about writing a Forth compiler for RISC architectures - which turns out to be harder than you'd think, given the simplicity of Forth.

Random walk in Python

An interesting article about the random walk hypothesis (quantitative finance) and an implementation in Python that passes the NIST tests for random algorithms. Very cool.

SCID

Source Code In Database. This was actually a passing reference at the very end of a humorous essay listing ways to write Java code that ensure your continued employment (i.e. writing easily misunderstood code). The essay is copied to a place not its original home, because frankly its original home is oddly formatted and broken up into multiple pages, and the copy was all one readable text extent.

But following that link brings you to this oddly-formatted list of excellent things you could do if you were dealing with a system that actually understood your code. Refactoring on the fly, as it were, because you've stored at least part of the code at the conceptual level instead of at the pure syntax level.

Very thought-provoking indeed! Some of them are sheer brilliance, obviously written by a guy who has done a lot of maintenance programming.

Stan, a probabilistic programming language

Wow. Stan is a "probabilistic programming language", apparently meaning that it is an engine for executing statistical models directly. My level of statistical naïveté being what it is, I can't actually even understand the manual very well, but it looks fascinating. Writing a set of tutorials for this would be a meaningful endeavor. As always, though, the HNN discussion is valuable, especially in regards to alternatives and books.

Donald Knuth's list of "readable programs"

Literate programming by the master.

Flowchart.js

Well, this is all kinds of attractive. A declarative language that is converted into flowcharts. You just can't beat that.

Evolving analog circuitry, and some not-very-related thoughts about compilation

So this thing here is all kinds of cool: given a problem, run a bunch of circuits through SPICE until a solution is found. That is just ... wow.

Weirdly, it kind of dovetails with some notions I was sorting through on my walk today. Modern GUI programs are heavy and use plugins because they tend towards a solution-for-all-problems approach. If I want to reuse the concept of "email", it's easier to write plugins for Thunderbird than start a new email client from scratch - or even a new quasi-email client. Like a workflow client - which is why Thunderbird, and other email clients, tends to spread towards task management, calendars, etc.

But this is because our software technology is primitive. Really, we should be thinking in terms of compiling special-purpose tools and then recompiling when our needs change, which was of course the original approach of Unix back when the world was young and resources scarce.

Resource consumption is serious business. This may be another motivation for semantic-level programming.

Well...

I know I said I was only going to write if I had something to say, but I just bought a new desktop machine to replace/augment my aging laptop. And since this new machine has ample capacity, I put Linux on it and I'm running Windows in a virtual machine (a real break for me - I've always run Windows on the desktop and Linux on the server, so this Wubuntu mashup is heady stuff!)

Anyway, so I'm using Firefox on the new machine, Chrome on the laptop. I like Chrome synching because I can cue things up to peruse later on my Android tablet, but synching between Chrome and Firefox? That would require third-party stuff I don't want to get into.

So now I've got two sets of queued links. I know, I know. More first-world problems. But my solution is just going to be to resume posting link posts, at least sometimes for things I find on Firefox that I also want to peruse on my tablet.

Spitbol

Spitbol is the fast version of SNOBOL, and it's being maintained by Dave Shields. Single-handedly. This came up on HNN a couple of weeks ago, and now Vice seems to have picked up on the story for a personal angle [hnn].

I've considered using Spitbol for some kind of NLP stuff, but after all the work I've already put into some pretty sophisticated tokenization I'm not sure it's worth the effort for my main language-handling tasks. We'll see. It's certainly tempting.

Why the semantic level?

For some reason I seem to have gotten off onto a philosophical tangent while walking the dog this evening, with the resulting epiphany: programming is a process of comprehension. And then documentation of that comprehension.

Back in the day, I spent several years working on two large systems (and here and there some other stuff, but the bulk of my programming work was on two systems). The first was a pharmaceutical document management system, and the second was a searchable online machine tool database. The first was for a corporate gig, obviously - pharmaceuticals are not something done in the garage by a startup - and so it was relatively well-managed and the technical debt was relatively well controlled.

The second was a startup, I was the technical lead and entire staff, and over the course of some dozen years I managed to run up a significant technical debt. Due to that debt, working on the system became an increasingly painful chore (every attempt to address one issue simply reminded me of the dozen related issues that were not going to be fixed because the customer had a very limited budget).

The process of leaving that situation and moving to technical translation took many years, and something in me fought it every step of the way. It was the last paid programming I ever did. Since that time, I've been a hanger-on in the startup programming community, but nothing ever gets off the ground, essentially because I have a fear of technical debt.

But why do we have technical debt? I'll tell you: because we commit to specific platforms and solutions during the programming process, and it is very difficult to undo those early decisions later. By programming at the syntactic level (some of which is of course unavoidable) we lock ourselves into low-level structure we can't easily back out of.

Addressing things at the semantic level - were it possible with existing tools - would avoid at least some of that technical debt. If we have semantic structure - if we are defining not software but the concepts behind the software - then the programming itself starts to look more like a compilation process. And just as we can recompile most code onto a new platform (maybe after fiddling with some flags and libraries), we could back out of syntactic-level, stack-level decisions by "recompiling" a set of concepts on a new platform.

Indeed, in a sense a new set of requirements would be a sort of recompilation.

It's a vague ideal, but this is essentially what I see as the promise of semantic-level programming. I earnestly hope I'll be able to make some forward progress on this over the next year.

Literate programming: contemplative versus exploratory programming styles

A couple of days ago, HN was asked, "Why did literate programming not catch on?" Predictably, answers ranged from "Because it's useless for real requirements" to "What do you mean? We use it all the time by policy." But a rough synthesis of the overall sense of the meeting, as it were, led me to consider that literate programming requires contemplation. And sometimes you just don't have the time, or sufficient knowledge, to contemplate.

In the typical startup environment, code is written quickly to address specific needs, and as the business pivots and refines, it mutates quickly. So many of the responses addressed that: literate styles don't react well to code churn, and you end up with a literate explanation of code that no longer matches the code (which is of course always the objection to documentation of any kind).

Reading actual code to determine its purpose is effectively reverse engineering. Sure, well-written code should be readable and so it feels odd to call that reverse engineering - but so much real-world code is unreadable that I think it's a good default attitude.

Ultimately, the exegetical stance of integrating literate programming with reverse engineering should support a pretty good overall software development style: quick prototypes to sound out a new task, followed by contemplation of lessons learned and a literate presentation of useful tooling. That's the goal.

Tic-tac-toe and the minimax algorithm: playing imperfect players

I saw an interesting here's-how-it-works post about the minimax algorithm in the context of Tic Tac Toe. I wrote a tic-tac-toe player myself a few years ago, for a HackerRank puzzle, and of course back in the Dark Ages when I was doing my MS in Computer Science.  But in this particular post, Jason Fox makes an interesting point: a perfect tic-tac-toe strategy is fatalistic.

While testing his player, he noticed that once the bot saw it couldn't win, it didn't seem to care how long it took to lose. He modified his algorithm to prefer longer fights, just to make it seem like it was putting up a fight.

Why would this behavior occur? And why does it bother us? Turns out it's simple: a perfect strategy assumes that its opponent is playing a perfect strategy as well, because that's easier to code. In other words, when scoring the different options, the perfect minimax strategy assigns its values by playing the other side perfectly as well. Mathematically, it doesn't matter whether we lose perfectly earlier or lose perfectly later, does it?

But that glosses over a fundamental truth out here in the world, and it's why it rankles to see this algorithm playing so oddly - when playing actual games, it's not at all uncommon for our opponent to screw up. And the longer we draw out the game, the more likely it is that they will. So intuitively, we want to fight a delaying action, because our own internal algorithms tell us that the longer we're in the game, the better off we are in terms of an eventual surprise win.

To reflect that, the tic-tac-toe minimax algorithm should really be modified to reflect the probability of error on the part of the opponent. Even a small change in that score should serve to weight longer fights a little higher, and that's enough for the algorithm to choose that path.

It would be instructive to write that up as a real article, but man, I've got stuff to do that's higher priority than that.

I've been busy

I see I haven't posted here since March...  Partly because I've been busy, but also because increasingly I find myself wanting to do something real instead of just posting a gallery of things others are doing. And since I'm slowly making some progress on that end - the third go at Decl is starting to come together into something that makes some sense - I think my brain cycles are going there instead of here.

That said, I have a ridiculous queue of interesting things to note. But new rules. I think I'm not going to post unless I have something to say. Which really shouldn't be a problem, should it? I've always got plenty to say.

Railway oriented programming

Here's a pretty fascinating talk about how to think of mature error handling as a kind of two-track railway, with errors forcing execution down onto a Failure track. I really like this.

The data engineering ecosystem

Interesting. Very interesting, actually. Sort of an architectural mapping approach.

Explorable explanations

This is a cool post on "the rise of explorable explanations". Very interesting collection of stuff there. On that note, a little musing about combining R and D3.