Research workflow

OK, so I don't nearly have time to write about this post in anything like the detail it deserves, but: he's a CS prof now, he's defined a pretty snazzy-looking approach to organizing research activities, it looks a whole lot like Hofstadter's model of mind, and I want to think a whole lot more about it.

So I'm posting it here, on the off chance I'll run across it again in a few months when I have some free moments. I don't even have time to read it all right now. Argh!

Also, from the same blog, the post that hit HNN: [here]. A good post, but not the one I'm most interested in.

Pump > Picasso

Pump > Picasso > blog example - a Pythonic Web framework that looks really nice and minimalistic. And the blog example is more or less a kata.

Machine learning tool list

Here's a tasty list of tools to investigate next time I have leisure to think about machine learning.

Article: The HTML5 Boom is Coming. Fast.

Well, this is good news: the desktop browser and mobile worlds are converging on HTML5 as their graphics platform of choice. Bye, Flash.

Time to whet those tools.

Target domain: testing

Unit tests. Something. Testing is probably related to workflow. Yes, I know this is overly terse.

Article: the Pain of JSON API Testing

A weird little article about Ruby JSON API testing and why it's difficult with Cucumber, which is a testing system in Ruby. Might I say: this Cucumber makes me see why Ruby.

FreeForm

A new programming language? Sorta kinda what I'm doing lately? FreeForm. [hnn]

Steve Yegge: the Universal Design Pattern

I think I need to reread this just to summarize it, but he starts from Doug Hoftstadter and goes to prototype-based programming, I think, is a reasonable one-sentence overview.

Miscellaneous software-component stuff

Link-dump time. These are pretty much all notes on software components. (Note to self: define somewhere that what I call software components don't seem to be the same thing as everybody else's software components. - Really these are slot fillers for particular roles in a system design or something.)

• WSGI in Python: The Pluggable Pipe Dream. [article]
• LevelDB, Google's key-value store, is now open source. [site]
• Using MySQL as NoSQL. (The circle is complete.) [article]
• Redesigning the Netflix API. [slideshow article] (the Programmable Web)

That's that.

Target domain: Windows 7 gadgets

A StackOverflow overview. tl;dr: they're HTML/CSS/Javascript and run in built-in IE in the Desktop.

Target app: Drag-and-drop Faxaway.com fax sender. That would rock!

Target domain: Telephony

Twilio [representative link] provides telephony as a service, with the ability to write applications that handle incoming calls and SMSs in ways I'm sure I don't yet fully understand. (Add Faxaway.com for incoming and outgoing faxes and I think you've got all the telephone-network-based forms of communication covered...)

Anyway, I need to do a little Twilio work for a side venture I've got going on, so I'll need to spend some time with it - the question I'm now asking myself is: how difficult would it be to put together a domain for Twilio apps? What exactly does that even mean? I want to spend some time thinking about that, because writing app domains feels like an important direction for Decl. The final output language could be anything, after all - including PHP or what have you - so this is kinda like Web programming, except different.

So. Marker in the sand here.

Polyglot programming

Here's an interesting article about a programming project that mixed C# and F# for the heavy lifting, and called external Ruby and Javascript. I think maybe the takeaway is that polyglot programming can work well when it's called for (each language has its own strengths and weaknesses).

On the other hand, I've heard comments by Rubyists that sometimes the proliferation of DSLs in typical Ruby programming makes comprehension more difficult. So ... another balance to be struck, perhaps.

Evolutionary program repair

Oh my God this makes me salivate.

The concept: use GA to perturb a syntax tree for a C program, use unit tests as the fitness criteria, sit back and watch code arise that meets all requirements. The link goes to fixed code. In real projects, apparently.

The mind boggles!

Parasail

A language for the development of parallel, high-integrity programs. Very interesting! Most interesting takeaway for me is that a "loop" in Parasail is what I called a checklist - if you don't specifically force sequentiality, it's not sequential. [specific post but the whole blog is probably worth a read through]

Users fill out forms faster with unified text fields

Of course, then you have more parsing to do - often of messy data indeed - but it's good to note that there is, in fact, a tradeoff that deserves careful consideration. [article]

Android programming and levels of abstraction

Here (on G+) is a thoughtful little essay by an iOS dev who's bought an Android. His own tl;dr is this: "within a few days of iOS programming, I had a working app. Within a few days of Android programming I was just beginning to understand the abstractions."

And his overall prescription is that Android, while it lets you do a lot more, makes you work harder for it: "We need more abstractions to get us (app developers) up and running in less time, with 95% accuracy."

And I'll bet you see the solution I would have in mind if I had the time to devote to it, can't you?

VVVV: a visual programming language

Now ported to Javascript. Very nice. Original implementation at [vvvv.org].

Example application description

Here's a fantastically detailed application specification from Guru.com (738799):

Description

I am looking for a fairly simple piece of software (you can make it as complex as you want, see below). It will track auctions and all bids and bidders for the item. In other words, software that will temporarily log members' bids on auctions.  This is strictly for the penny auction website quibids.com. Check it out and familiarize yourself with the job.  I don't care how it works just that it preforms this simple task but it must produce organized and readable info/data that is sortable and updates live with the auction.  It must also either have the ability to open the program multiple times for multiple auctions OR (will pay more) the ability to track 2 or 3 auctions with a single process.     PLEASE MENTION IN YOUR BID WHICH OF THESE OPTIONAL FEATURES YOU PLAN TO INCLUDE. (if any)  (Optional) Expect more money if you have the expertise to add these features, it is not required, however.:  - Ability to track multiple auctions (hoping for up to 5) in one process  - Checkbox to keep program on "top"  - Feature that allows me to "hide" users on the tracking list for an auction that have X number of minumum bids placed. for instance 10 bid minimum or that bidder wont show up on the populated list.  - Create alerts that creates a sound and popup to tell me when bidders are reaching maximum value for an item. For example on a 100 bid pack auction when all bidders are either under 10% value (10 bids or less) or over 90% (90 bids or more). Or maybe if theres only just one bidder left who is between 10-90%.

This should be trivial. I should be able to simply translate that into pseudocode and have it 90% written. That's the point of semantic programming. After I have Depatenting working, I think maybe I'll cycle back around to GUI work like this.

Jumpchart

A website charting/planning app? Maybe worth analyzing.

Link dump

Eesh, my browser tabs are getting too full again. Here, in no particular order of importance:

• Guru.com, another jobs site. No feed that I can see.
• Yottaflow, an online workflow site. Dammit.
• A little overview of SQLAlchemy, apparently a Python ORM.
• Github site for Anorm, a new very transparent PHP ORM.
• Wolfram have introduced the (apparently open) Computable Document Format.
• h5ai, a drop-in Apache index that really looks nice and works well.
• I can haz some...? A browser capability testing framework.
• An overview of possible military manipulation of social networks.

UI pattern database

Food for thought.

Schema creator

A JS tool to create schema.org entities. Very neat!

25 great links for data journalists

What the title says.

Code katas for Web developers

A nice little list of possible beginner Web projects - intended to be weekend projects - to wit:

1. Simple blog with comments.

2. Pastebin with short URLs.

3. Todo list site (sans JS).

4. Add authentication to 1 and 2.

5. Site to send text messages with Twilio. (API use.)

6. Upload target site on E3.

7a. AJAXify #3.

7b. Add user accounts to #3.

8. Create a JSON/XML API for #3 or whichever.

9. Wrap said API in a package in your language of choice.

10. Expense tracker with pretty graphs and export to CSV or Freshbooks or whatever.

There are other simple project ideas along these lines that I'd like to start to catalog, but this is a great start. Normal people would use these for learning a new language. I'm using them to build one.

Target application: wordstash.com

This should be relatively easy to clone, actually. And it's a great vocab flash card site!

Metaprogramming in Python

Short article.

Data parsing

Well, I want to read a DVD again, so I'm back to the notion of data parsing [earlier], which is creating a description of a data structure, then reading it into a nodal format for ... whatever.

This is essentially what I was doing with PDFs for reading, too (except there it's going through an API); I just need to formalize it into a coherent domain.

Wiimote interaction

Now this is cool! A Javascript library cleaning up the mess that is Wii interaction. Maybe I could get the boy interested.

PDFkit

PDF generation for node.js. Hmm.

Scaling: HTTP caching

Some good advice. "Varnish".

Blow-by-blow comparison of AI in a couple of languages

Fun article on implementing a Connect 4 engine in functional and imperative languages and wrapping it in a framework to play it. Nicely written up!

Signup form best practices

You had me at "best practices". [article]

Infinity

There are just a whole heap of good site components nowadays: smoke.js.

Underthesite.com

Cool little technology-stack database, apparently derived automatically, that indexes by URL and by technology. A lot of knowledge there. Makes me think I should probably archive it or something.

Similar tools from the HNN peanut gallery: http://lineofthought.com/, https://github.com/jpf/domain-profiler, https://github.com/mitsuhiko/probe, http://wappalyzer.com/, http://builtwith.com/.

Thoughts while doing sysadmin

Task: pruning some context tables that contain far too much very old information.

I'm struck again by the fact that my memory can't retain SQL syntax for longer than about a month. When I need to add an index, I have to look it up again. And of course there's no record of what I did last time unless I go back through anything I might have noted down on paper.

I need an administrative tool that can encapsulate actions taken - and log them. People have successfully used VCS systems to keep track of changes to SQL, but a workflow enactment kind of thing would be better. (Perhaps in combination with a VCS system.)

So again: the action.

Database administration systems are a dime a dozen, of course. It wouldn't be hard to slap something like that together based on the transaction patterns a couple of posts back, in combination with this notion here. Deserves some thought - especially given that the same concept could be extended to Unix commands in general. Administration of cron jobs, whatever.

In a larger sense, a sysadmin domain would be able to construct a picture of a system of some nature that could be used to inform the construction of this type of action. This thread of thought is still really vague, but I keep coming back to it. (And have for decades.)

Final solution: wrote a Perl script for each table that gently removes 10K records at a time, and put it in the crontab to run every 10 minutes. Building an index would lock the tables for hours, disabling the database for an unforeseeable period. This is the kind of thing that I should be able to reuse effortlessly.

The case for segregation of natural language

So this is something I thought of yesterday in connection with my as-yet-nascent natural language production library - in a way similar to the idea of separation of layout from program-generated code, there should really be a separation of "mechanical" language - language generated by lookup or traditional methods - from fluent language.

That's not really very clear, is it?

Here's what I'm proposing, essentially. The starting point is internationalization - which is hard if you want to do it right. It's hard because natural languages deal idiosyncratically with number and gender - just as a start. So you can't use simple fill-in-the-blank methods for generating phrases, once you start worrying about Japanese and Russian and Hebrew.

What I'm proposing instead is some kind of semantic structure that would express a given thought that should be expressed. This structure would then be expressed in a more sophisticated manner - and if necessary would require additional information, like the gender to be used for the addressee. (I presume that Hebrew normally assumes masculine gender if the addressee is unknown.)

But that means that we've got several layers of layout. First is the language-independent layout, into which we could insert lorem ipsum. Next is set pieces - paragraphs of explanatory "brochure" or instructional text that can be translated en masse and inserted wherever needed. And finally there are the interactive segments - error messages, responses to requests, and the like - that need more careful handling for fluency.

It's kind of inchoate, and less revolutionary than it seemed while I was walking the dog. But I don't want to forget it.

Flex and Exambuff

How did I never hear of Flex? It's an application-based compiler for Flash, effectively, used for building "Rich Internet Applications" (i.e. Flash apps) without the need for a lot of the animation stuff that makes Flash so weird to use for the software designer.

[sdk] [brief article]

The reason I even ran across it is that a chap named Tim Ruffles wrote an entire site (for commenting on scanned exam papers) that he's open sourced because it just wasn't working out for him. [article] [github] [site] The actual working-with-papers part is all in Flex.

I am very strongly considering making this another target for Decl.

d3.js - data-driven documents

Oh sweet Lord this is a nice-looking library that I plan to love for the rest of my life.

Programming language design

Interesting article on some of the design shortcomings of Go: Go rocks - How can we avoid something this bad in future?

Transaction patterns

From February on "Web app design":

Oh, wait: a mother lode of so-called "transaction patterns" used in a toolkit for building database administration sites. It is incredibly detailed. These transaction patterns, by the way, constitute a language for user-level interaction with Websites that is not restricted to the Web domain - they'll work just fine in the Wx context as well. So they should definitely be analyzed, but they can't just be relegated to WWW::Publisher. I'm not really sure where they should go. Probably a new domain. Database::Declarative? I just don't know!

It took me a few minutes to find this post just now, as I hadn't really marked it very effectively. Now I'm doing so. The concept of the transaction pattern is a good one.

Basic CSS of a grid layout

Nice breakdown. This is the kind of thing I want to look at macro-izing, like, next week. I've started work on CSS::Declarative again, and next on the checklist is macros.

Extended examination of app building in HTML5

One of the Opera devs is porting a C++ game into HTML5 as a kind of exercise. Worth reading.

databranches

Using git as a distributed database. Intriguing.

"Asynchronicity": checklist pattern

An interesting article about asynchronous building of page parts. Start off a checklist of parts (using, e.g. memcache), then time out at a given time, check the keys for parts that successfully got generated, and build your page with what you've got.

The point being that you end up with a much more resilient server - if a database is down for some reason, the rest of the page gets built and things degrade gracefully. Neat idea!

ClickEquations

Here's an interesting app which does ... I can't actually tell what it does. Looks cool, though, so I can't bear to lose the link. Seriously - some kind of data-oriented business stuff.

REST APIs

This is something I've given little thought to, so far: RESTful APIs. Nobody understands them. Another link from the same blog. Private media types - who knew?

Target domain: dates and times

Time.

Automata as macros

Interesting article on using Scheme macros to encode automata.

It goes through the steps from writing a crude interpreter for an automaton to writing a nice macro to produce higher-performance code. That's cool.

Target domain: cryptography

Lifetimes of cryptographic hash functions. This kind of ties back into the best-practices thing, but it's a solid domain knowledge target.

Web intents

Web intents seem to be a new way to link apps together to get tasks done. Worth a read when my mind is clear sometime.

Interesting programming language roundup

It's time for link dumping, boys and girls.

• Bandicoot, a set-based language. Looks kinda half-baked, but intriguing.
• OMeta, an object-oriented language for pattern matching. Very intriguing. [pdf]
• Another article on Coffeescript.

Target domain: Android apps

I mean, come on - this is the perfect target domain.

Tutorials.

Fexl: small functional programming language

Fexl. This guy even runs Fexl as an embedded interpreter in Perl. (Which is unfortunately not on CPAN.) Interesting stuff.

Joshfire framework

Wow, this is really close to what I want to do in the Web publishing sphere!

The Joshfire framework allows you to describe an app in terms of a data tree and a UI tree - and the app will then be expressed for different platforms! It's Javascript and HTML5 and entirely too close for comfort!

Very cool.