Factorization diagrams in Haskell

Here's an excellent post about factorization diagrams (arranging dots to represent the prime factorization of a number), implemented in Haskell.  It seems like this would be a great example for a mapping application.

Source maps in Javascript

Now here is a concept I love - the new JavaScript source map.  It allows you to start with a given source representation, then crunch and minify your script - but given the map, you can get back from a given execution state to the point in your code it reflects.

That's brilliant.

Backing up data on paper

I have utterly fallen in love with PaperBack.

Take a set of files.  Turn them into a graphic.  Print them and stick them in a drawer.  Later, scan them and restore the files.  Dear Lord, is that beautiful or what?

A little Word scripting

As you may know, I'm a technical translator by trade these days, so I work under Windows with Word documents a lot. Word has a scripting mechanism that I've often considered to be Microsoft's way of making it technically possible to program whatever you need in Word, without actually making it sensibly easy (making hard things possible and making easy things hard, thus ensuring grist for a robust consulting mill), but back in a previous life I spent a lot of paid time writing Word macros to do text processing. Profiting from that grist, I suppose.

Anyway, this week I have a huge pile of process definitions to be placed into Word, each consisting of a table on each page with a list of steps in a process to be carried out manually (many with barcodes for acknowledgement and so on). Each step has a unique identifier - by which I mean that each type of step has a unique identifier and there can be any number of tokens of each type of step.

And it's all in German and my job is to translate it to English. And the originals are all scanned PDFs, so I can't actually just use a regular translation tool to find repetitive text, or even cut and paste into Word; it all has to be typed. I do have properly formatted blank tables to start with for each document.

What I've ended up doing is putting each type of text into a table in a dictionary document, then writing a macro that looks up the unique ID I've typed into the given target document's first column and, if it's found, copies the already-translated row into the target document. If that type hasn't been encountered yet, then I translate it, and copy the row into the dictionary by hand.

I'm writing about it here, because at a semantic level that Word dictionary document is a database (a NoSQL database, if you want to be all cool-kids about it). If this macro were expressed in Decl, it would be much easier to get a lot of implications of that construal and work them into new ramifications.

In other words, construing one thing as another (a semantic mapping) is a form of recognition that could explicitly underlie the programming process in a semantic language. Also, I think I like the word "construal" as a technical term for this kind of mapping.

COLM: COmputer Language Manipulator

COLM is a language for parsing and tree transformation (like TXL before it). Bears watching.

(I'm pretty sure my concept of mapping is equivalent to a tree transformation with notes kept to permit reversibility.)

TXL: Tree Transformation Language

I essentially think of Decl macros or maps or whatever as falling into two categories: first is the text expression of a template, but the other works on tree structures.

Googling on "tree transformation" is a useful exercise, and five minutes perusal finds me two good links: TXL (the tree transformation language) is precisely what I want to be able to do (with a rule-based approach), and then there's an interesting paper ("A Language for Bidirectional Tree Transformations") by the Unison guys at UPenn about reversible lenses that act between tree structures, which is of course my map concept. (I should probably read everything they're doing.)

So those need to be read carefully. I'm sure there's more.

Boomerang: a synchronization language

A bidirectional textual language. It's the same thing as my concept of a mapping, which it turns out (at least some) other people call a "lens".

Mapping

A quick thought on mapping. A "document" object can map onto a PDF via a PDF builder, and a PDF object can map back onto a document object by means of some sort of thing that recognizes tables and things with less-than-perfect accuracy. Point being that we've got different tools in the two directions, and that one direction might be lossy.

Forgive me if that was obvious. I told you these were more like notes for myself than a regular blog.

Thoughts for the evening

So I tried "adapting" some of the examples for Flash into a pseudocode form, and as usual, everything I do points out the holes in my plan. Then I go walk the dog, and possible solutions occur. Today's harvest:

The sokoban example from the gazbming is rather oddly written; he defines some functions to create tiles and sprites, then creates 200 wall tiles, a bunch of package tiles, and so on. Then to build a map, he moves the tiles in Actionscript (and the maps are encoded in the Actionscript).

So fine. I can imagine writing a macro that expresses the map encodings as Actionscript, but really those 200 wall tiles are a map. I don't want to create them with Perl (though I could), because normally I'm not running Perl at define time, only building code snippets and running them after start. Maybe I could rethink that, but in this case, it seems more elegant to express the 200 wall tiles as a map from a sequence (from=1, to=200) through a template into a series of macro-inserted structures.

Now, this begs the question as to what, really, a map is. I've been going on the presumption that it's a nodal structure thing - but in this case, conceptually, I'm mapping from a sequence onto a series of nodes. I can imagine the same thing with a set of row data. Now, in the case of a sequence (which is immutable) this is a one-way trip. If I map to data, it's ... maybe not a one-way trip, but I probably have to define explicitly how the data should be updated if the nodal structure is changed, and certainly something like this is probably one-way.

But the point is, semantically, this is a map. Just because it's not my two-way nodal structure doesn't mean I shouldn't still call it a map.

Oy. You want to do something simple, define a new programming paradigm and a language to go with it, and revolutionize the world, and then it's just one thing and another.

Levels of abstraction as maps

So back in May I had a post on levels of abstraction in code, followed this month by some thoughts on finding mappings - these are actually the same thought.

In my invoice example, I imagine a very high-level abstract specification of a specific invoice, then a medium-level specification of the invoice as a generic document, then perhaps a lower-level specification of that document as a PDF. Followed, I guess, by the actual PDF. The point is that each of those specifications represents the same thing.

When I talk about levels of abstraction in a program, this is the same thing again - a high-level summary (or specification) of the parts of the program, and a lower-level specification of the actual program.

I just need to figure out how to express this understandably, and I'm good to go.

Mapping in real life

OK. So today's epiphany is as follows: Given a unit "invoice" which I have already specified (i.e. filled in), I can map that onto a "document" unit that might look like this:

document
  text title "Invoice #3"
  text customer
     Customer name
     Street 9
     D-whatever Germany
  text identifying
     Job # whatever
  table
     header
        text "Description"
        text "Units"
        text "Unit price"
        text "Price"
     row
        text "Translation DE-EN"
        text "4802 wds"
        text "See PO"
        text "438.00 €"
  text (align=right) "Total: 438.00 €"
  text bottom_boilerplate

This maps back and forth to the invoice, which is the abstract view of the same thing. And this can be specified even further with layout information, either in the abstract document or a different, more specific template, and then that structure can be mapped onto a Word document or PDF. (Or both.)

The point being that this map is then itself a live object that can be stored and represented, and that is semantic programming. This part is a lot like XSLT, because XSLT is all about mapping and transforming tree structures. But it's unlike XSLT because (1) it doesn't presume that the mapping is a one-way, one-time transformation, and (2) the organization of the maps and semantic units is organized in a lexical database somehow. That lexical database is itself the program. In some way.

I hope this cleared all that up.

HNN finds another interesting thing

Fabrik, an Apple visual programming toolkit from 1988, has lots of very intriguing features - including bidirectional mapping between objects, and a pervasive use of the dataflow paradigm. (!)

Another thought on levels of abstraction

Maintaining two views of the same structure (i.e. two levels of abstraction) is what I referred to earlier as a dynamic mapping. It's definitely something that has to end up in C::D.

Mapping

Consider this. A macro expression is a one-way street; given a macro instance, the macro engine expresses it as more ramified code, possibly affected by the environment, or parameters.

A more powerful macro engine could define a mapping between two structures, whereby a change in either could be reflected as a change in the other. For example, a mapping between the graphical boxes on a diagram and the underlying database could be set up that could modify the screen when the database was changed, and write (the significant) changes to the database when the screen image was manipulated.

This would be the equivalent of a mapping in cognitive science, the syntactic/semantic mapping we use to talk about how language expresses concepts. Or the same as the analogies Douglas Hofstadter uses to talk about ... everything.

A mapping in a declarative tree would look superficially a lot like an XSLT program. The only difference is that a mapping would remember where its results were, and could act as the trigger described above.

Another example: we've already got magic variables that look to a GUI field, for example, for reading and writing. An explicit mapping structure could define that in the language instead of just in the code; we could define this sort of mapping in our programs. This could be used to set up Excel-like functions in a spreadsheet. Or not in a spreadsheet if you don't like grids - just set up a dataflow program. (TODO: think about that a little more.)

So the to-do list for macros is:

• !tag defines a one-way map to a virtual tag. It expresses in place during built, and stops.
• !!tag defines an active one-way map; it expresses in place, but rebuilds whenever one of its parameters is changed (or potentially does; we could probably specify some active and some one-time parameters somehow).
• An explicit macro or template tag could also match specific parts of the tree, specify assertions about its structure that must be met, and coordinate expression in multiple points.
• A mapping would define that in two directions in some as-yet-unspecified manner.

The basic ones are going to be necessary to make PDF::Declarative make any sense for real-world use, because they allow us to specify a PDF structure that is parameterizable, say, with variable text. Then we can use PDF::Declarative to build a template that can be used again and again (e.g. as an invoice template, etc.) Do that, and my test case will be complete: I want to use PDF::Declarative to generate my translation invoices.