More thoughts on invoices

In addition to a "brittle" definition of a specific data structure representing an invoice, a semantic programming system should have some general, dare I say semantic knowledge concerning invoices, and business processes in general, documents, and so on. In other words, there should be a semantic web, one view of which might be a specific data structure definition, and the system could interact with e.g. Perl code using that specific data structure definition - but it could also modify it based on what it knows about invoices.

That's pretty darned hand-wavy, but again: it's where I want to go.

AI

I always go crazy for any AI I see. I'm happy to note there's a CPAN module AI::Genetic (searched for it after I saw an HNN post on genetic algorithms in Python). Anyway, this is a good target domain. Duh.

3D modeling

New domain of interest. ICE is an interactive composition environment that looks interesting. A blog.

Coffeescript

Interesting: compiles to Javascript. Slideshow.

SYMADE

An example of semantic-oriented programming. Interesting.

Finding mappings

Imagine, then, we have an invoice. We moreover have a task that somehow says, "Make a document from this invoice." The way to do that is to find a unit that is indexed as a map from an invoice onto a document. That map satisfies the need to make a document, so we set up the map, let it do its thing, and we now have a document.

Note that the task is thus a high-level description of the essence of the program. Again, this is akin to 4GL languages in that we state what to do, and the semantics tell us how to do it. This is truly declarative programming.

Mapping in real life

OK. So today's epiphany is as follows: Given a unit "invoice" which I have already specified (i.e. filled in), I can map that onto a "document" unit that might look like this:

document
  text title "Invoice #3"
  text customer
     Customer name
     Street 9
     D-whatever Germany
  text identifying
     Job # whatever
  table
     header
        text "Description"
        text "Units"
        text "Unit price"
        text "Price"
     row
        text "Translation DE-EN"
        text "4802 wds"
        text "See PO"
        text "438.00 €"
  text (align=right) "Total: 438.00 €"
  text bottom_boilerplate

This maps back and forth to the invoice, which is the abstract view of the same thing. And this can be specified even further with layout information, either in the abstract document or a different, more specific template, and then that structure can be mapped onto a Word document or PDF. (Or both.)

The point being that this map is then itself a live object that can be stored and represented, and that is semantic programming. This part is a lot like XSLT, because XSLT is all about mapping and transforming tree structures. But it's unlike XSLT because (1) it doesn't presume that the mapping is a one-way, one-time transformation, and (2) the organization of the maps and semantic units is organized in a lexical database somehow. That lexical database is itself the program. In some way.

I hope this cleared all that up.

Diggy/DGE Javascript/DHTML-based game platform

Another cool domain - Javascript games. (Also examine as prototype for Fireworks.)

Yahoo! design pattern library

Not the first time I've run across this, of course, but... it has semantics written all over it, so into the link heap it goes.

Another stab at "invoice"

I now dislike my earlier text-based notion of macros. The macro system should be native, and more importantly, needs to be at a semantic level. That is, we are describing to Class::Declarative what sort of node it should be building.

With this in mind, here's my current notion:

unit invoice
   has customer => customer
   has data items (description, price, unit, subtotal)
   assert usually count(items) > 0
   calc total = sum(price) from items
   has currency => currency default USD
   has comments

I need to find a more specifically macro-oriented structural definition and express it like this.

Now Lisp in PHP!

Of all things. I should just go ahead and finish my Perl one.

Another thought on files

I'm proposing a new approach to the formalization of knowledge about programming constructs. Instead of building a new module, I'm proposing the definition of heuristics for the use of the old ones. OK, sometimes your application justifies the creation of a new module (rather often, granted), but sometimes, given a script order, you either don't have the luxury of installing a module, or you just don't want to involve that kind of overhead.

In such a case, you want to be able to write a script that uses existing Perl infrastructure (say) but still manages to deal with sticky cases like UTF-8 BOM markers because the actual specific files in use for the case include them.

So the trajectory I went through: write the naive code, discover BOM markers, work out a way to deal with them, find one file in the list that didn't have BOM markers and write the appropriate conditional code to handle both cases - that trajectory is something that is amenable to automation.

Well. It's not really very amenable to automation right now. That's the point of semantic programming. That's what I want to automate. It didn't require a whole lot of human insight, just some techniques I've learned over the years and some basic logic. I don't think it's AI-complete; it's just one more corner to break off the brick, and it's the corner I have my eye on.

Couponing automation

Probably stupid, but Jeffrey believed that with couponing, he could live on $1 a day for food and have plenty to eat (via MeFi). He blogged it for a month, spending less than $30 and eating rather well - while donating food to a local food bank. Then he kept doing it.

This intrigues me. A lot of the coupon game involves planning and "deal detection" that might be amenable to automation. So ... maybe it's a good target?

If so: SavingAdvice.com coupon database. Jeffrey's FAQ. An explanation of blinkies. There are lots of fora, of course, where heuristics could be gleaned.

Worth thinking about.

MetaOptimize Q&A site

A stack-overflow clone for statistical methods, machine learning, etc.

A great domain for semantic programming.

Javascript local data store

Another interesting component in Javascript. Why, yes, I am in link dump mode, thank you.

Understanding systems

A blog post musing on understanding systems and how it makes us better programmers. This, again, is kind of where I want to be going with a semantic software system.

Intrusion detection

This blog post on intrusion detection probably has no place here, except that if semantic programming can't also include semantically motivated machine learning paradigms like intrusion detection, then I'm probably not thinking about the right thing.

Well. I find it interesting, so it gets bookmarked here.

UTF-8 files and Perl

I ran into some problems trying to load some files with UTF-8 text (German) using Perl. Thing is, these files had a three-byte byte order marker (BOM) of ef bb bf [here is another useful link] and Perl freaks out. You have to check the first line for those three bytes; if present, you toss them and keep a flag. Then for the rest of the file, you have to set the UTF-8 flag on each line read.

The code:

use Encode;

open F, "$d/$file";
$utf8_file = 0;
$firstline = scalar <F>;
if ($firstline) {
  if ($firstline =~ /^\xef\xbb\xbf/) {
     $firstline =~ s/^\xef\xbb\xbf//g;
     $utf8_file = 1;
     Encode::_utf8_on($firstline);
  }
  [ consume $firstline ];
}
while (<F>) {
  Encode::_utf8_on($_) if $utf8_file;
  [ consume $_ ];
}

This is tedious.

I see two approaches to dealing with this. The first is to create Yet Another Module (this includes adding code to Class::Declarative), then always use that module when coding. This is kind of the default, and ultimately it is unsatisfying.

The other approach is some kind of pattern / macro / template system that would include this knowledge and would somehow generate the appropriate code as needed. That's where semantic programming needs to be headed.

Boy, that's vague.

Web app ideas

A list of Web app ideas. Can't have too many. One or two of these look interesting.

Open-source Rails apps

Some example Rails apps. Seems as though a generic semantic platform could establish parallels between different frameworks, and kind of ... port ideas or features back and forth. Or something.