Decl and top-heaviness

Man, reading through all the stuff the v0.11 Decl::Node object supports, it's really no wonder I bogged down. It was just doing too much. I really hope that splitting things out into syntactic and semantic poles will make a difference.  (Or really, even more than just the two, given the declarative extraction phase in the middle.)

So yeah, I suppose a post on that is in order.  The new regime is finally getting underway, given that the last update to Decl was in 2011 and it's 2014 now.  I've started coding Decl::Syntax, which is the handling of syntactic nodes.

Note that a syntactic node is used to derive two different sets of semantics. The first is the machine semantics, the second being the human semantics. This is equivalent to the concept of literate programming, except that literate programming also parses the code chunks for indexing, which (initially, and maybe permanently) we will not be doing.

So the surface structure is the indented stuff. To derive the machine semantics, we go through two more phases.  Actually, three.

The first is markup. During markup, a Markdown ruleset is used to convert all Markdown nodes into X-expressions. The ruleset can be specified in the input, or can be one of a few named ones.

After markup comes declarative extraction. Here, we extract a tree of declarative nodes from the syntactic structure. These contain only the "true children" of each tag. X-expressions are converted to tag structures during this phase, and transclusions are resolved. Annotations are inserted into structured parameter values.  Macros might be expressed, I won't know this until I try expressing some things with prototypes.

The result of declarative extraction is a thinner tag structure that contains only machine-meaningful information. Anything explanatory is discarded, although obviously it's still available for examination if there's a need.

After extraction comes semantic mapping. Here, a set of vocabularies map declarative structure onto data structures. A default vocabulary might just map everything into vanilla Perl structures or objects, but more interesting vocabularies will build more interesting objects.

Finally, execution does whatever action is encoded by the semantic structure. This runs code, builds documents, activates the GUI, or whatever.

Keeping these phases strictly separate makes it possible to build all that detailed functionality into this system without losing sight of what's where. Or so I fervently hope.