Sunday, March 31, 2013
Quantopian
Cool. Algorithmic trading platform with arbitrary algorithms and lots of data. This is very cool. Here's the example that brought me to it.
Saturday, March 30, 2013
Put another way...
"A language should be designed in terms of an abstract syntax and it should have perhaps, several forms of concrete syntax: one which is easy to write and maybe quite abbreviated; another which is good to look at and maybe quite fancy... and another, which is easy to make computers manipulate... all should be based on the same abstract syntax... the abstract syntax is what the theoreticians will use and one or more of the concrete syntaxes is what the practitioners will use."
-- John McCarthy, creator of Lisp
-- John McCarthy, creator of Lisp
Alchemy and mechanical computing
Cool little article about a book of alchemy that contains a mysterious set of tables. It took four hundred years to reverse-engineer the algorithm for generating the tables, and this article is about not only that, but further reverse-engineering a mechanical computer that could have been built when the book was written.
Neat stuff! Very Italian in nature.
Neat stuff! Very Italian in nature.
Macros and washing machines
Well, here's a timely screed about macros and why they make sense - to which I can only say, "Yeah!"
Which brings me to discussion (again, and we'll keep discussing this until it comes out right!) of macros and code generation.
Let us imagine a system of articles and books that describe a codebase. Some of the codebase may be maintained outside this system; some of it within, because the articles include some literate programming tools that can generate sections of code. (This way the system can be used to start analyzing an existing codebase and slowly grow to encompass all of it, as needed.)
An article is equivalent to a book section, that is, a book consists of a hierarchical organization of multiple articles, presumably related. An article may still have hierarchical structure within it, though, because sometimes you just need that for clarity.
In general, though, a single article addresses a single "thing". That topic could be a feature or a specific function, or it could be a change request touching many different parts of the system. Ideally the maintenance of a complex system would thus have a narrative made up of multiple articles explaining the thinking at each stage.
OK. So in that context, let's assume that some of our literate programming-type tools include arbitrary macros that can be reused. (Literate programming can be seen as writing a number of single-use macros, so generalization of that to reusable macros is no great leap.) Some languages are easier to macro-ize than others, of course: we have to parse things to make truly effective use of macros without leaving the native syntax. But by extending the native syntax with a template language, of course (as we do in literate programming, actually) we can build macros for any language. The key is the code generation, you see.
It might be a good idea, though, if particularly questionable or novel macros were to be given a kind of "half-way existence", where the macro as well as its expansion are shown in the presentation. Maintenance then has a template or macro to work with, but the full code is shown for clarity. There are plenty of instances where that makes a lot of sense to me.
Which brings me to discussion (again, and we'll keep discussing this until it comes out right!) of macros and code generation.
Let us imagine a system of articles and books that describe a codebase. Some of the codebase may be maintained outside this system; some of it within, because the articles include some literate programming tools that can generate sections of code. (This way the system can be used to start analyzing an existing codebase and slowly grow to encompass all of it, as needed.)
An article is equivalent to a book section, that is, a book consists of a hierarchical organization of multiple articles, presumably related. An article may still have hierarchical structure within it, though, because sometimes you just need that for clarity.
In general, though, a single article addresses a single "thing". That topic could be a feature or a specific function, or it could be a change request touching many different parts of the system. Ideally the maintenance of a complex system would thus have a narrative made up of multiple articles explaining the thinking at each stage.
OK. So in that context, let's assume that some of our literate programming-type tools include arbitrary macros that can be reused. (Literate programming can be seen as writing a number of single-use macros, so generalization of that to reusable macros is no great leap.) Some languages are easier to macro-ize than others, of course: we have to parse things to make truly effective use of macros without leaving the native syntax. But by extending the native syntax with a template language, of course (as we do in literate programming, actually) we can build macros for any language. The key is the code generation, you see.
It might be a good idea, though, if particularly questionable or novel macros were to be given a kind of "half-way existence", where the macro as well as its expansion are shown in the presentation. Maintenance then has a template or macro to work with, but the full code is shown for clarity. There are plenty of instances where that makes a lot of sense to me.
Curation and CPAN
Construing CPAN as citations made it clear to me what isn't provided in the existing CPAN ecosystem.
That's an interesting thought.
That's an interesting thought.
Friday, March 29, 2013
New approach to Decl
I came to Decl from the wrong direction, last time.
My original thought was to build wxWidgets stuff in a way that didn't kill the beauty in my soul, and it kinda worked - but there was a lot of work involved, so I got distracted by some other shiny stuff like PDFs and Word documents and went off doing it. What I ended up with was a clunky interpreter instead of a semantic description language.
What I should have done instead was, well, a semantic description language. I've been giving some thought to that lately, because spring means it's time to start new things, or bring old ones back to new life.
What occurs to me is:
My original thought was to build wxWidgets stuff in a way that didn't kill the beauty in my soul, and it kinda worked - but there was a lot of work involved, so I got distracted by some other shiny stuff like PDFs and Word documents and went off doing it. What I ended up with was a clunky interpreter instead of a semantic description language.
What I should have done instead was, well, a semantic description language. I've been giving some thought to that lately, because spring means it's time to start new things, or bring old ones back to new life.
What occurs to me is:
- The basic parser into a data structure is a good thing. Let's keep that. Tags, good. Parameters, good (maybe drop the distinction between options and parameters because I couldn't keep them straight anyway). But the basic idea is good: a set of nested tags describing arbitrary structure that can then be mapped onto programming constructs, the tags representing nouns instead of the verbs we see in non-declarative programming.
- The semantics of tags need to be represented more explicitly. I did a lot of dancing around trying to map tags onto concepts behind the scenes, and it was unmanageable. Instead, I should explicitly state a mapping - as an appendix or footnote.
- Yeah, you heard me. We've developed an entire structure of documentation over centuries that is used to address semantic complexities by using out-of-band channels to clarify ambiguity or add information that could be distracting to the flow of presentation. Why should everything be linear in source code? Because it's easier for the compiler to understand? Balderdash. This is really close to where Knuth is going with literate programming - but Knuth worked with compiled languages, and I don't. I don't want to respect identifier uniqueness; my compiler should get from context what I mean. Knuth worked at too low a level. It's time to kick it up a notch.
- I should be able to use citations, too, to include semantic presentation that bears on a given solution topic. Citations here are just ("just") libraries - or macros. Boilerplate and templates. APIs to external functionality. CPAN modules. Anything that has already been worked out to address a solution space, can be a cited reference.
- Decl, although written in Perl, should not be Perl-bound. Ideally I should be able to use Decl to define a program in any language, or to define it in several languages at once. I should be able to use the same framework to use NLTK or data science stuff in Python, describe machine learning algorithms in Octave or R, define modules for CPAN, spin out a Web app in JavaScript, or write low-level things in C or assembler. Decl should be an approach, not a chain. The first time around, it was an interpreted language written in Perl. That was wrong. Decl should actually be a compiler - maybe a compiler on the fly, but still a compiler.
How's that for an Easter resolution?
Context restoration
One of the things I have the worst trouble with, in programming, is context restoration. I suspect this is a large part of what makes an IDE valuable, honestly.
The context of a task is just that - it's workflow. It's all the files involved in an activity, where you are in them, your notes and insights, bookmarks to documentation involved, and so on. It's something a workflow engine would have to provide anyway for any longer-running task.
But it can also be seen as the semantic context of a given task. A semantic programming system should include not only just the current code that addresses a given set of needs, but the history of that code, why changes were needed, and the thoughts of human programmers as they addressed them. There should be rich context for everything.
If you have to laboriously write it all up, though, you'll never get anything done. A semantic programming system should be a sort of assistant that (hopefully) understands what you're doing to the point that it can explain it to others.
The context of a task is just that - it's workflow. It's all the files involved in an activity, where you are in them, your notes and insights, bookmarks to documentation involved, and so on. It's something a workflow engine would have to provide anyway for any longer-running task.
But it can also be seen as the semantic context of a given task. A semantic programming system should include not only just the current code that addresses a given set of needs, but the history of that code, why changes were needed, and the thoughts of human programmers as they addressed them. There should be rich context for everything.
If you have to laboriously write it all up, though, you'll never get anything done. A semantic programming system should be a sort of assistant that (hopefully) understands what you're doing to the point that it can explain it to others.
Thursday, March 28, 2013
Web framework benchmark
Wow - here's a pretty amazing graph comparing speed of trivial JSON serialization of a freshly created object over about twenty different frameworks. There's a huge spread.
On that note, let's get a Web framework link dump here, OK?
On that note, let's get a Web framework link dump here, OK?
- Flask Megatutorial - a series on Flask that I think I'm going to work through.
- Getting started with Django
- and Django Best Practices
Wednesday, March 27, 2013
R: master troll
A good article on R and how it's essentially the uber-Perl in terms of having more than one way to do things.
Tuesday, March 26, 2013
Content generation
So apparently "content spinning" is a thing (like here) - take an article and munge the text so a search engine will accept it as being a different article. It's a ... I guess it's for SEO, to make blog posts look real. Or something.
I still think it would be fun to autogenerate articles using found content on the web. Give it a keyword or two, it finds some articles, spins them in kinda this way, writes a blog post. People would pay for that.
And I still want to write those paraphrasing tools for repurposing content from The AP.
I need a sabbatical, to get some of these projects off the ground.
I still think it would be fun to autogenerate articles using found content on the web. Give it a keyword or two, it finds some articles, spins them in kinda this way, writes a blog post. People would pay for that.
And I still want to write those paraphrasing tools for repurposing content from The AP.
I need a sabbatical, to get some of these projects off the ground.
PDF generation as a service
This is neat. Template generation into PDF documents as a service - very slick!
Diagramming link dump
I got a few diagramming links piling up.
- JS sequence diagrams, very nice look, parses a sequence diagram description language.
- Chart.js looks neat.
- A guy that blogs about UML, talking about different modes of use for it, which I find pretty enlightening. In case you're wondering, I favor "UML as programming language".
- A UML sharing service (UMLbin). Neat idea. But the only download option is a PNG - no source. So that's disappointing.
- And Visualizing Social Structures, a historical retrospective.
Python data tools
O'Reilly: "they keep getting better and better". Yeah. I'm gonna have to reimmerse myself in Python, I guess. *snf* I'll miss you, CPAN!
Monday, March 25, 2013
Expect
Expect is a neat Tcl extension that provides scripting on Unix for command-line programs, especially interactive ones. This is essentially what I'd like to do (unfortunately from my Windows box) through ssh, although it would be great to be able to do it on Windows as well.
Turns out Windows just basically doesn't permit this, so Windows ports don't work. Apparently at all, because Windows treats the console special in some way, and when invoking things in a pipe things don't stay interactive. It's complicated.
But still - the basic idea is quite sound. For really effective sysadmin work, I'd like to put together two ends: first, an "action worksheet" for complex command-line invocations, and second, just such an interactive back-and-forth. If it only works through ssh, then so be it - I've got a Cygwin ssh working fine that could probably do the trick.
[Also, note on using expect in noisy connection environments.]
Oh! Later, I found Net::SSH2::Expect (along with Net::SSH2::SCP and ::SFTP and the fact that Net::SSH2 is itself part of Strawberry, so part of my faith in the CPAN ecosystem was restored this day). [The Monks have some useful sample code. I think maybe SSH2 could use a tutorial.]
Turns out Windows just basically doesn't permit this, so Windows ports don't work. Apparently at all, because Windows treats the console special in some way, and when invoking things in a pipe things don't stay interactive. It's complicated.
But still - the basic idea is quite sound. For really effective sysadmin work, I'd like to put together two ends: first, an "action worksheet" for complex command-line invocations, and second, just such an interactive back-and-forth. If it only works through ssh, then so be it - I've got a Cygwin ssh working fine that could probably do the trick.
[Also, note on using expect in noisy connection environments.]
Oh! Later, I found Net::SSH2::Expect (along with Net::SSH2::SCP and ::SFTP and the fact that Net::SSH2 is itself part of Strawberry, so part of my faith in the CPAN ecosystem was restored this day). [The Monks have some useful sample code. I think maybe SSH2 could use a tutorial.]
Postgres
A couple of excellent slideshows about Postgres and its Django bindings that are really pretty convincing - brief, but that's because they're slideshows.
Subscribe to:
Posts (Atom)