Monday, February 28, 2011
Sunday, February 27, 2011
So Wesabe was a service to allow some kind of personal finance management - it failed. But they put together a pretty comprehensive set of code to slurp bank information out into a local database for you, and - lo! - they open-sourced it on Github. Here's an article about it.
Food for thought, if I can figure out how to make use of it.
So I'm looking for some really basic PHP examples to use for initial test cases. Here's a link dump with a few - it's surprising how translating some of this stuff into Declarative for publishing motivates the design work. Moral of the story: more examples! Lots of little apps!
- Tizag form example
- T. David's PHP script diary - older than dirt but still not a horrible way to get used to the language
- Well House Consultants has an example of uploading a file for storage as a blob in a database. Kind of a rudimentary document management feature. Neat, actually.
But those are really only for getting off the ground with PHP. Nobody writes stuff like that any more, it appears. They're really just good for a couple of proofs of concept. I've taken eight pages of paper notes on thoughts about just one signup form using jQuery; I'm getting pretty close to being able to start coding - so much for getting Depatenting done by the end of the month to be able to use the Google Adwords coupon I got.
More on this topic later.
(Later: also a list of open source PHP script modules. Either integrating these or analyzing them to distill commonalities for some basic modular features would be the way to go.)
Scaling. It's what's for dinner.
There seem to have been a lot of thought-provoking posts on scalability this week. Here is another area where a macro system makes sense. It's nice to have a simple prototype that doesn't need a whole lot of infrastructure, when you're hacking out an idea. But if that idea gets featured somewhere and it's written as a low-traffic app, you will suffer.
A good macro system will be able to bridge that gap, keeping the clean lines of the code understandable while allowing memcache to be shoehorned in all over the place.
This post is a link dump right now, but I want to come back to this topic and come up with an overall approach that works to describe scalable systems.
First, a great talk by Steve Huffman on lessons learned at Reddit. There are a lot of ideas packed into this talk; I've listened to the talk but haven't unpacked the ideas yet.
Second, a much simpler post from Dan Shipper about WhereMyFriends.be, featured on Mashable and CNN and thus buckling under the strain.
Programming tricks for memcached. Lots of thought needed there.
Photon, the high-performance PHP framework.
Couple of articles this week about doing your design in code instead of Photoshop. I'm thinking it would be cool to have a Photoshop-like drag-and-drop prototyping tool, of course, as part of Sproggler.
First was a post about the Quora design process, no less. This was followed up by an article by Chris Teso elaborating on why he agrees. Using the neologism "devigners". ...
Anyway, it's a good idea. And while I'm on the topic of design, a slight link dump: improving a specific design, a step-by-step case study.
Case in point, this article about asynchronous code patterns in node.js. Go ahead, read it.
Now here's the thing. If you read that code, the first version, the synchronous version, is comprehensible. The final version, an asynchronous version that actually works and handles exceptions rationally - well, you can understand it given the presentation, but I defy you to run across that in the wild and tell me what it does, clearly, without the need for fifteen minutes of error-prone thought.
Macros help us get around that. If we think of that final pattern as the compiled version of whatever concept we're trying to express, then it's a lot more palatable. And that's why a universal macro language like Declarative is going to be really useful for a lot of different things.
There are conceptually (for me, anyway) two different kinds of programming problem. First is the "set it up and let it run" kind that Declarative mostly addresses - the construction of data structures to model and simulate a particular system.
The second is solution algorithms - code that takes a particular question or problem in a problem space and derives a solution. This stuff often verges on artificial intelligence, and Declarative mostly doesn't address it. Nor should it, arguably - Declarative is intended to declare things and remove the hassle of setting all that framework stuff up. You can implement your algorithms in whatever language you like within that framework.
Anyway, this came up because of an interesting post on dynamic programming.
Saturday, February 26, 2011
Grist for the mill of "types of program" here on HNN, ever the most interesting forum out there.
- basic form
- form with multi-valued lines
- form with data set (multi-valued parent/children)
- form with grid
- tabbed form
- form with skins
- batch (loop thru something)
- batch update selected records in a table
- batch dump table records --> .txt or .xml file
- batch .txt or .xml file --> update table records
- batch file --> create & populate database table
- cut an email
- traverse internal tree function
- traverse file index(es)
- build html from parameters
- build .pdf from parameters
- build hp esc sequences from parameters
- benchmark a process
- how does that syntax work?
- which way is faster?
- batch string parser
- batch source code search
- batch source code changer
- batch parse source code, identify routine for reuse
Wednesday, February 23, 2011
Monday, February 21, 2011
The realm of Web app design is really pretty big. I've worked with it since the 90's, of course, but ... a lot has happened rather recently, and I'm still only scratching the surface. I think it's safe to say that it will continue to change.
The underlying structure of any Web application is, of course, a set of files on a disk somewhere. So at the lowest possible level, when specifying a Web application we're actually specifying a set of files. A structured set of files is a product. I'm going to call it that for now, because I want to drive home the idea that the notion of a product is a more general thing - it's a thing that properly belongs in C::Decl, not in a library domain. It's part and parcel of files and directories, because it is a set of files in directories.
Earlier today I posted on the "HTML5 boilerplate". This is at least part of what goes into a Web app template. I'd go so far as to say that the basic page template is also part of that macro. And make no mistake; the boilerplate is a macro that we will never see in our specific Web app definition.
The components of a Web app thus become: a specific database and table definitions within that database (aside: some table definitions will probably end up in boilerplate and component macros as well), a boilerplate and sets of imported components, some basic publishing actions to be carried out for different stages of the game (which could also go into boilerplate, actually), and the list of pages/application areas that will show up in the final application.
The only way to work this out is to start building Websites. My initial goal is the Depatentizer (8 days left, haha), but some smaller apps might also be a good idea. And then I want to rebuild Despammed, and maybe even Techspex.
I've been prowling around looking for insight, and so the rest of this post is a link dump about PHP site building (mostly).
Oh, wait: a mother lode of so-called "transaction patterns" used in a toolkit for building database administration sites. It is incredibly detailed. These transaction patterns, by the way, constitute a language for user-level interaction with Websites that is not restricted to the Web domain - they'll work just fine in the Wx context as well. So they should definitely be analyzed, but they can't just be relegated to WWW::Publisher. I'm not really sure where they should go. Probably a new domain. Database::Declarative? I just don't know!
Anyway, that link is brilliant. Brings up the MVC architectural pattern, too - again, this is a solid concept that should be highlighted somewhere, but not in WWW::Publisher. The link there is to the same site, and it includes more than MVC; it's an overview of his entire methodology, some of which I don't care for, but still - wow! Great find. (Building up to a PHP sample app, which is where I found it.)
Now for the link dump.
- DBI for PHP is PDO.
- CPAN for PHP is Pear: Pear coding standards.
- Writing web apps in UML... Mentions WebML (the Web modeling language) - neat!
- Units of Work (UoW pattern) in PHP.
- AJAX overview article. Haven't read all of it.
- Oh, oh, oh - this is also brilliant: a blueprint for PHP Web apps with a mindmap!
- A follow-up article from that series (which mostly never got written, sadly): the bootstrap. This would go into the boilerplate.
- Flourish library - most interesting to me for its abstraction layer over a variety of databases, allowing you to write a Standard Query Language that will be munged into what the specific database needs. That deserves some brain picking.
- Smarty is a PHP template library. The link is a sample app.
- Canvas whiteboard with chat using a web socket. That is a collaborative tool that would be useful on sproggler. And any tutoring site.
- A thoughtful article on prototyping and customer immediacy in the early days of a site. Some components for quick feedback should go into the boilerplate.
Saturday, February 19, 2011
Looks like this is starting to gain some momentum. I'm too tired to do anything with it just now beyond pointing and screaming, maybe hitting it with the thighbone of a dead animal - but later, boy howdy, later I'm going to understand the hell out of it.
Or something. It lets you make graphical representations of your code. Jesus Mary and all the saints, that is just stopping me in my tracks here... Literate programming done really right.
This is what I want for my birthday, Lord. Only in declarative wxPerl.
Monday, February 14, 2011
Sunday, February 13, 2011
Keynotopia is a quick startup that provides Keynote and PowerPoint templates for fast prototyping of apps - publish to a PDF and you get a prototype you can actually run on a phone. Which is a cool idea! So I want to do that with a more advanced prototype based on Decl.
Oooh, started with inubit, which is closed-source but has features that make my brain melt. This. I need to just scan down the menu on the left.
Saturday, February 12, 2011
Friday, February 11, 2011
- Rails AJAX frameworks - best practices at the moment.
- HTML5 canvas graphing solutions - Webdesigneraid might be something to follow.
- jsfx sound effects library in JS! Very fun!
- LimeJS game library.
- mxGraph - not open-source, but very, very feature-full. Good for inspiration.
When looking at an overall system ecology, we see a lot of applications in it that cover certain areas or regions of functionality. To a certain extent, those applications are interchangeable. What I'd like to do is to index some Web apps and open-source apps (see osalt.com for an example of this at a coarser grain than I mean) and reverse-engineer their specifications in some way that facilitates comparison.
Then, when putting together a specification list for a given business process (say), you could choose which elements to plug together, and identify places where interoperability projects might pay off.
I'm still not 100% clear where I'm going with this, but it deserves further thought.
This seems to be an interesting little niche: taking a design and spitting out a framework for coding. Obviously it's not automatizable, but what I want to be able to do is to specify such an app in a simple way (WWW::Publisher, of course).
Also: I'd really like to be able scrape a site of this nature into a specification for analysis and modification.
Tuesday, February 8, 2011
Monday, February 7, 2011
Sunday, February 6, 2011
So today I'm scripting the conversion of about 140 XLS and PPT files scattered in a directory tree on FTP into TTX in the same FTP directory tree. There's no way to do this except by calling an external Windows function and doing your best to automate it through the GUI.
This is a brittle process, and therein lies the difficulty.
Several of the XLS files are locked in Excel. TagEditor can still convert them, but Excel pops up a little dialog asking for the password - and since the TagEditor window never appears, the script runs afoul of my assumptions.
The way to get around this would be some sort of wrapping layer that explicitly states my assumptions and ensures they're met before proceeding. If something unexpected happens, ideally this facility would note it and diagnose it in a wait-a-minute record, and put the action back on the queue for intervention.
Which brings me to the other assumption I'm making, which is that a script is a sequential process; before one file is done, I don't go to the next (except of course that the script is sometimes not waiting for some reason, thus opening hundreds of TagEditor windows without checking whether it's appropriate or not, sigh.) Instead, there should be a parallel queue (this is a workflow concept); if one of the actions runs aground, others should proceed.
Both of these are higher-level approaches to the scripting task.
(Update: additional assumptions I was making: that the window closed immediately after being told to close, that the converted file was being written immediately, and that a window just opened is the topmost window after opening. All of these should be checked and accounted for in a mature script driver for Windows.)
Saturday, February 5, 2011
I have a spider app in mind. I was DuckDuckGoing on Toonbots, for whatever reason, and ran across a couple of links to "Toonbots forum, being able to avoid spammers and trolls and whatnot." I vaguely recall that conversation.
I know, I know, I'm a sucker for Web spidering, too. Sigh.
This is how it would work:
- Seed it with one or more of the target sentences.
- Google a sentence.
- Try to find its actual origin.
- Store all the other URLs and text; break the text up into sentences.
- Spread from there.
- Try to categorize the type of host (different breaking techniques were probably in use).
- Track down everybody making this possible, and fix each and every one of them. Oy.
Wouldn't that be cool?
- Toonbots forum, being able to avoid spammers and trolls and whatnot.
- The world is awash in fast money, he said, and it is changing the structure of capital markets.
- When you are pre-approved by spruce mortgage, you will have access to hundreds of loan programs.
- Many sellers would rather have a monthly check than a lump sum settlement when they sell.
Update 11/19/11: That particular link has been cleaned up, but the network as a whole is still there and still forwarding to the same ... actually, just a very similar site. Of course, there may have been multiple sites all along. So this project is still waiting to be done.
I'm down to the wire on table definitions and trying to come up with a good way to combine quick-and-dirty things with more careful solutions. I think the current implementation of the table tag is going to do the trick.
Here's a bit of a link dump.
- A useful comparison of popular field types in several different databases.
I guess I only had one link up about tables. Weird; I could have sworn there were two or three.
I just keep coming to this, don't I?
Mail::Box is a hueueuge mess. It doesn't load from CPAN on Windows, and reviews indicate that while it does a whole lot of things right, it makes a lot of very weird and undocumented assumptions that I, for one, just don't care for. So I'd like to do ... something ... for local email handling that starts more or less from scratch. Sigh. Then wrap it in Decl and a UI, and we're off to the races with something nobody cares about.
(I just really want to be able to write Perl filters for incoming mail, dammit.)
So ... Mail::MboxParser looks like a good start, maybe. Unfortunately, it relies on Mail::Mbox::MessageParser, which also assumes a Unix environment that may or may not pertain (CPAN testing fares poorly on Windows platforms - including mine). The reason here seems to be that it wants to incorporate compressed mailboxes using an external gzip. Why is that in the core module? Ugh.
I dunno. MIME::Tools is certainly definitive for handling individual messages, so perhaps I don't even want to worry about anything but that. Even MIME::Tools is horrible enough.
Email is a mess.
SMT engine. So help me, I'm just a sucker for this type of thing. Microsoft uses it to validate drivers (example). Anyway, Yices is an example. Background. And the trigger - a Haskell DSL for expression of SMT problems. This is the kind of thing I want to be doing here.