Why Perl didn't win

Whew. Yeah.

C::Blocks and Devel::Declare

There are a couple of different ways to hook into the Perl parser and define a keyword that flags your own funky syntax - neither of which I'd heard of until encountering the absolute genius that is C::Blocks.

C::Blocks embeds C code right into your Perl, using a cblock { this is C here } syntax that is just incredibly groovy. It's way more inline than Inline. OK, so it's not production-ready yet, but still - it will be. And I'm going to do that same thing for HBScheme.

So anyway, C::Blocks uses the shiny new pluggable keyword API to do that (introduced in 5.12) - but Devel::Declare lets you do pretty much the same thing, but less elegantly and safely. The difference appears to be that the pluggable keyword is available from XS and expects you to return opcodes (that is, it's a real live hook into the interpreter) while Devel::Declare runs in Perl and returns strings that will then be interpreted by the Perl interpreter. I might be wrong about that, but I haven't really gotten into it yet.

Yet.

Article: Perl and Windows UAC

New article written on the Vivtek site, a little in-depth investigation of Windows UAC and how to manipulate it from Perl, along with the release of a CPAN module.

Article: KeePass through SSL with Perl

New article at the Vivtek site on accessing KeePass using the KeePass plugin from Perl. I ran through progressively more elegant prototypes before coming up with a nice wrapper. I released the whole shebang as a CPAN module WWW::KeePassRest, which uses a new JSON API wrapper that is ... minimalistic in its design.

It'd be nice to be nice and principled about API wrappers on something like this basis, but that's definitely way down the priority list.

Win32::Exe

Oh, here's a cool little thing Mark Dootson did to manipulate executable files on Windows: Win32::Exe.

Log4Perl

Wow. Log::Log4Perl implements the perfect in-code logging system for all your Perl coding needs. It is a thing of sheer beauty.

Compling C++ to a DLL and calling it from Perl

Detailed instructions. Just in case.

COM stuff

And then there is just a whole passel of stuff about COM in Perl - again.

• Get the type name of a COM object
• A truly excellent rundown of C++ COMming.
• AutoHotKey can do it, too.
• Getting to the Registry in Perl is boring.
• Registry.
• ProgIDs in the Registry - this would probably be a smaller doable module.
• Registry. Registry.
• And finally Tcl for the win with COM!

GetMessage

I already used this page when writing Win32::MessageLoop. Easy stuff.

Scheme. In Perl?

You know, TinyScheme is essentially in the public domain, and would really be easy to embed into Perl. In fact, you could just use Perl's own scalars as a basic Scheme type and get most of the R5RS spec for free that way...

I think I want to think harder about that. There are a lot of things I want to do in Lisp-y environments without giving up my CPAN, and TinyScheme is really, really lightweight. It's quite conceivable that you could also use the Perl module structure (and CPAN) to organize your Scheme libraries. Hmm...

But even more fascinatingly, a Scheme embedded in XS could also be used to get some introspective (and on-the-fly!) access to other XS-defined items. Say, OLE.XS, which still has me stymied in my effort to get IEMech up and running in the modern age.

So - TinyScheme is probably the lightestweight choice, but there's also Chibi Scheme that looks a more full-featured alternative, plus of course there's already Inline::Guile for serious scheming.

(I'm going to go with TinyScheme, though, and make something self-contained, a la SQLite, that already has everything you need to start with Scheme.)

Oh, here's a link on what hygienic macros are. Makes sense.

Anyway, the motivation for all that is that Sussman physics-in-Scheme course. That thing is incredibly densely written! Fortunately I'm married to a theoretical physicist who, so far, appears impressed at my staying power and is more than happy to talk about anything I find unclear. At length.

Windows COM and OLE Automation

Remember how I was just going to get Win32::IE::Mechanize up and running again?  As if! Turns out newer versions of IE depend far more heavily on custom COM interfaces than did older IE versions, and Perl's Win32::OLE just doesn't do that.

So if I want to automate IE, I'm going to have to do that.

Unfortunately, the code for Win32::OLE is atrociously documented. Well, let's put it this way: the documentation principle used was RTFC. If you can't understand both Windows COM in C++ and perlguts manipulation of stashes in the same code, you clearly don't need to be doing COM in Perl, apparently.  It's horrible.

So down the rabbit hole I go. I've been reading a lot about COM and OLE Automation. Let me boil it down to the basic points for you.

• Windows uses COM [msdn] as its interprocess communication standard. It's actually pretty freaking neat, but Microsoft never met a set of documentation they thought wasn't obscure enough, so most of what Microsoft has written about how to use COM is horrible. And of course they also have no interest in enabling the use of their technology without purchase of their programming tools, so what you do find will largely assume you're using their MFC foundation classes for C++. That, or Visual Basic. Or now, .NET.
• COM calls communicate using interfaces. You create an object (or latch onto one running), and that object might be in a different process or even on a different machine. Windows handles all that for you; you just communicate with the interface, which might be a local stub. All data is encapsulated into VARIANTs, which are essentially little typed data blobs that work pretty much like Perl scalars.
• Interfaces can inherit from one another - with the restriction that only single inheritance is allowed, and new methods are just appended to the list of inherited ones.
• IUnknown is the basic interface.
• IDispatch is the interface for OLE Automation [wp]. It's the only interface supported by Perl's Win32::OLE. Therein lies our problem. IE is no longer built to be driven through IDispatch. Why? I'll tell you why: because Visual Basic now knows how to use non-IDispatch interfaces. Simple as that. Microsoft restricted their development to IDispatch only to give VB a chance to catch up. That catching up is .NET.
• A typelib is used to define the interfaces; in the absence of a typelib you just need documentation. If you don't have documentation, you can't use the interface; I believe this is by design, as Microsoft's business model relies on the ability to provide secrecy. IDispatch provides some reflection tools, but they're weak. And of course they only work if IDispatch is implemented. Win32::OLE::TypeLib provides some interface to that (see?) but it is undocumented. Urggh.
• An interface inherited from IDispatch is called dispatched; an interface inherited from IUnknown but not IDispatch is called custom. Win32::OLE explicitly does not cover custom interfaces; since the advent of .NET, though, Microsoft coding has increasingly wandered over there, because custom interfaces provide a reasonable namespace convention and also provide an easy upgrade path.

So if I want to rewrite Win32::IE::Mechanize to work with IE versions greater than about 7, I need to write Win32::COM, essentially from scratch.

Which is a fantastic opportunity! OK, it's not what I wanted to do in order to automate IE this year, but still - it's actually not that horrible. COM isn't as opaque as everybody makes it out to be, and I've been working towards this general thing anyway for years.

The basic guts of COM can actually be supported with very little code: I've run across a fantastic OLE tutorial written by Bartosz Milewski, who wrote and markets a neat code sharing product that works either with peer-to-peer or email connections. That's rad! Anyway, in the distant past, Milewski himself was at Microsoft, heading a product team, and had some good suggestions about how to organize COM - which Microsoft ignored. So now he has a tutorial that addresses OLE as it actually makes sense. And yeah, that's going into Perl now. Another take on how the whole thing works is here, by Chris Oakley. And there's a neat little library for C/C++ that takes a lot of the sting out of things, DispHelper.

(The rest of Milewski's tutorial is pretty salient, too, in terms of working with Windows, and honestly it should probably all end up in Perl. With a tutorial. And everything else on the Reliable Software site is equally fascinating. It's all good.)

Anyway, here's a little snippet of code [from here] showing how interfaces are explicitly referred to in VB:

'Create a html document class
Dim htmlDocument As IHTMLDocument2 = New HTMLDocumentClass()
htmlDocument.write(htmlToParse)
 
'Get all elements present in the document
Dim allElements As IHTMLElementCollection = htmlDocument.all

See that? IHTMLDocument2 isn't a type of object - it's an explicit specification of the interface to be used for the HTMLDocumentClass created. Same with the IHTMLElementCollection. Win32::OLE doesn't allow us to specify these interfaces and can't call them if we did.

Ideally, a new implementation of COM under Perl should permit not only consumption of COM interfaces, but provision as well. To a certain extent this is already solved in the event handling in Win32::OLE (which presents IDispatch to the outside world), but it's a weak solution. We can do better, if I can figure out how the whole process thing works. Then we can even register Perl to provide automation objects we can call from Word, for example. Wouldn't that be cool? Yes. Yes, it would be cool.

To make that happen the way I want to make it happen, I want to implement something like a declarative typelib provider. There's such a tool as a typelib viewer [here's the overview at AutoHotKey of the Microsoft OLEVIEW.exe tool]; not only does Microsoft provide that, but ActiveState Perl does as well. I'd much rather have one available on CPAN. That tool will be a command-line thing that outputs a report, and the report will be usable by the COM tools, both to provide an interface and to call it - and to document it.  I've been moving in that direction anyway for Office. This is the only real way to manage this stuff. This part is essentially database work.

Once that interface definition is there, we could also conceivably use it not only to do Perl things with it, but to generate code in other languages as well, for example to generate interface boilerplate in C++ a la this tool on Sourceforge or to build an XS framework for the same thing in a Perl module.

So we've got several things to provide, here:

• Win32::COM module that wraps up just the essentials of COM.
• Adapt or steal the other stuff that Win32::OLE provides, like typelibs, and for God's sake, let's document their use as well. Maybe we can just use them directly, I don't know. Whatever can be used instead of replaced, should obviously not be replaced.
• Better discovery and reflection in general of COM interfaces.
• Finally, a set of declarative tools for working with COM interfaces in Perl and possibly in other languages.

After the very first step, we can probably already get back to IEMech. Sheesh. Every time you think you're done pushing down the task stack, it turns out there's more cruft down there. I may have mentioned that I used to own a 140-year-old house. Perl is like that.

Hey, incidentally, Win32::OLE would be a great target for code understanding.

perlbrew

Multiple Perls on one system? Perlbrew! I'd never really heard of it until today. Also: Polish URL for the win in the Perl ecosystem!

OCR in Perl

Neat little DIY article that hits my sweet spots.

Yegge on Perl

Steve Yegge always makes you think, even when he's doing something so preposterous as challenging the One True Language.

not that he's wrong

What?!?

Yeah, I have to admit he's right. Perl is a convenient place to start with a lot of problems because as glue for CPAN it really gets you close to a quick solution for nearly anything on Earth. But ultimately I think we're going to find that any one programming language is simply not going to be sufficient for every need. That, after all, is the entire point of this very blog. So even though I have a kneejerk negative reaction to any criticism of Perl, my language of choice these past ten years or so, he's still right. Perl is an antique.

I just happen to like antiques.

And honestly, seeing the Perl expression of any idea as merely the cave-wall projection of the actual program, I have to say that starting out in Perl isn't such a bad way to start approaching a problem. It's just that I find myself drifting off the rails to language design fairly quickly in the course of any given larger project.

A possible way to combine the command line with Wx

Something that's bothered me for a while with Perl/Wx programming is that I'd really rather have a terminal that's easy to manage for command input. Since the DOS box is right there, it's irritated me that I can't just use it.

Maybe I can: Perlmonks from 2010. Then combine that with AnyEvent::Run for starting other programs...

Taking over CPAN modules

Here are some notes on taking over CPAN modules.  I can officially confirm that this works, because I have taken on co-maintainership of Iterator::Simple.  (Yeah, I'm still pretty enthused.)

• The official PAUSE guide to taking over a module.
• Identifying CPAN modules that need help

In the comments to the latter one, Neil Bowers describes his own procedure, which is the one I used:

• Find and solve a problem with a module.  (This is sometimes the hard part.)
• Post a patch to RT.
• Fork the module on Github (this is a little pie-in-the-sky, so put the module on Github, the point being to have an open repository you can refer to)
• Don't forget to update the repo with meta-information to point to itself.  (I didn't do this for Iterator::Simple, durr.)  More Github integration is a benefit to everyone.
• Two weeks after this, email the author at whatever email addresses you can find, remind him or her of the patch, point to the Github repo, and offer to take over co-maint.  Copy modules@perl.org on this email.
• One month after that, notify PAUSE that you haven't heard back.
• Wrap and upload your new version.

Wash, rinse, repeat: slow but certain world domination.

Moe

Moe is a new project to reimplement Perl 5 on Scala.  It's crazy enough it might just work!

Seriously, if it's compatible enough that it can run CPAN, I'm moving.  To that end:

• Github (my fork)
• Scala
• A presentation of the plan

Showing a repository link on MetaCPAN

Szabó Gábor weighs in to plug a gap in documentation.

Maybe Perl 6 will be kinda cool

You know I'm not ready for Perl 6 yet, but still.... This can be defined in Perl 6:

die "resistance is futile" if !($resistance ~~ 4.7kΩ ± 5%);

Stanford Core NLP via Perl

Well this is neat!