Identifying programmers

Another application of NLP techniques to source code. Interesting.

Article up, and also: exegesis and code analysis/understanding

My first article kinda-sorta based on an exegetical approach (of my own prototype code) is up on the Vivtek site, and the six-day code rush to write code to build the article from a note database has really whetted my appetite for more of the same. There are all kinds of exegetical efforts I want to make, getting into code reading in a big way.

So I trolled around Google for salient things. Here's a list of interesting things.

• TipsForReadingCode at c2.com was quite helpful as a set of ... well, tips for reading code.
• Code comprehension tools at grok2.com; the vast majority of this class of tool is closed-source and rather expensive. I think this is largely because large codebases are not typical for open source projects, but rather enterprise code, and enterprises have the money to pay for expensive tools. That's my reading, anyway. But:
• cscope is a venerable tool for static analysis of C code, anyway, and has been perverted to handle other large-scale grep-like analyses of large numbers of files. The tool itself may or may not be something I'm interested in, but its approach is probably pretty valid.
• cflow is another flow dependency analysis tool, also open source.
• Perl is, as always, a special case.
• Well, what about static code analysis in general? Here's another list of tools.
• There's a clang-based analyzer.
• This moribund project on code "aspects" looks fascinating.
• Finally, a book on pattern-based OO refactoring, which also looks pretty fascinating.

So, as always, lots of people are doing things tangentially related to what I want to do, but nothing is 100% there - because what I want to do is an extract-then-literate-programming kind of thing. We'll see how much sense this approach makes. Wish me luck.

GitHub's Linguist

Linguist is what GitHub uses to figure out the (programming) language in a given file.

Decompiling, reverse engineering tools

More on code analysis!  Apparently April was the month for it. Note that this is an HNN post, not an article; the article pointed to is not actually all that interesting but the discussion is.

Valgrind

Open-source code analysis and profiling tool.

ydiff: structural diff for code

Interesting.

Grok

Here's a fascinating little article about a Google project called Grok - led by Steve Yegge, who's been doing some equally interesting bloviation about programmer mentality lately (of which more in a separate post).

Grok is still close to the vest, but it's presumably about code analysis as expressed in an improved and unified build system.  It has subsumed the no-longer-public Google Code (durnit).  In short, it's making me salivate.

John Carmack on static code analysis

Short version: DO IT.

SourceMap: finding where the JavaScript comes from

Apparently SourceMap is a code project tracker that, given a JavaScript error, can find out which of your many files in PHP and Coffescript and what-have-you is actually responsible for the code that produce the error. Nice!

Codebot bounty

Here's a neat open-source bounty on a code bot.

Codefixbot

I just had the neatest idea. One of the nicest things about Perl/CPAN is the CPAN Testers Network - if you package your module with tests and put it on CPAN, hundreds of automated testing systems running different versions of Perl on different machines under different operating systems test it for you and email you the results.

I can't say how outstanding that is for code quality.

So. Code quality. I posted a couple of days ago about code quality. Here's the first iteration of my idea: a generic code quality tester that would crawl open-source repositories (aside: since the demise of Google Code, there is no universal code index, and that should change), identify problems, and if on e.g. Github, automatically create and submit a pull request to fix common errors. But otherwise attempt to format an easy patch and get in touch with the authors.

That's cool enough - a sort of universal code quality assurance system that would just ... fix everything it finds. But then I came to the second iteration of my Good Idea, which is something even more interesting: a code generator bot. Let's say I have some kind of idea and a language to express it in - the bot could come by and generate, say, C++ or Java code to implement my idea.

OK, granted, that's vague. But surely there's some kind of continuum there from the easy to imagine (code quality automation on the loose) to the Singularity (write a blog post about something you want, and the Internet implements it for you and links to it from a comment). And honestly - what a philanthropic opportunity!

So now I know what I want to do this year. Just gotta jam it onto the priority list.

Javascript pitfall: missing var

A heartrending account of mistaken globality. Killer comment from HNN: jshint, stupid.

So.... Code quality tools in general. I want to build a framework. Gauntlet thrown.

An aside on machine learning, and open-source contribution

So having forced my brain to code a vectorized cost function in Octave starting from the equation - a task that truly taxed skills that had lain dusty for decades, and involved a brief discussion with my private theoretical physicist - I've started to think maybe I might be capable of learning a new trick or two. This Stanford class just barely scratches the surface, of course, and my mathematical background is essentially nil, so I've got a steep hill to climb.

But. There are open-source machine learning projects out there. Perhaps it might be best to start contributing. So on that note: the mloss.org project database. 334 projects and counting.

And one of the things that caught my eye this week on the software development front is PVS-Studio, a static C/C++ code analyzer that finds common coding errors. There was briefly an article on it listing 91 such errors, but it was deleted. Of course, it would be a hell of a lot more interesting to have an open-source equivalent. If there isn't one, I intend to damn well start one, with a curated set of flags (this may be why the article disappeared, of course...).

Update after reading this: OK, so I'm an idiot. Sometimes it's easy to forget the last twenty years and the Internet and all. [also]

Anyway, the whole concept of static code analysis fits well with my vague idea of a "code understander" set loose on open-source code.