The Organized Crime and Corruption Reporting Project

The OCCRP is another of those data journalism efforts that make me drool. Text analysis, oooh.

ETL

Saw a post on the Perl jobs list yesterday for an ETL expert. Apparently this is the name Big Data people give to batch data transformations: Extract, Transform, Load.

What I just call "indexing" in the greater sense.

Cool.

Analytics applied to MarioKart

This is pretty fun: Kartlytics. It's a Joyent fun article presenting their product Manta, which is an object store apparently for Big-Data-as-a-Service (BDaaS, I like that).

Cool stuff all the way down.

Hancock

Hancock is a language developed at AT&T in the late 90's to do searching of incoming data on a bulk basis, essentially to implement metadata surveillance. Wired (2007) and HNN commentary.

H2O math for Hadoop

The H2O project provides a math runtime for Hadoop, extending it for big data, statistics, machine learning, all that jazz.

PSPP

PSPP is an open source alternative to SPSS, IBM's statistical analysis package.

Using metadata to find Paul Revere

Analysis of membership data to find influential colonial terrorists.

Data sources

Another cool post about finding neat data sets.

Big Data in 2013

Nice picture of the state of Big Data.

Data science architecture

Now here's a useful post: an overview of Big Data architecture.

12 online tools for big data

Big Data, as a buzzword, is starting to leak.  I wonder what will come next?  Good list of tools/startups, though.

Berkeley: Big Data with Twitter

A blog post at Berkeley apparently recapping a class just completed.

Interesting big data/textual analysis blog

Another Word For It.  The author is interested in topic maps, an automatic semantic structuring technique for large document collections.

My problem: I have never developed the habit or a toolset for consuming news.  If something bubbles up on HNN, that's historically been enough for me (before HNN, it was just whether I encountered something in other online communities; HNN is a lot more efficient in finding things I like, though).

I need to do something about that.

Update 2012-11-06: And then I find CPAN module "TM" (topic maps) today - freaky!

Common Crawl code contest

Cool.  They want volunteers (I don't (yet) qualify as a paid coder there).  I should really look into that.

AMP Camp

AMP Camp was a Berkeley thing about big data that I didn't have time for.  Maybe soon.

Python for data analysis

O'Reilly has a new book out on data analysis.  I'm sorely tempted.

Getting experience with big data

A list of suggestions of how to get initial experience with large datasets.

Big Data links

A few things about big data:

• A book on best practices by Marz and Ritchie. You can pre-read it as well. I'm thinking about it. [hnn thread]
• The data science toolkit, a specialized Linux VM for download with lots of juicy databases and tools preinstalled and a common API.
• Real-time feed processing with Storm. Storm abstracts out the queue-workers pattern into an infrastructure component. Interesting stuff.

Feedback models of big data usage

Interesting article at O'Reilly on modeling the use of data in companies. Feedback being good.

Big data predictions for 2012

Interesting article, for which I have time measured in microseconds. It's more time than I have just to post a link to it here.