Sunday, July 10, 2011

Thoughts while doing sysadmin

Task: pruning some context tables that contain far too much very old information.

I'm struck again by the fact that my memory can't retain SQL syntax for longer than about a month. When I need to add an index, I have to look it up again. And of course there's no record of what I did last time unless I go back through anything I might have noted down on paper.

I need an administrative tool that can encapsulate actions taken - and log them. People have successfully used VCS systems to keep track of changes to SQL, but a workflow enactment kind of thing would be better. (Perhaps in combination with a VCS system.)

So again: the action.

Database administration systems are a dime a dozen, of course. It wouldn't be hard to slap something like that together based on the transaction patterns a couple of posts back, in combination with this notion here. Deserves some thought - especially given that the same concept could be extended to Unix commands in general. Administration of cron jobs, whatever.

In a larger sense, a sysadmin domain would be able to construct a picture of a system of some nature that could be used to inform the construction of this type of action. This thread of thought is still really vague, but I keep coming back to it. (And have for decades.)

Final solution: wrote a Perl script for each table that gently removes 10K records at a time, and put it in the crontab to run every 10 minutes. Building an index would lock the tables for hours, disabling the database for an unforeseeable period. This is the kind of thing that I should be able to reuse effortlessly.

No comments:

Post a Comment