Fuzz testing

Fuzz testing is throwing randomly perturbed inputs at a given piece of software to see what breaks. I was entirely unaware of the state of the art of fuzz testing, though. afl-fuzz is a tool that watches the execution traces of its target while reacting to the input. It can synthesize a legal bash script from nothing, by watching how bash reacts to different byte sequences. (And it discovered that bash vulnerability that made everybody upgrade last month or so.)

To which I can only say: holy Toledo. I have seen the future.

It discovered CDATA sections in XML. Randomly. Against the expectations of its author, who says, "it's an example of the fuzzer defiantly and secretly working around one of its intentional and explicit design limitations". Evolution is weird. Almost magic.

It made a legal JPG from the seed string "Hello" - again, by noticing different execution paths taken in response to different bytes of input. Here are some test sets for graphics. Interesting stuff.