## Matasano - Programming Lessons Learned

From: andrew cooke <andrew@...>

Date: Sun, 5 May 2013 23:07:47 -0400

I just finished the crypto challenge.

I (vaguely) remember, many years ago, sitting down to write a library that
would help me with cryptanalysis (I've always be interested in codes and the
like).

Back then I wasn't (I hope) as good a programmer as I am now.  Or, at least,
not as experienced.  And I remember getting stuck in analysis paralysis quite
quickly.  The project never got very far.

Looking at the code for the crypto challenge, I have effectively done what I
set out to do years ago.  And I think there are some lessons to learn from
that.

Most obvious, a general point about software development - it's much easier to
write code when you're solving a real problem (rather than trying to develop
something to address an abstract set of issues).  This is much of the essence
of "Agile".

But there are also some specific details that I am pleased with:

- I used a very general approach for "rating" the results from using various
keys, based on pluggable tokenizers and dictionaries of scores.  This let
me use the same code for letter and word frequency analysis.  The Python
Counter class was particularly useful, as was a simple hack to print the
contents as a histogram (on it's side, each "column" a row of stars).

- Coroutines as actors worked well for simulating protocols
http://www.acooke.org/cute/UsingCorou0.html

- Higher order functions worked nicely for constructing operations from
simpler primitives - both building cipher modes from ECB and hashes via
Merkel Damgard http://www.acooke.org/cute/PurePython0.html

And one where I could have done better:

- I didn't find a good balance between generators and sequences.  Generators
are great for working with sequences of bits, bytes, words.  And for
composing pipelines.  But at a higher level (eg output from hashes) you
want byte arrays that you can add together like strings.  Instead I let
Andrew