C[omp]ute

From: andrew cooke <andrew@...>

Date: Sun, 5 May 2013 23:07:47 -0400

I just finished the crypto challenge.

I (vaguely) remember, many years ago, sitting down to write a library that
would help me with cryptanalysis (I've always be interested in codes and the
like).

Back then I wasn't (I hope) as good a programmer as I am now.  Or, at least,
not as experienced.  And I remember getting stuck in analysis paralysis quite
quickly.  The project never got very far.

Looking at the code for the crypto challenge, I have effectively done what I
set out to do years ago.  And I think there are some lessons to learn from
that.

Most obvious, a general point about software development - it's much easier to
write code when you're solving a real problem (rather than trying to develop
something to address an abstract set of issues).  This is much of the essence
of "Agile".

But there are also some specific details that I am pleased with:

 - I used a very general approach for "rating" the results from using various
   keys, based on pluggable tokenizers and dictionaries of scores.  This let
   me use the same code for letter and word frequency analysis.  The Python
   Counter class was particularly useful, as was a simple hack to print the
   contents as a histogram (on it's side, each "column" a row of stars).

 - Coroutines as actors worked well for simulating protocols
   http://www.acooke.org/cute/UsingCorou0.html

 - Higher order functions worked nicely for constructing operations from
   simpler primitives - both building cipher modes from ECB and hashes via
   Merkel Damgard http://www.acooke.org/cute/PurePython0.html

And one where I could have done better:

 - I didn't find a good balance between generators and sequences.  Generators
   are great for working with sequences of bits, bytes, words.  And for
   composing pipelines.  But at a higher level (eg output from hashes) you
   want byte arrays that you can add together like strings.  Instead I let
   generators "get too high" and then had to explicitly add bytes().

Andrew