## A Heavyweight Python Cache

From: andrew cooke <andrew@...>

Date: Wed, 6 Feb 2013 19:12:49 -0300

Calling EchoNest (see previous post) is expensive.  So I have written a very
general cache, backed by an SQLAlchemy database.

The code is at
https://github.com/andrewcooke/uykfg/tree/master/src/uykfg/support/cache

It has the following features:

* Transparently wraps function calls (should only be used with pure
functions, obviously!)

* Can be used with multiple functions (called "owners" in the database
schema).

* Caches both values and exceptions.

* Has separate expiry lifetimes for values and expcetions (typically

* Uses fuzzy expiry lifetime to avoid cache stampede (+/-50%)

* Supports LRU rejection when size exceeds some limit (data are stored
as compressed binary blobs in the database)

* Size and lifetimes can be specified separately for each owner

The main downside is that it is pretty heavyweight.  It's a no-brainer for web
calls, but you wouldn't want to use it to cache values that can be
re-calculated locally.

Also, it cannot cache values or exceptions that cannot be "pickled".

Andrew

### Mods to Cache

From: andrew cooke <andrew@...>

Date: Sun, 10 Feb 2013 20:03:13 -0300

Since writing the above I've changed the cache slightly.

Expired values now remain in the cache (unless size becomes too large and they
are ejected by the LRU mechanism).  When the key to expired value is requested
the call is made and, if successful, the expried value is replaced.  But if
the call fails and the exceptin raised is "Fallback", then the expired value
is retained and returned as a result.

This is useful for web services and the like which may fail due to network
issues.  If those errors are caught and wrapped in "Fallback" then the
existign cache value is used, since it's better to use an old value than none
at all.

If the call raises an exception that is not a Fallback instance then that is
treated as a valid "result", cached (replacing the expired value) and
returned.  As before.

Andrew