C[omp]ute

From: "andrew cooke" <andrew@...>

Date: Sat, 27 Sep 2008 04:34:37 -0400 (CLT)

The Application

I am writing a Java application that presents information, stored in a
database, via a web server.  The presentation does not modify the data,
but the database does evolve, slowly, on disk (modified by other
processes).

The data themselves can be divided into two groups: first a set of related
objects that describe the system being measured (in this context they
could be considered "metadata"); second a set of measurements.  There are
many more measurements than there are metadata.

Some iBatis Details

iBatis provides very simple caching at the SQL statement level.  So if a
query is repeated with the same parameters then the result may be
retrieved from the cache.  The advantage of this approach is that it is
simple to implement and understand.  The disadvantage is that it is not
"intelligent", in that it does not recognise object identities.

For example, if I have two different queries that both return an instance
of the same class, iBatis will cache two different instances of the same
object for each key - one from each query.

iBatis is also limited in how it will map graphs - it does not provide
transparent retrieval of related objects.  So if one class references
another class then either both must be mapped in a single query to iBatis,
or a second call must be made explicitly (in which case the first instance
presumably contains a key that will be used to find the correct value).

This limitation can be removed by using a second layer - for example
Spring AOP.

My Solution

I have decided to use a very simple mapping approach with two levels of
cache.  The first level of cache is the iBatis statement cache; the second
is a set of maps from keys to instances for the "metadata" objects.  This
second level holds all metadata instances in-memory.  The caches are in a
singleton "database interface" class.

The connections between related objects are not explicit in the objects
themselves.  Instead, the "database interface" provides methods that
retrieve related objects (for example "List<Foo> getFoosForBaz(Baz)").

One drawback to this approach is that it is not very "OO".  The instances
do not form a graph that can be traversed directly.  For this particular
application that is not an issue - there is very little "analysis" of the
data (it is just "dumb presentation").

Another drawback is that for each metadata class the ORM manages two
different types - instances and keys.  iBatis must return instances when
asked for "all instances" to populate the cache, but must return only keys
when asked about relations between objects (the keys are then used to
retrieve instances from the second level cache).  In practice, with some
basic engineering (return factory objects rather than keys; these take the
"database interface" and extract the cached value), the extra work is
minimal.

The advantages are that the scheme is fast, compact, and relatively easy
to understand.  It is fast because almost all information is cached:
metadata instances are cached in the second level caches; relationships
are cached (as lists of keys) in the iBatis cache.  It is compact because
no duplicate instances are created (except when a cache expires).

The simplicity is not so obvious until you start to consider details like
cache expiry.  Because there are no explicit relations each pool of
instances (caches are grouped by class) can expire independently.  There
is no worry about instances being "trapped" in complex graphs.  There
should be no memory leaks.

This approach is very different to how I used SQLAlchemy in Python.  The
approach I used there was much more "sophisticated", with transparent,
lazy retrieval of related objects.  In that case I was writing a client
application that did not need to be long-lived, reliable, or efficient
(within reason, of course).

It's possible that I am being too conservative in this case.  Spring AOP
(or Hibernate) might have made the code more transparent at no extra cost
in practice.  But I think this was a reasonable approach to use here,
given the circumstances (both the application - particularly the "read
only" nature - and my limited knowledge).

Andrew