C[omp]ute

From: andrew cooke <andrew@...>

Date: Wed, 13 Feb 2013 23:04:35 -0300

[This is from http://stackoverflow.com/a/14789952/181772]

EchoNest have a simple rate limit.  Each HTTP repsonse includes header values
indicating the current max connections per minute, and the total number made
so far in "this" minute.

If you have a distributed application it's not immediately clear what the best
approach is to managing your connections so that they make efficient, fair use
of the service without exceeding the limit.

My first approach was a complicated affair based on estimating external
request rates while measuring local ones.  But that's not stable and only
works well when clients are very regular.

A much better approach is to use the following formula after making a request.

  barrier = now() + 60 / max(1, (limit - count))**greedy

This tells you when you can next make a request (after barrier).  It's easy to
understand intuitively - as count approaches limit the gap tends towards 60
seconds (since count resets at 60s this is a natural upper limit).

The greedy parameter, when greater than 1, lets requests happen more quickly
when count is low.  This helps "bursty" clients when there is a low load.
I've found that a value of about 1.2 works well.

In practice this is much more stable than the earlier approach, and also so
much simpler to implement (and so, I hope, bug free).

Andrew