Rate limits for non-badge-image requests

David A. Wheeler

Some overeager people are trying to spider the entire best practices site all at once. This can cause trouble for everyone else. Our current rate limits don’t trigger soon enough, because they cover *all* requests, and we can handle many badge image requests.

So I propose adding a new rate limit for anything OTHER than badge images & static files. Details here:

The default rate limit I’m proposing is up to 15 requests every 15 seconds. That short time window will let us detect, far more quickly, when someone is making too many requests at once. It could be different, e.g., 30 requests / 15 seconds or 20 requests / 10 seconds. Recommendations welcome. The goal is to make it invisible to “normal” users, but stop abuses quickly.

I’d especially like to hear from anyone whose dashboard might be negatively impacted. If you just serve CII badge images it shouldn’t impact you at all.

If we use the CDN to serve the JSON data about individual projects we could exclude that as well, but that would be a different change.

--- David A. Wheeler

Join CII-badges@lists.coreinfrastructure.org to automatically receive all group messages.