Topics

Rate limits for non-badge-image requests


David Wheeler
 

Some overeager people are trying to spider the entire best practices site all at once. This can cause trouble for everyone else. Our current rate limits don’t trigger soon enough, because they cover *all* requests, and we can handle many badge image requests.

So I propose adding a new rate limit for anything OTHER than badge images & static files. Details here:
https://github.com/coreinfrastructure/best-practices-badge/issues/1475
https://github.com/coreinfrastructure/best-practices-badge/pull/1478

The default rate limit I’m proposing is up to 15 requests every 15 seconds. That short time window will let us detect, far more quickly, when someone is making too many requests at once. It could be different, e.g., 30 requests / 15 seconds or 20 requests / 10 seconds. Recommendations welcome. The goal is to make it invisible to “normal” users, but stop abuses quickly.

I’d especially like to hear from anyone whose dashboard might be negatively impacted. If you just serve CII badge images it shouldn’t impact you at all.

If we use the CDN to serve the JSON data about individual projects we could exclude that as well, but that would be a different change.

--- David A. Wheeler


Kate Stewart
 

Adding Sean to this thread, as CHAOSS risk metrics have a dashboard
that uses the CII badge information.

Sean - any impact expected from your perspective?

Thanks, Kate

On Thu, Oct 1, 2020 at 7:30 PM David Wheeler
<dwheeler@...> wrote:

Some overeager people are trying to spider the entire best practices site all at once. This can cause trouble for everyone else. Our current rate limits don’t trigger soon enough, because they cover *all* requests, and we can handle many badge image requests.

So I propose adding a new rate limit for anything OTHER than badge images & static files. Details here:
https://github.com/coreinfrastructure/best-practices-badge/issues/1475
https://github.com/coreinfrastructure/best-practices-badge/pull/1478

The default rate limit I’m proposing is up to 15 requests every 15 seconds. That short time window will let us detect, far more quickly, when someone is making too many requests at once. It could be different, e.g., 30 requests / 15 seconds or 20 requests / 10 seconds. Recommendations welcome. The goal is to make it invisible to “normal” users, but stop abuses quickly.

I’d especially like to hear from anyone whose dashboard might be negatively impacted. If you just serve CII badge images it shouldn’t impact you at all.

If we use the CDN to serve the JSON data about individual projects we could exclude that as well, but that would be a different change.

--- David A. Wheeler