@Arnold Nipper Is this still relevant? The email thread is from 2022...
Chriztoffer Hansen | PeeringDB Volunteer - AdminCom Chair chriztoffer(@)peeringdb(.)com | www.peeringdb.com On Thu, 11 Apr 2024 at 13:28, Arnold Nipper <[email protected]> wrote: > > fyi, Arnold > > > -------- Weitergeleitete Nachricht -------- > Betreff: Re: [PDB Tech] new rate limiting mechanism is too strict > Datum: Tue, 17 May 2022 16:54:59 +0000 (UTC) > Von: Chris Caputo <[email protected]> > Antwort an: Chris Caputo <[email protected]> > An: [email protected] > Kopie (CC): Theo de Raadt <[email protected]> > > All, > > I am behind the throttling rollout in the last 24 hours, and have worked > with Theo to loosen things up for now. I've also reached out to pierky > re changes requested for arouteserver and will endeavor to delay > resumption of the same throttling until after arouteserver users have > had reasonable time to upgrade. > > Highlights for all client developers: > > - Implement support for PeeringDB API keys: > > https://docs.peeringdb.com/howto/api_keys/ > > The idea being that we will throttle users using API keys less. > > - Add a delay in between queries that is randomly between 2 and 2.5 > seconds, to reduce thundering herd. This delay will mean a client > queries PeeringDB at most 30 hits per minute, which will be unthrottled > if they are authenticated using API keys and not making identical > requests. > > - Highly preferred over separate queries: If you don't need non-public > contact info from PeeringDB, is that you implement peeringdb-py > (peeringdb-py: http://peeringdb.github.io/peeringdb-py/) client-side > caching. Doing so enables you to locally query the heck out of a local > sqlite (or whatever) database. The start time of a peeringdb-py run > should be randomized per the docs > (http://peeringdb.github.io/peeringdb-py/cli/). At the SeattleIX we use > peeringdb-py and here is what the once per hour update looks like for > all of PeeringDB: > > [17/May/2022:15:40:09 +0000] "GET /api/org?since=1652794724&depth=0 > HTTP/1.1" 200 392 "-" "PeeringDB/1.2.1.1 django_peeringdb/2.13.0" 0.423 > [17/May/2022:15:40:10 +0000] "GET /api/fac?since=1652773361&depth=0 > HTTP/1.1" 200 24 "-" "PeeringDB/1.2.1.1 django_peeringdb/2.13.0" 0.409 > [17/May/2022:15:40:10 +0000] "GET /api/net?since=1652796557&depth=0 > HTTP/1.1" 200 1695 "-" "PeeringDB/1.2.1.1 django_peeringdb/2.13.0" 0.426 > [17/May/2022:15:40:11 +0000] "GET /api/ix?since=1652397370&depth=0 > HTTP/1.1" 200 24 "-" "PeeringDB/1.2.1.1 django_peeringdb/2.13.0" 0.397 > [17/May/2022:15:40:11 +0000] "GET /api/ixfac?since=1652763759&depth=0 > HTTP/1.1" 200 24 "-" "PeeringDB/1.2.1.1 django_peeringdb/2.13.0" 0.414 > [17/May/2022:15:40:12 +0000] "GET /api/ixlan?since=1652781160&depth=0 > HTTP/1.1" 200 24 "-" "PeeringDB/1.2.1.1 django_peeringdb/2.13.0" 0.399 > [17/May/2022:15:40:12 +0000] "GET /api/ixpfx?since=1652429334&depth=0 > HTTP/1.1" 200 24 "-" "PeeringDB/1.2.1.1 django_peeringdb/2.13.0" 0.408 > [17/May/2022:15:40:13 +0000] "GET /api/netfac?since=1652790428&depth=0 > HTTP/1.1" 200 318 "-" "PeeringDB/1.2.1.1 django_peeringdb/2.13.0" 0.553 > [17/May/2022:15:40:14 +0000] "GET /api/netixlan?since=1652796556&depth=0 > HTTP/1.1" 200 399 "-" "PeeringDB/1.2.1.1 django_peeringdb/2.13.0" 0.590 > [17/May/2022:15:40:14 +0000] "GET /api/poc?since=1652785835&depth=0 > HTTP/1.1" 200 24 "-" "PeeringDB/1.2.1.1 django_peeringdb/2.13.0" 0.640 > > It is fast because, as I understand it, django serializes PeeringDB > changes, and the timestamp (since last update) results in only the > changes being delivered. > > Finally: My apology to those disrupted by this. Please feel free to > reach out to me with any questions or concerns. > > Thanks, > Chris > _______________________________________________ > Pdb-tech mailing list > [email protected] > https://lists.peeringdb.com/cgi-bin/mailman/listinfo/pdb-tech > > -- > Arnold Nipper > Chief Technology Evangelist and Co-Founder > > DE-CIX Management GmbH > Lindleystraße 12 | 60314 Frankfurt a.M. | Germany > Phone +49 69 1730902 22 | Mobile +49 172 2650958 > [email protected] | www.de-cix.net > Geschaeftsfuehrer Ivaylo Ivanov und Sebastian Seifert > Registergericht AG Koeln HRB 51135 > > Want to work at DE-CIX: https://de-cix.net/en/about-de-cix/careers > _______________________________________________ > Pdb-tech mailing list > [email protected] > https://lists.peeringdb.com/cgi-bin/mailman/listinfo/pdb-tech _______________________________________________ Pdb-tech mailing list [email protected] https://lists.peeringdb.com/cgi-bin/mailman/listinfo/pdb-tech
