All,
I am behind the throttling rollout in the last 24 hours, and have worked
with Theo to loosen things up for now. I've also reached out to pierky re
changes requested for arouteserver and will endeavor to delay resumption
of the same throttling until after arouteserver users have had reasonable
time to upgrade.
Highlights for all client developers:
- Implement support for PeeringDB API keys:
https://docs.peeringdb.com/howto/api_keys/
The idea being that we will throttle users using API keys less.
- Add a delay in between queries that is randomly between 2 and 2.5
seconds, to reduce thundering herd. This delay will mean a client
queries PeeringDB at most 30 hits per minute, which will be unthrottled
if they are authenticated using API keys and not making identical
requests.
- Highly preferred over separate queries: If you don't need non-public
contact info from PeeringDB, is that you implement peeringdb-py
(peeringdb-py: http://peeringdb.github.io/peeringdb-py/) client-side
caching. Doing so enables you to locally query the heck out of a local
sqlite (or whatever) database. The start time of a peeringdb-py run
should be randomized per the docs
(http://peeringdb.github.io/peeringdb-py/cli/). At the SeattleIX we use
peeringdb-py and here is what the once per hour update looks like for
all of PeeringDB:
[17/May/2022:15:40:09 +0000] "GET /api/org?since=1652794724&depth=0 HTTP/1.1"
200 392 "-" "PeeringDB/1.2.1.1 django_peeringdb/2.13.0" 0.423
[17/May/2022:15:40:10 +0000] "GET /api/fac?since=1652773361&depth=0 HTTP/1.1"
200 24 "-" "PeeringDB/1.2.1.1 django_peeringdb/2.13.0" 0.409
[17/May/2022:15:40:10 +0000] "GET /api/net?since=1652796557&depth=0 HTTP/1.1"
200 1695 "-" "PeeringDB/1.2.1.1 django_peeringdb/2.13.0" 0.426
[17/May/2022:15:40:11 +0000] "GET /api/ix?since=1652397370&depth=0 HTTP/1.1"
200 24 "-" "PeeringDB/1.2.1.1 django_peeringdb/2.13.0" 0.397
[17/May/2022:15:40:11 +0000] "GET /api/ixfac?since=1652763759&depth=0 HTTP/1.1"
200 24 "-" "PeeringDB/1.2.1.1 django_peeringdb/2.13.0" 0.414
[17/May/2022:15:40:12 +0000] "GET /api/ixlan?since=1652781160&depth=0 HTTP/1.1"
200 24 "-" "PeeringDB/1.2.1.1 django_peeringdb/2.13.0" 0.399
[17/May/2022:15:40:12 +0000] "GET /api/ixpfx?since=1652429334&depth=0 HTTP/1.1"
200 24 "-" "PeeringDB/1.2.1.1 django_peeringdb/2.13.0" 0.408
[17/May/2022:15:40:13 +0000] "GET /api/netfac?since=1652790428&depth=0
HTTP/1.1" 200 318 "-" "PeeringDB/1.2.1.1 django_peeringdb/2.13.0" 0.553
[17/May/2022:15:40:14 +0000] "GET /api/netixlan?since=1652796556&depth=0
HTTP/1.1" 200 399 "-" "PeeringDB/1.2.1.1 django_peeringdb/2.13.0" 0.590
[17/May/2022:15:40:14 +0000] "GET /api/poc?since=1652785835&depth=0 HTTP/1.1"
200 24 "-" "PeeringDB/1.2.1.1 django_peeringdb/2.13.0" 0.640
It is fast because, as I understand it, django serializes PeeringDB
changes, and the timestamp (since last update) results in only the
changes being delivered.
Finally: My apology to those disrupted by this. Please feel free to reach
out to me with any questions or concerns.
Thanks,
Chris
_______________________________________________
Pdb-tech mailing list
Pdb-tech@lists.peeringdb.com
https://lists.peeringdb.com/cgi-bin/mailman/listinfo/pdb-tech
