Best bet is anycast routing. BGP peers distributed among the world, all
advertising the same block of addresses brings natual geographical
proximity as long as bgp as-path hops are somewhat valid. Done in a /24
or larger (2+ for disparity), that becomes your "anycast" subnet,
advertising to as many peers as you can justify/afford.
This is what dns shops do, as well hosting providers that have to absorb
large bursts of traffic (ahem, ddos) to distribute among infrastructure
to dissect and mitigate it. Likewise you just setup farms of
reverse-proxy forwarders behind each block, from there you can snat,
x-forward-for inject the packets along anywhere you need to on back-end
data center interconnects (or DSR load-balance if frisky). Put the
intelligence of the source ip/client at the reverse proxy layer.
Global-dns can only do so much, relying on recursion, but better to
field the requests first without the dns tricks in anycast ip routing,
as it's the only thing upstream from dns protocol itself for providers
to give resiliency and distributing load "globally", naturally.
-mb
On 08/06/2014 11:17 PM, David Schwartz wrote:
Here’s something interesting for the infrastructure geeks on the list ...
How would you approach setting up a service that had to sink around, oh … say …
10-20 million small HTTP POST requests per minute throughout the day, from
sources geographically distributed around the country?
To do development and get the logic working, a small server is sufficient. But
it needs to scale quickly once it’s launched.
There will be a high degree of geo-locality, so servers could be set up to
handle requests from different geographic areas. HTTP requests from a given
area would be routed to whatever server is dedicated for that area. I guess
their IP address could be used for that purpose?
(How granular is the location data for IP addresses on mobile devices? Are they
reliable? We could add a location geotag to the packet headers if that would
help.)
Note that the servers don’t need to be physically LOCATED in the area; rather,
they're dedicated to SERVING a well-defined geographic area.
There’s no need for cross-talk, either. That is, there’s no need for a server
serving, say, the LA area to cross-post with one in San Diego, except in a very
small overlapping area which is easy to address.
Can this sort of routing be done with a DNS service? (eg., DNSMadeEasy.com is
one I’m familiar with)
Or is something more massive needed?
Also note that this would be an automated service. It has a very steady stream
of small incoming packets, peaking at various times of the day, with limited
responses. No ads, no graphics, no user interactions at all.
I know there are infrastructure services in place to handle this kind of thing,
like what Amazon offers, and others. I’m looking for any specific pointers to
services that might fit this use case profile.
-David
---------------------------------------------------
PLUG-discuss mailing list - [email protected]
To subscribe, unsubscribe, or to change your mail settings:
http://lists.phxlinux.org/mailman/listinfo/plug-discuss
---------------------------------------------------
PLUG-discuss mailing list - [email protected]
To subscribe, unsubscribe, or to change your mail settings:
http://lists.phxlinux.org/mailman/listinfo/plug-discuss
