RE: Geoip lookup
Here's a few more resources: http://www.ipdeny.com/ipblocks/ http://www.nirsoft.net/countryip/ Frank -Original Message- From: shawn wilson [mailto:ag4ve...@gmail.com] Sent: Saturday, May 25, 2013 11:27 PM To: i...@cymru.com Cc: North American Network Operators Group Subject: Re: Geoip lookup If anyone is interrested, here's a little Perl CLI util to lookup what countries registered networks within a block. There's no documentation yet, it's a .pl where it should probably be a command with a makefile installer, and Net::CIDR overlaps Net::IP. At any rate, hopefully it is useful to someone. https://github.com/ag4ve/geocidr PS - do note the -mask option (where you can define say, a 20 or 21 or 22) so that you're not sitting there banging on their DNS looking up tons of /32s for blocks CYMRU doesn't have any information on. On Sat, May 25, 2013 at 6:44 AM, John Curran wrote: > On May 24, 2013, at 10:47 AM, David Conrad wrote: > >> I replied privately to Owen, but might as well share: >> >> On May 23, 2013, at 11:57 PM, Owen DeLong wrote: >>>>>> True, according to (at least some of) the RIRs they reside in regions... >>>>> Really? Which ones? I thought they were only issued to organizations that >>>>> had operations in regions. >>> That was exactly my point, Bill... If you have operations in RIPE and ARIN >>> regions, it is entirely possible for you to obtain addresses from RIPE or >>> ARIN and use them in both locations, or, obtain addresses from both RIPE >>> and ARIN and use them in their respective regions, or mix and match in just >>> about any imaginable way. Thus, IP addresses don't reside in regions, >>> either. They are merely issued somewhat regionally. >> >> A direct quote from a recent interaction with ARIN (this was requested by >> ARIN staff as part of the back and forth for requesting address space): >> >> "Please reply and verify that you will be using the requested number >> resources within the ARIN region and announcing all routing prefixes of the >> requested space from within the ARIN region. In accordance with section 2.2 >> of the NRPM, ARIN issues number resources only for use within its region. >> ARIN is therefore only able to provide for your in-region numbering needs." >> >> I believe AfriNIC and LACNIC have similar limitations on use but am too lazy >> to look it up (and I don't really care all that much: just thought it was >> amusing). > > Indeed. This was covered in more detail in the Policy Experience Report > given at the ARIN 31, in which it was noted that we are seeing an increase > in requests for IPv4 address space from parties who have infrastructure in > the region, but for customers entirely from outside the region. This has > resulted in a significant change in the issuance rate and therefore any > estimates for regional free pool depletion. ARIN has sought guidance from > the community regarding what constitutes appropriate in-region use, should > this be based on infrastructure or served customers, and whether incidental > use outside the region is appropriate. (This topic was also on this list on > 26 April 2012 - see attached email from that thread) Policy proposals in > this area to bring further clarity in address management are encouraged. > > FYI, > /John > > John Curran > President and CEO > ARIN > > === > Begin forwarded message: > >> From: John Curran >> Subject: Re: "It's the end of the world as we know it" -- REM >> Date: April 26, 2013 10:43:51 AM EDT >> To: "nanog@nanog.org Group" >> >> On Apr 26, 2013, at 10:23 AM, Chris Grundemann wrote: >>> >>> One interesting twist in all of this is that several of these new >>> "slow-start" players in the ARIN region seem to be servicing customers >>> outside of the region with equipment and services hosted here inside >>> the ARIN region (see slide 12 on the ARIN 31 "Policy Implementation >>> and Experience Report" >>> https://www.arin.net/participate/meetings/reports/ARIN_31/PDF/monday/nobile_policy.pdf). >> >> NANOG Folks - >> >> Please read this slide deck, section noted by Chris. It explains the >> "situation"... (I would not call the sudden acceleration in IP address >> issuance a problem, per se, as that is an judgement for the community >> either way.) >> >> FYI, >> /John >> >> John Curran >> President and CEO >> ARIN > > > >
Re: Geoip lookup
If anyone is interrested, here's a little Perl CLI util to lookup what countries registered networks within a block. There's no documentation yet, it's a .pl where it should probably be a command with a makefile installer, and Net::CIDR overlaps Net::IP. At any rate, hopefully it is useful to someone. https://github.com/ag4ve/geocidr PS - do note the -mask option (where you can define say, a 20 or 21 or 22) so that you're not sitting there banging on their DNS looking up tons of /32s for blocks CYMRU doesn't have any information on. On Sat, May 25, 2013 at 6:44 AM, John Curran wrote: > On May 24, 2013, at 10:47 AM, David Conrad wrote: > >> I replied privately to Owen, but might as well share: >> >> On May 23, 2013, at 11:57 PM, Owen DeLong wrote: >> True, according to (at least some of) the RIRs they reside in regions... > Really? Which ones? I thought they were only issued to organizations that > had operations in regions. >>> That was exactly my point, Bill... If you have operations in RIPE and ARIN >>> regions, it is entirely possible for you to obtain addresses from RIPE or >>> ARIN and use them in both locations, or, obtain addresses from both RIPE >>> and ARIN and use them in their respective regions, or mix and match in just >>> about any imaginable way. Thus, IP addresses don't reside in regions, >>> either. They are merely issued somewhat regionally. >> >> A direct quote from a recent interaction with ARIN (this was requested by >> ARIN staff as part of the back and forth for requesting address space): >> >> "Please reply and verify that you will be using the requested number >> resources within the ARIN region and announcing all routing prefixes of the >> requested space from within the ARIN region. In accordance with section 2.2 >> of the NRPM, ARIN issues number resources only for use within its region. >> ARIN is therefore only able to provide for your in-region numbering needs." >> >> I believe AfriNIC and LACNIC have similar limitations on use but am too lazy >> to look it up (and I don't really care all that much: just thought it was >> amusing). > > Indeed. This was covered in more detail in the Policy Experience Report > given at the ARIN 31, in which it was noted that we are seeing an increase > in requests for IPv4 address space from parties who have infrastructure in > the region, but for customers entirely from outside the region. This has > resulted in a significant change in the issuance rate and therefore any > estimates for regional free pool depletion. ARIN has sought guidance from > the community regarding what constitutes appropriate in-region use, should > this be based on infrastructure or served customers, and whether incidental > use outside the region is appropriate. (This topic was also on this list on > 26 April 2012 - see attached email from that thread) Policy proposals in > this area to bring further clarity in address management are encouraged. > > FYI, > /John > > John Curran > President and CEO > ARIN > > === > Begin forwarded message: > >> From: John Curran >> Subject: Re: "It's the end of the world as we know it" -- REM >> Date: April 26, 2013 10:43:51 AM EDT >> To: "nanog@nanog.org Group" >> >> On Apr 26, 2013, at 10:23 AM, Chris Grundemann wrote: >>> >>> One interesting twist in all of this is that several of these new >>> "slow-start" players in the ARIN region seem to be servicing customers >>> outside of the region with equipment and services hosted here inside >>> the ARIN region (see slide 12 on the ARIN 31 "Policy Implementation >>> and Experience Report" >>> https://www.arin.net/participate/meetings/reports/ARIN_31/PDF/monday/nobile_policy.pdf). >> >> NANOG Folks - >> >> Please read this slide deck, section noted by Chris. It explains the >> "situation"... (I would not call the sudden acceleration in IP address >> issuance a problem, per se, as that is an judgement for the community >> either way.) >> >> FYI, >> /John >> >> John Curran >> President and CEO >> ARIN > > > >
Re: Geoip lookup
On May 24, 2013, at 10:47 AM, David Conrad wrote: > I replied privately to Owen, but might as well share: > > On May 23, 2013, at 11:57 PM, Owen DeLong wrote: > True, according to (at least some of) the RIRs they reside in regions... Really? Which ones? I thought they were only issued to organizations that had operations in regions. >> That was exactly my point, Bill... If you have operations in RIPE and ARIN >> regions, it is entirely possible for you to obtain addresses from RIPE or >> ARIN and use them in both locations, or, obtain addresses from both RIPE and >> ARIN and use them in their respective regions, or mix and match in just >> about any imaginable way. Thus, IP addresses don't reside in regions, >> either. They are merely issued somewhat regionally. > > A direct quote from a recent interaction with ARIN (this was requested by > ARIN staff as part of the back and forth for requesting address space): > > "Please reply and verify that you will be using the requested number > resources within the ARIN region and announcing all routing prefixes of the > requested space from within the ARIN region. In accordance with section 2.2 > of the NRPM, ARIN issues number resources only for use within its region. > ARIN is therefore only able to provide for your in-region numbering needs." > > I believe AfriNIC and LACNIC have similar limitations on use but am too lazy > to look it up (and I don't really care all that much: just thought it was > amusing). Indeed. This was covered in more detail in the Policy Experience Report given at the ARIN 31, in which it was noted that we are seeing an increase in requests for IPv4 address space from parties who have infrastructure in the region, but for customers entirely from outside the region. This has resulted in a significant change in the issuance rate and therefore any estimates for regional free pool depletion. ARIN has sought guidance from the community regarding what constitutes appropriate in-region use, should this be based on infrastructure or served customers, and whether incidental use outside the region is appropriate. (This topic was also on this list on 26 April 2012 - see attached email from that thread) Policy proposals in this area to bring further clarity in address management are encouraged. FYI, /John John Curran President and CEO ARIN === Begin forwarded message: > From: John Curran > Subject: Re: "It's the end of the world as we know it" -- REM > Date: April 26, 2013 10:43:51 AM EDT > To: "nanog@nanog.org Group" > > On Apr 26, 2013, at 10:23 AM, Chris Grundemann wrote: >> >> One interesting twist in all of this is that several of these new >> "slow-start" players in the ARIN region seem to be servicing customers >> outside of the region with equipment and services hosted here inside >> the ARIN region (see slide 12 on the ARIN 31 "Policy Implementation >> and Experience Report" >> https://www.arin.net/participate/meetings/reports/ARIN_31/PDF/monday/nobile_policy.pdf). > > NANOG Folks - > > Please read this slide deck, section noted by Chris. It explains the > "situation"... (I would not call the sudden acceleration in IP address > issuance a problem, per se, as that is an judgement for the community > either way.) > > FYI, > /John > > John Curran > President and CEO > ARIN
Re: Geoip lookup
On May 24, 2013, at 3:27 AM, Paul Kelly :: Blacknight wrote: >>> >>> Just because I have operations in one region does not preclude me >> from having operations >>> in other regions. YMMV of course. >>> >>> /bill >> >> That was exactly my point, Bill... If you have operations in RIPE and ARIN >> regions, it is entirely possible for you to obtain addresses from RIPE or >> ARIN >> and use them in both locations, or, obtain addresses from both RIPE and >> ARIN and use them in their respective regions, or mix and match in just about >> any imaginable way. Thus, IP addresses don't reside in regions, either. They >> are merely issued somewhat regionally. >> > > In theory Maxmind is quite accurate. From 1 x /20 that we own we tag > different space with the country: flag in the RIPE db. Maxmind picks this up > after approx 30 days and says it's in Country X vrs country Y. Interesting. I wonder how many days it takes the RIPE db to report the correct country for those more specifics that you've tagged... TV > e.g. > > $ geoiplookup 81.17.247.64 > GeoIP Country Edition: US, United States > $ geoiplookup 81.17.247.1 > GeoIP Country Edition: IE, Ireland > > Obviously the RIPE db structure makes this simple. As for other RIRs it's not > as easy. Like someone else said, it's going to be an 80% solution and its > really down to good administration from a network operator point of view. > i.e. if you route some of your RIPE space in ARIN territory you should > specify the country. There's numerous reasons for this but it's just good > practice IMO. > > Pk >
Re: Geoip lookup
You're thinking like an engineer. Think like a marketer. They expect less than 1% response on paper mail advertising. Now, compare and contrast your idea of a reasonable confidence level and theirs. -- -Barry Shein The World | b...@theworld.com | http://www.TheWorld.com Purveyors to the Trade | Voice: 800-THE-WRLD| Dial-Up: US, PR, Canada Software Tool & Die| Public Access Internet | SINCE 1989 *oo*
Re: Geoip lookup
On May 24, 2013, at 01:13 , shawn wilson wrote:
> I knew this would come up. Actually I'm surprised and glad it waited until I
> got a solution first.
>
> I'll address a few points:
> - this is mainly to stop stupid things from sending packets from countries we
> will probably never want to do business with (I'm looking mainly at that big
> country under APNIC).
>
I can't tell you how much I enjoyed all the hoops I had to jump through in
order to access my online banking while traveling in that country.
Assuming that your local customers aren't in that location isn't a valid
assumption to begin with. Making life difficult for those that do travel will
not earn you brownie points with them. (I am no longer with the financial
institution that made this most difficult).
> - I'd prefer a solution that blocks all traffic that is routed through those
> countries so that they could never see data from us (and when Jin-rong has a
> configuration mess up and rerouts ~10% of traffic through them for a half
> hour, I don't see any of that traffic). Since I have no idea how one would go
> about doing this, just blocking traffic from IP addresses registered in
> certain countries is good enough.
>
That's hard to do. Unless you require "record-route" on all packets and have
some way to validate the contents of the route recording header (and enough
space in the header to record all hops every time), it's not going to be
possible. Further, even if it were, there's no way to ensure that all of your
client's packets will get retransmitted on a path that works, so you would have
the potential to severely degrade customer service in non-intuitive and
hard-to-diagnose ways.
If you are my competitor, then I encourage you to try this.
> - it is well known (I think everyone on this list at least) that you can
> evade geographic placement of your origin by tunneling. Given this, I fail to
> see the point in bringing up that "GeoIP" doesn't work. Also, if it doesn't
> work, why do content providers, CDNs, google, and streaming services rely on
> it as part of their business model? The sad truth of the mater is it does
> work and surprisingly well. We just don't like it because it's brittle and a
> user can fool us (I know Akami and the like look at trip time and the like
> because they know there are issues). Given all of this, how often is looking
> at the country an IP address originates from via what is listed for the
> particular ASN actually fiction?
>
Asking why providers rely on GeoIP in the face of it's flaws is like asking why
people continue to buy Windows. It's a cross between inertia and a lack of
better solutions at comparable cost. The sad truth of the matter is that it
doesn't work. It works well enough to give the illusion of working. Deeper
analysis, however, reveals that it works just well enough to keep honest people
honest some of the time. Further, victims of it not working have little or no
recourse available to them even if they understand what is happening. For the
average user, it just looks like some portion of the internet is
{permanently|temporarily} broken again for reasons passing understanding and
they go somewhere else.
Owen
> Again, the input was invaluable for getting me where I wanted to be so thanks
> again.
>
> On May 24, 2013 2:59 AM, "Owen DeLong" wrote:
>
> On May 23, 2013, at 23:49 , bmann...@vacation.karoshi.com wrote:
>
> > On Thu, May 23, 2013 at 11:39:12PM -0700, Owen DeLong wrote:
> >>
> >> On May 23, 2013, at 23:17 , David Conrad wrote:
> >>
> >>> On May 23, 2013, at 10:53 PM, Andreas Larsen
> >>> wrote:
> The whole idea of Geoip is flawed.
> >>>
> >>> Sure, but pragmatically, it's an 80% solution.
> >>>
> IP dosen't reside in countries,
> >>>
> >>> True, according to (at least some of) the RIRs they reside in regions...
> >>>
> >>
> >> Really? Which ones? I thought they were only issued to organizations that
> >> had operations in regions.
> >>
> >> Owen
> >
> > Just because I have operations in one region does not preclude me
> > from having operations
> > in other regions. YMMV of course.
> >
> > /bill
>
> That was exactly my point, Bill... If you have operations in RIPE and ARIN
> regions, it is entirely possible for you to obtain addresses from RIPE or
> ARIN and use them in both locations, or, obtain addresses from both RIPE and
> ARIN and use them in their respective regions, or mix and match in just about
> any imaginable way. Thus, IP addresses don't reside in regions, either. They
> are merely issued somewhat regionally.
>
> Owen
>
>
Re: Geoip lookup
On May 24, 2013, at 00:28 , Jean-Francois Mezei wrote: > On 13-05-24 02:57, Owen DeLong wrote: > >> That was exactly my point, Bill... If you have operations in RIPE and ARIN >> regions, it is entirely possible for you to obtain addresses from RIPE or >> ARIN and use them in both locations, or, obtain addresses from both RIPE and >> ARIN and use them in their respective regions, or mix and match in just >> about any imaginable way. Thus, IP addresses don't reside in regions, >> either. They are merely issued somewhat regionally. > > Correct. > But the fact remains that a lot of services assume geolocation works and > do so in terms of restricting access to their content (oftent due to > legacy content rights that require geolocation). > The fact remains that a certain percentage of the population robs banks. Neither is a particularly good thing, IMHO. > One extreme example. A sports equipment retailer operates under a > different banner in the province of Québec than in the rest of Canada. > They geolocate the user's province and prevent québeckers from accessing > the "rest of canada" web site. > And the quebeckers that care use a tunnel to get an address that doesn't geolocate to quebec. > So residents of ontario who subscribe to an ISP based in Québec are > blocked from the web site because that web site thinks they are based in > Québec. > Which goes to prove my point wrt. bank robbery. > The problem is with many web designers and managers who do not > understand geolocation and the ISP business and how they are structured. > So called experts who remain rather ignorant in their field of "expertise" are a problem across a wide variety of fields. The internet is not unique in this regard and geolocation is just one aspect of this problem on the internet. > In the case of the sports equipment chain. there is no real need to > geoblock. (perhaps to prevent Québeckers from seeing the prices in the > rest of canada ?) > Yep... And even if there were a reason, geoblocking is a joke anyway because it is trivially subverted by anyone who cares and more of a problem for people who should have access but their IP doesn't match their actual geography. > But in the case of entertainment, rights to programs are purchased with > strict geolocation requirements. One example are pay TV channels TMN > (Astral) and Movie Central (Shaw). The first has eastern canada, the > later has western Canada. But IP geolocation doesn't help in either of these cases. Those wanting to subvert the programming restrictions simply use a tunnel to do so. On the other hand, a customer who lives near the boundary and gets his internet service from the "wrong side" of the boundary has access to the service from the wrong geography and not the correct geography. > an IPTV BDU (regulated "cable" carrier (aka: cable competitor) must > therefore ensure that a customer to whom it delivers the IPTV feed for > "TMN" is located in the region for which TMN has rights. Same for all > channels. And there is also pesky channel substitution requirements > rhat are based on your location. In Canada, we are not allowed to watch > a program on a US channel if a local TV channel carries the same program > at same time. And geolocation by IP doesn't actually work to enforce any of these restrictions because tunnels easily circumvent it for those that want to circumvent it. OTOH, it also breaks the process for those who happen to be victims of unfortunate mismatches between topology and geography. Where the IPTV provider is also the ISP, this isn't really much of a problem, but in that case, geo IP is kind of redundant. Where the IPTV provider is not the ISP, it gets very strange very quickly. > The better solution is to do like satellite BDUs do: billing address. > But some web based systems ignore the unreliable geolocation services > and use them to geolocate their customers. Yep... Again, see above comments about ignorance and bank robbery. > It is probably the fault of geolocation services which misrepresent the > accuracy of their data. But if you can't beat them, you might as well > join them, and that may mean separate IP blocks for different > provinces/states and separate registrations so geoocation companies can > at least get province/state right. Why would I want to help them? I'd much rather give my customers the option of where they want to pretend to be. If I were running a provider that crossed such regional boundaries, I'd likely offer a service (for a fee) where a customer could tunnel through whichever region got them access to the content they wanted at any given time. > It will get much worse if governments start to tax purchases/services > based on gelocation. ROFLMAO... Indeed... That will likely lead to some very interesting lawsuits and consumer complaints about invalid taxation due to inaccuracies in the geolocation database. Owen
Re: Geoip lookup
I replied privately to Owen, but might as well share: On May 23, 2013, at 11:57 PM, Owen DeLong wrote: True, according to (at least some of) the RIRs they reside in regions... >>> Really? Which ones? I thought they were only issued to organizations that >>> had operations in regions. > That was exactly my point, Bill... If you have operations in RIPE and ARIN > regions, it is entirely possible for you to obtain addresses from RIPE or > ARIN and use them in both locations, or, obtain addresses from both RIPE and > ARIN and use them in their respective regions, or mix and match in just about > any imaginable way. Thus, IP addresses don't reside in regions, either. They > are merely issued somewhat regionally. A direct quote from a recent interaction with ARIN (this was requested by ARIN staff as part of the back and forth for requesting address space): "Please reply and verify that you will be using the requested number resources within the ARIN region and announcing all routing prefixes of the requested space from within the ARIN region. In accordance with section 2.2 of the NRPM, ARIN issues number resources only for use within its region. ARIN is therefore only able to provide for your in-region numbering needs." I believe AfriNIC and LACNIC have similar limitations on use but am too lazy to look it up (and I don't really care all that much: just thought it was amusing). Regards, -drc
Re: Geoip lookup
On May 24, 2013, at 2:34 AM, Andreas Larsen wrote: > If we continue to support and build tools around this geolocation based > ip-dravel, we give people a false notion that this is something we should > do. > ... > > Or just get rid of the whole idea and realize that the internet is global > and reaches everywhere no matter what your IP currently is. While the Internet is global and reaches everywhere, the same is not true about most businesses and governments... As a result, there are many use cases that we may not like, but are seen as basic requirements by those organizations. Examples include laws and business contracts that require different behavior depending on the location of the user, and from the view of these organizations, the Internet almost gives the impression of shoddy workmanship to omit such an obvious capability. Luckily, many organizations did come up with workarounds, and the lack of a 100% reliable solution did not prevent them from distributing content (software, music, movies, articles, etc.) that they only had rights to do so in a particular region. If the approximate geolocation approaches had not been used, we'd would not have had the region-restricted experimentation in content distribution that underlies quite a bit of the industry even today. One can argue that regionally-based business models should be changed, but the fact is that the not-quite-reliable geolocation services are actually has been pretty important in enabling traditional content in making it onto the Internet. (It is left as a exercise for the reader as to whether more highly reliable geolocation would meaningfully help the situation, or simply enable its use in non-commercial contexts to the detriment of the global user community.) /John Disclaimer: My views alone (& for folks who wish to filter this email based on my geolocation, it is presently Northern Virginia USA ;-)
Re: Geoip lookup
I knew this would come up. Actually I'm surprised and glad it waited until I got a solution first. I'll address a few points: - this is mainly to stop stupid things from sending packets from countries we will probably never want to do business with (I'm looking mainly at that big country under APNIC). - I'd prefer a solution that blocks all traffic that is routed through those countries so that they could never see data from us (and when Jin-rong has a configuration mess up and rerouts ~10% of traffic through them for a half hour, I don't see any of that traffic). Since I have no idea how one would go about doing this, just blocking traffic from IP addresses registered in certain countries is good enough. - it is well known (I think everyone on this list at least) that you can evade geographic placement of your origin by tunneling. Given this, I fail to see the point in bringing up that "GeoIP" doesn't work. Also, if it doesn't work, why do content providers, CDNs, google, and streaming services rely on it as part of their business model? The sad truth of the mater is it does work and surprisingly well. We just don't like it because it's brittle and a user can fool us (I know Akami and the like look at trip time and the like because they know there are issues). Given all of this, how often is looking at the country an IP address originates from via what is listed for the particular ASN actually fiction? Again, the input was invaluable for getting me where I wanted to be so thanks again. On May 24, 2013 2:59 AM, "Owen DeLong" wrote: > > On May 23, 2013, at 23:49 , bmann...@vacation.karoshi.com wrote: > > > On Thu, May 23, 2013 at 11:39:12PM -0700, Owen DeLong wrote: > >> > >> On May 23, 2013, at 23:17 , David Conrad wrote: > >> > >>> On May 23, 2013, at 10:53 PM, Andreas Larsen < > andreas.lar...@ip-only.se> wrote: > The whole idea of Geoip is flawed. > >>> > >>> Sure, but pragmatically, it's an 80% solution. > >>> > IP dosen't reside in countries, > >>> > >>> True, according to (at least some of) the RIRs they reside in > regions... > >>> > >> > >> Really? Which ones? I thought they were only issued to organizations > that had operations in regions. > >> > >> Owen > > > > Just because I have operations in one region does not preclude me > from having operations > > in other regions. YMMV of course. > > > > /bill > > That was exactly my point, Bill... If you have operations in RIPE and ARIN > regions, it is entirely possible for you to obtain addresses from RIPE or > ARIN and use them in both locations, or, obtain addresses from both RIPE > and ARIN and use them in their respective regions, or mix and match in just > about any imaginable way. Thus, IP addresses don't reside in regions, > either. They are merely issued somewhat regionally. > > Owen > > >
Re: Geoip lookup
On 13-05-24 02:57, Owen DeLong wrote: > That was exactly my point, Bill... If you have operations in RIPE and ARIN > regions, it is entirely possible for you to obtain addresses from RIPE or > ARIN and use them in both locations, or, obtain addresses from both RIPE and > ARIN and use them in their respective regions, or mix and match in just about > any imaginable way. Thus, IP addresses don't reside in regions, either. They > are merely issued somewhat regionally. Correct. But the fact remains that a lot of services assume geolocation works and do so in terms of restricting access to their content (oftent due to legacy content rights that require geolocation). One extreme example. A sports equipment retailer operates under a different banner in the province of Québec than in the rest of Canada. They geolocate the user's province and prevent québeckers from accessing the "rest of canada" web site. So residents of ontario who subscribe to an ISP based in Québec are blocked from the web site because that web site thinks they are based in Québec. The problem is with many web designers and managers who do not understand geolocation and the ISP business and how they are structured. In the case of the sports equipment chain. there is no real need to geoblock. (perhaps to prevent Québeckers from seeing the prices in the rest of canada ?) But in the case of entertainment, rights to programs are purchased with strict geolocation requirements. One example are pay TV channels TMN (Astral) and Movie Central (Shaw). The first has eastern canada, the later has western Canada. an IPTV BDU (regulated "cable" carrier (aka: cable competitor) must therefore ensure that a customer to whom it delivers the IPTV feed for "TMN" is located in the region for which TMN has rights. Same for all channels. And there is also pesky channel substitution requirements rhat are based on your location. In Canada, we are not allowed to watch a program on a US channel if a local TV channel carries the same program at same time. The better solution is to do like satellite BDUs do: billing address. But some web based systems ignore the unreliable geolocation services and use them to geolocate their customers. It is probably the fault of geolocation services which misrepresent the accuracy of their data. But if you can't beat them, you might as well join them, and that may mean separate IP blocks for different provinces/states and separate registrations so geoocation companies can at least get province/state right. It will get much worse if governments start to tax purchases/services based on gelocation.
RE: Geoip lookup
> > > > Just because I have operations in one region does not preclude me > from having operations > > in other regions. YMMV of course. > > > > /bill > > That was exactly my point, Bill... If you have operations in RIPE and ARIN > regions, it is entirely possible for you to obtain addresses from RIPE or ARIN > and use them in both locations, or, obtain addresses from both RIPE and > ARIN and use them in their respective regions, or mix and match in just about > any imaginable way. Thus, IP addresses don't reside in regions, either. They > are merely issued somewhat regionally. > In theory Maxmind is quite accurate. From 1 x /20 that we own we tag different space with the country: flag in the RIPE db. Maxmind picks this up after approx 30 days and says it's in Country X vrs country Y. e.g. $ geoiplookup 81.17.247.64 GeoIP Country Edition: US, United States $ geoiplookup 81.17.247.1 GeoIP Country Edition: IE, Ireland Obviously the RIPE db structure makes this simple. As for other RIRs it's not as easy. Like someone else said, it's going to be an 80% solution and its really down to good administration from a network operator point of view. i.e. if you route some of your RIPE space in ARIN territory you should specify the country. There's numerous reasons for this but it's just good practice IMO. Pk
Re: Geoip lookup
On May 23, 2013, at 23:49 , bmann...@vacation.karoshi.com wrote: > On Thu, May 23, 2013 at 11:39:12PM -0700, Owen DeLong wrote: >> >> On May 23, 2013, at 23:17 , David Conrad wrote: >> >>> On May 23, 2013, at 10:53 PM, Andreas Larsen >>> wrote: The whole idea of Geoip is flawed. >>> >>> Sure, but pragmatically, it's an 80% solution. >>> IP dosen't reside in countries, >>> >>> True, according to (at least some of) the RIRs they reside in regions... >>> >> >> Really? Which ones? I thought they were only issued to organizations that >> had operations in regions. >> >> Owen > > Just because I have operations in one region does not preclude me from > having operations > in other regions. YMMV of course. > > /bill That was exactly my point, Bill... If you have operations in RIPE and ARIN regions, it is entirely possible for you to obtain addresses from RIPE or ARIN and use them in both locations, or, obtain addresses from both RIPE and ARIN and use them in their respective regions, or mix and match in just about any imaginable way. Thus, IP addresses don't reside in regions, either. They are merely issued somewhat regionally. Owen
Re: Geoip lookup
On Thu, May 23, 2013 at 11:39:12PM -0700, Owen DeLong wrote: > > On May 23, 2013, at 23:17 , David Conrad wrote: > > > On May 23, 2013, at 10:53 PM, Andreas Larsen > > wrote: > >> The whole idea of Geoip is flawed. > > > > Sure, but pragmatically, it's an 80% solution. > > > >> IP dosen't reside in countries, > > > > True, according to (at least some of) the RIRs they reside in regions... > > > > Really? Which ones? I thought they were only issued to organizations that had > operations in regions. > > Owen Just because I have operations in one region does not preclude me from having operations in other regions. YMMV of course. /bill
Re: Geoip lookup
On May 23, 2013, at 23:17 , David Conrad wrote: > On May 23, 2013, at 10:53 PM, Andreas Larsen > wrote: >> The whole idea of Geoip is flawed. > > Sure, but pragmatically, it's an 80% solution. > >> IP dosen't reside in countries, > > True, according to (at least some of) the RIRs they reside in regions... > Really? Which ones? I thought they were only issued to organizations that had operations in regions. Owen
Re: Geoip lookup
If we continue to support and build tools around this geolocation based ip-dravel, we give people a false notion that this is something we should do. Identify users with some other means that Geoip Couple of things comes to mind. * normal postage mail that they have to collect at their home and send back confirming that they are indeed in the country from where their IP is * Passports scanned. * Fingerprinting Or just get rid of the whole idea and realize that the internet is global and reaches everywhere no matter what your IP currently is. Med vänlig hälsning Andreas Larsen IP-Only Telecommunication AB| Postadress: 753 81 UPPSALA | Besöksadress: S:t Persgatan 6, Uppsala | Telefon: +46 (0)18 843 10 00 | Direkt: +46 (0)18 843 10 56 www.ip-only.se Den 2013-05-24 08:17 skrev David Conrad : >On May 23, 2013, at 10:53 PM, Andreas Larsen >wrote: >> The whole idea of Geoip is flawed. > >Sure, but pragmatically, it's an 80% solution. > >> IP dosen't reside in countries, > >True, according to (at least some of) the RIRs they reside in regions... > >Regards, >-drc >
Re: Geoip lookup
On May 23, 2013, at 10:53 PM, Andreas Larsen wrote: > The whole idea of Geoip is flawed. Sure, but pragmatically, it's an 80% solution. > IP dosen't reside in countries, True, according to (at least some of) the RIRs they reside in regions... Regards, -drc
Re: Geoip lookup
The whole idea of Geoip is flawed. IP dosen't reside in countries, they are routable adresses that can reside everywhere, I guess soon on mars even. Med vänlig hälsning Andreas Larsen IP-Only Telecommunication AB| Postadress: 753 81 UPPSALA | Besöksadress: S:t Persgatan 6, Uppsala | Telefon: +46 (0)18 843 10 00 | Direkt: +46 (0)18 843 10 56 www.ip-only.se Den 2013-05-24 02:54 skrev Rob Seastrom : > >This may be just a case of getting what you pay for, but Maxmind marks >entire netblocks as proxies, puts 'em in the wrong country, and >ignores repeated efforts by the registrant of the address space to set >the record straight. The problem comes when people actually do stuff >with the information, like block access to legitimate web sites >because the're in "proxy space" and therefore assumed to be bad guys >(believe it or not this practice is widespread by well-intentioned but >clueless folk). Caveat utilitor. > >-r > >chip writes: > >> I've used the MaxMind Lite geo-ip database plus some perl modules and a >>BGP >> table to get something fairly close. Anything in the BGP table that was >> larger than a /20 I split into /20's. For my use case, this was close >> enough. I then grabbed 30 or so IP's within the range and geo-ip mapped >> them. You can then apply some algebra and get a general idea of where >> things are or are not. >> >> Things I used: >> http://search.cpan.org/~plonka/Net-Patricia-1.014/Patricia.pm - For >> ip/prefix/lat-lon mapping >> http://search.cpan.org/~borisz/Geo-IP-1.41/lib/Geo/IP.pm - For Geo-IP >> lat/lon data >> http://dev.maxmind.com/geoip/legacy/geolite - Maxmind's city database >> http://data.caida.org/datasets/routing/routeviews-prefix2as/ - for BGP >> prefix/mask + src ASN info >> >> Good luck! >> >> --chip >> >> >> >> >> >> >> On Thu, May 23, 2013 at 3:47 PM, shawn wilson >>wrote: >> >>> What's the best way to find the networks in a country? I was thinking >>>of >>> writing some perl with Net::Whois::ARIN or some such module and loop >>> through the block. But I think I'll have to be smarter than just a >>>simple >>> loop not to get blocked and I figure I'm not the first to want to do >>>this. >>> >>> I've noticed some paid databases out there. They don't cost much but >>>are >>> they even worth what they charge? Ie, countryipblocks.net doesn't list >>> quite a few addresses from a country I've looked at blocking. Isn't >>>this >>> information free from the different *NICs anyway? >>> >>> This is probably two questions: a program that smartly looks for >>>country's >>> blocks in a block and are GeoIP services worth anything? >>> >> >> >> >> -- >> Just my $.02, your mileage may vary, batteries not included, etc >
Re: Geoip lookup
This may be just a case of getting what you pay for, but Maxmind marks entire netblocks as proxies, puts 'em in the wrong country, and ignores repeated efforts by the registrant of the address space to set the record straight. The problem comes when people actually do stuff with the information, like block access to legitimate web sites because the're in "proxy space" and therefore assumed to be bad guys (believe it or not this practice is widespread by well-intentioned but clueless folk). Caveat utilitor. -r chip writes: > I've used the MaxMind Lite geo-ip database plus some perl modules and a BGP > table to get something fairly close. Anything in the BGP table that was > larger than a /20 I split into /20's. For my use case, this was close > enough. I then grabbed 30 or so IP's within the range and geo-ip mapped > them. You can then apply some algebra and get a general idea of where > things are or are not. > > Things I used: > http://search.cpan.org/~plonka/Net-Patricia-1.014/Patricia.pm - For > ip/prefix/lat-lon mapping > http://search.cpan.org/~borisz/Geo-IP-1.41/lib/Geo/IP.pm - For Geo-IP > lat/lon data > http://dev.maxmind.com/geoip/legacy/geolite - Maxmind's city database > http://data.caida.org/datasets/routing/routeviews-prefix2as/ - for BGP > prefix/mask + src ASN info > > Good luck! > > --chip > > > > > > > On Thu, May 23, 2013 at 3:47 PM, shawn wilson wrote: > >> What's the best way to find the networks in a country? I was thinking of >> writing some perl with Net::Whois::ARIN or some such module and loop >> through the block. But I think I'll have to be smarter than just a simple >> loop not to get blocked and I figure I'm not the first to want to do this. >> >> I've noticed some paid databases out there. They don't cost much but are >> they even worth what they charge? Ie, countryipblocks.net doesn't list >> quite a few addresses from a country I've looked at blocking. Isn't this >> information free from the different *NICs anyway? >> >> This is probably two questions: a program that smartly looks for country's >> blocks in a block and are GeoIP services worth anything? >> > > > > -- > Just my $.02, your mileage may vary, batteries not included, etc
Re: Geoip lookup
On Thu, May 23, 2013 at 5:36 PM, Joe Abley wrote: > > On 2013-05-23, at 16:56, shawn wilson wrote: > >> It looks you're right and everyone does have the same data in >> historical format. Looks like RIPE has everything compiled into what >> is current. So if a block hasn't changed for 10 years, it'll be in the >> RIPE dataset vs with the others, I'd have to write something to >> overlay the data through out time to get current? > > Could be. You've looked at this more than I have, now -- I was mainly trying > to point out that bulk data retrieval is a possible option so you could avoid > whois-hammering Actually, I can't find anything better, so I think i'm going to query the bottom of ranges like so: % dig +short 0.0.66.77.origin.asn.cymru.com TXT "16245 | 77.66.0.0/17 | DK | ripencc | 2007-01-24" % dig +short 0.0.65.77.origin.asn.cymru.com TXT "13110 | 77.65.0.0/17 | PL | ripencc | 2007-01-17" According to their web site, they won't block me if I don't do anything stupid "If you are planning on implementing the use of this service in any software, application, or device PLEASE let us know in advance. We would like to adequately plan for capacity and make sure that we can adequately handle the load. If at all possible, PLEASE use the DNS based service since it is faster and more efficient, particularly for larger deployments of individual IP based queries." http://www.team-cymru.org/Services/ip-to-asn.html#dns and use this https://metacpan.org/module/MRSAM/Net-CIDR-0.17/CIDR.pm to find my upper ranges. So, I'm pretty much thinking about whois-hammering still :( Also, I just picked those IPs at random (ie, start at one end of the number, hit twice, dot, next number) nothing particularly interresting about whoever that is AFAIK.
Re: Geoip lookup
On 2013-05-23, at 16:56, shawn wilson wrote: > It looks you're right and everyone does have the same data in > historical format. Looks like RIPE has everything compiled into what > is current. So if a block hasn't changed for 10 years, it'll be in the > RIPE dataset vs with the others, I'd have to write something to > overlay the data through out time to get current? Could be. You've looked at this more than I have, now -- I was mainly trying to point out that bulk data retrieval is a possible option so you could avoid whois-hammering :-) Joe
Re: Geoip lookup
I've used the MaxMind Lite geo-ip database plus some perl modules and a BGP table to get something fairly close. Anything in the BGP table that was larger than a /20 I split into /20's. For my use case, this was close enough. I then grabbed 30 or so IP's within the range and geo-ip mapped them. You can then apply some algebra and get a general idea of where things are or are not. Things I used: http://search.cpan.org/~plonka/Net-Patricia-1.014/Patricia.pm - For ip/prefix/lat-lon mapping http://search.cpan.org/~borisz/Geo-IP-1.41/lib/Geo/IP.pm - For Geo-IP lat/lon data http://dev.maxmind.com/geoip/legacy/geolite - Maxmind's city database http://data.caida.org/datasets/routing/routeviews-prefix2as/ - for BGP prefix/mask + src ASN info Good luck! --chip On Thu, May 23, 2013 at 3:47 PM, shawn wilson wrote: > What's the best way to find the networks in a country? I was thinking of > writing some perl with Net::Whois::ARIN or some such module and loop > through the block. But I think I'll have to be smarter than just a simple > loop not to get blocked and I figure I'm not the first to want to do this. > > I've noticed some paid databases out there. They don't cost much but are > they even worth what they charge? Ie, countryipblocks.net doesn't list > quite a few addresses from a country I've looked at blocking. Isn't this > information free from the different *NICs anyway? > > This is probably two questions: a program that smartly looks for country's > blocks in a block and are GeoIP services worth anything? > -- Just my $.02, your mileage may vary, batteries not included, etc
Re: Geoip lookup
On Thu, May 23, 2013 at 4:40 PM, shawn wilson wrote: > On Thu, May 23, 2013 at 4:32 PM, Joe Abley wrote: >> >> On 2013-05-23, at 15:47, shawn wilson wrote: >> >> >> ftp://ftp.apnic.net/public/apnic/stats/apnic/ >> ftp://ftp.ripe.net/ripe/dbase/ >> ftp://ftp.lacnic.net/pub/stats/lacnic/ >> ftp://ftp.afrinic.net/stats/afrinic/ >> ftp://ftp.arin.net/pub/stats/arin/ >> It looks you're right and everyone does have the same data in historical format. Looks like RIPE has everything compiled into what is current. So if a block hasn't changed for 10 years, it'll be in the RIPE dataset vs with the others, I'd have to write something to overlay the data through out time to get current?
Re: Geoip lookup
On Thu, May 23, 2013 at 4:32 PM, Joe Abley wrote: > > On 2013-05-23, at 15:47, shawn wilson wrote: > >> What's the best way to find the networks in a country? I was thinking of >> writing some perl with Net::Whois::ARIN or some such module and loop >> through the block. But I think I'll have to be smarter than just a simple >> loop not to get blocked and I figure I'm not the first to want to do this. > > If you are looking for registration data, try looking in one or more of > > ftp://ftp.apnic.net/public/apnic/stats/apnic/ > ftp://ftp.ripe.net/ripe/dbase/ > ftp://ftp.lacnic.net/pub/stats/lacnic/ > ftp://ftp.afrinic.net/stats/afrinic/ > ftp://ftp.arin.net/pub/stats/arin/ > > (poke around and see what you can find; I didn't spend much time trying, but > several/all of the RIRs seem to mirror data from all the others) Thanks > > Note that "networks in a country" is a funny phrase. The sets > > - address space assigned to all organisations located in country X > - routes visible in country X (from some viewpoint) > - all addresses assigned to devices physically located within country X > - routes that are considered "in-country" in places where billing is aligned > with the necessity to traverse a long bit of wet glass > > are frequently incongruent. If this matters, you might want to consider a > more detailed specification of "networks in a country". > I had somewhat considered the second and the fourth point. I assumed by using whois data, I am getting the second of those options and that was good enough. If there's a way to (somewhat easily) implement the third option, I'm all ears.
Re: Geoip lookup
On 2013-05-23, at 15:47, shawn wilson wrote: > What's the best way to find the networks in a country? I was thinking of > writing some perl with Net::Whois::ARIN or some such module and loop > through the block. But I think I'll have to be smarter than just a simple > loop not to get blocked and I figure I'm not the first to want to do this. If you are looking for registration data, try looking in one or more of ftp://ftp.apnic.net/public/apnic/stats/apnic/ ftp://ftp.ripe.net/ripe/dbase/ ftp://ftp.lacnic.net/pub/stats/lacnic/ ftp://ftp.afrinic.net/stats/afrinic/ ftp://ftp.arin.net/pub/stats/arin/ (poke around and see what you can find; I didn't spend much time trying, but several/all of the RIRs seem to mirror data from all the others) Note that "networks in a country" is a funny phrase. The sets - address space assigned to all organisations located in country X - routes visible in country X (from some viewpoint) - all addresses assigned to devices physically located within country X - routes that are considered "in-country" in places where billing is aligned with the necessity to traverse a long bit of wet glass are frequently incongruent. If this matters, you might want to consider a more detailed specification of "networks in a country". Joe
Geoip lookup
What's the best way to find the networks in a country? I was thinking of writing some perl with Net::Whois::ARIN or some such module and loop through the block. But I think I'll have to be smarter than just a simple loop not to get blocked and I figure I'm not the first to want to do this. I've noticed some paid databases out there. They don't cost much but are they even worth what they charge? Ie, countryipblocks.net doesn't list quite a few addresses from a country I've looked at blocking. Isn't this information free from the different *NICs anyway? This is probably two questions: a program that smartly looks for country's blocks in a block and are GeoIP services worth anything?
