Re: /27 the new /24

[Joe Abley]

On 12 Oct 2015, at 11:23, Todd Underwood wrote:


it's also not entirely obvious what the point of having local IXes
that serve these kinds of collections of people.


I think that's true. But I don't think it's always the case this means 
there is no point.


When Citylink (incubated by the City Council) started stringing fibre 
around central Wellington, New Zealand and building fast ethernet access 
into buildings in the mid-90s it wasn't obvious what benefit that would 
give anybody at a time when most of the country was hanging off a 
collection of E1 half-circuit leases to the west-coast US. But it led to 
a cottage industry in teleworking, off-site data storage, digital print 
shops and video conferencing in the city that would not have been 
possible otherwise.


What Citylink built was part access network, part exchange, but to their 
credit they didn't spend too much time worrying about what to call what 
they were building: they just built it.


Part of keeping the network stupid is giving people at the edge room to 
innovate. Sometimes when you build it, they come.



Joe

Re: /27 the new /24

[Royce Williams]

On Mon, Oct 12, 2015 at 7:23 AM, Todd Underwood  wrote:
>
> it's also not entirely obvious what the point of having local IXes
> that serve these kinds of collections of people.
>
> how much inter-ASN traffic is there generally for a city of 100k
> people, even if they all have 1Gb/s connections?  are they all
> torrenting, accessing local business web pages that are hosted
> locally, streaming video from local streaming caches?  if a local IX
> is a good place for a llnw, akamai, ggc, netflix cache node, i can see
> it, but that's about it.

They are microcosms of larger areas - think gamers, B2B, local
support/RDP, etc.  When there's ~25ms extra latency ANC<->SEA, every
little bit helps.

It also lets us function locally when there are major external outages.

Finally, there's a psychological benefit. It drove me nuts to have my
traffic shipped to Seattle to end up literally one block away.  :)
It's like when I tracked my Macbook from China -- stopping briefly in
its palletized/containerized form in Anchorage on its way to Kentucky
(UPS distribution), and then back to Anchorage ... days later. :)

Royce

Re: /27 the new /24

[Todd Underwood]

it's also not entirely obvious what the point of having local IXes
that serve these kinds of collections of people.

how much inter-ASN traffic is there generally for a city of 100k
people, even if they all have 1Gb/s connections?  are they all
torrenting, accessing local business web pages that are hosted
locally, streaming video from local streaming caches?  if a local IX
is a good place for a llnw, akamai, ggc, netflix cache node, i can see
it, but that's about it.

t

On Mon, Oct 12, 2015 at 10:33 AM, joel jaeggli  wrote:
> On 10/12/15 1:57 AM, Henrik Thostrup Jensen wrote:
>> On Fri, 9 Oct 2015, Jeremy Austin wrote:
>>
>>> Juneau, I'm not so surprised; how many other cities that small and
>>> isolated
>>> have IXes? I'm curious. It's an interesting prospect, at least for some
>>> value of $location.
>>
>> Several small cities in Sweden have IXes. Not sure than any of them are
>> quite as small as Juneau, but some (Borås, Luleå, Sundsvall) are sub
>> 100k people, and other cities (Umeå, Uppsala) are just over 100k
>> inhabitants. Umeå and Luleå are releativly isolated - at least by
>> European standards.
>>
>> Most of these are probably just a switch or two, and are probably there
>> to provide better quality of service, and not because it makes for a
>> good business.
>
> Sweden's  IX infrastructure is not entirely unique but are certainly
> borne out of a particular set of circumstances and public private
> partnerships that  don't generally exist elsewhere.
>
> https://en.wikipedia.org/wiki/Netnod
>
>>
>> Best regards, Henrik
>>
>>  Henrik Thostrup Jensen 
>>  Software Developer, NORDUnet
>>
>
>

Re: /27 the new /24

[joel jaeggli]

On 10/12/15 1:57 AM, Henrik Thostrup Jensen wrote:
> On Fri, 9 Oct 2015, Jeremy Austin wrote:
> 
>> Juneau, I'm not so surprised; how many other cities that small and
>> isolated
>> have IXes? I'm curious. It's an interesting prospect, at least for some
>> value of $location.
> 
> Several small cities in Sweden have IXes. Not sure than any of them are
> quite as small as Juneau, but some (Borås, Luleå, Sundsvall) are sub
> 100k people, and other cities (Umeå, Uppsala) are just over 100k
> inhabitants. Umeå and Luleå are releativly isolated - at least by
> European standards.
> 
> Most of these are probably just a switch or two, and are probably there
> to provide better quality of service, and not because it makes for a
> good business.

Sweden's  IX infrastructure is not entirely unique but are certainly
borne out of a particular set of circumstances and public private
partnerships that  don't generally exist elsewhere.

https://en.wikipedia.org/wiki/Netnod

> 
> Best regards, Henrik
> 
>  Henrik Thostrup Jensen 
>  Software Developer, NORDUnet
> 




signature.asc
Description: OpenPGP digital signature

Re: /27 the new /24

[Henrik Thostrup Jensen]

On Fri, 9 Oct 2015, Jeremy Austin wrote:


Juneau, I'm not so surprised; how many other cities that small and isolated
have IXes? I'm curious. It's an interesting prospect, at least for some
value of $location.


Several small cities in Sweden have IXes. Not sure than any of them are 
quite as small as Juneau, but some (Borås, Luleå, Sundsvall) are sub 100k 
people, and other cities (Umeå, Uppsala) are just over 100k inhabitants. 
Umeå and Luleå are releativly isolated - at least by European standards.


Most of these are probably just a switch or two, and are probably there to 
provide better quality of service, and not because it makes for a good 
business.



Best regards, Henrik

 Henrik Thostrup Jensen 
 Software Developer, NORDUnet

Re: /27 the new /24

[Christopher Morrow]

On Mon, Oct 12, 2015 at 11:23 AM, Todd Underwood  wrote:
> it's also not entirely obvious what the point of having local IXes
> that serve these kinds of collections of people.
>

one might consider that localized services or peer-to-peer traffic
might not want to burden the long-haul links, for the cost of a 1g or
10g port on a local switch/router.

this conversation is sort of like the ipv6 part earlier though... 'if
people want to do this, cool! if they don't or can't for $REASONS also
cool.'

> how much inter-ASN traffic is there generally for a city of 100k
> people, even if they all have 1Gb/s connections?  are they all
> torrenting, accessing local business web pages that are hosted
> locally, streaming video from local streaming caches?  if a local IX
> is a good place for a llnw, akamai, ggc, netflix cache node, i can see
> it, but that's about it.

it's not clear that a flix cache would rate showing up at an IX in
(for example) juneau... though it'd sure be interesting to know the
share of traffic your 4 exemplars contribute to the overall longhaul
traffic mix.

> t
>
> On Mon, Oct 12, 2015 at 10:33 AM, joel jaeggli  wrote:
>> On 10/12/15 1:57 AM, Henrik Thostrup Jensen wrote:
>>> On Fri, 9 Oct 2015, Jeremy Austin wrote:
>>>
 Juneau, I'm not so surprised; how many other cities that small and
 isolated
 have IXes? I'm curious. It's an interesting prospect, at least for some
 value of $location.
>>>
>>> Several small cities in Sweden have IXes. Not sure than any of them are
>>> quite as small as Juneau, but some (Borås, Luleå, Sundsvall) are sub
>>> 100k people, and other cities (Umeå, Uppsala) are just over 100k
>>> inhabitants. Umeå and Luleå are releativly isolated - at least by
>>> European standards.
>>>
>>> Most of these are probably just a switch or two, and are probably there
>>> to provide better quality of service, and not because it makes for a
>>> good business.
>>
>> Sweden's  IX infrastructure is not entirely unique but are certainly
>> borne out of a particular set of circumstances and public private
>> partnerships that  don't generally exist elsewhere.
>>
>> https://en.wikipedia.org/wiki/Netnod
>>
>>>
>>> Best regards, Henrik
>>>
>>>  Henrik Thostrup Jensen 
>>>  Software Developer, NORDUnet
>>>
>>
>>

Re: /27 the new /24

[Christopher Morrow]

On Mon, Oct 12, 2015 at 1:19 PM, Todd Underwood  wrote:
> all,
>
> On Mon, Oct 12, 2015 at 1:15 PM, Christopher Morrow
>  wrote:
>> On Mon, Oct 12, 2015 at 11:23 AM, Todd Underwood  wrote:
>>> it's also not entirely obvious what the point of having local IXes
>>> that serve these kinds of collections of people.
>>>
>>
>> this conversation is sort of like the ipv6 part earlier though... 'if
>> people want to do this, cool! if they don't or can't for $REASONS also
>> cool.'
>
> oh, for sure.  anyone who wants to should, of course.
>
> i'm just pointing out (in opposition to the drumbeat of "MOAR IXes
> EVERYWHERE!!!" message) that IXes are often not that useful and people
> should critically evaluate whether they need one and would benefit
> from the cost.

sure... folk in a position to do so might want to look at their
netflow/etc data and decide to where they send/receive the most
traffic, if it's their neighbor consider saying: "Howdy neighbor! how
about we uncongest our longhaul and send these bits over a local
ethernet? Oh! jane's also in the mix, let's get together on a hp
switch and win!"

> so far, the "coolness", "psychological", "possible future industry"
> benefits are all cited.  that's fine.  but there's often zero business
> case for an IX outside of major fibre confluences.

'major' perhaps depends on your perspective here, right?

Sure, in Chicago where boatloads of east/west (and some North/South)
fiber shares conduit it sure seems clearly a win to have an IX
there... but I bet if you have 1g to SEA from ANK... losing 200mbps to
crappy-gammer-uturn traffic would be nice to avoid too, eh? Or hell
email even...

anyway, sure more numbers and metrics and thought seems like a good
plan, just like in the v6 discussion earlier.

Re: /27 the new /24

[Mike]

On 10/09/2015 05:22 AM, Lee Howard wrote:

NO, THERE IS NOT. We operate in rural and underserved areas and WE DO
NOT HAVE realistic choices. Can you see me from your ivory tower?
I looked up tiedyenetworks.com, and I think he¹s 100 miles from Sacramento.
I hope some sales person from a transit provider is giving him a call right
now, but it¹s entirely possible that there are no big providers in his
neighborhood. Sorry, Mike, wish I could help you there.



Thats not even the half of it.

My personal heroics in solving the connectivity problem here, is that we 
became a CLEC in order to take the bull on by the short and curlys and 
build facilities. But the problem is even bigger than just getting dark 
fiber strands and collocating in a few select telco offices; My entire 
county is woefully underserved. Connectivity here is expensive as all 
hell. So on top of the nearly $1mln now sunk in this part of the 
venture, I am STILL looking at several more $$mln to build out of this 
dank hole and connect up at those carrier hotels in the far off fantasy 
world where connectivity options abound.


It sounds like you do have some concern about the transition, and you 
know there¹s a bug, at least with OS downloads. Please do report those 
issues you know about. Usually, Happy Eyeballs masks problems in dual 
stack, whether that¹s good or bad. If we can get your upstream(s) to 
support IPv6, then maybe we can leverage them to help troubleshoot MTU 
problems, so you don¹t have to spend a lot of time on them. Or maybe 
they go away when you¹re no longer tunnelling.


No, the problem is that v6 isn't ready for prime time. Period.


I can't remember the last time I saw a site stall due to reaching it
over IPv6 it is that long ago.

It happens every day for me, which only amplifies my perception that v6
IS NOT READY FOR PRIME TIME.

Would you at least keep a list of places you have these problems, even if
you never follow up on it? Again, I¹m wondering if tunnelling is the
problem, and once you have native dual-stack, you could refer to the list
and see if problems just dry up.


No, the problem is that v6 isn't ready for prime time, period. Im not 
going to consider rolling it out to my customers until the point comes 
where it stops going down and stops malfunctioning and stops being 'a 
problem' that has to be dealt with by disabling the v6 stack on the 
afflicted host. Until then, it's going to continue to be treated as 
academic masturbation likely to be replaced with something competently 
designed based on technical merits alone.

Damm maddening. Can't imagine the screaming I'll
hear if a home user ever ran into similar so I am quite gun shy about
the prospect. Secondly, the the dodgy nature of the CPE connected to
our
network and the terminally buggy fw they all run is sure to be a never
ending source of stupidity.

CPE devices are buggy for IPv4 as well.  Bugs in CPE devices are
only found and fixed if the code paths are exercised.

Not my job. v4 works, v6 does not. I am a provider not a developer.

I would guess it is your job in IPv4.
I would also guess, based on gateways I¹ve seen, than 10% of CPE
has critical IPv4 bugs, and 25% of CPE has critical Ipv6 bugs. I agree with
you that the difference is too high, and maybe waiting a year helps get
those ratios aligned.

CPE vendors: step it up!


I think the major pain points in CPE gear is really less than 'ipv4' 
bugs and more just bad design in general. They 'lock up' and stop 
forwarding, requiring end users to 'power cycle' the equipment in order 
to attain functionality again. And, they still stupidly have a 'reset' 
button, which users still think will help them when it's "locked up" but 
in fact simply "wipes out required settings", causing further problems 
for the poor hapless user. They are quite big on shiny flashing lights 
and starship or battle ship shaped plastic housings, but long term 
reliability is about as good as trusting v6 for anything, which is to 
say they are not trustworthy at all.





Figure out how long until you think you really need all of your customers
to have IPv6. Subtract your CPE replacement time. Start replacing CPE then.
e.g., if users need IPv6 in 2018, and you replace all CPE on a 5 year
schedule, you should begin providing IPv6-capable CPE in 2013.



Again, requires me to be a developer and not a provider. Working CPE do 
not exist yet.



Thirdly, some parts of my network are
wireless, and multicast is a huge, huge problem on wireless (the 802.11
varities anyways). The forwarding rates for multicast are sickeningly
low for many brand of gear - yes, it's at the bottom of the barrel no
matter how good or hot your signal is - and I honestly expect v6 to
experience enough disruption over wireless as to render it unusable for
exactly this reason alone.

You expect but haven't tested.

Based on observation and experience, I think v6 will wipe out the 802.11
portion of my network and no, Im not going to 'test' it, 

Re: /27 the new /24

[Todd Underwood]

all,

On Mon, Oct 12, 2015 at 1:15 PM, Christopher Morrow
 wrote:
> On Mon, Oct 12, 2015 at 11:23 AM, Todd Underwood  wrote:
>> it's also not entirely obvious what the point of having local IXes
>> that serve these kinds of collections of people.
>>
>
> this conversation is sort of like the ipv6 part earlier though... 'if
> people want to do this, cool! if they don't or can't for $REASONS also
> cool.'

oh, for sure.  anyone who wants to should, of course.

i'm just pointing out (in opposition to the drumbeat of "MOAR IXes
EVERYWHERE!!!" message) that IXes are often not that useful and people
should critically evaluate whether they need one and would benefit
from the cost.

so far, the "coolness", "psychological", "possible future industry"
benefits are all cited.  that's fine.  but there's often zero business
case for an IX outside of major fibre confluences.

t

Re: /27 the new /24

[Lee Howard]

On 10/12/15, 1:49 PM, "NANOG on behalf of Mike"  wrote:

>
>Thats not even the half of it.
>
>My personal heroics in solving the connectivity problem here, is that we
>became a CLEC in order to take the bull on by the short and curlys and
>build facilities. But the problem is even bigger than just getting dark
>fiber strands and collocating in a few select telco offices; My entire
>county is woefully underserved. Connectivity here is expensive as all
>hell. So on top of the nearly $1mln now sunk in this part of the
>venture, I am STILL looking at several more $$mln to build out of this
>dank hole and connect up at those carrier hotels in the far off fantasy
>world where connectivity options abound.

You have my sympathies. Which gets you nothing but consolation.

>
>> It sounds like you do have some concern about the transition, and you
>> know there¹s a bug, at least with OS downloads. Please do report those
>> issues you know about. Usually, Happy Eyeballs masks problems in dual
>> stack, whether that¹s good or bad. If we can get your upstream(s) to
>> support IPv6, then maybe we can leverage them to help troubleshoot MTU
>> problems, so you don¹t have to spend a lot of time on them. Or maybe
>> they go away when you¹re no longer tunnelling.
>
>No, the problem is that v6 isn't ready for prime time. Period.

That statement is too broad. Lots of companies are using IPv6 in prime
time.

There may be some features with some bugs. I don’t know how we shake those
out if people don’t try those features and report the bugs.

>
 I can't remember the last time I saw a site stall due to reaching it
 over IPv6 it is that long ago.
>>> It happens every day for me, which only amplifies my perception that v6
>>> IS NOT READY FOR PRIME TIME.
>> Would you at least keep a list of places you have these problems, even
>>if
>> you never follow up on it? Again, I¹m wondering if tunnelling is the
>> problem, and once you have native dual-stack, you could refer to the
>>list
>> and see if problems just dry up.
>
>No, the problem is that v6 isn't ready for prime time, period. Im not
>going to consider rolling it out to my customers until the point comes
>where it stops going down and stops malfunctioning and stops being 'a
>problem' that has to be dealt with by disabling the v6 stack on the
>afflicted host. Until then, it's going to continue to be treated as
>academic masturbation likely to be replaced with something competently
>designed based on technical merits alone.

IPv4 malfunctions, too. I haven’t seen anything to suggest that IPv6 is
less robust than IPv4. One does have to climb the learning curve and
develop support processes, but that’s true with anything, including IPv4.


> Damm maddening. Can't imagine the screaming I'll
> hear if a home user ever ran into similar so I am quite gun shy about
> the prospect. Secondly, the the dodgy nature of the CPE connected to
> our
> network and the terminally buggy fw they all run is sure to be a
>never
> ending source of stupidity.
 CPE devices are buggy for IPv4 as well.  Bugs in CPE devices are
 only found and fixed if the code paths are exercised.
>>> Not my job. v4 works, v6 does not. I am a provider not a developer.
>> I would guess it is your job in IPv4.
>> I would also guess, based on gateways I¹ve seen, than 10% of CPE
>> has critical IPv4 bugs, and 25% of CPE has critical Ipv6 bugs. I agree
>>with
>> you that the difference is too high, and maybe waiting a year helps get
>> those ratios aligned.
>>
>> CPE vendors: step it up!
>
>I think the major pain points in CPE gear is really less than 'ipv4'
>bugs and more just bad design in general. They 'lock up' and stop
>forwarding, requiring end users to 'power cycle' the equipment in order
>to attain functionality again. And, they still stupidly have a 'reset'
>button, which users still think will help them when it's "locked up" but
>in fact simply "wipes out required settings", causing further problems
>for the poor hapless user. They are quite big on shiny flashing lights
>and starship or battle ship shaped plastic housings, but long term
>reliability is about as good as trusting v6 for anything, which is to
>say they are not trustworthy at all.

So, CPE is buggy and sucks. I wish there was money to be made in
delivering quality CPE code.


>>
>> Figure out how long until you think you really need all of your
>>customers
>> to have IPv6. Subtract your CPE replacement time. Start replacing CPE
>>then.
>> e.g., if users need IPv6 in 2018, and you replace all CPE on a 5 year
>> schedule, you should begin providing IPv6-capable CPE in 2013.
>>
>
>Again, requires me to be a developer and not a provider. Working CPE do
>not exist yet.

You said above that you thought that CPE reliability is as good as IPv6.
There are quite a few models that work as well with IPv6 as IPv4. But not
all.

But I didn’t ask you to be a 

Re: /27 the new /24

[Baldur Norddahl]

On 12 October 2015 at 19:49, Mike  wrote:

> No, it's not going to help. v6 over current wireless doesn't work for the
> reasons that multicast is a gaping hole.


Why is IPv6 multicast any different than IPv4 broadcast (required for ARP
and many other things)? If you run without MLD snooping IPv6 multicast is
implemented as broadcast at the layer 2 level, so it should behave exactly
the same.

You also have the option of using something like 6RD so you avoid IPv6 in
your backbone entirely. Or use MPLS for the same effect.

Regards,

Baldur

Re: /27 the new /24

[Jeremy Austin]

On Sat, Oct 10, 2015 at 12:51 PM, Todd Underwood 
wrote:

>
> you already know that that's not how the internet in the rural west works.
>  it's fine.  smile and nod and pretend that they are making sensible claims
> and move back to trying to figure out how to make things work on your own
> network.
>

Thank you, Todd. While I must take some exception to your use of the word
'hinterlands' [1] rather than 'frontier', you're right on the mark
everywhere else. :)

With all the talk around updating BCPs, perhaps we also need IUPs --
Interesting Uncommon Practices: the edge cases which contrast to, but do
not invalidate, the middle.

-J

[1] Kleinfeld, "The Frontier Romance"
http://www.newsminer.com/features/sundays/book_reviews/kleinfeld-s-book-explores-the-romance-of-the-frontier/article_57da7bda-e15c-11e2-9281-0019bb30f31a.html

Re: /27 the new /24

[Eric Kuhnke]

As Jeremy has described in detail, the problem is at OSI layer 1. Not a
lack of peering exchanges such as the VANIX. There is no dark fiber route
from Alaska via the Yukon to Vancouver.

I know where most of the Telus (ILEC) and Northwestel (Bell) fiber is in
northern BC and none of interconnects with Alaska.

Network topologically all locations in Alaska which are fiber fed via the
existing submarine cable routes (not on geostationary C/Ku-band satellite)
are a suburb of Seattle. Imagine an island with a population of about
600,000 people located somewhere in Puget Sound with various DWDM circuits
that have their other ends in the Westin Building. Various IP transit,
peering, transport and IX connections at that location.

Other satellite fed singlehomed locations in Alaska can be logically just
about anywhere thanks to the way bent-pipe relay via geostationary
transponders work. There's at least a couple of dozen large teleports in
the US 48 states with 7.3m and larger C-band dishes that support two way
TDMA and SCPC services into Alaska. In such case the sites are
indistinguishable from very low bandwidth singlehomed FDD microwave sites
which happen to have at minimum 495ms latency.



On Fri, Oct 9, 2015 at 1:04 PM, Owen DeLong  wrote:

>
> > On Oct 8, 2015, at 11:24 PM, Jeremy Austin  wrote:
> >
> > On Thu, Oct 8, 2015 at 3:25 PM, James Jun  wrote:
> >
> >>
> >> If you want choices in your transit providers, you should get a
> transport
> >> circuit (dark, wave or EPL) to a nearby carrier hotel/data center.  Once
> >> you do that, you will suddenly find that virtually almost everyone in
> the
> >> competitive IP transit market will provide you with dual-stacked
> IPv4/IPv6
> >> service.
> >>
> >
> > The future is here, but it isn't evenly distributed yet. I'm in North
> > America, but there are no IXPs in my *state*, let alone in my *continent*
> > -- from an undersea fiber perspective. There is no truly competitive IP
> > transit market within Alaska that I am aware of. Would love to be proved
> > wrong. Heck, GCI and ACS (the two providers with such fiber) only
> directly
> > peered a handful of years ago.
>
> Alaska is in the same continent as Canda and the Contiguous US.
>
> VANIX (Vancouver), CIX (Calgary), Manitoba-IX (Winnipeg), WPGIX
> (WInnipeg), TORIX (Toronto),
> and an exchange in Montreal (I forget the name) exist as well as a few
> others in Canada (I think
> there’s even one out in the maritimes).
>
> There are tons of exchanges all over the contiguous US.
>
> I’m surprised that there isn’t yet an exchange point in Juneau or
> Anchorage, but that
> does, indeed, appear to be the case. Perhaps you should work with some
> other ISPs
> in your state to form one.
>
> According to this:
> http://www.alaskaunited.com 
>
> There is subsea fiber to several points in AK from Seattle and beyond.
>
> And on a continental basis, quite a bit of undersea fiber in other landing
> stations
> around the coastal areas of the contiguous 48.
>
> >> If you are buying DIA circuit from some $isp to your rural location that
> >> you call "head-end" and are expecting to receive a competitive service,
> >> and support for IPv6, well, then your expectations are either
> unreasonable,
> >> ignorant or both.
> >>
> >
> > Interestingly both statewide providers *do* provide both IPv4 and IPv6
> > peering. The trick is to find a spot where there's true price
> competition.
> > The 3 largest statewide ISPs have fiber that meets a mere three city
> blocks
> > from one of my POPs, but there's no allowable IX. I'm looking at you,
> AT
>
> I’m not sure what you mean by “allowable IX”, to the best of my knowledge,
> anyone
> can build an IX anywhere.
>
> Owen
>
>
>

Re: /27 the new /24

[Todd Underwood]

In general, most of NANOG recipients live in the populated metros and know
very little about what it's like to try to provide internet access in the
hinterlands.  do not pay attention to there magical claims of 'just connect
to some IX and everything will be fine'.

you already know that that's not how the internet in the rural west works.
 it's fine.  smile and nod and pretend that they are making sensible claims
and move back to trying to figure out how to make things work on your own
network.

cheers,


t
On Oct 10, 2015 2:43 PM, "Eric Kuhnke"  wrote:

> As Jeremy has described in detail, the problem is at OSI layer 1. Not a
> lack of peering exchanges such as the VANIX. There is no dark fiber route
> from Alaska via the Yukon to Vancouver.
>
> I know where most of the Telus (ILEC) and Northwestel (Bell) fiber is in
> northern BC and none of interconnects with Alaska.
>
> Network topologically all locations in Alaska which are fiber fed via the
> existing submarine cable routes (not on geostationary C/Ku-band satellite)
> are a suburb of Seattle. Imagine an island with a population of about
> 600,000 people located somewhere in Puget Sound with various DWDM circuits
> that have their other ends in the Westin Building. Various IP transit,
> peering, transport and IX connections at that location.
>
> Other satellite fed singlehomed locations in Alaska can be logically just
> about anywhere thanks to the way bent-pipe relay via geostationary
> transponders work. There's at least a couple of dozen large teleports in
> the US 48 states with 7.3m and larger C-band dishes that support two way
> TDMA and SCPC services into Alaska. In such case the sites are
> indistinguishable from very low bandwidth singlehomed FDD microwave sites
> which happen to have at minimum 495ms latency.
>
>
>
> On Fri, Oct 9, 2015 at 1:04 PM, Owen DeLong  wrote:
>
> >
> > > On Oct 8, 2015, at 11:24 PM, Jeremy Austin  wrote:
> > >
> > > On Thu, Oct 8, 2015 at 3:25 PM, James Jun  wrote:
> > >
> > >>
> > >> If you want choices in your transit providers, you should get a
> > transport
> > >> circuit (dark, wave or EPL) to a nearby carrier hotel/data center.
> Once
> > >> you do that, you will suddenly find that virtually almost everyone in
> > the
> > >> competitive IP transit market will provide you with dual-stacked
> > IPv4/IPv6
> > >> service.
> > >>
> > >
> > > The future is here, but it isn't evenly distributed yet. I'm in North
> > > America, but there are no IXPs in my *state*, let alone in my
> *continent*
> > > -- from an undersea fiber perspective. There is no truly competitive IP
> > > transit market within Alaska that I am aware of. Would love to be
> proved
> > > wrong. Heck, GCI and ACS (the two providers with such fiber) only
> > directly
> > > peered a handful of years ago.
> >
> > Alaska is in the same continent as Canda and the Contiguous US.
> >
> > VANIX (Vancouver), CIX (Calgary), Manitoba-IX (Winnipeg), WPGIX
> > (WInnipeg), TORIX (Toronto),
> > and an exchange in Montreal (I forget the name) exist as well as a few
> > others in Canada (I think
> > there’s even one out in the maritimes).
> >
> > There are tons of exchanges all over the contiguous US.
> >
> > I’m surprised that there isn’t yet an exchange point in Juneau or
> > Anchorage, but that
> > does, indeed, appear to be the case. Perhaps you should work with some
> > other ISPs
> > in your state to form one.
> >
> > According to this:
> > http://www.alaskaunited.com 
> >
> > There is subsea fiber to several points in AK from Seattle and beyond.
> >
> > And on a continental basis, quite a bit of undersea fiber in other
> landing
> > stations
> > around the coastal areas of the contiguous 48.
> >
> > >> If you are buying DIA circuit from some $isp to your rural location
> that
> > >> you call "head-end" and are expecting to receive a competitive
> service,
> > >> and support for IPv6, well, then your expectations are either
> > unreasonable,
> > >> ignorant or both.
> > >>
> > >
> > > Interestingly both statewide providers *do* provide both IPv4 and IPv6
> > > peering. The trick is to find a spot where there's true price
> > competition.
> > > The 3 largest statewide ISPs have fiber that meets a mere three city
> > blocks
> > > from one of my POPs, but there's no allowable IX. I'm looking at you,
> > AT
> >
> > I’m not sure what you mean by “allowable IX”, to the best of my
> knowledge,
> > anyone
> > can build an IX anywhere.
> >
> > Owen
> >
> >
> >
>

Re: /27 the new /24

[Denis Fondras]

> >>Plus one to that. We are such a provider, and IPv6 is on my list of
> >>things to implement, but the barriers are still plenty high. Firstly, I
> >>do have an Ipv6 assignmnt and bgp (v4) and an asn, but until I can get
> >>IPv6 transit,
> >
> >There are lots of transit providers that provide IPv6.  It really is
> >time to name and shame transit providers that don't provide IPv6.
> 
> Unless he's buying from Bob's Bait, Tackle, and Internet (who's reselling
> service off his Brighthouse cable modem connection), I find it hard to
> believe there are "transit providers" in the NANOG region who still cannot
> provide dual-stack addressing and BGP for DIA.
> 

Speaking of HE, they can provide IPv6 transit (for some definition of
transit) to anyone with an ASN for almost free.

Re: /27 the new /24

[Jeremy Austin]

On Fri, Oct 9, 2015 at 12:04 PM, Owen DeLong  wrote:

>
>
> The future is here, but it isn't evenly distributed yet. I'm in North
> America, but there are no IXPs in my *state*, let alone in my *continent*
> -- from an undersea fiber perspective. There is no truly competitive IP
> transit market within Alaska that I am aware of. Would love to be proved
> wrong. Heck, GCI and ACS (the two providers with such fiber) only directly
> peered a handful of years ago.
>
>
> Alaska is in the same continent as Canda and the Contiguous US.
>

Geographically yes, but not IP-topologically. It may strictly speaking be
an exaggeration to speak of continental latencies, but we do feel a bit cut
off up here. From me to Ohio is just about twice as far as from me to CA.
The distance from the eastern US to Portugal is only about twice as long as
the Anchorage to Seattle route.


> VANIX (Vancouver), CIX (Calgary), Manitoba-IX (Winnipeg), WPGIX
> (WInnipeg), TORIX (Toronto),
> and an exchange in Montreal (I forget the name) exist as well as a few
> others in Canada (I think
> there’s even one out in the maritimes).
>

If there were ever an Alaska-to-Canada pipeline or gas line built, no doubt
there could be fiber. To my knowledge no non-Arctic Alaska to Yukon route
exists or is in public planning. I think AT may have some microwave. The
Yukon has less overall population than the city of Fairbanks, AK, and it
would be difficult to justify a fiber build, say, from Tok to Whitehorse,
without other reasons. I'm not looking at great circle routes at the
moment, but an overland route would probably be *longer* from Anchorage to
Vancouver than the current undersea routes.


> There are tons of exchanges all over the contiguous US.
>

Exactly. Now imagine an area — Alaska not including Anchorage — twice the
size of Texas, with the population of Pittsburgh, in tiny clumps far apart.
It is *possible* that the lack of IX in Alaska is due solely to geography
and not, say, to an inadequately competitive ISP environment.

I’m surprised that there isn’t yet an exchange point in Juneau or
> Anchorage, but that
> does, indeed, appear to be the case. Perhaps you should work with some
> other ISPs
> in your state to form one.
>

Juneau, I'm not so surprised; how many other cities that small and isolated
have IXes? I'm curious. It's an interesting prospect, at least for some
value of $location. Anyone interested, hit me up.

According to this:
> http://www.alaskaunited.com
>
> There is subsea fiber to several points in AK from Seattle and beyond.
>

Said undersea fiber is owned by GCI and ACS. There are some pending routes
west and north, I believe.


>
> And on a continental basis, quite a bit of undersea fiber in other landing
> stations
> around the coastal areas of the contiguous 48.
>
> If you are buying DIA circuit from some $isp to your rural location that
> you call "head-end" and are expecting to receive a competitive service,
> and support for IPv6, well, then your expectations are either unreasonable,
> ignorant or both.
>
> Interestingly both statewide providers *do* provide both IPv4 and IPv6
> peering. The trick is to find a spot where there's true price competition.
> The 3 largest statewide ISPs have fiber that meets a mere three city blocks
> from one of my POPs, but there's no allowable IX. I'm looking at you, AT
>
>
> I’m not sure what you mean by “allowable IX”, to the best of my knowledge,
> anyone
> can build an IX anywhere.
>

 I should have been more clear. No allowable IX *at the nearest fiber
meetup to me*.

It would be illuminating to see what minimum peak hour per-capita bw is
necessary to make rural IX pay, and for what value of $rural.

"Alaska suffers from… an abject lack of density." —Joe Freddoso, Mighty
River/USAC

Re: /27 the new /24

[Mike Hammett]

As I "know" Jeremy from elsewhere, I reached out to him this morning about 
this. I would like to speak with any other Alaskers? Alaskites? people from 
Alaska. 




- 
Mike Hammett 
Intelligent Computing Solutions 
http://www.ics-il.com 



Midwest Internet Exchange 
http://www.midwest-ix.com 


- Original Message -

From: "Jeremy Austin" <jhaus...@gmail.com> 
To: "Owen DeLong" <o...@delong.com> 
Cc: nanog@nanog.org, "James Jun" <ja...@towardex.com> 
Sent: Friday, October 9, 2015 3:51:12 PM 
Subject: Re: /27 the new /24 

On Fri, Oct 9, 2015 at 12:04 PM, Owen DeLong <o...@delong.com> wrote: 

> 
> 
> The future is here, but it isn't evenly distributed yet. I'm in North 
> America, but there are no IXPs in my *state*, let alone in my *continent* 
> -- from an undersea fiber perspective. There is no truly competitive IP 
> transit market within Alaska that I am aware of. Would love to be proved 
> wrong. Heck, GCI and ACS (the two providers with such fiber) only directly 
> peered a handful of years ago. 
> 
> 
> Alaska is in the same continent as Canda and the Contiguous US. 
> 

Geographically yes, but not IP-topologically. It may strictly speaking be 
an exaggeration to speak of continental latencies, but we do feel a bit cut 
off up here. From me to Ohio is just about twice as far as from me to CA. 
The distance from the eastern US to Portugal is only about twice as long as 
the Anchorage to Seattle route. 


> VANIX (Vancouver), CIX (Calgary), Manitoba-IX (Winnipeg), WPGIX 
> (WInnipeg), TORIX (Toronto), 
> and an exchange in Montreal (I forget the name) exist as well as a few 
> others in Canada (I think 
> there’s even one out in the maritimes). 
> 

If there were ever an Alaska-to-Canada pipeline or gas line built, no doubt 
there could be fiber. To my knowledge no non-Arctic Alaska to Yukon route 
exists or is in public planning. I think AT may have some microwave. The 
Yukon has less overall population than the city of Fairbanks, AK, and it 
would be difficult to justify a fiber build, say, from Tok to Whitehorse, 
without other reasons. I'm not looking at great circle routes at the 
moment, but an overland route would probably be *longer* from Anchorage to 
Vancouver than the current undersea routes. 


> There are tons of exchanges all over the contiguous US. 
> 

Exactly. Now imagine an area — Alaska not including Anchorage — twice the 
size of Texas, with the population of Pittsburgh, in tiny clumps far apart. 
It is *possible* that the lack of IX in Alaska is due solely to geography 
and not, say, to an inadequately competitive ISP environment. 

I’m surprised that there isn’t yet an exchange point in Juneau or 
> Anchorage, but that 
> does, indeed, appear to be the case. Perhaps you should work with some 
> other ISPs 
> in your state to form one. 
> 

Juneau, I'm not so surprised; how many other cities that small and isolated 
have IXes? I'm curious. It's an interesting prospect, at least for some 
value of $location. Anyone interested, hit me up. 

According to this: 
> http://www.alaskaunited.com 
> 
> There is subsea fiber to several points in AK from Seattle and beyond. 
> 

Said undersea fiber is owned by GCI and ACS. There are some pending routes 
west and north, I believe. 


> 
> And on a continental basis, quite a bit of undersea fiber in other landing 
> stations 
> around the coastal areas of the contiguous 48. 
> 
> If you are buying DIA circuit from some $isp to your rural location that 
> you call "head-end" and are expecting to receive a competitive service, 
> and support for IPv6, well, then your expectations are either unreasonable, 
> ignorant or both. 
> 
> Interestingly both statewide providers *do* provide both IPv4 and IPv6 
> peering. The trick is to find a spot where there's true price competition. 
> The 3 largest statewide ISPs have fiber that meets a mere three city blocks 
> from one of my POPs, but there's no allowable IX. I'm looking at you, AT 
> 
> 
> I’m not sure what you mean by “allowable IX”, to the best of my knowledge, 
> anyone 
> can build an IX anywhere. 
> 

I should have been more clear. No allowable IX *at the nearest fiber 
meetup to me*. 

It would be illuminating to see what minimum peak hour per-capita bw is 
necessary to make rural IX pay, and for what value of $rural. 

"Alaska suffers from… an abject lack of density." —Joe Freddoso, Mighty 
River/USAC 

Re: AW: AW: AW: /27 the new /24

[Jérôme Nicolle]

Hello James,

Le 05/10/2015 06:45, James Jun a écrit :
> I'm not aware of any carrier-grade network that operates on these things.

With the availability of a 80Gbps model and upcoming updates to the
routing process (in RouterOS 7), chances are these boxes will drag much
more attention in the next 2-3 years.

Still, it looks like their products are widely used in developping
countries where cheap hardware and flexible / low power requirements
(you'd run a CCR1009 off a car battery for a solid 2 weeks - no
regulator needed) makes them the only viable choice.

I know of at least a dozen ISP running these as well, here in western
Europe. It solved many space and power issues in dense carrier hotels,
and is a cheap and efficient way out of a 6500/7600 (in sub 20Gbps
scenarios) based network.

I wouldn't sell transit or provide criticial services with a
mikrotik-based network just yet, mostly for the lack of enough personnal
confidence and experience with them, but havin endured nights of
debugging with poor quality code in recent major player's routers, I
doubt they're as misfits as you suggest.

Best regards,

-- 
Jérôme Nicolle

Re: /27 the new /24

[Lee Howard]

On 10/8/15, 6:45 PM, "NANOG on behalf of Mike"  wrote:

>
>
>On 10/08/2015 02:41 PM, Mark Andrews wrote:
>>
>>
>> Plus one to that. We are such a provider, and IPv6 is on my list of
>> things to implement, but the barriers are still plenty high. Firstly, I
>> do have an Ipv6 assignmnt and bgp (v4) and an asn, but until I can get
>> IPv6 transit,
>>
>> There are lots of transit providers that provide IPv6.  It really is
>> time to name and shame transit providers that don't provide IPv6.
>
>NO, THERE IS NOT. We operate in rural and underserved areas and WE DO
>NOT HAVE realistic choices. Can you see me from your ivory tower?

I looked up tiedyenetworks.com, and I think he¹s 100 miles from Sacramento.
I hope some sales person from a transit provider is giving him a call right
now, but it¹s entirely possible that there are no big providers in his
neighborhood. Sorry, Mike, wish I could help you there.

However, you can still mock your upstreams here. Then we can offer them
help to support IPv6.

>
>>> there is not much point in my putting a lot of effort into
>>> enabling IPv6 for my subscribers. Yes I have a HE tunnel and yes it's
>>> working, but it's not the same as running native v6 and with my own
>>> address space. Second, on the group of servers that have v6 thru the HE
>>> tunnel, I still run into problems all the time where some operations
>>> over v6 simply fail inexplictly, requireing me to turn off v6 on that
>>> host so whatever it is I'm doing can proceed over v4.
>>> Stuff like OS updates for example.
>> Then complain to the OS vendor.  It is most probably someone breaking
>> PMTU discover by filtering PTB.  Going native will hide these
>> problems until the MTU between the DC and the rest of the net
>> increases.  You could also just lower the advertised MTU internally
>> to match the tunnel MTU which would let you simulate better what a
>> native experience would be.
>Not my job. v4 works, v6 does not, end of story.

It sounds like you do have some concern about the transition, and you know
there¹s a bug, at least with OS downloads. Please do report those
issues you know about.

Usually, Happy Eyeballs masks problems in dual stack, whether that¹s good
or bad. 

If we can get your upstream(s) to support IPv6, then maybe we can leverage
them to help troubleshoot MTU problems, so you don¹t have to spend a lot
of time on them. Or maybe they go away when you¹re no longer tunnelling.

>
>>
>> I can't remember the last time I saw a site stall due to reaching it
>> over IPv6 it is that long ago.
>
>It happens every day for me, which only amplifies my perception that v6
>IS NOT READY FOR PRIME TIME.

Would you at least keep a list of places you have these problems, even if
you never follow up on it? Again, I¹m wondering if tunnelling is the
problem, and once you have native dual-stack, you could refer to the list
and see if problems just dry up.

>
>>> Damm maddening. Can't imagine the screaming I'll
>>> hear if a home user ever ran into similar so I am quite gun shy about
>>> the prospect. Secondly, the the dodgy nature of the CPE connected to
>>>our
>>> network and the terminally buggy fw they all run is sure to be a never
>>> ending source of stupidity.
>> CPE devices are buggy for IPv4 as well.  Bugs in CPE devices are
>> only found and fixed if the code paths are exercised.
>
>Not my job. v4 works, v6 does not. I am a provider not a developer.

I would guess it is your job in IPv4.
I would also guess, based on gateways I¹ve seen, than 10% of CPE
has critical IPv4 bugs, and 25% of CPE has critical Ipv6 bugs. I agree with
you that the difference is too high, and maybe waiting a year helps get
those ratios aligned.

CPE vendors: step it up!

>> That said IPv6 worked fine for me with the shipped image (old version
>> of OpenWRT) using 6to4 before I reflashed it to a modern version
>> of OpenWRT as I wanted to use the HE tunnel rather than 6to4.  I
>> know that is only one CPE device.
>  And will you be providing all of my end users with replacement CPE
>that meets all of the other requirements too? No? Because no such
>devices exist yet? OHHH yeah thats right, I'm a provider not a
>developer, so again, not a solution for my business.

Ugh, 6to4 is a bad idea anyway. Deprecated, even.
There are loads of gateways that support native dual-stack and several
transition mechanisms (DS-lite, 6rd, and MAP pretty soon), but native
dual-stack is the way to go if you possibly can.
If you provide gateways as part of your service, you should at least make
sure you¹re providing devices that at least *claim* IPv6 support now (and
the IPv6 CPE Ready logo is meaningful here), so that as old equipment
ages out, you¹re not stuck replacing newer boxes.

Figure out how long until you think you really need all of your customers
to have IPv6. Subtract your CPE replacement time. Start replacing CPE then.
e.g., if users need IPv6 in 2018, and you replace 

Re: /27 the new /24

[Mike Hammett]

Agreed. Often times people forget that budgets aren't unlimited and not 
everyone is in One Wilshire, 350 Cermak or 60 Hudson. 




- 
Mike Hammett 
Intelligent Computing Solutions 
http://www.ics-il.com 



Midwest Internet Exchange 
http://www.midwest-ix.com 


- Original Message -

From: "Jason Baugher" <ja...@thebaughers.com> 
To: "James Jun" <ja...@towardex.com> 
Cc: "NANOG" <nanog@nanog.org> 
Sent: Thursday, October 8, 2015 7:21:05 PM 
Subject: Re: /27 the new /24 

This thread, while originally interesting and helpful, seems to have 
degraded to a contest to see who can be the most arrogant, condescending 
and insulting. Congrats. 
On Oct 8, 2015 6:25 PM, "James Jun" <ja...@towardex.com> wrote: 

> On Thu, Oct 08, 2015 at 03:45:38PM -0700, Mike wrote: 
> > 
> > NO, THERE IS NOT. We operate in rural and underserved areas and WE DO 
> > NOT HAVE realistic choices. Can you see me from your ivory tower? 
> 
> Who is your upstream provider? 
> 
> I think you're confused on how the IP transit industry works. 
> 
> If you want choices in your transit providers, you should get a transport 
> circuit (dark, wave or EPL) to a nearby carrier hotel/data center. Once 
> you do that, you will suddenly find that virtually almost everyone in the 
> competitive IP transit market will provide you with dual-stacked IPv4/IPv6 
> service. 
> 
> If you are buying DIA circuit from some $isp to your rural location that 
> you call "head-end" and are expecting to receive a competitive service, 
> and support for IPv6, well, then your expectations are either unreasonable, 
> ignorant or both. 
> 
> Best, 
> James 
> 

Re: Mikrotik in the DFZ (Was Re: AW: AW: /27 the new /24)

[Jérôme Nicolle]

-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1

Hello William,

Le 03/10/2015 10:23, William Waites a écrit :
> I wish it were possible today to run different software on their 
> larger boxes. If some like-minded small providers wanted to get 
> together with us to fund a FreeBSD port to the CCR routers that
> would be great. Please contact me off-list if you are interested in
> this, I'll coordinate.

One of my contacts has worked on a similar path, and we encountered
many issues that makes it quite difficult.

Here is what I gathered, while I never had no access to the
NDA-covered material.

The Tilera architecture and its SDK is a great way to start such
project but Mikrotik didn't just use an off-the-shelf chip as
recommended, they also made slight changes to how the network
interfaces operates, and didn't provide any documentation.

It's not as easy as swapping a driver and rebuild a kernel, more like
changing how the programmable logic in Tilera's interface blocks
dispatch frames among the core's interconnexion grid.

Also, the cores are not fully compliant with MIPS specifications,
aren't combined as an SMP assembly at all (rather a NUMA grid with
added glue logic) and you can't even load the first instruction at 0x0
without using Tilera's own proprietary init code to allocate
ressources, initialize cores and setup the multiple "containers" (kind
of hardware virtualization).

So it's not quite about porting an OS than it has to do with tight
coupling of proprietary control code, bare-metal and FPGA logic, and a
specific data-plane implementation.

Still, the attempts have gone as far as booting a cutom linux kernel
spanned among a single CPU instance made of all 36 cores, but it has
no access to the network interfaces on the mikrotik board.

It does, however, work flawlessly on Tilera's developpment boards and
appliances, though neither the Linux kernel's data plane or
DPDK-derived code are yet able to take advantage of the specificities
of Tilera's architecture.

Nevertheless, if you want do get deeper and have enough motivation to
get past the technical difficulties, I'd gladly try to help into
bringing an alternative OS to these box.

Best regards,

- -- 
Jérôme Nicolle
-BEGIN PGP SIGNATURE-
Version: GnuPG/MacGPG2 v2
Comment: GPGTools - https://gpgtools.org

iEYEARECAAYFAlYXtDUACgkQbt+nwQamihu3eACdFRkX/yXrEJHJHm9F7HD0ClV4
2ikAnjy6a7KRheMlKTfFRaccfuYQInfc
=qRKm
-END PGP SIGNATURE-

Re: AW: AW: AW: /27 the new /24

[Mike Hammett]

I know of literally hundreds of ISPs using them in the US and I'm sure that 
number is in the thousands. After hearing complaints from larger networks of 
their larger gear... it's the same shit everyone else deals with. 




- 
Mike Hammett 
Intelligent Computing Solutions 
http://www.ics-il.com 



Midwest Internet Exchange 
http://www.midwest-ix.com 


- Original Message -

From: "Jérôme Nicolle" <jer...@ceriz.fr> 
To: nanog@nanog.org 
Sent: Friday, October 9, 2015 7:10:31 AM 
Subject: Re: AW: AW: AW: /27 the new /24 

Hello James, 

Le 05/10/2015 06:45, James Jun a écrit : 
> I'm not aware of any carrier-grade network that operates on these things. 

With the availability of a 80Gbps model and upcoming updates to the 
routing process (in RouterOS 7), chances are these boxes will drag much 
more attention in the next 2-3 years. 

Still, it looks like their products are widely used in developping 
countries where cheap hardware and flexible / low power requirements 
(you'd run a CCR1009 off a car battery for a solid 2 weeks - no 
regulator needed) makes them the only viable choice. 

I know of at least a dozen ISP running these as well, here in western 
Europe. It solved many space and power issues in dense carrier hotels, 
and is a cheap and efficient way out of a 6500/7600 (in sub 20Gbps 
scenarios) based network. 

I wouldn't sell transit or provide criticial services with a 
mikrotik-based network just yet, mostly for the lack of enough personnal 
confidence and experience with them, but havin endured nights of 
debugging with poor quality code in recent major player's routers, I 
doubt they're as misfits as you suggest. 

Best regards, 

-- 
Jérôme Nicolle 

Re: /27 the new /24

[Mike]

On 10/08/2015 07:58 PM, Owen DeLong wrote:


I can't remember the last time I saw a site stall due to reaching it over IPv6 
it is that long ago.

It happens every day for me, which only amplifies my perception that v6 IS NOT 
READY FOR PRIME TIME.

Yet you refuse to troubleshoot your issues with it that are not shared by 
others and blame the protocol for whatever is probably wrong with your own 
network. Interesting tactic.


Thats invalid. It matters not that you claim these isses are not 'shared 
by others' - they are experienced routinely by others, and it's growing 
worse as more 'services' are transitioned to 'v6' but then the attendant 
support such as monitoring and operational knowledge/experience hasn't 
caught up and those transitioned services fail on v6 silently for long 
periods of time. That is the majority of the v6 world today and it's 
useless to claim otherwise.



Mike-

The continuing IPv6 discussion (was: /27 the new /24)

[Stephen Satchell]

I have been reviewing the proposed submission to the FCC ET Docket No. 
15-170, regarding the requirement that vendors of wireless equipment 
"lock down" updates, and find this quote in that submission particularly 
apropos to the ongoing IPv6-on-wireless discussion:


"Most Wi-Fi routers, even the newest ones, do not or only barely support 
IPv6, with poor implementations of IPv6 leading to problems with 
interoperability"


Quoting this thread, they are, in their submission!

Does it make the assertion that "existing CPE isn't ready for prime 
time" accurate?  That's still open for debate.  What is "settled 
science" is that, after 17 years, we are still learning just how to deal 
with IPv6's quirks and foibles, just as the community did with IPv4 in 
the last quarter of the previous century (and continuing into this one). 
 After 17 years, the community at large has a huge hurdle educating the 
vendors, the providers, and the consumers to the point that everyone 
(not just a few geeks) has a satisfactory comfort level with the new 
protocol.


The heat and flame in this thread shows the range of comfort levels. 
Most of the conflict exposed in this thread concerns the differences in 
views by providers of general transit, versus providers of "last mile" 
and enterprise connectivity.  The positions taken by the two camps are 
equally valid FOR THEIR MARKETS.


As one of the participants in the latter market category, I have learned 
a great deal about IPv6.  Moreover, I have been given pointers by people 
in this forum to sources of additional information, particularly sources 
that are more up to date.  Amazon will be delivering a big box around 
the 17th.


As for my own network, I have discovered that my upstream, Charter 
Communications, has put together some support pages for their deployment 
of IPv6.  Once I have the appropriate firewall in place to mirror my 
IPv4 firewall I can start getting dual-stack IPv4/IPv6 up and going.

Re: /27 the new /24

[Scott Weeks]

--- morrowc.li...@gmail.com wrote:
From: Christopher Morrow 

... and trying to jam your favorite flavor of spam 
down the other person's throat is only going to make 
them hate hawaii.
--


FYI...  :-)

http://www.thehawaiiplan.com/why-do-hawaiians-love-spam

"One tidbit of information that seems to get around is 
that people in Hawaii really like SPAM. And it’s true: 
about 6 million cans of SPAM are eaten each year in 
Hawaii. That’s around 5 cans per person in Hawaii."


scott

Re: /27 the new /24

[Owen DeLong]

> On Oct 8, 2015, at 11:24 PM, Jeremy Austin  wrote:
> 
> On Thu, Oct 8, 2015 at 3:25 PM, James Jun  wrote:
> 
>> 
>> If you want choices in your transit providers, you should get a transport
>> circuit (dark, wave or EPL) to a nearby carrier hotel/data center.  Once
>> you do that, you will suddenly find that virtually almost everyone in the
>> competitive IP transit market will provide you with dual-stacked IPv4/IPv6
>> service.
>> 
> 
> The future is here, but it isn't evenly distributed yet. I'm in North
> America, but there are no IXPs in my *state*, let alone in my *continent*
> -- from an undersea fiber perspective. There is no truly competitive IP
> transit market within Alaska that I am aware of. Would love to be proved
> wrong. Heck, GCI and ACS (the two providers with such fiber) only directly
> peered a handful of years ago.

Alaska is in the same continent as Canda and the Contiguous US.

VANIX (Vancouver), CIX (Calgary), Manitoba-IX (Winnipeg), WPGIX (WInnipeg), 
TORIX (Toronto),
and an exchange in Montreal (I forget the name) exist as well as a few others 
in Canada (I think
there’s even one out in the maritimes).

There are tons of exchanges all over the contiguous US.

I’m surprised that there isn’t yet an exchange point in Juneau or Anchorage, 
but that
does, indeed, appear to be the case. Perhaps you should work with some other 
ISPs
in your state to form one.

According to this:
http://www.alaskaunited.com 

There is subsea fiber to several points in AK from Seattle and beyond.

And on a continental basis, quite a bit of undersea fiber in other landing 
stations
around the coastal areas of the contiguous 48.

>> If you are buying DIA circuit from some $isp to your rural location that
>> you call "head-end" and are expecting to receive a competitive service,
>> and support for IPv6, well, then your expectations are either unreasonable,
>> ignorant or both.
>> 
> 
> Interestingly both statewide providers *do* provide both IPv4 and IPv6
> peering. The trick is to find a spot where there's true price competition.
> The 3 largest statewide ISPs have fiber that meets a mere three city blocks
> from one of my POPs, but there's no allowable IX. I'm looking at you, AT

I’m not sure what you mean by “allowable IX”, to the best of my knowledge, 
anyone
can build an IX anywhere.

Owen

Re: /27 the new /24

[Owen DeLong]

> On Oct 9, 2015, at 10:22 AM, Mike  wrote:
> 
> On 10/08/2015 07:58 PM, Owen DeLong wrote:
>> 
>> I can't remember the last time I saw a site stall due to reaching it over 
>> IPv6 it is that long ago.
>>> It happens every day for me, which only amplifies my perception that v6 IS 
>>> NOT READY FOR PRIME TIME.
>> Yet you refuse to troubleshoot your issues with it that are not shared by 
>> others and blame the protocol for whatever is probably wrong with your own 
>> network. Interesting tactic.
> 
> Thats invalid. It matters not that you claim these isses are not 'shared by 
> others' - they are experienced routinely by others, and it's growing worse as 
> more 'services' are transitioned to 'v6' but then the attendant support such 
> as monitoring and operational knowledge/experience hasn't caught up and those 
> transitioned services fail on v6 silently for long periods of time. That is 
> the majority of the v6 world today and it's useless to claim otherwise.

Permit me to rephrase….

…not experienced by those with functioning networks…

That is… It is not an inherent problem in the protocol, but rather a local 
misconfiguration  in those networks experiencing these problems.

There is sufficient evidence to back this up as the symptoms describe well 
known problems with well known solutions. The choice of particular network 
operators to complain about IPv6 being broken rather than research and apply 
those well known solutions is, in fact, a problem with those operators and not 
a problem with IPv6.

Owen

Re: /27 the new /24

[Christopher Morrow]

(I'm going to regret this but...)

On Fri, Oct 9, 2015 at 10:22 AM, Mike  wrote:
> On 10/08/2015 07:58 PM, Owen DeLong wrote:
>>
>>
>> I can't remember the last time I saw a site stall due to reaching it over
>> IPv6 it is that long ago.
>>>
>>> It happens every day for me, which only amplifies my perception that v6
>>> IS NOT READY FOR PRIME TIME.
>>
>> Yet you refuse to troubleshoot your issues with it that are not shared by
>> others and blame the protocol for whatever is probably wrong with your own
>> network. Interesting tactic.
>
>
> Thats invalid. It matters not that you claim these isses are not 'shared by
> others' - they are experienced routinely by others, and it's growing worse
> as more 'services' are transitioned to 'v6' but then the attendant support
> such as monitoring and operational knowledge/experience hasn't caught up and
> those transitioned services fail on v6 silently for long periods of time.
> That is the majority of the v6 world today and it's useless to claim
> otherwise.

The sense I get from the the thread bits I read is generally:
  1) some (one, few, etc) folks are upset with v6 in their network or
their deployment or their experience
  2) some (many?) folks are pushing to 'move to v6!'
  3) anger

I think we should remember that:
  1) your network, your rules
  2) if you don't want to add v6 that's totally your call
  3) the v4 internet will start getting less used over time, and more v6
  stuff will appear
  4) eventually users on only v4 will get degraded/no service for things
  they want to do.

putting your head in the sand (on either side) isn't helpful here, and
trying to jam your favorite flavor of spam down the other person's
throat is only going to make them hate hawaii.

-chris
(I'm sure there's a Dune quote to be used here somewhere as well...)

Re: /27 the new /24

[Stephen Satchell]

On 10/09/2015 08:18 AM, Christopher Morrow wrote:

(I'm going to regret this but...)


No good deed ever goes unpunished.


(I'm sure there's a Dune quote to be used here somewhere as well...)


Indeed:

   "A beginning is the time for taking the
most delicate care that the balances
are correct."
   -- _Dune_ (1965) Frank Herbert

Re: /27 the new /24

[Matthew Kaufman]

On 10/7/15 7:00 AM, Mark Andrews wrote:
I don't see anyone wishing it went differnetly. I see someone pointing 
out the reality that lots of ISP's are way too late to delivering 
IPv6. *Every* ISP should have been planning to deliver IPv6 by the 
time the first RIR ran out of IPv4 addresses. 


Look, I'm as much a supporter of delivering IPv6 as anyone. I've had 
IPv6 enabled on my home network (and the small data center I run in my 
garage) for over a decade now. In 2004, I made sure that IPv6 was fully 
supported in the peer-to-peer stack I developed and that eventually 
became RFC 7016. And for the last 5 years I've been pushing for IPv6 
support in the product I work on for my employer.


But the reality is that there's a whole lot of small and medium-sized 
ISPs run by fine, upstanding individuals serving their communities -- 
even in and around the San Francisco Bay Area -- that have either no or 
very limited (tunnels only) support for IPv6. That's the reality of the 
transition. And threatening these folks with the attorney general isn't 
the way to get them to adopt IPv6, nor is shaming them. They will add 
IPv6 support when it is easy to do, when their staff has the time, and 
when the economics make sense.


Meanwhile we have app developers trying to use cloud platforms that 
don't support IPv6 well (or at all), writing code while sitting in 
offices that don't have IPv6 service due either to their ISP or their 
internal IT department... and so there's another reason ISPs need to 
keep concentrating on IPv4 as their first priority.


And so, in the current actual Internet, not some hypothetical one, if 
you want your website to be seen, you get it an IPv4 address. And with 
IPv4 going for $6-$8 each and it being possible to support hundreds or 
thousands of websites on a single IPv4 address, there's really no excuse.


Will this be different in the future? I sure hope so. But we're not 
there yet.


Matthew Kaufman

Re: /27 the new /24

[Valdis . Kletnieks]

On Wed, 07 Oct 2015 17:49:44 -0400, Matthew Kaufman said:
>
>
> > On Oct 7, 2015, at 4:15 PM, Mark Andrews  wrote:
> >
> >
> > I don't have to.  I'm sure some AG will do so soon enough.
>
> There's always an optimist around.
>
> Good luck with that.

And I happened to get a big thing of very good microwave popcorn the other
night.  What perfect timing


pgpakoRc2vxBK.pgp
Description: PGP signature

Re: /27 the new /24

[Mark Andrews]

In message <56166c30.3070...@matthew.at>, Matthew Kaufman writes:
> 
> 
> On 10/7/15 7:00 AM, Mark Andrews wrote:
> > I don't see anyone wishing it went differnetly. I see someone pointing 
> > out the reality that lots of ISP's are way too late to delivering 
> > IPv6. *Every* ISP should have been planning to deliver IPv6 by the 
> > time the first RIR ran out of IPv4 addresses. 
> 
> Look, I'm as much a supporter of delivering IPv6 as anyone. I've had 
> IPv6 enabled on my home network (and the small data center I run in my 
> garage) for over a decade now. In 2004, I made sure that IPv6 was fully 
> supported in the peer-to-peer stack I developed and that eventually 
> became RFC 7016. And for the last 5 years I've been pushing for IPv6 
> support in the product I work on for my employer.

And I've done tunnelled (home) and native (office in RWC) for over
a decade.

> But the reality is that there's a whole lot of small and medium-sized 
> ISPs run by fine, upstanding individuals serving their communities -- 
> even in and around the San Francisco Bay Area -- that have either no or 
> very limited (tunnels only) support for IPv6. That's the reality of the 
> transition. And threatening these folks with the attorney general isn't 
> the way to get them to adopt IPv6, nor is shaming them. They will add 
> IPv6 support when it is easy to do, when their staff has the time, and 
> when the economics make sense.

I'm happy if they advertise "IPv4 Internet Only", just don't lie
by claiming you deliever the Internet.  To deliver the Internet you
need to be delivering both IPv4 and IPv6.

> Meanwhile we have app developers trying to use cloud platforms that 
> don't support IPv6 well (or at all), writing code while sitting in 
> offices that don't have IPv6 service due either to their ISP or their 
> internal IT department... and so there's another reason ISPs need to 
> keep concentrating on IPv4 as their first priority.

Just because some of the ISP's customers are happy with IPv4 is not
a reason to neglect the customers that need IPv6.  You may not want
to call the Sudan often but would you want your telco to be incapable
of delivering calls to the Sudan?

> And so, in the current actual Internet, not some hypothetical one, if 
> you want your website to be seen, you get it an IPv4 address. And with 
> IPv4 going for $6-$8 each and it being possible to support hundreds or 
> thousands of websites on a single IPv4 address, there's really no excuse.

And there you go assuming that a hosted web site is what someone
needs rather than the ability to get back to their machines that
are behind a CGN for IPv4 but are reachable over IPv6.

This is today's reality and ISP's are not meeting today's needs.
It's not just about having enough IPv4 addresses.  It's about
providing the infrastructure to allow your customers to connect to
everyone.

> Will this be different in the future? I sure hope so. But we're not 
> there yet.
> 
> Matthew Kaufman
-- 
Mark Andrews, ISC
1 Seymour St., Dundas Valley, NSW 2117, Australia
PHONE: +61 2 9871 4742 INTERNET: ma...@isc.org

Re: /27 the new /24

[Christian de Larrinaga]

Around 2004 I noted that the fear was without v4 something in the
network would break. (It was considered crazy then to consider v6 only).

Now I'm seeing concern that something in the applications will break.
The difference is that networks can't guarantee to push static IPv4 to
those problems like they could. New networks can't establish let alone
grow unless they are essentially v6 only with v4 translation. But I'm
seeing concern that some of these newer IETF transition mechanisms are
too complex or expensive  - i.e., off-putting enough so a smaller ISP is
forced to consider CGNAT. 

I'm not sure if this is just an isolated case or if there is something
missing needed by smaller and growing ISPs . 


Christian


Matthew Kaufman wrote:
>
>
> On 10/7/15 7:00 AM, Mark Andrews wrote:
>> I don't see anyone wishing it went differnetly. I see someone
>> pointing out the reality that lots of ISP's are way too late to
>> delivering IPv6. *Every* ISP should have been planning to deliver
>> IPv6 by the time the first RIR ran out of IPv4 addresses. 
>
> Look, I'm as much a supporter of delivering IPv6 as anyone. I've had
> IPv6 enabled on my home network (and the small data center I run in my
> garage) for over a decade now. In 2004, I made sure that IPv6 was
> fully supported in the peer-to-peer stack I developed and that
> eventually became RFC 7016. And for the last 5 years I've been pushing
> for IPv6 support in the product I work on for my employer.
>
> But the reality is that there's a whole lot of small and medium-sized
> ISPs run by fine, upstanding individuals serving their communities --
> even in and around the San Francisco Bay Area -- that have either no
> or very limited (tunnels only) support for IPv6. That's the reality of
> the transition. And threatening these folks with the attorney general
> isn't the way to get them to adopt IPv6, nor is shaming them. They
> will add IPv6 support when it is easy to do, when their staff has the
> time, and when the economics make sense.
>
> Meanwhile we have app developers trying to use cloud platforms that
> don't support IPv6 well (or at all), writing code while sitting in
> offices that don't have IPv6 service due either to their ISP or their
> internal IT department... and so there's another reason ISPs need to
> keep concentrating on IPv4 as their first priority.
>
> And so, in the current actual Internet, not some hypothetical one, if
> you want your website to be seen, you get it an IPv4 address. And with
> IPv4 going for $6-$8 each and it being possible to support hundreds or
> thousands of websites on a single IPv4 address, there's really no excuse.
>
> Will this be different in the future? I sure hope so. But we're not
> there yet.
>
> Matthew Kaufman

-- 
Christian de Larrinaga  FBCS, CITP,
-
@ FirstHand
-
+44 7989 386778
c...@firsthand.net
-

Re: /27 the new /24

[David Barak via NANOG]

On Thu, 10/8/15, Mark Andrews  wrote:

> This is today's reality and ISP's are not meeting
> today's needs.
> It's not just about
> having enough IPv4 addresses.  It's about
> providing the infrastructure to allow your
> customers to connect to
> everyone.

I think you should s/everyone/everyone they care about/

That roughly explains why there is no particular consumer outcry (which isn't 
about speed/bandwidth or mobile coverage, anyway).

 David Barak

Re: /27 the new /24

[Mike]

On 10/08/2015 06:14 AM, Matthew Kaufman wrote:



On 10/7/15 7:00 AM, Mark Andrews wrote:
I don't see anyone wishing it went differnetly. I see someone 
pointing out the reality that lots of ISP's are way too late to 
delivering IPv6. *Every* ISP should have been planning to deliver 
IPv6 by the time the first RIR ran out of IPv4 addresses. 


Look, I'm as much a supporter of delivering IPv6 as anyone. I've had 
IPv6 enabled on my home network (and the small data center I run in my 
garage) for over a decade now. In 2004, I made sure that IPv6 was 
fully supported in the peer-to-peer stack I developed and that 
eventually became RFC 7016. And for the last 5 years I've been pushing 
for IPv6 support in the product I work on for my employer.


But the reality is that there's a whole lot of small and medium-sized 
ISPs run by fine, upstanding individuals serving their communities -- 
even in and around the San Francisco Bay Area -- that have either no 
or very limited (tunnels only) support for IPv6. That's the reality of 
the transition. And threatening these folks with the attorney general 
isn't the way to get them to adopt IPv6, nor is shaming them. They 
will add IPv6 support when it is easy to do, when their staff has the 
time, and when the economics make sense.





Plus one to that. We are such a provider, and IPv6 is on my list of 
things to implement, but the barriers are still plenty high. Firstly, I 
do have an Ipv6 assignmnt and bgp (v4) and an asn, but until I can get 
IPv6 transit, there is not much point in my putting a lot of effort into 
enabling IPv6 for my subscribers. Yes I have a HE tunnel and yes it's 
working, but it's not the same as running native v6 and with my own 
address space. Second, on the group of servers that have v6 thru the HE 
tunnel, I still run into problems all the time where some operations 
over v6 simply fail inexplictly, requireing me to turn off v6 on that 
host so whatever it is I'm doing can proceed over v4. Stuff like OS 
updates for example. Damm maddening. Can't imagine the screaming I'll 
hear if a home user ever ran into similar so I am quite gun shy about 
the prospect. Secondly, the the dodgy nature of the CPE connected to our 
network and the terminally buggy fw they all run is sure to be a never 
ending source of stupidity. Thirdly, some parts of my network are 
wireless, and multicast is a huge, huge problem on wireless (the 802.11 
varities anyways). The forwarding rates for multicast are sickeningly 
low for many brand of gear - yes, it's at the bottom of the barrel no 
matter how good or hot your signal is - and I honestly expect v6 to 
experience enough disruption over wireless as to render it unusable for 
exactly this reason alone.


The wired portion of my subscriber network is only slightly better, im 
pretty sure it can deal with v6 in the middle, but the question is still 
wether specfic CPE models can and which set of bugs I'll hit on my 
access concentrators passing our v6 over PPPoE. I just read about a 
cisco bug where enabling rp-filtering on v6 causes a router reload, 
which I would hit immediately since rp-filtering is a standard 
subscriber profile option here (trying to be a good netizen). How many 
other network destroying bugs await? The longer I wait on v6, the less 
work I will have to do dealing with bugs. So, as the original posted 
said, we'll do v6 when it's easy, when we have time, and when the 
economics make sense.


Mike-

Re: /27 the new /24

[James Jun]

On Thu, Oct 08, 2015 at 03:45:38PM -0700, Mike wrote:
> 
> NO, THERE IS NOT. We operate in rural and underserved areas and WE DO 
> NOT HAVE realistic choices. Can you see me from your ivory tower?

Who is your upstream provider?

I think you're confused on how the IP transit industry works.

If you want choices in your transit providers, you should get a transport 
circuit (dark, wave or EPL) to a nearby carrier hotel/data center.  Once
you do that, you will suddenly find that virtually almost everyone in the
competitive IP transit market will provide you with dual-stacked IPv4/IPv6
service.

If you are buying DIA circuit from some $isp to your rural location that
you call "head-end" and are expecting to receive a competitive service,
and support for IPv6, well, then your expectations are either unreasonable, 
ignorant or both.

Best,
James

Re: /27 the new /24

[Mike]

On 10/08/2015 02:41 PM, Mark Andrews wrote:



Plus one to that. We are such a provider, and IPv6 is on my list of
things to implement, but the barriers are still plenty high. Firstly, I
do have an Ipv6 assignmnt and bgp (v4) and an asn, but until I can get
IPv6 transit,

There are lots of transit providers that provide IPv6.  It really is
time to name and shame transit providers that don't provide IPv6.


NO, THERE IS NOT. We operate in rural and underserved areas and WE DO 
NOT HAVE realistic choices. Can you see me from your ivory tower?



there is not much point in my putting a lot of effort into
enabling IPv6 for my subscribers. Yes I have a HE tunnel and yes it's
working, but it's not the same as running native v6 and with my own
address space. Second, on the group of servers that have v6 thru the HE
tunnel, I still run into problems all the time where some operations
over v6 simply fail inexplictly, requireing me to turn off v6 on that
host so whatever it is I'm doing can proceed over v4.
Stuff like OS updates for example.

Then complain to the OS vendor.  It is most probably someone breaking
PMTU discover by filtering PTB.  Going native will hide these
problems until the MTU between the DC and the rest of the net
increases.  You could also just lower the advertised MTU internally
to match the tunnel MTU which would let you simulate better what a
native experience would be.

Not my job. v4 works, v6 does not, end of story.



I can't remember the last time I saw a site stall due to reaching it 
over IPv6 it is that long ago.


It happens every day for me, which only amplifies my perception that v6 
IS NOT READY FOR PRIME TIME.



Damm maddening. Can't imagine the screaming I'll
hear if a home user ever ran into similar so I am quite gun shy about
the prospect. Secondly, the the dodgy nature of the CPE connected to our
network and the terminally buggy fw they all run is sure to be a never
ending source of stupidity.

CPE devices are buggy for IPv4 as well.  Bugs in CPE devices are
only found and fixed if the code paths are exercised.


Not my job. v4 works, v6 does not. I am a provider not a developer.

That said IPv6 worked fine for me with the shipped image (old version
of OpenWRT) using 6to4 before I reflashed it to a modern version
of OpenWRT as I wanted to use the HE tunnel rather than 6to4.  I
know that is only one CPE device.
 And will you be providing all of my end users with replacement CPE 
that meets all of the other requirements too? No? Because no such 
devices exist yet? OHHH yeah thats right, I'm a provider not a 
developer, so again, not a solution for my business.



Thirdly, some parts of my network are
wireless, and multicast is a huge, huge problem on wireless (the 802.11
varities anyways). The forwarding rates for multicast are sickeningly
low for many brand of gear - yes, it's at the bottom of the barrel no
matter how good or hot your signal is - and I honestly expect v6 to
experience enough disruption over wireless as to render it unusable for
exactly this reason alone.

You expect but haven't tested.


Based on observation and experience, I think v6 will wipe out the 802.11 
portion of my network and no, Im not going to 'test' it, recovery would 
be near impossible and in any event I don't experiment with paying 
customers. I won't move until the underlaying issues are resolved, and 
that means fixing multicast in wireless, which won't be done by me again 
because, you guessed it, I am a provider and not a developer.

The wired portion of my subscriber network is only slightly better, im
pretty sure it can deal with v6 in the middle, but the question is still
wether specfic CPE models can and which set of bugs I'll hit on my
access concentrators passing our v6 over PPPoE. I just read about a
cisco bug where enabling rp-filtering on v6 causes a router reload,
which I would hit immediately since rp-filtering is a standard
subscriber profile option here (trying to be a good netizen). How many
other network destroying bugs await? The longer I wait on v6, the less
work I will have to do dealing with bugs. So, as the original posted
said, we'll do v6 when it's easy, when we have time, and when the
economics make sense.

And is there a fix available yet?  All code has bugs in it.  They
exist in both the IPv4 code paths and the IPv6 code paths.  There
are lots of places that are going IPv6 only internally and only
having IPv4 at the fringe.  You can't do that if routers are flakey
when pushing IPv6 packets.  This is basically just fear overriding
rational decisions.


I am a provider and not a developer, and I am likely only going to use 
what I know works and what is within my sphere of control and influence. 
The flakey crappy state of v6 today means I am not putting it out 
anywhere a customer would have any exposure to it. I don't play games 
with my customers that way.

Re: /27 the new /24

[Mark Andrews]

In message <561699f3.1070...@tiedyenetworks.com>, Mike writes:
> On 10/08/2015 06:14 AM, Matthew Kaufman wrote:
> >
> >
> > On 10/7/15 7:00 AM, Mark Andrews wrote:
> >> I don't see anyone wishing it went differnetly. I see someone 
> >> pointing out the reality that lots of ISP's are way too late to 
> >> delivering IPv6. *Every* ISP should have been planning to deliver 
> >> IPv6 by the time the first RIR ran out of IPv4 addresses. 
> >
> > Look, I'm as much a supporter of delivering IPv6 as anyone. I've had 
> > IPv6 enabled on my home network (and the small data center I run in my 
> > garage) for over a decade now. In 2004, I made sure that IPv6 was 
> > fully supported in the peer-to-peer stack I developed and that 
> > eventually became RFC 7016. And for the last 5 years I've been pushing 
> > for IPv6 support in the product I work on for my employer.
> >
> > But the reality is that there's a whole lot of small and medium-sized 
> > ISPs run by fine, upstanding individuals serving their communities -- 
> > even in and around the San Francisco Bay Area -- that have either no 
> > or very limited (tunnels only) support for IPv6. That's the reality of 
> > the transition. And threatening these folks with the attorney general 
> > isn't the way to get them to adopt IPv6, nor is shaming them. They 
> > will add IPv6 support when it is easy to do, when their staff has the 
> > time, and when the economics make sense.
> >
> 
> 
> Plus one to that. We are such a provider, and IPv6 is on my list of 
> things to implement, but the barriers are still plenty high. Firstly, I 
> do have an Ipv6 assignmnt and bgp (v4) and an asn, but until I can get 
> IPv6 transit,

There are lots of transit providers that provide IPv6.  It really is
time to name and shame transit providers that don't provide IPv6.

> there is not much point in my putting a lot of effort into 
> enabling IPv6 for my subscribers. Yes I have a HE tunnel and yes it's 
> working, but it's not the same as running native v6 and with my own 
> address space. Second, on the group of servers that have v6 thru the HE 
> tunnel, I still run into problems all the time where some operations 
> over v6 simply fail inexplictly, requireing me to turn off v6 on that 
> host so whatever it is I'm doing can proceed over v4.

> Stuff like OS updates for example.

Then complain to the OS vendor.  It is most probably someone breaking
PMTU discover by filtering PTB.  Going native will hide these
problems until the MTU between the DC and the rest of the net
increases.  You could also just lower the advertised MTU internally
to match the tunnel MTU which would let you simulate better what a
native experience would be.

I can't remember the last time I saw a site stall due to reaching
it over IPv6 it is that long ago.

> Damm maddening. Can't imagine the screaming I'll 
> hear if a home user ever ran into similar so I am quite gun shy about 
> the prospect. Secondly, the the dodgy nature of the CPE connected to our 
> network and the terminally buggy fw they all run is sure to be a never 
> ending source of stupidity.

CPE devices are buggy for IPv4 as well.  Bugs in CPE devices are
only found and fixed if the code paths are exercised.

That said IPv6 worked fine for me with the shipped image (old version
of OpenWRT) using 6to4 before I reflashed it to a modern version
of OpenWRT as I wanted to use the HE tunnel rather than 6to4.  I
know that is only one CPE device.

> Thirdly, some parts of my network are 
> wireless, and multicast is a huge, huge problem on wireless (the 802.11 
> varities anyways). The forwarding rates for multicast are sickeningly 
> low for many brand of gear - yes, it's at the bottom of the barrel no 
> matter how good or hot your signal is - and I honestly expect v6 to 
> experience enough disruption over wireless as to render it unusable for 
> exactly this reason alone.

You expect but haven't tested.

> The wired portion of my subscriber network is only slightly better, im 
> pretty sure it can deal with v6 in the middle, but the question is still 
> wether specfic CPE models can and which set of bugs I'll hit on my 
> access concentrators passing our v6 over PPPoE. I just read about a 
> cisco bug where enabling rp-filtering on v6 causes a router reload, 
> which I would hit immediately since rp-filtering is a standard 
> subscriber profile option here (trying to be a good netizen). How many 
> other network destroying bugs await? The longer I wait on v6, the less 
> work I will have to do dealing with bugs. So, as the original posted 
> said, we'll do v6 when it's easy, when we have time, and when the 
> economics make sense.

And is there a fix available yet?  All code has bugs in it.  They
exist in both the IPv4 code paths and the IPv6 code paths.  There
are lots of places that are going IPv6 only internally and only
having IPv4 at the fringe.  You can't do that if routers are flakey
when pushing IPv6 packets.  This is basically just fear 

Re: /27 the new /24

[Jason Baugher]

This thread, while originally interesting and helpful, seems to have
degraded to a contest to see who can be the most arrogant, condescending
and insulting. Congrats.
On Oct 8, 2015 6:25 PM, "James Jun"  wrote:

> On Thu, Oct 08, 2015 at 03:45:38PM -0700, Mike wrote:
> >
> > NO, THERE IS NOT. We operate in rural and underserved areas and WE DO
> > NOT HAVE realistic choices. Can you see me from your ivory tower?
>
> Who is your upstream provider?
>
> I think you're confused on how the IP transit industry works.
>
> If you want choices in your transit providers, you should get a transport
> circuit (dark, wave or EPL) to a nearby carrier hotel/data center.  Once
> you do that, you will suddenly find that virtually almost everyone in the
> competitive IP transit market will provide you with dual-stacked IPv4/IPv6
> service.
>
> If you are buying DIA circuit from some $isp to your rural location that
> you call "head-end" and are expecting to receive a competitive service,
> and support for IPv6, well, then your expectations are either unreasonable,
> ignorant or both.
>
> Best,
> James
>

Re: /27 the new /24

[Jeremy Austin]

On Thu, Oct 8, 2015 at 3:25 PM, James Jun  wrote:

>
> If you want choices in your transit providers, you should get a transport
> circuit (dark, wave or EPL) to a nearby carrier hotel/data center.  Once
> you do that, you will suddenly find that virtually almost everyone in the
> competitive IP transit market will provide you with dual-stacked IPv4/IPv6
> service.
>

The future is here, but it isn't evenly distributed yet. I'm in North
America, but there are no IXPs in my *state*, let alone in my *continent*
-- from an undersea fiber perspective. There is no truly competitive IP
transit market within Alaska that I am aware of. Would love to be proved
wrong. Heck, GCI and ACS (the two providers with such fiber) only directly
peered a handful of years ago.


> If you are buying DIA circuit from some $isp to your rural location that
> you call "head-end" and are expecting to receive a competitive service,
> and support for IPv6, well, then your expectations are either unreasonable,
> ignorant or both.
>

Interestingly both statewide providers *do* provide both IPv4 and IPv6
peering. The trick is to find a spot where there's true price competition.
The 3 largest statewide ISPs have fiber that meets a mere three city blocks
from one of my POPs, but there's no allowable IX. I'm looking at you, AT

-- 
Jeremy Austin
Whitestone Power & Communications, Alaska

Re: /27 the new /24

[Ricky Beam]

On Thu, 08 Oct 2015 18:45:38 -0400, Mike   
wrote:

WE DO NOT HAVE realistic choices.


Or, apparently, realistic expectations.

You, do, indeed, deserve public shaming for your complete lack of  
willingness to support IPv6. Your customers have no "realistic choices"  
either. How many other ISPs do they have to choice from? If you cannot be  
bothered to support IPv6, how are they supposed to? ("use a tunnel broker"  
is the *WRONG* answer)


You are an ISP. You don't get to say "NO!" to IPv6. It is what the global  
internet is moving towards. You _WILL_ support it, or you will be left  
behind, and your customers who have little or no other options will suffer  
for it.


https://youtu.be/g1GF4Gnb-D0

I can't remember the last time I saw a site stall due to reaching it  
over IPv6 it is that long ago.


It happens every day for me, which only amplifies my perception that v6  
IS NOT READY FOR PRIME TIME.


This is just *your* flawed perception. Have you bothered to be an engineer  
and figure out _WHY_ it doesn't work? Or do you like keeping your head in  
the sand mumbling "it's not my job"?



Thirdly, some parts of my network are
wireless, and multicast ...

Based on observation and experience, I think v6 will wipe out the 802.11


If you are providing customer access via 802.11 technology, then yes, you  
do have a serious problem. But it's a problem you already have with v4 as  
well... or do you block _ALL_ broadcast traffic from your 802.11 network?  
Have you checked those networks, because I'm pretty sure there's multicast  
on them already. (windows and mac generate multicast by default)


My experience shows multicast is a problem for 99% of WiFi gear. It's  
handled like all other broadcast and sent at "basic rate" to all stations,  
which is going to be slow as crap. However, IPv6 ND (and RAs) do not  
amount to a volume that will, under normal conditions, kill a WiFi  
network. I run IPv6 over my 802.11a/b/g/n networks; no one has even  
noticed! (even with Truly Ancient Hardware(tm))


--Ricky

Re: /27 the new /24

[Jon Lewis]

On Fri, 9 Oct 2015, Mark Andrews wrote:


Plus one to that. We are such a provider, and IPv6 is on my list of
things to implement, but the barriers are still plenty high. Firstly, I
do have an Ipv6 assignmnt and bgp (v4) and an asn, but until I can get
IPv6 transit,


There are lots of transit providers that provide IPv6.  It really is
time to name and shame transit providers that don't provide IPv6.


Unless he's buying from Bob's Bait, Tackle, and Internet (who's reselling 
service off his Brighthouse cable modem connection), I find it hard to 
believe there are "transit providers" in the NANOG region who still cannot 
provide dual-stack addressing and BGP for DIA.



there is not much point in my putting a lot of effort into
enabling IPv6 for my subscribers. Yes I have a HE tunnel and yes it's


With some OS's (Apple) preferring v6 if it's there, it would actually be a 
bad idea to enable IPv6 for your subscribers before you have 
stable/reliable v6 connectivity hooked up for the network.



address space. Second, on the group of servers that have v6 thru the HE
tunnel, I still run into problems all the time where some operations
over v6 simply fail inexplictly, requireing me to turn off v6 on that
host so whatever it is I'm doing can proceed over v4.


v6 routing doesn't always get the same level of scrutiny as v4.  i.e. 
Suboptimal v6 paths might get used for some time before someone with 
enough clue to notice speaks up.  Presumably, that will change as v6 
adoption gets more widespread.


--
 Jon Lewis, MCP :)   |  I route
 |  therefore you are
_ http://www.lewis.org/~jlewis/pgp for PGP public key_

Re: /27 the new /24

[Stephen Satchell]

On 10/08/2015 05:50 PM, Ricky Beam wrote:

You are an ISP. You don't get to say "NO!" to IPv6. It is what the
global internet is moving towards. You _WILL_ support it, or you will be
left behind, and your customers who have little or no other options will
suffer for it.


ISP == "Internet Service Provider".  The key word here is "service". 
tiedyenetworks.com is a provider of services to customers, and I suspect 
those are retail customers.  What he just told you is that the service 
he provides, in his experience, does not play well with IPv6 AS 
CURRENTLY IMPLEMENTED IN AVAILABLE EQUIPMENT.  On the one hand, IPv6 is 
"the future" (I just invested a fair amount of cred to get the books 
recommended to me here on NANOG to get up to speed) but like early 
versions of just about every thing and every product, there are still a 
few potholes.


tiedyenetworks.com, from my reading of this thread, has elected to limit 
his service offerings to his customers that he can reasonably support. 
That's good, solid business sense.  Nothing is worse than providing a 
product that does not work as expected or advertised.  VW, anyone?



(windows and mac generate multicast by default)


And unless there is a damn good need for that multicast traffic, it gets 
blocked.  From my edge network, I block multicasts and broadcasts both 
inbound and outbound.  When I was network admin for the web hosting 
company I worked for, I also blocked a number of ports at my edge, ports 
that had no business being used in the general case.  I had *one* 
customer that needed to come in using 3309; I punched a hole in the ACLs 
for that one customer, and damn carefully.



This is just *your* flawed perception. Have you bothered to be an
engineer and figure out _WHY_ it doesn't work?


Maybe you missed his earlier declaration:  "I'm a provider, not a 
developer."  He expects the equipment to work.  It doesn't.  Did he ask 
his vendor?  I don't know, but my personal experience with 
wireless-equipment vendors is not encouraging.  Some people don't have 
the money, resources, or time to winkle out all the wrinkles, so they go 
with what works in their situation.  Consider the rural market:  damn 
few customers, so $150K engineers are out of the question.



I run IPv6 over my 802.11a/b/g/n networks; no one has even noticed!
(even with Truly Ancient Hardware(tm))


That's your experience.  He has a different experience.  I suspect your 
customer base is considerably more dense than tiedyenetwork.com's base. 
 Did you say you are primarily a rural provider?  Mike did.  Your 
earlier traffic suggests your base of operations is more in a city or 
suburban environment.  Apples and oranges, if true.

Re: /27 the new /24

[Owen DeLong]

> On Oct 8, 2015, at 3:45 PM, Mike  wrote:
> 
> 
> 
> On 10/08/2015 02:41 PM, Mark Andrews wrote:
>> 
>> 
>> Plus one to that. We are such a provider, and IPv6 is on my list of
>> things to implement, but the barriers are still plenty high. Firstly, I
>> do have an Ipv6 assignmnt and bgp (v4) and an asn, but until I can get
>> IPv6 transit,
>> 
>> There are lots of transit providers that provide IPv6.  It really is
>> time to name and shame transit providers that don't provide IPv6.
> 
> NO, THERE IS NOT. We operate in rural and underserved areas and WE DO NOT 
> HAVE realistic choices. Can you see me from your ivory tower?

Um… 

There ARE LOTS of transit providers that provide IPv6. It may be true that none 
of them serve your locality or overlap locations where you have presence, but 
that does not mean that they do not exist.

> 
>>> there is not much point in my putting a lot of effort into
>>> enabling IPv6 for my subscribers. Yes I have a HE tunnel and yes it's
>>> working, but it's not the same as running native v6 and with my own
>>> address space. Second, on the group of servers that have v6 thru the HE
>>> tunnel, I still run into problems all the time where some operations
>>> over v6 simply fail inexplictly, requireing me to turn off v6 on that
>>> host so whatever it is I'm doing can proceed over v4.
>>> Stuff like OS updates for example.
>> Then complain to the OS vendor.  It is most probably someone breaking
>> PMTU discover by filtering PTB.  Going native will hide these
>> problems until the MTU between the DC and the rest of the net
>> increases.  You could also just lower the advertised MTU internally
>> to match the tunnel MTU which would let you simulate better what a
>> native experience would be.
> Not my job. v4 works, v6 does not, end of story.

Hmmm… Let’s see if you can still say that in a few years.

>> I can't remember the last time I saw a site stall due to reaching it over 
>> IPv6 it is that long ago.
> 
> It happens every day for me, which only amplifies my perception that v6 IS 
> NOT READY FOR PRIME TIME.

Yet you refuse to troubleshoot your issues with it that are not shared by 
others and blame the protocol for whatever is probably wrong with your own 
network. Interesting tactic.

Best of luck with that as your network gradually becomes an IPv4 island no 
longer connected to the majority of the internet.

Owen

Re: /27 the new /24

[Mark Andrews]

In message <56172237.5030...@satchell.net>, Stephen Satchell writes:
> On 10/08/2015 05:50 PM, Ricky Beam wrote:
> > You are an ISP. You don't get to say "NO!" to IPv6. It is what the
> > global internet is moving towards. You _WILL_ support it, or you will be
> > left behind, and your customers who have little or no other options will
> > suffer for it.
> 
> ISP == "Internet Service Provider".  The key word here is "service". 
> tiedyenetworks.com is a provider of services to customers, and I suspect 
> those are retail customers.  What he just told you is that the service 
> he provides, in his experience, does not play well with IPv6 AS 
> CURRENTLY IMPLEMENTED IN AVAILABLE EQUIPMENT.  On the one hand, IPv6 is 
> "the future" (I just invested a fair amount of cred to get the books 
> recommended to me here on NANOG to get up to speed) but like early 
> versions of just about every thing and every product, there are still a 
> few potholes.
> 
> tiedyenetworks.com, from my reading of this thread, has elected to limit 
> his service offerings to his customers that he can reasonably support. 
> That's good, solid business sense.  Nothing is worse than providing a 
> product that does not work as expected or advertised.  VW, anyone?
> 
> > (windows and mac generate multicast by default)
> 
> And unless there is a damn good need for that multicast traffic, it gets 
> blocked.  From my edge network, I block multicasts and broadcasts both 
> inbound and outbound.  When I was network admin for the web hosting 
> company I worked for, I also blocked a number of ports at my edge, ports 
> that had no business being used in the general case.  I had *one* 
> customer that needed to come in using 3309; I punched a hole in the ACLs 
> for that one customer, and damn carefully.
> 
> > This is just *your* flawed perception. Have you bothered to be an
> > engineer and figure out _WHY_ it doesn't work?
> 
> Maybe you missed his earlier declaration:  "I'm a provider, not a 
> developer."  He expects the equipment to work.  It doesn't.  Did he ask 
> his vendor?  I don't know, but my personal experience with 
> wireless-equipment vendors is not encouraging.  Some people don't have 
> the money, resources, or time to winkle out all the wrinkles, so they go 
> with what works in their situation.  Consider the rural market:  damn 
> few customers, so $150K engineers are out of the question.

I also saw that he was using a tunnel yet was unwilling to configure
the local network to account for this when testing yet was willing
to bag IPv6 due to the side effects of being behind a tunnel.

IPv4 also works poorly when you introduce a tunnel and the people
you connect to are idiots that block / don't handle PTB messages.

Do like for like testing before bagging the protocol.

20% of the US eyeballs have working native IPv6 without lots of
complaints.  If you are have problems over a tunnel and they aren't
you may want to re-evalute your opinion of IPv6 and look to getting
native connections.

IPv6 really does work as well as IPv4 give like for like connections.

Mark

> > I run IPv6 over my 802.11a/b/g/n networks; no one has even noticed!
> > (even with Truly Ancient Hardware(tm))
> 
> That's your experience.  He has a different experience.  I suspect your 
> customer base is considerably more dense than tiedyenetwork.com's base. 
>   Did you say you are primarily a rural provider?  Mike did.  Your 
> earlier traffic suggests your base of operations is more in a city or 
> suburban environment.  Apples and oranges, if true.
-- 
Mark Andrews, ISC
1 Seymour St., Dundas Valley, NSW 2117, Australia
PHONE: +61 2 9871 4742 INTERNET: ma...@isc.org

Re: /27 the new /24

[Mark Andrews]

In message <520ce953-012c-4599-a85b-69517e090...@matthew.at>, Matthew Kaufman w
rites:
>>
>>
>> On Oct 7, 2015, at 7:00 AM, Mark Andrews  wrote:
>>
>>
>> In message , Matthew
>> Kaufman w
>> rites:
>>>
>>>
 On Oct 7, 2015, at 5:01 AM, Owen DeLong  wrote:



 Instead, the followup question is needed…
 “That’s great, but how does that help me reach a web site that
 doesn’t have and can’t get an IPv4 address?”

 Owen

>>>
>>> At the present time, a web site that doesn't have and can't get an
>>> IPv4 address isn't "on the Internet".
>>
>> It's on the Internet.  ISP's that fail to supply IPv6 at this point
>> in time are committing fraud if they claim to supply Internet
>> connection.
>
>Good luck prosecuting them for that.

I don't have to.  I'm sure some AG will do so soon enough. 

>Along with all the internal IT departments that are failing to deliver v6
>to wifi and desktops.

What does a companies decision about whether to use IPv6 internally
have to do with the matter?  It's not fraud to not use IPv6.  It
is fraud to claim that you supply "the Internet" and don't supply
IPv6.

>>> That may change in the future, but right now this is the web site's
>>> fault, not your ISP's.
>>
>> No, it isn't the site's fault.  The internet ran out of IPv4 addresses
>> years ago.  Not everyone can get a public adddress.
>>
>Right. Now it is only people who can afford about $8 one time. (The going
>rate for IPv4 on the transfer market at modest block sizes)
>
>> There are
>> millions of customers without a public IPv4 address that can host
>> a site because they are behind a CGN which is only needed because
>> of the short sightedness of lots of ISPs failing to deliver IPv6
>> to their customers.
>>
>I think you meant cannot.

Yes, "cannot host a site".

>Most consumer ISPs also prevent this as a matter of policy. Good luck
>getting those policies changed.

Actually most don't.  Remember a "site" can be the remote controls for
equipement at home.

>>> Wishing that the IPv6 transition had gone differently does not change
>>> reality.
>>
>> I don't see anyone wishing it went differnetly.  I see someone
>> pointing out the reality that lots of ISP's are way too late to
>> delivering IPv6.
>>
>Sure, they're too late. Which is why, until there's more progress, a
>website not reachable over IPv4 is fairly useless if the goal is to serve
>"most of the users on the Internet"
>>
>>
>> *Every* ISP should have been planning to deliver IPv6 by the time
>> the first RIR ran out of IPv4 addresses.  That would have been
>> just-in-time engineering.  It's not like they didn't have over a
>> decade to plan to do it,  It's not like there wern't reasonable
>> accurate forcecasts for when that would happen.
>>
>Yeah, totally agree. Didn't happen. Still hasn't happened. Won't happen
>tomorrow.

Which still doesn't make it the fault of the site that can only get
IPv6.

>> It was not hard to see what would happen if you didn't deliver IPv6
>> before the first RIR ran out.
>>
>> No instead most of then stuck their heads in the sand and said "we
>> have enough IPv4 addresses" without looking at whom they need to
>> connect with.
>>
>Last I checked, things are still working out just fine for all of them.
>Despite the obvious concerns about the future.

Usually only because they abuse their oligopoly position.  The FCC
(US) / ACCC (AUS) etc. should be prosecuting ISP's that don't supply
IPv6 at this point in time.

>Matthew Kaufman
>>
>(Sent from my iPhone)
>
-- 
Mark Andrews, ISC
1 Seymour St., Dundas Valley, NSW 2117, Australia
PHONE: +61 2 9871 4742 INTERNET: ma...@isc.org

Re: /27 the new /24

[Ray Soucy]

Here is a quick starting point for filtering IPv6 on a Linux host system if
you don't feel comfortable opening up all ICMPv6 traffic:

http://soucy.org/tmp/v6firewall/ip6tables.txt

I haven't really re-visited it in a while, so if I'm forgetting something
let me know.

On Wed, Oct 7, 2015 at 9:13 AM, Stephen Satchell  wrote:

> This is excellent feedback, thank you.
>
> On 10/07/2015 04:54 AM, Owen DeLong wrote:
>
>>
>> On Oct 4, 2015, at 7:49 AM, Stephen Satchell  wrote:
>>>
>>> My bookshelf is full of books describing IPv4. Saying "IPv6 just
>>> works" ignores the issues of configuring intelligent firewalls to block
>>> the ne-er-do-wells using the new IP-level protocol.
>>>
>>
>> You will need most of the same blockages in IPv6 that you needed in IPv4,
>> actually.
>>
>> There are some important differences for ICMP (don’t break PMTU-D or
>> ND), but otherwise, really not much difference between your IPv4
>> security policy and your IPv6 security policy.
>>
>> In fact, on my linux box, I generate my IPv4 iptables file using
>> little more than a global search and replace on the IPv6 iptables
>> configuration which replaces the IPv6 prefixes/addresses with the
>> corresponding IPv4 prefixes/addresses. (My IPv6 addresses for things
>> that take incoming connections have an algorithmic map to IPv4 addresses
>> for things that have them.)
>>
>
> On my box, I have a librry of shell functions that do the generation,
> driven by parameter tables.  If I'm reading you correctly, I can just
> augment the parameter tables and those functions to generate the
> appropriate corresponding ip6table commands in parallel with the iptable
> commands.
>
> Question: should I still rate-limit ICMP packets in IPv6?  Also, someone
> on this list pointed me to NIST SP800-119, "Guidelines for the Secure
> Deployment of IPv6", the contents of which which I will incorporate.
>
> There is limited IPv6 support in many of the GUIs still,
>> unfortunately, but the command line tools are all there and for the
>> most part work pretty much identically for v4 and v6, the difference
>> often being as little as ping vs ping6 or   vs.
>>  -6 .
>>
>
> I've not been happy with the GUIs, because getting them to do what I want
> is a royal pain.  For example, I'm forced to use port-based redirection in
> one edge firewall application -- I blew a whole weekend figuring out how to
> do that with the CentOS 7 firewalld corkscrew, for a customer who outgrew
> the RV-220 he used for the application.  At least that didn't need IPv6!
>
> Primarily it involves changing the IPv4 addresses and/or prefixes
>> into IPv6 addresses and/or prefixes.
>>
>
> What about fragmented packets?  And adjusting the parameters in ip6table
> filters to detect the DNS "ANY" requests used in the DDoS amplification
> attacks?
>
> I'm not asking NANOG to go past its charter, but I am asking the
>>> IPv6fanatics on this mailing list to recognize that, even though the net
>>> itself may be running IPv6, the support and education infrastructure is
>>> still behind the curve. Reading RFCs is good, reading man pages is good,
>>> but there is no guidance about how to implement end-network policies in
>>> the wild yet...at least not that I've been able to find.
>>>
>>
>> There is actually quite a bit of information out there. Sylvia
>> Hagen’sIPv6 book covers a lot of this (O’Reilly publishes it).
>>
>
> Um, that would be "books".  Which one do you recommend I start with?
>
> * IPv6 Essentials (3rd Edition), 2014, ASIN: B00RWSNEKG
> * Planning for IPv6 (1st Edition), 2011,  ISBN-10: 1449305393
>
> (I would assume the first, as the NIST document probably covers the
> contents of the second)
>
> There are also several other good IPv6 books.
>>
>
> Recommendations?
>
> "ipv6.disable" will be changed to zero when I know how to set the
>>> firewall to implement the policies I need to keep other edge networks
>>> from disrupting mine.
>>>
>>
>> You do. You just don’t realize that you do. See above.
>>
>
> That's encouraging.  Being able to leverage the knowledge from IPv4 to
> project the same policies into IPv6 makes it easier for me, as I'm already
> using programmatic methods of generating the firewalls.  I expected that
> the implementation of existing applications-level policies would be
> parallel; it's the policies further down the stack that was my concern.
>
> Also, I have a lot of IP level blocks (like simpler Cisco access control
> lists) to shut out those people who like to bang on my SSH front door. I
> believe that people who are so rude as to try to break through dozens or
> hundreds of time a day will have other bad habits, and don't deserve to be
> allowed for anything.  (I have similar blocks for rabid spammers not in the
> DNSBLs, but that's a different story.)  I would expect to maintain a
> separate list of IPv6 subnets, based on experience.
>
> Which brings up another question:  should I block IPv6 access to port 25
> on my 

Re: /27 the new /24

[Matthew Kaufman]

> On Oct 7, 2015, at 4:15 PM, Mark Andrews  wrote:
> 
> 
> I don't have to.  I'm sure some AG will do so soon enough. 

There's always an optimist around.

Good luck with that.

Matthew Kaufman

(Sent from my iPhone)

Re: /27 the new /24

[Mel Beckman]

We know. I recommend you read the whole thread before reacting.  

 -mel beckman

> On Oct 7, 2015, at 4:56 AM, Owen DeLong  wrote:
> 
> 
>> On Oct 4, 2015, at 7:52 AM, Mel Beckman  wrote:
>> 
>> If it doesn't support IPSec, it's not really IPv6. Just as if it failed to 
>> support any other mandatory IPv6 specification, such as RA.
> 
> Not true. IPSec is recommended, not mandatory.
> 
> This change was made in favor of resource constrained nodes (think micro 
> controllers with very small memories).
> 
>> There's really no excuse for not supporting IPSec, as it's a widely 
>> available open source component that costs nothing to incorporate into an 
>> IPv6 stack.
> 
> Simply not true. There are nodes which have no need for it and are resource 
> constrained.
> 
>> Your observation simply means that users must be informed when buying IPv6 
>> devices, just as they must with any product. You can buy either genuine IPv6 
>> or half-baked IPv6 products. When I speak of IPv6, I speak only of the 
>> genuine article.
> 
> This is true. If you need the device to support IPv6, you should definitely 
> make sure that it does, but that is ordinary reality with any feature of any 
> product rather than anything specific to IPv6.
> 
> Owen
> 

Re: /27 the new /24

[Owen DeLong]

> On Oct 4, 2015, at 7:49 AM, Stephen Satchell  wrote:
> 
> On 10/04/2015 06:40 AM, Matthias Leisi wrote:
>> Fully agree. But the current state of IPv6 outside "professional“
>> networks/devices is sincerely limited by a lot of poor CPE and
>> consumer device implementations.
> 
> I have to ask:  where is the book _IPv6 for Dummies_ or equivalent?
> 
> Specifically, is http://www.xnetworks.es/contents/Infoblox/IPv6forDummies.pdf 
> any good? (I just downloaded it to inspect.)
> 
> My bookshelf is full of books describing IPv4.  Saying "IPv6 just works" 
> ignores the issues of configuring intelligent firewalls to block the 
> ne-er-do-wells using the new IP-level protocol.

You will need most of the same blockages in IPv6 that you needed in IPv4, 
actually.

There are some important differences for ICMP (don’t break PMTU-D or ND), but 
otherwise, really not much difference between your IPv4 security policy and 
your IPv6 security policy.

In fact, on my linux box, I generate my IPv4 iptables file using little more 
than a global search and replace on the IPv6 iptables configuration which 
replaces the IPv6 prefixes/addresses with the corresponding IPv4 
prefixes/addresses. (My IPv6 addresses for things that take incoming 
connections have an algorithmic map to IPv4 addresses for things that have 
them.)

> I use CentOS, the community version of Red Hat Enterprise.  I looked around 
> for useful books on building IPv6 firewalls with the same granularity as the 
> above-mentioned book for IPv4, and haven't found anything useful as yet.  In 
> particular, I'm looking for material that lays out how to build a 
> mostly-closed firewall and DMZ in IPv6.  The lack of IPv6 support goes 
> further:  I didn't find anything useful in Red Hat (CentOS) firewall tools 
> that provides IPv6 support...but that's probably because I don't know what 
> I'm looking for.  (Also, that GUI software is intended for use on individual 
> servers/computers, not in a edge-firewall with forwarding and DMZ 
> responsibilities.)

Where you have an iptables file, you add an ip6tables file and change the 
prefixes and addresses. Otherwise, it’s really pretty much the same.

There is limited IPv6 support in many of the GUIs still, unfortunately, but the 
command line tools are all there and for the most part work pretty much 
identically for v4 and v6, the difference often being as little as ping vs 
ping6 or   vs.  -6 .

> Building a secure firewall takes more than just knowing how to issue ip6table 
> commands; one also needs to know exactly what goes into those commands.  
> NANOG concentrates on network operators who need to provide a good Internet 
> experience to all their downstream customers, which is why I see the bias 
> toward openness...as it should be.  Those of us who run edge networks have 
> different problems to solve.

If you know what goes into the iptables commands, then there’s very little 
difference for the ip6tables commands.

Primarily it involves changing the IPv4 addresses and/or prefixes into IPv6 
addresses and/or prefixes. The rest of the commands are very much literally the 
same… An example from my actual configurations:

iptables:
-A RH-Firewall-1-INPUT -m state --state ESTABLISHED,RELATED -j ACCEPT
-A RH-Firewall-1-INPUT -m state --state NEW -m tcp -p tcp --dport 22 -j ACCEPT
-A RH-Firewall-1-INPUT -s 192.159.10.0/24 -m state --state NEW -m tcp -p tcp 
--dport 53 -j ACCEPT
-A RH-Firewall-1-INPUT -m state --state NEW -m udp -p udp --dport 53 -m limit 
--limit 30/minute --limit-burst 90 -j ACCEPT
-A RH-Firewall-1-INPUT -m state --state NEW -m tcp -p tcp --dport 53 -m limit 
--limit 30/minute --limit-burst 90 -j ACCEPT
-A RH-Firewall-1-INPUT -m state --state NEW -m tcp -p tcp --dport 5900 -j ACCEPT

ip6tables:
-A RH-Firewall-1-INPUT -m state --state ESTABLISHED,RELATED -j ACCEPT
-A RH-Firewall-1-INPUT -m state --state NEW -m tcp -p tcp --dport 22 -j ACCEPT
-A RH-Firewall-1-INPUT -s 2001:470:1f00:3142::/64 -m state --state NEW -m udp 
-p udp --dport 53 -j ACCEPT
-A RH-Firewall-1-INPUT -m state --state NEW -m udp -p udp --dport 53 -m limit 
--limit 30/minute --limit-burst 90 -j ACCEPT
-A RH-Firewall-1-INPUT -m state --state NEW -m tcp -p tcp --dport 53 -m limit 
--limit 30/minute --limit-burst 90 -j ACCEPT
-A RH-Firewall-1-INPUT -m state --state NEW -m tcp -p tcp --dport 5900 -j ACCEPT


This is not my entire configuration (which is somewhat complex and in need of 
some cruft removal due to organic growth over time), but these 6 lines do 
provide a reasonably representative sample of things and include rate-limiting 
DNS queries from outsiders.

> I'm not asking NANOG to go past its charter, but I am asking the IPv6 
> fanatics on this mailing list to recognize that, even though the net itself 
> may be running IPv6, the support and education infrastructure is still behind 
> the curve.  Reading RFCs is good, reading man pages is good, but there is no 
> guidance about how to implement 

Re: /27 the new /24

[Owen DeLong]

> On Oct 4, 2015, at 8:33 AM, Jon Lewis  wrote:
> 
> On Sun, 4 Oct 2015, Mel Beckman wrote:
> 
>> If it doesn't support IPSec, it's not really IPv6. Just as if it failed to 
>> support any other mandatory IPv6 specification, such as RA.
> 
> Go tell cisco that.  IIRC, the first network I dual-stacked, I was kind of 
> surprised when I found I could not use authentication in OSPFv3, because 
> OSPFv3 assumes IPv6 will supply the IPSec to do auth...but these routers 
> didn't support IPSec.  They still managed to route IPv6 and support IPv6 
> customers...so it really was IPv6...just not the full suite of everything 
> you'd expect from IPv6.

A router with OSPFv3 and no IPSec for securing the OSPFv3 sessions really is an 
incomplete implementation.

This is one case where IPSec really should be considered mandatory rather than 
recommended.

> 
>> Your observation simply means that users must be informed when buying IPv6 
>> devices, just as they must with any product. You can buy either genuine IPv6 
>> or half-baked IPv6 products. When I speak of IPv6, I speak only of the 
>> genuine article.
> 
> Does anyone buy "IPv6 devices”?

Yes… For some definitions of that term.

> The biggest hurdle I've seen with IPv6 adoption (i.e. going dual-stack, with 
> the idea that we'll gradually transition most things / most traffic from v4 
> to v6) is the number of end-user network providers that don't offer v6 at 
> all.  My home cable internet provider still doesn't offer IPv6.
>  When I asked one of their support people about it recently, I was told not 
> to worry, they have plenty of v4 addresses left, but it was implied that they 
> do plan to start offering v6 sometime soon.  They should have started rolling 
> out IPv6 to any customers that wanted it years ago, so that by today, it 
> would be standard for all their installations to be dual-stack.  But here we 
> are, nearly 2016, and they don't have a single IPv6 customer (AFAIK) yet.

Yeah, lots of providers still don’t get it like that.

The problem is we’ve also done a poor job training people who call them up 
asking for IPv6.

Many accept “We have plenty of IPv4 addresses” as an answer.

Instead, the followup question is needed… “That’s great, but how does that help 
me reach a web site that doesn’t have and can’t get an IPv4 address?”

Owen

Re: /27 the new /24

[Owen DeLong]

> On Oct 4, 2015, at 7:52 AM, Mel Beckman  wrote:
> 
> If it doesn't support IPSec, it's not really IPv6. Just as if it failed to 
> support any other mandatory IPv6 specification, such as RA. 

Not true. IPSec is recommended, not mandatory.

This change was made in favor of resource constrained nodes (think micro 
controllers with very small memories).

> There's really no excuse for not supporting IPSec, as it's a widely available 
> open source component that costs nothing to incorporate into an IPv6 stack. 

Simply not true. There are nodes which have no need for it and are resource 
constrained.

> Your observation simply means that users must be informed when buying IPv6 
> devices, just as they must with any product. You can buy either genuine IPv6 
> or half-baked IPv6 products. When I speak of IPv6, I speak only of the 
> genuine article. 

This is true. If you need the device to support IPv6, you should definitely 
make sure that it does, but that is ordinary reality with any feature of any 
product rather than anything specific to IPv6.

Owen

Re: /27 the new /24

[Mark Andrews]

In message , Matthew Kaufman w
rites:
> 
> 
> > On Oct 7, 2015, at 5:01 AM, Owen DeLong  wrote:
> >=20
> >=20
> >=20
> > Instead, the followup question is needed=E2=80=A6 =E2=80=9CThat=E2=80=99s g
> =
> reat, but how does that help me reach a web site that doesn=E2=80=99t have a=
> nd can=E2=80=99t get an IPv4 address?=E2=80=9D
> >=20
> > Owen
> >=20
> 
> At the present time, a web site that doesn't have and can't get an IPv4 addr=
> ess isn't "on the Internet".

It's on the Internet.  ISP's that fail to supply IPv6 at this point
in time are committing fraud if they claim to supply Internet
connection.

> That may change in the future, but right now this is the web site's fault, n=
> ot your ISP's.

No, it isn't the site's fault.  The internet ran out of IPv4 addresses
years ago.  Not everyone can get a public adddress.  There are
millions of customers without a public IPv4 address that can host
a site because they are behind a CGN which is only needed because
of the short sightedness of lots of ISPs failing to deliver IPv6
to their customers.

> Wishing that the IPv6 transition had gone differently does not change
> reality.

I don't see anyone wishing it went differnetly.  I see someone
pointing out the reality that lots of ISP's are way too late to
delivering IPv6.

*Every* ISP should have been planning to deliver IPv6 by the time
the first RIR ran out of IPv4 addresses.  That would have been
just-in-time engineering.  It's not like they didn't have over a
decade to plan to do it,  It's not like there wern't reasonable
accurate forcecasts for when that would happen.

It was not hard to see what would happen if you didn't deliver IPv6
before the first RIR ran out.

No instead most of then stuck their heads in the sand and said "we
have enough IPv4 addresses" without looking at whom they need to
connect with.

Mark

> Matthew Kaufman
>
> (Sent from my iPhone)
-- 
Mark Andrews, ISC
1 Seymour St., Dundas Valley, NSW 2117, Australia
PHONE: +61 2 9871 4742 INTERNET: ma...@isc.org

Re: /27 the new /24

[Joe Abley]

On 7 Oct 2015, at 9:29, Matthew Kaufman wrote:


On Oct 7, 2015, at 5:01 AM, Owen DeLong  wrote:

Instead, the followup question is needed… “That’s great, but 
how does that help me reach a web site that doesn’t have and 
can’t get an IPv4 address?”


At the present time, a web site that doesn't have and can't get an 
IPv4 address isn't "on the Internet".


By the only definition of the Internet that matters, the function "is on 
the Internet" is very much in the eye of the beholder.


Using this definition, v6-only services are most certainly "on the 
Internet" for people who have v6 connectivity. Similarly, various 
instances of the Pirate Bay that have v4 reachability are not "on the 
Internet" for end-users in draconian jurisdictions like the UK. Trying 
to make assertions about what "on the Internet" means in a general, 
global sense is an effort doomed to failure.


I realise you're talking in pragmatic terms about services that have a 
general, dispersed, global end-user population, but not all services are 
like that.



Joe

[1] Internet, n: “the largest equivalence class in the reflexive 
transitive symmetric closure of the relationship ‘can be reached by an 
IP packet from’” (Seth Breidbart)

Re: /27 the new /24

[Owen DeLong]

Memory footprint is still an issue in lots of things like ESP8266 (which 
doesn’t yet support IPv6, but hopefully will soon).

Not everything is a cell phone or larger. There are lots of cool new things 
coming out in the SoC world where you’ve got a micro controller, GPIOs, CAN, 
SPI, WiFi, and more all in a single chip or module.

Another example (also currently IPv4 only, but hopefully that will get fixed) 
is particle.io.

These are $10-$20 (and sometimes even less) complete systems with very small 
memories and very low power consumption which are great for deploying things 
like remote sensors and the like.

Owen

> On Oct 4, 2015, at 11:15 AM, Mel Beckman  wrote:
> 
> Stefann,
> 
> You're right. I remember hearing rumblings of vendors requesting this change, 
> mostly because embedded processors of the time had difficulty performing well 
> with IPv6. I see that in 2011 rfc6434 lowered IPSec from "must" to "should". 
> Nevertheless, plenty of products produced before 2011 included IPSec and the 
> vast majority of IPv6-capable nodes on the Internet have it today. 
> Performance is no longer an issue. 
> 
> -mel beckman
> 
>> On Oct 4, 2015, at 8:58 AM, Sander Steffann  wrote:
>> 
>> Hi,
>> 
>>> Op 4 okt. 2015, om 16:52 heeft Mel Beckman  het volgende 
>>> geschreven:
>>> 
>>> If it doesn't support IPSec, it's not really IPv6. Just as if it failed to 
>>> support any other mandatory IPv6 specification, such as RA.
>> 
>> I think you're still looking at an old version of the IPv6 Node 
>> Requirements. Check https://tools.ietf.org/html/rfc6434#section-11, 
>> specifically this bit:
>> 
>> """
>> Previously, IPv6 mandated implementation of IPsec and recommended the key 
>> management approach of IKE.  This document updates that recommendation by 
>> making support of the IPsec Architecture a SHOULD for all IPv6 nodes.
>> """
>> 
>> This was published in December 2011.
>> 
>> Cheers,
>> Sander
>> 

Re: /27 the new /24

[Matthew Kaufman]

> On Oct 7, 2015, at 5:01 AM, Owen DeLong  wrote:
> 
> 
> 
> Instead, the followup question is needed… “That’s great, but how does that 
> help me reach a web site that doesn’t have and can’t get an IPv4 address?”
> 
> Owen
> 

At the present time, a web site that doesn't have and can't get an IPv4 address 
isn't "on the Internet".

That may change in the future, but right now this is the web site's fault, not 
your ISP's.

Wishing that the IPv6 transition had gone differently does not change reality.

Matthew Kaufman

(Sent from my iPhone)

Re: /27 the new /24

[Owen DeLong]

> On Oct 7, 2015, at 6:29 AM, Matthew Kaufman  wrote:
> 
> 
> 
>> On Oct 7, 2015, at 5:01 AM, Owen DeLong  wrote:
>> 
>> 
>> 
>> Instead, the followup question is needed… “That’s great, but how does that 
>> help me reach a web site that doesn’t have and can’t get an IPv4 address?”
>> 
>> Owen
>> 
> 
> At the present time, a web site that doesn't have and can't get an IPv4 
> address isn't "on the Internet".
> 
> That may change in the future, but right now this is the web site's fault, 
> not your ISP's.
> 
> Wishing that the IPv6 transition had gone differently does not change reality.
> 
> Matthew Kaufman
> 
> (Sent from my iPhone)

Swift boating aside, repeating the same assertion doesn’t make it true.

Owen

Re: /27 the new /24

[Stephen Satchell]

This is excellent feedback, thank you.

On 10/07/2015 04:54 AM, Owen DeLong wrote:



On Oct 4, 2015, at 7:49 AM, Stephen Satchell  wrote:

My bookshelf is full of books describing IPv4. Saying "IPv6 just
works" ignores the issues of configuring intelligent firewalls to block
the ne-er-do-wells using the new IP-level protocol.


You will need most of the same blockages in IPv6 that you needed in IPv4, 
actually.

There are some important differences for ICMP (don’t break PMTU-D or
ND), but otherwise, really not much difference between your IPv4
security policy and your IPv6 security policy.

In fact, on my linux box, I generate my IPv4 iptables file using
little more than a global search and replace on the IPv6 iptables
configuration which replaces the IPv6 prefixes/addresses with the
corresponding IPv4 prefixes/addresses. (My IPv6 addresses for things
that take incoming connections have an algorithmic map to IPv4 addresses
for things that have them.)


On my box, I have a librry of shell functions that do the generation, 
driven by parameter tables.  If I'm reading you correctly, I can just 
augment the parameter tables and those functions to generate the 
appropriate corresponding ip6table commands in parallel with the iptable 
commands.


Question: should I still rate-limit ICMP packets in IPv6?  Also, someone 
on this list pointed me to NIST SP800-119, "Guidelines for the Secure 
Deployment of IPv6", the contents of which which I will incorporate.



There is limited IPv6 support in many of the GUIs still,
unfortunately, but the command line tools are all there and for the
most part work pretty much identically for v4 and v6, the difference
often being as little as ping vs ping6 or   vs.
 -6 .


I've not been happy with the GUIs, because getting them to do what I 
want is a royal pain.  For example, I'm forced to use port-based 
redirection in one edge firewall application -- I blew a whole weekend 
figuring out how to do that with the CentOS 7 firewalld corkscrew, for a 
customer who outgrew the RV-220 he used for the application.  At least 
that didn't need IPv6!



Primarily it involves changing the IPv4 addresses and/or prefixes
into IPv6 addresses and/or prefixes.


What about fragmented packets?  And adjusting the parameters in ip6table 
filters to detect the DNS "ANY" requests used in the DDoS amplification 
attacks?



I'm not asking NANOG to go past its charter, but I am asking the
IPv6fanatics on this mailing list to recognize that, even though the net
itself may be running IPv6, the support and education infrastructure is
still behind the curve. Reading RFCs is good, reading man pages is good,
but there is no guidance about how to implement end-network policies in
the wild yet...at least not that I've been able to find.


There is actually quite a bit of information out there. Sylvia
Hagen’sIPv6 book covers a lot of this (O’Reilly publishes it).


Um, that would be "books".  Which one do you recommend I start with?

* IPv6 Essentials (3rd Edition), 2014, ASIN: B00RWSNEKG
* Planning for IPv6 (1st Edition), 2011,  ISBN-10: 1449305393

(I would assume the first, as the NIST document probably covers the 
contents of the second)



There are also several other good IPv6 books.


Recommendations?


"ipv6.disable" will be changed to zero when I know how to set the
firewall to implement the policies I need to keep other edge networks
from disrupting mine.


You do. You just don’t realize that you do. See above.


That's encouraging.  Being able to leverage the knowledge from IPv4 to 
project the same policies into IPv6 makes it easier for me, as I'm 
already using programmatic methods of generating the firewalls.  I 
expected that the implementation of existing applications-level policies 
would be parallel; it's the policies further down the stack that was my 
concern.


Also, I have a lot of IP level blocks (like simpler Cisco access control 
lists) to shut out those people who like to bang on my SSH front door. 
I believe that people who are so rude as to try to break through dozens 
or hundreds of time a day will have other bad habits, and don't deserve 
to be allowed for anything.  (I have similar blocks for rabid spammers 
not in the DNSBLs, but that's a different story.)  I would expect to 
maintain a separate list of IPv6 subnets, based on experience.


Which brings up another question:  should I block IPv6 access to port 25 
on my mail servers, and not announce a  record for it?  Postfix 
handles IPv6, but I've seen discussion that e-mail service is going to 
be IPv4 only for quite a while.  Should I even enable IPv6 on my mail 
server at this time?  Or is that a question I should post elsewhere?


As an aside, my day job is converting to Python programming, so my first 
Python project may well be the conversion of my existing firewall 
generator into that language.

Re: /27 the new /24

[Stephen Satchell]

On 10/07/2015 06:29 AM, Matthew Kaufman wrote:

On Oct 7, 2015, at 5:01 AM, Owen DeLong  wrote:


Instead, the followup question is needed… “That’s great, but how
does that help me reach a web site that doesn’t have and can’t get an
IPv4 address?”



At the present time, a web site that doesn't have and can't get an IPv4 address isn't 
"on the Internet".


Boy, this is drifting rather badly from the NANOG charter.

That said, let be disabuse you of a notion:   if someone wants to put up 
a web site, it's very, very easy to find a place that can provide an 
IPv4 IP address for the service.  It won't be a *private* IPv4 address, 
but...  Frankly, there are plenty of Web hosts that provide the service. 
 Not good enough?  Try using a cloud service to host your private WWW 
server.


Still not good enough?  Use a Web host and its IPv4 access to layer-7 
proxy your IPv6-only web site.  Finding a Web hosting company with IPv6 
support is a little more effort.  Start with this list: 
https://www.sixxs.net/wiki/IPv6_Enabled_Hosting


When I was working for a Web hosting company, along with 60 or so 
shared-hosting servers, we had one monster CPanel server (offering 
dirt-cheap low-performance hosting) with more than 1,000 sites on it, 
served by a single IP address.

Re: /27 the new /24

[t...@pelican.org]

On Wednesday, 7 October, 2015 12:54, "Owen DeLong"  said:

> There are some important differences for ICMP (don’t break PMTU-D or ND),
> but otherwise, really not much difference between your IPv4 security policy 
> and
> your IPv6 security policy.

The IPv4 world would have been nicer without quite so much of the "ICMP is 
evil!" nonsense, but agreed, it's somewhat more fundamental in IPv6.

> In fact, on my linux box, I generate my IPv4 iptables file using little more 
> than
> a global search and replace on the IPv6 iptables configuration which replaces 
> the
> IPv6 prefixes/addresses with the corresponding IPv4 prefixes/addresses. (My 
> IPv6
> addresses for things that take incoming connections have an algorithmic map to
> IPv4 addresses for things that have them.)

Similarly for at least some supplied tools on top of iptables.  'ufw' Just 
Works with both - 'ufw allow 25/tcp' will insert the appropriate rule into both 
iptables and ip6tables, for example.

Regards,
Tim.

Re: /27 the new /24

[Matthew Kaufman]

> On Oct 7, 2015, at 7:00 AM, Mark Andrews  wrote:
> 
> 
> In message , Matthew Kaufman 
> w
> rites:
>> 
>> 
>>> On Oct 7, 2015, at 5:01 AM, Owen DeLong  wrote:
>>> =20
>>> =20
>>> =20
>>> Instead, the followup question is needed=E2=80=A6 =E2=80=9CThat=E2=80=99s g
>> =
>> reat, but how does that help me reach a web site that doesn=E2=80=99t have a=
>> nd can=E2=80=99t get an IPv4 address?=E2=80=9D
>>> =20
>>> Owen
>>> =20
>> 
>> At the present time, a web site that doesn't have and can't get an IPv4 addr=
>> ess isn't "on the Internet".
> 
> It's on the Internet.  ISP's that fail to supply IPv6 at this point
> in time are committing fraud if they claim to supply Internet
> connection.

Good luck prosecuting them for that.

Along with all the internal IT departments that are failing to deliver v6 to 
wifi and desktops.

> 
>> That may change in the future, but right now this is the web site's fault, n=
>> ot your ISP's.
> 
> No, it isn't the site's fault.  The internet ran out of IPv4 addresses
> years ago.  Not everyone can get a public adddress.  

Right. Now it is only people who can afford about $8 one time. (The going rate 
for IPv4 on the transfer market at modest block sizes)

> There are
> millions of customers without a public IPv4 address that can host
> a site because they are behind a CGN which is only needed because
> of the short sightedness of lots of ISPs failing to deliver IPv6
> to their customers.

I think you meant cannot.

Most consumer ISPs also prevent this as a matter of policy. Good luck getting 
those policies changed.

> 
>> Wishing that the IPv6 transition had gone differently does not change
>> reality.
> 
> I don't see anyone wishing it went differnetly.  I see someone
> pointing out the reality that lots of ISP's are way too late to
> delivering IPv6.

Sure, they're too late. Which is why, until there's more progress, a website 
not reachable over IPv4 is fairly useless if the goal is to serve "most of the 
users on the Internet"

> 
> *Every* ISP should have been planning to deliver IPv6 by the time
> the first RIR ran out of IPv4 addresses.  That would have been
> just-in-time engineering.  It's not like they didn't have over a
> decade to plan to do it,  It's not like there wern't reasonable
> accurate forcecasts for when that would happen.

Yeah, totally agree. Didn't happen. Still hasn't happened. Won't happen 
tomorrow.

> 
> It was not hard to see what would happen if you didn't deliver IPv6
> before the first RIR ran out.
> 
> No instead most of then stuck their heads in the sand and said "we
> have enough IPv4 addresses" without looking at whom they need to
> connect with.

Last I checked, things are still working out just fine for all of them. Despite 
the obvious concerns about the future.

Matthew Kaufman

(Sent from my iPhone)

Re: AW: AW: AW: /27 the new /24

[James Jun]

On Sat, Oct 03, 2015 at 08:10:36AM -0500, Mike Hammett wrote:
> 
> People keep thinking I want Level 3 to replace a loaded 6500 with a CCR and 
> that's simply not what I'm saying at all. The point of rattling off the 
> newer\smaller hardware was to say that if the site doesn't require 40G\100G, 
> doesn't have the revenue to support an MX480, etc. you should put in a 
> smaller\cheaper box. 
> Cost is a non-issue at that point because the smaller gear that's all you 
> need will have far less operational cost. Someone thought a particular POP 
> was going to be a big hit... and wasn't. 

In an SP environment, there is an escalating operating cost and network 
complexity to having small full-featured routers (ie. MX80, ASR9001, CER2k, 
etc) at every data center, POP or anywhere you need to terminate customers.  
The reality is that small routers (even if you were to use ghetto routers) have 
poor economics in port density.  It's feasible for a startup ISP to spam MX80 
or equivalent anytime they need more ports, but there comes a point where 
plopping a big chassis is the way to go.

At my place, we started with MX80s to cheap out on router ports anytime we had 
to light a data center.  That only got us far and we ended up having to migrate 
to ASR 9010s and start phasing out small routers.  The increasing complexity of 
having dozens of small routers and managing LSP mesh to remainder of the 
network is ugly.  Moreover, full-table BGP routers are also the places where 
you exercise edge policy with complex routing policies.  Even with automation, 
managing dozens of those in a region that could have been served by only 2 
routers is annoying.  It's easier to haul IP customers to fewer, but more 
reliable large-chassis platforms and use packet-optical network to get to the 
customer premise.

Between the above and the lack of control-plane redundancy on most small 
routers, there are operational complexities & economic realities to keep in 
mind; it's not strictly about whether a site requires 40G/100G.  


> On the flip side, if there are 200 ports of customers chances are you need 
> the big interfaces that aren't on the old boxes. You have the bigger revenue. 
> Heck, the new big boxes probably still use less power than the old big boxes 
> anyway. 

The idea has its merits, however in practice, it isn't feasible.  People don't 
put in line cards into their router with expectation that they need to be 
replaced 2 years down the road because FIB TCAM ran out.  Even if you have the 
revenue to justify new line cards, constant migration of customer interfaces 
means disruptive maintenance for that customer.  We'd prefer IP network to be 
as reliable as dial-tone, if possible.

The global routing table is approaching 600k today.  Lot of line cards in 
installed base today only handle ~1.0/1.3 to ~1.8 million IPv4.  When you start 
replacing those line cards (and mind you, a 24x10GE line card has a list price 
running into $300k range), the next logical level is 4 million IPv4.  With all 
the deaggs with /24s, just how long of time are we going to have with /27 
explosion before 4 million FIB runs out of space?

I can see /25 being contemplated, but the cost of moving to /27 just isn't 
worth it at the moment.


> 
> What I learned from this thread: Once you mention MT\UBNT routers, people 
> assume you're using a MT\UBNT hammer everywhere. 

I'm not aware of any carrier-grade network that operates on these things.


Best,
james

Re: /27 the new /24

[Mel Beckman]

Keep in mind that IPv6 has IPSec VPN built into the protocol. It doesn't need 
to be in the router. 

Unlike IPv4, where the IPSec VPN protocol is an add-on, optional service, with 
IPv6 it's built into every device, because IPsec is a mandatory component for 
IPv6, and therefore, the IPsec security model is required to be supported for 
all IPv6 implementations. 

Thus it is a true end-to-end secure transport between two nodes -- even when 
those nodes are behind a firewall. You can still created IPv6 VPNs from 
site-to-site (called "tunnel mode"), but the idea with IPv6 is that since you 
can directly encrypt every TCP session, eventually the need for tunnels will 
diminish, if not go away completely. 

Interestingly, IPsec came out of funding from Clinton administration for 
securely hosting the whitehouse.gov email server. Trusted Information Systems 
software engineer Wei Xu started researching IP security methods in July 1994, 
and ultimately developed the first rendition of IPSec. He ported it to several 
server OSes of the time. 

 -mel beckman

> On Oct 4, 2015, at 6:41 AM, Matthias Leisi  wrote:
> 
> The built-in VPN which only supports IPv4 (that one specifically on an Asus 
> router).

Re: /27 the new /24

[Mel Beckman]

If it doesn't support IPSec, it's not really IPv6. Just as if it failed to 
support any other mandatory IPv6 specification, such as RA. 

There's really no excuse for not supporting IPSec, as it's a widely available 
open source component that costs nothing to incorporate into an IPv6 stack. 

Your observation simply means that users must be informed when buying IPv6 
devices, just as they must with any product. You can buy either genuine IPv6 or 
half-baked IPv6 products. When I speak of IPv6, I speak only of the genuine 
article. 

 -mel beckman

On Oct 4, 2015, at 7:41 AM, "sth...@nethelp.no"  wrote:

>> Keep in mind that IPv6 has IPSec VPN built into the protocol. It doesn't 
>> need to be in the router. 
>> 
>> Unlike IPv4, where the IPSec VPN protocol is an add-on, optional service, 
>> with IPv6 it's built into every device, because IPsec is a mandatory 
>> component for IPv6, and therefore, the IPsec security model is required to 
>> be supported for all IPv6 implementations.
> 
> If you really believe all IPv6 devices support IPsec, I can only
> conclude that your IPv6 experience is limited...
> 
> Steinar Haug, Nethelp consulting, sth...@nethelp.no

Re: /27 the new /24

[Mel Beckman]

Randy,

Your claim is a red herring. IPSec has nothing to do with IPv6 deployment. 
Deployment doesn't require global IPSec, which need only reside in endpoint 
nodes. It's not needed at all in the routjg and distribution infrastructure, 
which is where deployment happens

The vast majority of IPv6 nodes -- which is where the IPSec requirement exists 
-- have IPSec built in: Linux, Mac OSX, and Windows. Devices that sometimes act 
as nodes, such as firewalls terminating IPSec tunnels, also obviously need 
IPSec. Devices that are simply IPv6 pass-through, such as consumer-grade 
routers, don't.

Users can buy whatever level of functionality they need at the edges. If you 
don't need IPSec tunnel support in your firewall, you can buy one without it. 
Deployment cares nothing about IPSec.

 -mel beckman

On Oct 4, 2015, at 8:05 AM, Randy Bush > 
wrote:

If it doesn't support IPSec, it's not really IPv6.

by that criterion, ipv6 deployment is effectively zero

Re: /27 the new /24

[Sander Steffann]

Hi,

> Op 4 okt. 2015, om 16:52 heeft Mel Beckman  het volgende 
> geschreven:
> 
> If it doesn't support IPSec, it's not really IPv6. Just as if it failed to 
> support any other mandatory IPv6 specification, such as RA. 

I think you're still looking at an old version of the IPv6 Node Requirements. 
Check https://tools.ietf.org/html/rfc6434#section-11, specifically this bit:

"""
Previously, IPv6 mandated implementation of IPsec and recommended the key 
management approach of IKE.  This document updates that recommendation by 
making support of the IPsec Architecture a SHOULD for all IPv6 nodes.
"""

This was published in December 2011.

Cheers,
Sander

Re: /27 the new /24

[Denis Fondras]

> Building a secure firewall takes more than just knowing how to issue
> ip6table commands; one also needs to know exactly what goes into those
> commands.  NANOG concentrates on network operators who need to provide a
> good Internet experience to all their downstream customers, which is why I
> see the bias toward openness...as it should be.  Those of us who run edge
> networks have different problems to solve.
> 

NIST has very good publication on this subject :
http://csrc.nist.gov/publications/nistpubs/800-119/sp800-119.pdf
(Table 3-7 is a must read for any IPv6 newbie)

Denis

Re: /27 the new /24

[Mel Beckman]

I recommend any of a number of online courses for a quick understanding of 
IPv6. But nothing beats making your own IPv6 lab and getting hands-on 
experience. Here's a course I built walking you through that process:

http://windowsitpro.com/build-your-own-ipv6-lab-and-become-ipv6-guru-demand

 -mel beckman

> On Oct 4, 2015, at 7:49 AM, Stephen Satchell  wrote:
> 
>> On 10/04/2015 06:40 AM, Matthias Leisi wrote:
>> Fully agree. But the current state of IPv6 outside "professional“
>> networks/devices is sincerely limited by a lot of poor CPE and
>> consumer device implementations.
> 
> I have to ask:  where is the book _IPv6 for Dummies_ or equivalent?
> 
> Specifically, is http://www.xnetworks.es/contents/Infoblox/IPv6forDummies.pdf 
> any good? (I just downloaded it to inspect.)
> 
> My bookshelf is full of books describing IPv4.  Saying "IPv6 just works" 
> ignores the issues of configuring intelligent firewalls to block the 
> ne-er-do-wells using the new IP-level protocol.
> 
> In Robert L. Ziegler's book _Linux Firewalls_ Second Edition (2002, ISBN 
> 0-7357-1099-6), the *only* mention of IPv6 is in the discussion of NAT, and 
> that discussion is limited to "NAT is a stopgap until IPv6 achieves wide 
> implementation.  A preview of the Third Edition fails to mention ip6tables at 
> all, the same lack that the Second Edition has.
> 
> I use CentOS, the community version of Red Hat Enterprise.  I looked around 
> for useful books on building IPv6 firewalls with the same granularity as the 
> above-mentioned book for IPv4, and haven't found anything useful as yet.  In 
> particular, I'm looking for material that lays out how to build a 
> mostly-closed firewall and DMZ in IPv6.  The lack of IPv6 support goes 
> further:  I didn't find anything useful in Red Hat (CentOS) firewall tools 
> that provides IPv6 support...but that's probably because I don't know what 
> I'm looking for.  (Also, that GUI software is intended for use on individual 
> servers/computers, not in a edge-firewall with forwarding and DMZ 
> responsibilities.)
> 
> Building a secure firewall takes more than just knowing how to issue ip6table 
> commands; one also needs to know exactly what goes into those commands.  
> NANOG concentrates on network operators who need to provide a good Internet 
> experience to all their downstream customers, which is why I see the bias 
> toward openness...as it should be.  Those of us who run edge networks have 
> different problems to solve.
> 
> I'm not asking NANOG to go past its charter, but I am asking the IPv6 
> fanatics on this mailing list to recognize that, even though the net itself 
> may be running IPv6, the support and education infrastructure is still behind 
> the curve.  Reading RFCs is good, reading man pages is good, but there is no 
> guidance about how to implement end-network policies in the wild yet...at 
> least not that I've been able to find.
> 
> "ipv6.disable" will be changed to zero when I know how to set the firewall to 
> implement the policies I need to keep other edge networks from disrupting 
> mine.
> 

Re: /27 the new /24

[Randy Bush]

> Keep in mind that IPv6 has IPSec VPN built into the protocol.

yet another ipv6 fantasy.  it may be in the powerpoint but it is not in
the implementations.

Re: /27 the new /24

[sthaug]

> Keep in mind that IPv6 has IPSec VPN built into the protocol. It doesn't need 
> to be in the router. 
> 
> Unlike IPv4, where the IPSec VPN protocol is an add-on, optional service, 
> with IPv6 it's built into every device, because IPsec is a mandatory 
> component for IPv6, and therefore, the IPsec security model is required to be 
> supported for all IPv6 implementations. 

If you really believe all IPv6 devices support IPsec, I can only
conclude that your IPv6 experience is limited...

Steinar Haug, Nethelp consulting, sth...@nethelp.no

Re: /27 the new /24

[Stephen Satchell]

On 10/04/2015 06:40 AM, Matthias Leisi wrote:

Fully agree. But the current state of IPv6 outside "professional“
networks/devices is sincerely limited by a lot of poor CPE and
consumer device implementations.


I have to ask:  where is the book _IPv6 for Dummies_ or equivalent?

Specifically, is 
http://www.xnetworks.es/contents/Infoblox/IPv6forDummies.pdf any good? 
(I just downloaded it to inspect.)


My bookshelf is full of books describing IPv4.  Saying "IPv6 just works" 
ignores the issues of configuring intelligent firewalls to block the 
ne-er-do-wells using the new IP-level protocol.


In Robert L. Ziegler's book _Linux Firewalls_ Second Edition (2002, ISBN 
0-7357-1099-6), the *only* mention of IPv6 is in the discussion of NAT, 
and that discussion is limited to "NAT is a stopgap until IPv6 achieves 
wide implementation.  A preview of the Third Edition fails to mention 
ip6tables at all, the same lack that the Second Edition has.


I use CentOS, the community version of Red Hat Enterprise.  I looked 
around for useful books on building IPv6 firewalls with the same 
granularity as the above-mentioned book for IPv4, and haven't found 
anything useful as yet.  In particular, I'm looking for material that 
lays out how to build a mostly-closed firewall and DMZ in IPv6.  The 
lack of IPv6 support goes further:  I didn't find anything useful in Red 
Hat (CentOS) firewall tools that provides IPv6 support...but that's 
probably because I don't know what I'm looking for.  (Also, that GUI 
software is intended for use on individual servers/computers, not in a 
edge-firewall with forwarding and DMZ responsibilities.)


Building a secure firewall takes more than just knowing how to issue 
ip6table commands; one also needs to know exactly what goes into those 
commands.  NANOG concentrates on network operators who need to provide a 
good Internet experience to all their downstream customers, which is why 
I see the bias toward openness...as it should be.  Those of us who run 
edge networks have different problems to solve.


I'm not asking NANOG to go past its charter, but I am asking the IPv6 
fanatics on this mailing list to recognize that, even though the net 
itself may be running IPv6, the support and education infrastructure is 
still behind the curve.  Reading RFCs is good, reading man pages is 
good, but there is no guidance about how to implement end-network 
policies in the wild yet...at least not that I've been able to find.


"ipv6.disable" will be changed to zero when I know how to set the 
firewall to implement the policies I need to keep other edge networks 
from disrupting mine.

Re: /27 the new /24

[Randy Bush]

i give

< plonk >

Re: /27 the new /24

[Matthias Leisi]

> One or more of these things will be the death of IPv4:

IPv4 will not die, it will be superseded by something better :) 

What I have found to be the greatest obstacle to IPv6 adoption is the state of 
IPv6 support in various types of CPEs / network equipment. The support is 
mostly OK in higher-end gear. But have you checked the support in home- or 
small-office devices? 

In the handful of devices I recently had to deal with, IPv6 is always at best a 
„second class citizen“. First, there is some GUI for setup around IPv4. Then, 
if you are lucky, there are some poorly and inconsistently labelled  „Advanced“ 
settings that may or may not enable IPv6. Or may enable some semi-consistent 
state that has been in an obscure lab setup once upon a time. 

The built-in VPN which only supports IPv4 (that one specifically on an Asus 
router). A printer that behaves differently at different times under IPv4 than 
under IPv6. A NAS that works with IPv6 - *most* of the time. 

While I can personally work around most of these issues, it simply does not 
scale to even a small office environment with some semi-technical people. 
That’s basic stuff that just needs to work, regardless of whether it runs on 
IPv4 or IPv6. 

> Fortunately, in this case, we have a nice new body that we can transplant the 
> internet into that has many fruitful years ahead
> of it. So… Do whatever you have to to survive in the meantime, but focus on 
> getting your stuff onto the IPv6 internet so that
> we can all let IPv4 go gently into that good night and have it’s well 
> deserved final rest.

Fully agree. But the current state of IPv6 outside "professional“ 
networks/devices is sincerely limited by a lot of poor CPE and consumer device 
implementations. 

— Matthias



smime.p7s
Description: S/MIME cryptographic signature

Re: /27 the new /24

[Randy Bush]

> If it doesn't support IPSec, it's not really IPv6.

by that criterion, ipv6 deployment is effectively zero

Re: /27 the new /24

[Jon Lewis]

On Sun, 4 Oct 2015, Mel Beckman wrote:

If it doesn't support IPSec, it's not really IPv6. Just as if it failed 
to support any other mandatory IPv6 specification, such as RA.


Go tell cisco that.  IIRC, the first network I dual-stacked, I was kind of 
surprised when I found I could not use authentication in OSPFv3, because 
OSPFv3 assumes IPv6 will supply the IPSec to do auth...but these routers 
didn't support IPSec.  They still managed to route IPv6 and support IPv6 
customers...so it really was IPv6...just not the full suite of everything 
you'd expect from IPv6.


Your observation simply means that users must be informed when buying 
IPv6 devices, just as they must with any product. You can buy either 
genuine IPv6 or half-baked IPv6 products. When I speak of IPv6, I speak 
only of the genuine article.


Does anyone buy "IPv6 devices"?

The biggest hurdle I've seen with IPv6 adoption (i.e. going dual-stack, 
with the idea that we'll gradually transition most things / most traffic 
from v4 to v6) is the number of end-user network providers that don't 
offer v6 at all.  My home cable internet provider still doesn't offer 
IPv6.  When I asked one of their support people about it recently, I was 
told not to worry, they have plenty of v4 addresses left, but it was 
implied that they do plan to start offering v6 sometime soon.  They should 
have started rolling out IPv6 to any customers that wanted it years ago, 
so that by today, it would be standard for all their installations to be 
dual-stack.  But here we are, nearly 2016, and they don't have a single 
IPv6 customer (AFAIK) yet.


--
 Jon Lewis, MCP :)   |  I route
 |  therefore you are
_ http://www.lewis.org/~jlewis/pgp for PGP public key_

Re: /27 the new /24

[Mel Beckman]

What Cisco routers, and what vintage IOS, are you finding have no IPSec 
support? I've not run into that problem. 

 -mel beckman

> On Oct 4, 2015, at 8:33 AM, Jon Lewis  wrote:
> 
>> On Sun, 4 Oct 2015, Mel Beckman wrote:
>> 
>> If it doesn't support IPSec, it's not really IPv6. Just as if it failed to 
>> support any other mandatory IPv6 specification, such as RA.
> 
> Go tell cisco that.  IIRC, the first network I dual-stacked, I was kind of 
> surprised when I found I could not use authentication in OSPFv3, because 
> OSPFv3 assumes IPv6 will supply the IPSec to do auth...but these routers 
> didn't support IPSec.  They still managed to route IPv6 and support IPv6 
> customers...so it really was IPv6...just not the full suite of everything 
> you'd expect from IPv6.
> 
>> Your observation simply means that users must be informed when buying IPv6 
>> devices, just as they must with any product. You can buy either genuine IPv6 
>> or half-baked IPv6 products. When I speak of IPv6, I speak only of the 
>> genuine article.
> 
> Does anyone buy "IPv6 devices"?
> 
> The biggest hurdle I've seen with IPv6 adoption (i.e. going dual-stack, with 
> the idea that we'll gradually transition most things / most traffic from v4 
> to v6) is the number of end-user network providers that don't offer v6 at 
> all.  My home cable internet provider still doesn't offer IPv6.  When I asked 
> one of their support people about it recently, I was told not to worry, they 
> have plenty of v4 addresses left, but it was implied that they do plan to 
> start offering v6 sometime soon.  They should have started rolling out IPv6 
> to any customers that wanted it years ago, so that by today, it would be 
> standard for all their installations to be dual-stack.  But here we are, 
> nearly 2016, and they don't have a single IPv6 customer (AFAIK) yet.
> 
> --
> Jon Lewis, MCP :)   |  I route
> |  therefore you are
> _ http://www.lewis.org/~jlewis/pgp for PGP public key_

Re: /27 the new /24

[Nick Hilliard]

On 04/10/2015 16:03, Randy Bush wrote:
> yet another ipv6 fantasy.  it may be in the powerpoint but it is not in
> the implementations.

the ipsec tickbox was removed from ipv6 in rfc6434 (2011).

Nick

Re: /27 the new /24

[Mel Beckman]

Stefann,

You're right. I remember hearing rumblings of vendors requesting this change, 
mostly because embedded processors of the time had difficulty performing well 
with IPv6. I see that in 2011 rfc6434 lowered IPSec from "must" to "should". 
Nevertheless, plenty of products produced before 2011 included IPSec and the 
vast majority of IPv6-capable nodes on the Internet have it today. Performance 
is no longer an issue. 

 -mel beckman

> On Oct 4, 2015, at 8:58 AM, Sander Steffann  wrote:
> 
> Hi,
> 
>> Op 4 okt. 2015, om 16:52 heeft Mel Beckman  het volgende 
>> geschreven:
>> 
>> If it doesn't support IPSec, it's not really IPv6. Just as if it failed to 
>> support any other mandatory IPv6 specification, such as RA.
> 
> I think you're still looking at an old version of the IPv6 Node Requirements. 
> Check https://tools.ietf.org/html/rfc6434#section-11, specifically this bit:
> 
> """
> Previously, IPv6 mandated implementation of IPsec and recommended the key 
> management approach of IKE.  This document updates that recommendation by 
> making support of the IPsec Architecture a SHOULD for all IPv6 nodes.
> """
> 
> This was published in December 2011.
> 
> Cheers,
> Sander
> 

Re: /27 the new /24

[Jon Lewis]

sup720-3bxl, but this was a number of years ago.  I don't recall the 
exact version.  It was probably 12.2SXI-something.


On Sun, 4 Oct 2015, Mel Beckman wrote:


What Cisco routers, and what vintage IOS, are you finding have no IPSec 
support? I've not run into that problem.

-mel beckman



--
 Jon Lewis, MCP :)   |  I route
 |  therefore you are
_ http://www.lewis.org/~jlewis/pgp for PGP public key_

Re: /27 the new /24

[Mel Beckman]

 A lot has changed since 12.2 :)

I believe all shipping gear supports IPSec in IPv6. 

 -mel beckman

> On Oct 4, 2015, at 11:48 AM, Jon Lewis  wrote:
> 
> sup720-3bxl, but this was a number of years ago.  I don't recall the exact 
> version.  It was probably 12.2SXI-something.
> 
>> On Sun, 4 Oct 2015, Mel Beckman wrote:
>> 
>> What Cisco routers, and what vintage IOS, are you finding have no IPSec 
>> support? I've not run into that problem.
>> 
>> -mel beckman
>> 
> 
> --
> Jon Lewis, MCP :)   |  I route
> |  therefore you are
> _ http://www.lewis.org/~jlewis/pgp for PGP public key_

AW: AW: /27 the new /24

[Jürgen Jaritsch]

Hi Mike,

it's not a bureaucracy problem ... if you're a big player and you have to 
decide about a 2-3 Mio invest to upgrade only a few of your POPs (and let's say 
you have hundreds of POPs) it will be hard to find the "right" decision.

Some questions  these decision makers have to think about:

#) What are the future plans for this POP?
#) How upgradeable / expandable is the new equipment?
#) Does our engineers know everything they need to run & debug & fix this new 
equipment?
#) TOC incl support contract over the complete lifetime?
#) Product life cycle? (Is it outdated in two years??)
#) Will we keep spare parts onsite or nearby?
#) How long needs the vendor to deliver everything I need?
#) Is it compatible with all the already installed equipment?
#) Migration plan to move existing customers to the new equipment?

There are a ton of additional questions ... but I guess I pointed out some of 
the most important. Big players can't only calculate the price of the equipment 
- most of the time all the surrounding stuff (installation, new cabinets, 
migrations, training of engineers, etc) is producing 0,5x to 1x of the 
equipment costs. To get some easy numbers: take the discounted price (no one 
pays list prices ...) of an equipment and take this price x2 => that will be a 
realistic number to get the box onsite, up and running.

It's not all the time something simple like a router with 20 patch cords :(.

Best regards

Jürgen Jaritsch
Head of Network & Infrastructure

ANEXIA Internetdienstleistungs GmbH

Telefon: +43-5-0556-300
Telefax: +43-5-0556-500

E-Mail: jjarit...@anexia-it.com 
Web: http://www.anexia-it.com 

Anschrift Hauptsitz Klagenfurt: Feldkirchnerstraße 140, 9020 Klagenfurt
Geschäftsführer: Alexander Windbichler
Firmenbuch: FN 289918a | Gerichtsstand: Klagenfurt | UID-Nummer: AT U63216601

-Ursprüngliche Nachricht-
Von: NANOG [mailto:nanog-boun...@nanog.org] Im Auftrag von Mike Hammett
Gesendet: Samstag, 03. Oktober 2015 04:53
Cc: NANOG <nanog@nanog.org>
Betreff: Re: AW: /27 the new /24

A better truth may be that I have no idea about bureaucracies... which I'll 
happily admit to. 




- 
Mike Hammett 
Intelligent Computing Solutions 
http://www.ics-il.com 



Midwest Internet Exchange 
http://www.midwest-ix.com 


- Original Message -

From: "Jürgen Jaritsch" <j...@anexia.at> 
To: "Mike Hammett" <na...@ics-il.net>, "NANOG" <nanog@nanog.org> 
Sent: Friday, October 2, 2015 2:25:10 PM 
Subject: AW: /27 the new /24 

> Stop using old shit. 

Sorry, but the truth is: you have no idea about how earning revenue works and 
you obviously also have no idea about carrier grade networks. 




Jürgen Jaritsch 
Head of Network & Infrastructure 

ANEXIA Internetdienstleistungs GmbH 

Telefon: +43-5-0556-300 
Telefax: +43-5-0556-500 

E-Mail: jjarit...@anexia-it.com 
Web: http://www.anexia-it.com 

Anschrift Hauptsitz Klagenfurt: Feldkirchnerstraße 140, 9020 Klagenfurt 
Geschäftsführer: Alexander Windbichler 
Firmenbuch: FN 289918a | Gerichtsstand: Klagenfurt | UID-Nummer: AT U63216601 

-Ursprüngliche Nachricht- 
Von: NANOG [mailto:nanog-boun...@nanog.org] Im Auftrag von Mike Hammett 
Gesendet: Freitag, 02. Oktober 2015 20:38 
An: NANOG <nanog@nanog.org> 
Betreff: Re: /27 the new /24 

Chances are the revenue passing scales to some degree as well. Small business 
with small bandwidth needs buys small and has small revenue. Big business with 
big bandwidth needs buys big and has big revenue to support big router. 

I can think of no reason why ten years goes by and you haven't had a need to 
throw out the old network for new. If your business hasn't scaled with the 
times, then you need to get rid of your Cat 6500 and get something more power, 
space, heat, etc. efficient. 


I saw someone replace a stack of Mikrotik CCRs with a pair of old Cisco 
routers. I don't know what they were at the moment, but they had GBICs, so they 
weren't exactly new. Each router had two 2500w power supplies. They'll be worse 
in every way (other than *possibly* BGP convergence). The old setup consumed at 
most 300 watts. The new setup requires $500/month in power... and is worse. 

Stop using old shit. 




- 
Mike Hammett 
Intelligent Computing Solutions 
http://www.ics-il.com 



Midwest Internet Exchange 
http://www.midwest-ix.com 


- Original Message - 

From: "William Herrin" <b...@herrin.us> 
To: "Mike Hammett" <na...@ics-il.net> 
Cc: "NANOG" <nanog@nanog.org> 
Sent: Friday, October 2, 2015 1:09:16 PM 
Subject: Re: /27 the new /24 

On Fri, Oct 2, 2015 at 11:50 AM, Mike Hammett <na...@ics-il.net> wrote: 
> How many routers out there have this limitation? A $100 router 
> I bought ten years ago could manage many full tables. If 
> someone's network can't match that today, should I really have 
> any pity for them? 

Hi

Re: Mikrotik in the DFZ (Was Re: AW: AW: /27 the new /24)

[Mike Hammett]

Sure MT has issues, but so does everyone. As someone that has used them for 10+ 
years, the past six months has seen a bit of a re-awakening over there. You can 
see this in the time to completion of many feature requests, bug fixes, new 
features, etc. I'm not sure they're going to do everything everyone is after, 
but they certainly have shown a huge increase in willingness to go the right 
direction. 

Of course it's easy for someone running big iron to scoff at the lack of 
feature X or feature Y. To that I say, what are the capabilities of your $200 
router? Your $2k router? I haven't priced out new low-end gear from the big 
iron vendors, but I can't imagine at what price point you need to be at to have 
a multi-gig capable VPLS router. For Mikrotik you're in the $200 - $1k range, 
depending on what you mean by "multi-gig". One thing I miss as I start to use 
more non-Mikrotik hardware... Torch. I wish everything had Torch. Put Packet 
Sniffer in the list of things I'd like to see everywhere. I don't want port 
mirror as who's to say I have something to mirror to everywhere that can also 
capture? Put a few basic filters and drop the PCAP right on the damn box. Now 
obviously with something running BSD you could code up whatever you'd like or 
have an array of open-source packages to work with , but that wouldn't have the 
nice feature integration of a router OS. 

I have no problem running Mikrotik in the DFZ. Mine pull down full tables in 30 
- 35 seconds, can handle somewhere in the 30 - 60 gb range when firewall rules 
are applied and so on. They'd cost under $1,500 new, but I got mine put 
together for a fraction of that. They're so cheap you can run two. Run two and 
now you have the advantage of being able to do maintenance without downtime. 
It's a little kludgey, but can get get the job done at a price point the others 
can't. Maybe with newer CCRs and ROS7 I could drop the need for the x86 boxes. 
We'll see. 




- 
Mike Hammett 
Intelligent Computing Solutions 
http://www.ics-il.com 



Midwest Internet Exchange 
http://www.midwest-ix.com 


- Original Message -

From: "William Waites" <wwai...@tardis.ed.ac.uk> 
To: j...@anexia.at 
Cc: na...@ics-il.net, nanog@nanog.org 
Sent: Saturday, October 3, 2015 3:23:49 AM 
Subject: Mikrotik in the DFZ (Was Re: AW: AW: /27 the new /24) 

On Fri, 2 Oct 2015 23:11:47 +, Jürgen Jaritsch <j...@anexia.at> said: 

> Regarding the words "I have a small router which handles 
> multiple full tables ...": push and pull a few full tables at 
> the same time and you'll see what's happening: the CCRs are 
> SLOW. And why? Because the software is not as good as it could 
> be: the BGP daemon uses only one core of a 36(?) core CPU. 

To expand on this, the problem is worse than being single-threaded. I 
had one of these in the lab and fed it 2x full tables. Sure it wasn't 
the fastest at accepting them but then I noticed that even in steady 
state one of the CPUs was pegged. What was happening -- and this was 
confirmed by Mikrotik -- was that it was recalculating the *entire* 
FIB for each update. The general background noise of announce / 
withdraw messages means it is doing this all the time. Any churn and 
it would have a very hard time. 

There are other serious bugs such as not doing recursive next hop 
lookup for IPv6 (it does for IPv4). This makes them unuseable as BGP 
routers even for partial tables with most non-trivial iBGP 
topologies. All of which may be fixed one day in version 7 of their 
operating system, which will inevitably have many bugs as any software 
project .0 release will, so we'll have to wait for 7.x for it to be 
reasonably safe to use. 

That said, we use a lot of Mikrotik kit for our rural 
networks. They're weird and quirky but you can't beat them on price, 
port density and power consumption. With 16 ports and 36 cores surely 
they should be capable of pushing several Gbps of traffic with a few 
full tables. 

I wish it were possible today to run different software on their 
larger boxes. If some like-minded small providers wanted to get 
together with us to fund a FreeBSD port to the CCR routers that would 
be great. Please contact me off-list if you are interested in this, 
I'll coordinate. 

As it is we don't let them anywhere near the DFZ, that's done with PCs 
running FreeBSD and BIRD which can easily do the job but is still an 
order of magnitude more expensive (and an order of magnitude less 
expensive than what you need if you want 10s of Gbps). 

-w 

-- 
William Waites <wwai...@tardis.ed.ac.uk> | School of Informatics 
http://tardis.ed.ac.uk/~wwaites/ | University of Edinburgh 
https://hubs.net.uk/ | HUBS AS60241 

The University of Edinburgh is a charitable body, registered in 
Scotland, with registration number SC005336. 

Re: AW: AW: AW: /27 the new /24

[Mike Hammett]

I don't think we are talking different things, though I think we are talking in 
circles and thus the thread probably needs to die. 


People keep thinking I want Level 3 to replace a loaded 6500 with a CCR and 
that's simply not what I'm saying at all. The point of rattling off the 
newer\smaller hardware was to say that if the site doesn't require 40G\100G, 
doesn't have the revenue to support an MX480, etc. you should put in a 
smaller\cheaper box. Cost is a non-issue at that point because the smaller gear 
that's all you need will have far less operational cost. Someone thought a 
particular POP was going to be a big hit... and wasn't. On the flip side, if 
there are 200 ports of customers chances are you need the big interfaces that 
aren't on the old boxes. You have the bigger revenue. Heck, the new big boxes 
probably still use less power than the old big boxes anyway. 



What I learned from this thread: Once you mention MT\UBNT routers, people 
assume you're using a MT\UBNT hammer everywhere. 




- 
Mike Hammett 
Intelligent Computing Solutions 
http://www.ics-il.com 



Midwest Internet Exchange 
http://www.midwest-ix.com 


- Original Message -

From: "Jürgen Jaritsch" <j...@anexia.at> 
To: "Mike Hammett" <na...@ics-il.net> 
Cc: "NANOG" <nanog@nanog.org> 
Sent: Saturday, October 3, 2015 6:06:59 AM 
Subject: AW: AW: AW: /27 the new /24 

Hi Mike, 

> but the boxes that have been there for 10 years have more than paid for 
> themselves (unless they're a shitty business). 

No question about that! But why should they throw them away if they can still 
print $$$ with these boxes? They have to change nothing till the global routing 
table reaches at least 768k ... so let's say this will happen in 12-18 months. 
They have enough time to prepare, migrate, etc ... and while all the side 
stories are happening they are still able to print $$$ with the "old shit". 

> What I was saying is that my little business with meager means (and revenues) 
> can afford a box to do it. 

This is definitely a question about sizing. Replacing a box with ~200 connected 
customers (only at this box!) is way more complex and this is nothing 
unrealistic. 

> If their business hasn't boomed, maybe it's time to replace that old 6500 
> with a 4500x or a QFX-5100 or an x670 or whatever. 

4500x => no MPLS features 

QFX-5100 => very nice box (I'm a big fan) but complicate (and expensive!) 
licensing. 

Extreme x670 => nice box too - we also use this. But it's simply too small and 
the BGP configuration on these boxes is horrible. It's also not possible to 
provide Ethernet over MPLS with LACP BPDU forwarding ... too less features. 
Nice for aggregation and POP interconnect. 

All three models are new and shiny but they can't replace a 6500/7600. Too less 
port density and too less features (people are still using SDH. You need SDH in 
an 6500/7600? Simply install the required line card ...). If you really plan to 
replace a 6509 or even a 6513 you have to go with something like Juniper 
MX480/960 (I'm in love ... :D) or Cisco Nexus 7k/9k. 

One thing that will more and more happen: physical separation. There will be 
boxes with 10G/40G/100G only and boxes with 100M/1G only. Why? It's easier for 
vendors to remove old compatibility requirements (like electrical interfaces). 
So what we did in the past 3 years (replacing old boxes with new boxes with 
1G/10G interfaces) was useless - we'll get our "old shit" back in place and 
bring them up and running. Of course: the "old shit" will be reduced to do 
aggregation layer or to something like "multihop instance" to transport the 
customers access port to the "real big and powerful router". Solving this with 
Layer2 extensions (like VLANs) is not practicable because you'll ran into other 
problems (like STP instances, etc). Probably it makes sense to solve it with 
Layer2VPN (Ethernet over MPLS, etc) to transport the physical interface to a 
virtual interface. 

Lots of things to think about :(. 


> Your decreased power bill alone will pay it off. If it has boomed, then ten 
> years of revenues should get you whatever the bigger Ciscos are or an MX or 
> whatever the bigger Extremes are. 

Power is no argument. You get power starting at 0,10 Eur /kWh. Another 0,10 Eur 
/ kWh for cooling and we talk about 0,20 Eur / kWh => Cisco 6513 (configured 
with 11 line cards + 2x SUP) with 2x 6kW PSU uses 3,8kW. 3,8kW * 24 hours * 30 
days = 2.736 kWh per month. 2.736 * 0,20 Eur = 547,2 Eur per month for power 
consumption + cooling. If you have a good sales engineer you earn the revenue 
for this "side cost" with 1 customer :). Realistic calculation is: 10 customers 
are required to earn the money for the footprint. 


> Don't whine about my choices in gear I mentioned. I was just throwing things 
> out there. Ol

Re: AW: /27 the new /24

[Baldur Norddahl]

Except we might very well reach 1+ million routes soon without accepting
longer prefixes than /24. Also route updates is a concern - do I really
need to be informed every time someone on the other end of the world resets
a link?

On 3 October 2015 at 12:57, William Waites  wrote:

> On Sat, 3 Oct 2015 12:42:01 +0200, Baldur Norddahl <
> baldur.nordd...@gmail.com> said:
>
> > 2 million routes will not be enough if we go full /27. This is
> > not a scalable solution. Something else is needed to provide
> > multihoming for small networks (LISP?).
>
> It's not too far off though. One way of looking at it is, for each
> extra bit we allow, we potentially double the table size. So with 500k
> routes and a /24 limit now, we might expect 4 million with /27. Not
> exactly because it depends strongly on the distribution of prefix
> lengths, but probably not a bad guess.
>
> Also there are optimisations that I wonder if the vendors are doing to
> preserve TCAM such as aggregating adjacent networks with the same next
> hop into the supernet. That would mitigate the impact of wanton
> deaggregation at least and the algorithm doesn't look too hard. Do the
> big iron vendors do this?
>
> -w
>
> --
> William Waites   |  School of Informatics
>http://tardis.ed.ac.uk/~wwaites/   | University of Edinburgh
>  https://hubs.net.uk/ |  HUBS AS60241
>
> The University of Edinburgh is a charitable body, registered in
> Scotland, with registration number SC005336.
>

AW: AW: /27 the new /24

[Jürgen Jaritsch]

Hi,

yeah, of course there are newer models ... I mentioned the older ones (from the 
past 3-5 years). There are also Cisco routers available that are able to handle 
more than 1 Mio routes - of course also from Juniper.




Jürgen Jaritsch
Head of Network & Infrastructure

ANEXIA Internetdienstleistungs GmbH

Telefon: +43-5-0556-300
Telefax: +43-5-0556-500

E-Mail: jjarit...@anexia-it.com 
Web: http://www.anexia-it.com 

Anschrift Hauptsitz Klagenfurt: Feldkirchnerstraße 140, 9020 Klagenfurt
Geschäftsführer: Alexander Windbichler
Firmenbuch: FN 289918a | Gerichtsstand: Klagenfurt | UID-Nummer: AT U63216601

-Ursprüngliche Nachricht-
Von: Youssef Bengelloun-Zahr [mailto:yous...@720.fr] 
Gesendet: Samstag, 03. Oktober 2015 11:03
An: Jürgen Jaritsch <j...@anexia.at>
Cc: nanog@nanog.org; max...@netassist.ua
Betreff: Re: AW: /27 the new /24

Hi,

FYI, newer linecard models from BROCADE can hold 2 million routes. Probably 
others can do that now too.

Disclaimer : I'm not working for them or defending them, just setting an 
information straight.

My 2 cents.



> Le 3 oct. 2015 à 10:33, Jürgen Jaritsch <j...@anexia.at> a écrit :
> 
> As mentioned before: even the new SUP2T from Cisco is limited to 1Mio routes 
> ...
> 
> There are MANY other vendors with the same limitations: Juniper, Brocade, etc
> 
> And the solt equipment is not the 99USD trash from the super market at the 
> corner ...
> 
> 
> Jürgen Jaritsch
> Head of Network & Infrastructure
> 
> ANEXIA Internetdienstleistungs GmbH
> 
> Telefon: +43-5-0556-300
> Telefax: +43-5-0556-500
> 
> E-Mail: j...@anexia.at
> Web: http://www.anexia.at
> 
> Anschrift Hauptsitz Klagenfurt: Feldkirchnerstraße 140, 9020 Klagenfurt
> Geschäftsführer: Alexander Windbichler
> Firmenbuch: FN 289918a | Gerichtsstand: Klagenfurt | UID-Nummer: AT U63216601
> 
> 
> -Original Message-
> From: Max Tulyev [max...@netassist.ua]
> Received: Samstag, 03 Okt. 2015, 9:11
> To: nanog@nanog.org [nanog@nanog.org]
> Subject: Re: AW: /27 the new /24
> 
> Which routers? DIR-300 with OpenWRT/Quagga? :)
> 
> I think all above-the-trash level routers supports >1M routes, isn't it?
> 
>> On 02.10.15 17:45, Jürgen Jaritsch wrote:
>> Hi,
>> 
>> this would at least help to get rid of many old routing engines around the 
>> world :) ... or people would keep their "learn nothing smaller than /24" 
>> filters in place. Also an option - but not for companies who act as an IP 
>> transit provider.
>> 
>> 
>> best regards
>> 
>> Jürgen Jaritsch
>> Head of Network & Infrastructure
>> 
>> ANEXIA Internetdienstleistungs GmbH
>> 
>> Telefon: +43-5-0556-300
>> Telefax: +43-5-0556-500
>> 
>> E-Mail: jjarit...@anexia-it.com
>> Web: http://www.anexia-it.com
>> 
>> Anschrift Hauptsitz Klagenfurt: Feldkirchnerstraße 140, 9020 Klagenfurt
>> Geschäftsführer: Alexander Windbichler
>> Firmenbuch: FN 289918a | Gerichtsstand: Klagenfurt | UID-Nummer: AT U63216601
>> 
>> 
>> -Ursprüngliche Nachricht-
>> Von: NANOG [mailto:nanog-boun...@nanog.org] Im Auftrag von Justin Wilson - 
>> MTIN
>> Gesendet: Freitag, 02. Oktober 2015 16:32
>> An: NANOG
>> Betreff: /27 the new /24
>> 
>> I was in a discussion the other day and several Tier2 providers were talking 
>> about the idea of adjusting their BGP filters to accept prefixes smaller 
>> than a /24.  A few were saying they thought about going down to as small as 
>> a /27.  This was mainly due to more networks coming online and not having 
>> even a /24 of IPv4 space.  The first argument is against this is the 
>> potential bloat the global routing table could have.  Many folks have worked 
>> hard for years to summarize and such. others were saying they would do a /26 
>> or bigger.
>> 
>> However, what do we do about the new networks which want to do BGP but only 
>> can get small allocations from someone (either a RIR or one of their 
>> upstreams)?
>> 
>> Just throwing that out there. Seems like an interesting discussion.
>> 
>> 
>> Justin Wilson
>> j...@mtin.net
>> 
>> ---
>> http://www.mtin.net Owner/CEO
>> xISP Solutions- Consulting – Data Centers - Bandwidth
>> 
>> http://www.midwest-ix.com  COO/Chairman
>> Internet Exchange - Peering - Distributed Fabric
> 

Re: AW: /27 the new /24

[William Waites]

On Sat, 3 Oct 2015 12:42:01 +0200, Baldur Norddahl  
said:

> 2 million routes will not be enough if we go full /27. This is
> not a scalable solution. Something else is needed to provide
> multihoming for small networks (LISP?).

It's not too far off though. One way of looking at it is, for each
extra bit we allow, we potentially double the table size. So with 500k
routes and a /24 limit now, we might expect 4 million with /27. Not
exactly because it depends strongly on the distribution of prefix
lengths, but probably not a bad guess.

Also there are optimisations that I wonder if the vendors are doing to
preserve TCAM such as aggregating adjacent networks with the same next
hop into the supernet. That would mitigate the impact of wanton
deaggregation at least and the algorithm doesn't look too hard. Do the
big iron vendors do this?

-w

--
William Waites   |  School of Informatics
   http://tardis.ed.ac.uk/~wwaites/   | University of Edinburgh
 https://hubs.net.uk/ |  HUBS AS60241

The University of Edinburgh is a charitable body, registered in
Scotland, with registration number SC005336.


pgpQpvBY0HsM0.pgp
Description: PGP signature

AW: AW: AW: /27 the new /24

[Jürgen Jaritsch]

-500

E-Mail: jjarit...@anexia-it.com 
Web: http://www.anexia-it.com 

Anschrift Hauptsitz Klagenfurt: Feldkirchnerstraße 140, 9020 Klagenfurt
Geschäftsführer: Alexander Windbichler
Firmenbuch: FN 289918a | Gerichtsstand: Klagenfurt | UID-Nummer: AT U63216601

-Ursprüngliche Nachricht-
Von: NANOG [mailto:nanog-boun...@nanog.org] Im Auftrag von Mike Hammett
Gesendet: Samstag, 03. Oktober 2015 02:52
Cc: NANOG <nanog@nanog.org>
Betreff: Re: AW: AW: /27 the new /24

I don't expect carriers to be running UBNT\Mikrotik, but the boxes that have 
been there for 10 years have more than paid for themselves (unless they're a 
shitty business). It's time to rip and replace with whatever is appropriate for 
that site. No, I obviously don't think I'm going to change anyone's opinion on 
the matter (at least not anyone that matters in one of these networks). What I 
was saying is that my little business with meager means (and revenues) can 
afford a box to do it. They can too. 



I don't doubt their situation sucks... but either you fix it or you don't. Time 
and the rest of the Internet won't wait for them. 


If their business hasn't boomed, maybe it's time to replace that old 6500 with 
a 4500x or a QFX-5100 or an x670 or whatever. Your decreased power bill alone 
will pay it off. If it has boomed, then ten years of revenues should get you 
whatever the bigger Ciscos are or an MX or whatever the bigger Extremes are. 

Don't whine about my choices in gear I mentioned. I was just throwing things 
out there. Old big, new small if no money or old big new big if money. 


BTW: ROS 7 won't have multi-threaded BGP, but will be optimized to handle full 
table imports in a significantly reduced time. Oh, and I'm not sure that you 
couldn't do at least three nines with MT\UBNT. Well, no experience with the 
EdgeRouters yet. 




- 
Mike Hammett 
Intelligent Computing Solutions 
http://www.ics-il.com 



Midwest Internet Exchange 
http://www.midwest-ix.com 


- Original Message -

From: "Jürgen Jaritsch" <j...@anexia.at> 
To: "Mike Hammett" <na...@ics-il.net> 
Cc: "NANOG" <nanog@nanog.org> 
Sent: Friday, October 2, 2015 6:11:47 PM 
Subject: AW: AW: /27 the new /24 

Hi Mike, 

sorry, this was probably sent to quick ... let me please explain my POV of your 
statement: 

I want to concentrate my detailed answer only to the backbone situation which 
is often handled by the 6500/7600 - I guess all of us know that the 6500/7600 
has a ton of additional features ... 


6-7 years in the past carriers (and/or big ISPs) had only n*1G backbone 
capacities built with platforms that only had n*100M interfaces another 3-5 
years before. Their only invest in these 3-5 years was to add the Gig line 
cards, install some software updates and add new fibre optics (GBICs). Chassis, 
cabling, management interfaces etc could be remain in the cabinet - they only 
had to replace ONE line card (let's say for a few thousand bucks) and with this 
invest they were able to scale up their capacities. Of course: at some point 
they also had to replace the SUPs, PSUs, FANs, etc. But the invest in the 
surrounding stuff is nothing compared with completely new machines. 

So what all these companies did was buying a machine with an basic 
configuration and since 10(!) years they are able to expand this machines with 
(more or less) small and cheap upgrades. 

In backbone situations the 6500/7600 are definitely at the end of the resources 
the platform can provide. Most of the carriers (and of course also the bigger 
ISPs) had a real chance to evaluate a new model/vendor to ran future networks 
(with possibly also a very good scale-up path and scaling- and 
upgrade-options). Most of the before mentioned are already in an migration 
process (let's take a look at Seabone ... they are migration from Cisco to a 
mix of Juniper and Huawei). 

Summary: there are strict limitations within the Cisco 6500/7600 platform and 
these limitations forces the big players to move this boxes out (or move them 
into other parts of their network). The limitation with 1Mio routes is not a 
secret and the admins of these boxes decide what they want to use (e.g. 768k 
routes for IPv4 unicast and 256k routes for MPLS+VRF, etc). If the global 
routing table reaches the 768k mark (I guess this will happen in the next 
12-18months) most of the boxes will crash again (as it happened in Aug 2014). 


Regarding the words "I have a small router which handles multiple full tables 
...": push and pull a few full tables at the same time and you'll see what's 
happening: the CCRs are SLOW. And why? Because the software is not as good as 
it could be: the BGP daemon uses only one core of a 36(?) core CPU. Same 
problem in the past with the EoIP daemon (not sure if they fixed it on the CCRs 
- they fixed it on x86). 

Routerboards are nice and cool and to be honest: I'm a big fan of this stuff 
(also Ubiquiti).

Re: AW: /27 the new /24

[Baldur Norddahl]

2 million routes will not be enough if we go full /27. This is not a
scalable solution. Something else is needed to provide multihoming for
small networks (LISP?).

Regards,

Baldur


On 3 October 2015 at 11:03, Youssef Bengelloun-Zahr <yous...@720.fr> wrote:

> Hi,
>
> FYI, newer linecard models from BROCADE can hold 2 million routes.
> Probably others can do that now too.
>
> Disclaimer : I'm not working for them or defending them, just setting an
> information straight.
>
> My 2 cents.
>
>
>
> > Le 3 oct. 2015 à 10:33, Jürgen Jaritsch <j...@anexia.at> a écrit :
> >
> > As mentioned before: even the new SUP2T from Cisco is limited to 1Mio
> routes ...
> >
> > There are MANY other vendors with the same limitations: Juniper,
> Brocade, etc
> >
> > And the solt equipment is not the 99USD trash from the super market at
> the corner ...
> >
> >
> > Jürgen Jaritsch
> > Head of Network & Infrastructure
> >
> > ANEXIA Internetdienstleistungs GmbH
> >
> > Telefon: +43-5-0556-300
> > Telefax: +43-5-0556-500
> >
> > E-Mail: j...@anexia.at
> > Web: http://www.anexia.at
> >
> > Anschrift Hauptsitz Klagenfurt: Feldkirchnerstraße 140, 9020 Klagenfurt
> > Geschäftsführer: Alexander Windbichler
> > Firmenbuch: FN 289918a | Gerichtsstand: Klagenfurt | UID-Nummer: AT
> U63216601
> >
> >
> > -Original Message-
> > From: Max Tulyev [max...@netassist.ua]
> > Received: Samstag, 03 Okt. 2015, 9:11
> > To: nanog@nanog.org [nanog@nanog.org]
> > Subject: Re: AW: /27 the new /24
> >
> > Which routers? DIR-300 with OpenWRT/Quagga? :)
> >
> > I think all above-the-trash level routers supports >1M routes, isn't it?
> >
> >> On 02.10.15 17:45, Jürgen Jaritsch wrote:
> >> Hi,
> >>
> >> this would at least help to get rid of many old routing engines around
> the world :) ... or people would keep their "learn nothing smaller than
> /24" filters in place. Also an option - but not for companies who act as an
> IP transit provider.
> >>
> >>
> >> best regards
> >>
> >> Jürgen Jaritsch
> >> Head of Network & Infrastructure
> >>
> >> ANEXIA Internetdienstleistungs GmbH
> >>
> >> Telefon: +43-5-0556-300
> >> Telefax: +43-5-0556-500
> >>
> >> E-Mail: jjarit...@anexia-it.com
> >> Web: http://www.anexia-it.com
> >>
> >> Anschrift Hauptsitz Klagenfurt: Feldkirchnerstraße 140, 9020 Klagenfurt
> >> Geschäftsführer: Alexander Windbichler
> >> Firmenbuch: FN 289918a | Gerichtsstand: Klagenfurt | UID-Nummer: AT
> U63216601
> >>
> >>
> >> -Ursprüngliche Nachricht-
> >> Von: NANOG [mailto:nanog-boun...@nanog.org] Im Auftrag von Justin
> Wilson - MTIN
> >> Gesendet: Freitag, 02. Oktober 2015 16:32
> >> An: NANOG
> >> Betreff: /27 the new /24
> >>
> >> I was in a discussion the other day and several Tier2 providers were
> talking about the idea of adjusting their BGP filters to accept prefixes
> smaller than a /24.  A few were saying they thought about going down to as
> small as a /27.  This was mainly due to more networks coming online and not
> having even a /24 of IPv4 space.  The first argument is against this is the
> potential bloat the global routing table could have.  Many folks have
> worked hard for years to summarize and such. others were saying they would
> do a /26 or bigger.
> >>
> >> However, what do we do about the new networks which want to do BGP but
> only can get small allocations from someone (either a RIR or one of their
> upstreams)?
> >>
> >> Just throwing that out there. Seems like an interesting discussion.
> >>
> >>
> >> Justin Wilson
> >> j...@mtin.net
> >>
> >> ---
> >> http://www.mtin.net Owner/CEO
> >> xISP Solutions- Consulting – Data Centers - Bandwidth
> >>
> >> http://www.midwest-ix.com  COO/Chairman
> >> Internet Exchange - Peering - Distributed Fabric
> >
>

Re: AW: /27 the new /24

[Youssef Bengelloun-Zahr]

Hi,

FYI, newer linecard models from BROCADE can hold 2 million routes. Probably 
others can do that now too.

Disclaimer : I'm not working for them or defending them, just setting an 
information straight.

My 2 cents.



> Le 3 oct. 2015 à 10:33, Jürgen Jaritsch <j...@anexia.at> a écrit :
> 
> As mentioned before: even the new SUP2T from Cisco is limited to 1Mio routes 
> ...
> 
> There are MANY other vendors with the same limitations: Juniper, Brocade, etc
> 
> And the solt equipment is not the 99USD trash from the super market at the 
> corner ...
> 
> 
> Jürgen Jaritsch
> Head of Network & Infrastructure
> 
> ANEXIA Internetdienstleistungs GmbH
> 
> Telefon: +43-5-0556-300
> Telefax: +43-5-0556-500
> 
> E-Mail: j...@anexia.at
> Web: http://www.anexia.at
> 
> Anschrift Hauptsitz Klagenfurt: Feldkirchnerstraße 140, 9020 Klagenfurt
> Geschäftsführer: Alexander Windbichler
> Firmenbuch: FN 289918a | Gerichtsstand: Klagenfurt | UID-Nummer: AT U63216601
> 
> 
> -Original Message-
> From: Max Tulyev [max...@netassist.ua]
> Received: Samstag, 03 Okt. 2015, 9:11
> To: nanog@nanog.org [nanog@nanog.org]
> Subject: Re: AW: /27 the new /24
> 
> Which routers? DIR-300 with OpenWRT/Quagga? :)
> 
> I think all above-the-trash level routers supports >1M routes, isn't it?
> 
>> On 02.10.15 17:45, Jürgen Jaritsch wrote:
>> Hi,
>> 
>> this would at least help to get rid of many old routing engines around the 
>> world :) ... or people would keep their "learn nothing smaller than /24" 
>> filters in place. Also an option - but not for companies who act as an IP 
>> transit provider.
>> 
>> 
>> best regards
>> 
>> Jürgen Jaritsch
>> Head of Network & Infrastructure
>> 
>> ANEXIA Internetdienstleistungs GmbH
>> 
>> Telefon: +43-5-0556-300
>> Telefax: +43-5-0556-500
>> 
>> E-Mail: jjarit...@anexia-it.com
>> Web: http://www.anexia-it.com
>> 
>> Anschrift Hauptsitz Klagenfurt: Feldkirchnerstraße 140, 9020 Klagenfurt
>> Geschäftsführer: Alexander Windbichler
>> Firmenbuch: FN 289918a | Gerichtsstand: Klagenfurt | UID-Nummer: AT U63216601
>> 
>> 
>> -Ursprüngliche Nachricht-
>> Von: NANOG [mailto:nanog-boun...@nanog.org] Im Auftrag von Justin Wilson - 
>> MTIN
>> Gesendet: Freitag, 02. Oktober 2015 16:32
>> An: NANOG
>> Betreff: /27 the new /24
>> 
>> I was in a discussion the other day and several Tier2 providers were talking 
>> about the idea of adjusting their BGP filters to accept prefixes smaller 
>> than a /24.  A few were saying they thought about going down to as small as 
>> a /27.  This was mainly due to more networks coming online and not having 
>> even a /24 of IPv4 space.  The first argument is against this is the 
>> potential bloat the global routing table could have.  Many folks have worked 
>> hard for years to summarize and such. others were saying they would do a /26 
>> or bigger.
>> 
>> However, what do we do about the new networks which want to do BGP but only 
>> can get small allocations from someone (either a RIR or one of their 
>> upstreams)?
>> 
>> Just throwing that out there. Seems like an interesting discussion.
>> 
>> 
>> Justin Wilson
>> j...@mtin.net
>> 
>> ---
>> http://www.mtin.net Owner/CEO
>> xISP Solutions- Consulting – Data Centers - Bandwidth
>> 
>> http://www.midwest-ix.com  COO/Chairman
>> Internet Exchange - Peering - Distributed Fabric
> 

Re: AW: /27 the new /24

[Randy Bush]

> It's not too far off though. One way of looking at it is, for each
> extra bit we allow, we potentially double the table size.

that is math.  in reality

table size is proportional to

multihoming + traffic engineering

randy

Re: /27 the new /24

[Owen DeLong]

The race is on…

One or more of these things will be the death of IPv4:

1.  Not enough addresses
2.  Routing Table Bloat due to one or more of:
A.  Traffic Engineering
B.  Stupid configuration
C.  Address Fragmentation through transfers and 
microallocations of “Transition space”
D.  Stupid network designs (where the physical deployment 
forces something like B rather than just software stupidity)
3.  Some highly desirable feature or application which simply can’t 
be implemented effectively on IPv4.

As such, I say go ahead and route whatever. ARIN provides a nice constrained 
prefix for these transitional allocations
so you can at least limit the bloat from that factor, but even at the /24 
level, we’re faced with upwards of 16 million routes
that we are nowhere near ready to handle.

In case you haven’t been paying attention for the last 25 years, there are a 
number of ways in which IPv4 DOES NOT SCALE.

We’ve been keeping it alive on life support for a VERY LONG time. As with all 
patients on life support, the longer it continues,
the more it becomes a losing battle and the harder (and more resource intensive 
and more expensive) it becomes to keep
the patient alive.

Fortunately, in this case, we have a nice new body that we can transplant the 
internet into that has many fruitful years ahead
of it. So… Do whatever you have to to survive in the meantime, but focus on 
getting your stuff onto the IPv6 internet so that
we can all let IPv4 go gently into that good night and have it’s well deserved 
final rest.

Owen

> On Oct 3, 2015, at 06:18 , Baldur Norddahl  wrote:
> 
> Except we might very well reach 1+ million routes soon without accepting
> longer prefixes than /24. Also route updates is a concern - do I really
> need to be informed every time someone on the other end of the world resets
> a link?
> 
> On 3 October 2015 at 12:57, William Waites  wrote:
> 
>> On Sat, 3 Oct 2015 12:42:01 +0200, Baldur Norddahl <
>> baldur.nordd...@gmail.com> said:
>> 
>>> 2 million routes will not be enough if we go full /27. This is
>>> not a scalable solution. Something else is needed to provide
>>> multihoming for small networks (LISP?).
>> 
>> It's not too far off though. One way of looking at it is, for each
>> extra bit we allow, we potentially double the table size. So with 500k
>> routes and a /24 limit now, we might expect 4 million with /27. Not
>> exactly because it depends strongly on the distribution of prefix
>> lengths, but probably not a bad guess.
>> 
>> Also there are optimisations that I wonder if the vendors are doing to
>> preserve TCAM such as aggregating adjacent networks with the same next
>> hop into the supernet. That would mitigate the impact of wanton
>> deaggregation at least and the algorithm doesn't look too hard. Do the
>> big iron vendors do this?
>> 
>> -w
>> 
>> --
>> William Waites   |  School of Informatics
>>   http://tardis.ed.ac.uk/~wwaites/   | University of Edinburgh
>> https://hubs.net.uk/ |  HUBS AS60241
>> 
>> The University of Edinburgh is a charitable body, registered in
>> Scotland, with registration number SC005336.
>> 

Re: AW: /27 the new /24

[Max Tulyev]

Which routers? DIR-300 with OpenWRT/Quagga? :)

I think all above-the-trash level routers supports >1M routes, isn't it?

On 02.10.15 17:45, Jürgen Jaritsch wrote:
> Hi,
> 
> this would at least help to get rid of many old routing engines around the 
> world :) ... or people would keep their "learn nothing smaller than /24" 
> filters in place. Also an option - but not for companies who act as an IP 
> transit provider.
> 
> 
> best regards
> 
> Jürgen Jaritsch
> Head of Network & Infrastructure
> 
> ANEXIA Internetdienstleistungs GmbH
> 
> Telefon: +43-5-0556-300
> Telefax: +43-5-0556-500
> 
> E-Mail: jjarit...@anexia-it.com 
> Web: http://www.anexia-it.com 
> 
> Anschrift Hauptsitz Klagenfurt: Feldkirchnerstraße 140, 9020 Klagenfurt
> Geschäftsführer: Alexander Windbichler
> Firmenbuch: FN 289918a | Gerichtsstand: Klagenfurt | UID-Nummer: AT U63216601
> 
> 
> -Ursprüngliche Nachricht-
> Von: NANOG [mailto:nanog-boun...@nanog.org] Im Auftrag von Justin Wilson - 
> MTIN
> Gesendet: Freitag, 02. Oktober 2015 16:32
> An: NANOG
> Betreff: /27 the new /24
> 
> I was in a discussion the other day and several Tier2 providers were talking 
> about the idea of adjusting their BGP filters to accept prefixes smaller than 
> a /24.  A few were saying they thought about going down to as small as a /27. 
>  This was mainly due to more networks coming online and not having even a /24 
> of IPv4 space.  The first argument is against this is the potential bloat the 
> global routing table could have.  Many folks have worked hard for years to 
> summarize and such. others were saying they would do a /26 or bigger.  
> 
> However, what do we do about the new networks which want to do BGP but only 
> can get small allocations from someone (either a RIR or one of their 
> upstreams)?
> 
> Just throwing that out there. Seems like an interesting discussion.
> 
> 
> Justin Wilson
> j...@mtin.net
> 
> ---
> http://www.mtin.net Owner/CEO
> xISP Solutions- Consulting – Data Centers - Bandwidth
> 
> http://www.midwest-ix.com  COO/Chairman
> Internet Exchange - Peering - Distributed Fabric
> 

Mikrotik in the DFZ (Was Re: AW: AW: /27 the new /24)

[William Waites]

On Fri, 2 Oct 2015 23:11:47 +, Jürgen Jaritsch  said:

> Regarding the words "I have a small router which handles
> multiple full tables ...": push and pull a few full tables at
> the same time and you'll see what's happening: the CCRs are
> SLOW. And why? Because the software is not as good as it could
> be: the BGP daemon uses only one core of a 36(?) core CPU.

To expand on this, the problem is worse than being single-threaded. I
had one of these in the lab and fed it 2x full tables. Sure it wasn't
the fastest at accepting them but then I noticed that even in steady
state one of the CPUs was pegged. What was happening -- and this was
confirmed by Mikrotik -- was that it was recalculating the *entire*
FIB for each update. The general background noise of announce /
withdraw messages means it is doing this all the time. Any churn and
it would have a very hard time.

There are other serious bugs such as not doing recursive next hop
lookup for IPv6 (it does for IPv4). This makes them unuseable as BGP
routers even for partial tables with most non-trivial iBGP
topologies. All of which may be fixed one day in version 7 of their
operating system, which will inevitably have many bugs as any software
project .0 release will, so we'll have to wait for 7.x for it to be
reasonably safe to use.

That said, we use a lot of Mikrotik kit for our rural
networks. They're weird and quirky but you can't beat them on price,
port density and power consumption. With 16 ports and 36 cores surely 
they should be capable of pushing several Gbps of traffic with a few
full tables.

I wish it were possible today to run different software on their
larger boxes. If some like-minded small providers wanted to get
together with us to fund a FreeBSD port to the CCR routers that would
be great. Please contact me off-list if you are interested in this,
I'll coordinate.

As it is we don't let them anywhere near the DFZ, that's done with PCs
running FreeBSD and BIRD which can easily do the job but is still an
order of magnitude more expensive (and an order of magnitude less
expensive than what you need if you want 10s of Gbps).

-w

--
William Waites   |  School of Informatics
   http://tardis.ed.ac.uk/~wwaites/   | University of Edinburgh
 https://hubs.net.uk/ |  HUBS AS60241

The University of Edinburgh is a charitable body, registered in
Scotland, with registration number SC005336.


pgpkqJVGALdoZ.pgp
Description: PGP signature

RE: AW: /27 the new /24

[Jürgen Jaritsch]

As mentioned before: even the new SUP2T from Cisco is limited to 1Mio routes ...

There are MANY other vendors with the same limitations: Juniper, Brocade, etc

And the solt equipment is not the 99USD trash from the super market at the 
corner ...


Jürgen Jaritsch
Head of Network & Infrastructure

ANEXIA Internetdienstleistungs GmbH

Telefon: +43-5-0556-300
Telefax: +43-5-0556-500

E-Mail: j...@anexia.at
Web: http://www.anexia.at

Anschrift Hauptsitz Klagenfurt: Feldkirchnerstraße 140, 9020 Klagenfurt
Geschäftsführer: Alexander Windbichler
Firmenbuch: FN 289918a | Gerichtsstand: Klagenfurt | UID-Nummer: AT U63216601


-Original Message-
From: Max Tulyev [max...@netassist.ua]
Received: Samstag, 03 Okt. 2015, 9:11
To: nanog@nanog.org [nanog@nanog.org]
Subject: Re: AW: /27 the new /24

Which routers? DIR-300 with OpenWRT/Quagga? :)

I think all above-the-trash level routers supports >1M routes, isn't it?

On 02.10.15 17:45, Jürgen Jaritsch wrote:
> Hi,
>
> this would at least help to get rid of many old routing engines around the 
> world :) ... or people would keep their "learn nothing smaller than /24" 
> filters in place. Also an option - but not for companies who act as an IP 
> transit provider.
>
>
> best regards
>
> Jürgen Jaritsch
> Head of Network & Infrastructure
>
> ANEXIA Internetdienstleistungs GmbH
>
> Telefon: +43-5-0556-300
> Telefax: +43-5-0556-500
>
> E-Mail: jjarit...@anexia-it.com
> Web: http://www.anexia-it.com
>
> Anschrift Hauptsitz Klagenfurt: Feldkirchnerstraße 140, 9020 Klagenfurt
> Geschäftsführer: Alexander Windbichler
> Firmenbuch: FN 289918a | Gerichtsstand: Klagenfurt | UID-Nummer: AT U63216601
>
>
> -Ursprüngliche Nachricht-
> Von: NANOG [mailto:nanog-boun...@nanog.org] Im Auftrag von Justin Wilson - 
> MTIN
> Gesendet: Freitag, 02. Oktober 2015 16:32
> An: NANOG
> Betreff: /27 the new /24
>
> I was in a discussion the other day and several Tier2 providers were talking 
> about the idea of adjusting their BGP filters to accept prefixes smaller than 
> a /24.  A few were saying they thought about going down to as small as a /27. 
>  This was mainly due to more networks coming online and not having even a /24 
> of IPv4 space.  The first argument is against this is the potential bloat the 
> global routing table could have.  Many folks have worked hard for years to 
> summarize and such. others were saying they would do a /26 or bigger.
>
> However, what do we do about the new networks which want to do BGP but only 
> can get small allocations from someone (either a RIR or one of their 
> upstreams)?
>
> Just throwing that out there. Seems like an interesting discussion.
>
>
> Justin Wilson
> j...@mtin.net
>
> ---
> http://www.mtin.net Owner/CEO
> xISP Solutions- Consulting – Data Centers - Bandwidth
>
> http://www.midwest-ix.com  COO/Chairman
> Internet Exchange - Peering - Distributed Fabric
>

/27 the new /24

[Justin Wilson - MTIN]

I was in a discussion the other day and several Tier2 providers were talking 
about the idea of adjusting their BGP filters to accept prefixes smaller than a 
/24.  A few were saying they thought about going down to as small as a /27.  
This was mainly due to more networks coming online and not having even a /24 of 
IPv4 space.  The first argument is against this is the potential bloat the 
global routing table could have.  Many folks have worked hard for years to 
summarize and such. others were saying they would do a /26 or bigger.  

However, what do we do about the new networks which want to do BGP but only can 
get small allocations from someone (either a RIR or one of their upstreams)?

Just throwing that out there. Seems like an interesting discussion.


Justin Wilson
j...@mtin.net

---
http://www.mtin.net Owner/CEO
xISP Solutions- Consulting – Data Centers - Bandwidth

http://www.midwest-ix.com  COO/Chairman
Internet Exchange - Peering - Distributed Fabric

AW: /27 the new /24

[Jürgen Jaritsch]

Hi,

this would at least help to get rid of many old routing engines around the 
world :) ... or people would keep their "learn nothing smaller than /24" 
filters in place. Also an option - but not for companies who act as an IP 
transit provider.


best regards

Jürgen Jaritsch
Head of Network & Infrastructure

ANEXIA Internetdienstleistungs GmbH

Telefon: +43-5-0556-300
Telefax: +43-5-0556-500

E-Mail: jjarit...@anexia-it.com 
Web: http://www.anexia-it.com 

Anschrift Hauptsitz Klagenfurt: Feldkirchnerstraße 140, 9020 Klagenfurt
Geschäftsführer: Alexander Windbichler
Firmenbuch: FN 289918a | Gerichtsstand: Klagenfurt | UID-Nummer: AT U63216601


-Ursprüngliche Nachricht-
Von: NANOG [mailto:nanog-boun...@nanog.org] Im Auftrag von Justin Wilson - MTIN
Gesendet: Freitag, 02. Oktober 2015 16:32
An: NANOG
Betreff: /27 the new /24

I was in a discussion the other day and several Tier2 providers were talking 
about the idea of adjusting their BGP filters to accept prefixes smaller than a 
/24.  A few were saying they thought about going down to as small as a /27.  
This was mainly due to more networks coming online and not having even a /24 of 
IPv4 space.  The first argument is against this is the potential bloat the 
global routing table could have.  Many folks have worked hard for years to 
summarize and such. others were saying they would do a /26 or bigger.  

However, what do we do about the new networks which want to do BGP but only can 
get small allocations from someone (either a RIR or one of their upstreams)?

Just throwing that out there. Seems like an interesting discussion.


Justin Wilson
j...@mtin.net

---
http://www.mtin.net Owner/CEO
xISP Solutions- Consulting – Data Centers - Bandwidth

http://www.midwest-ix.com  COO/Chairman
Internet Exchange - Peering - Distributed Fabric

Re: /27 the new /24

[William Herrin]

On Fri, Oct 2, 2015 at 10:32 AM, Justin Wilson - MTIN  wrote:
> However, what do we do about the new networks which
> want to do BGP but only can get small allocations from
> someone (either a RIR or one of their upstreams)?

Hi Justin,

Rent or sell them a /24 and make money. If they can't afford a /24 at
today's market rate, why should the rest of us spend much more money
upgrading routers to accommodate their advertisement?

The annual systemic cost of carrying that prefix is still more than
double the one-time cost of acquiring a /24. No doubt that gap will
close, but there's no cost justification to change the /24 filters
just yet.

Regards,
Bill Herrin



-- 
William Herrin  her...@dirtside.com  b...@herrin.us
Owner, Dirtside Systems . Web: 

Re: /27 the new /24

[Mike Hammett]

Much m ore than I'm willing to spend. ;-) 




- 
Mike Hammett 
Intelligent Computing Solutions 
http://www.ics-il.com 



Midwest Internet Exchange 
http://www.midwest-ix.com 


- Original Message -

From: "Matthew Kaufman" <matt...@matthew.at> 
To: "Justin Wilson - MTIN" <li...@mtin.net> 
Cc: "NANOG" <nanog@nanog.org> 
Sent: Friday, October 2, 2015 9:48:33 AM 
Subject: Re: /27 the new /24 

A /24 isn't that expensive yet... 

Matthew Kaufman 

(Sent from my iPhone) 

> On Oct 2, 2015, at 7:32 AM, Justin Wilson - MTIN <li...@mtin.net> wrote: 
> 
> I was in a discussion the other day and several Tier2 providers were talking 
> about the idea of adjusting their BGP filters to accept prefixes smaller than 
> a /24. A few were saying they thought about going down to as small as a /27. 
> This was mainly due to more networks coming online and not having even a /24 
> of IPv4 space. The first argument is against this is the potential bloat the 
> global routing table could have. Many folks have worked hard for years to 
> summarize and such. others were saying they would do a /26 or bigger. 
> 
> However, what do we do about the new networks which want to do BGP but only 
> can get small allocations from someone (either a RIR or one of their 
> upstreams)? 
> 
> Just throwing that out there. Seems like an interesting discussion. 
> 
> 
> Justin Wilson 
> j...@mtin.net 
> 
> --- 
> http://www.mtin.net Owner/CEO 
> xISP Solutions- Consulting – Data Centers - Bandwidth 
> 
> http://www.midwest-ix.com COO/Chairman 
> Internet Exchange - Peering - Distributed Fabric 
> 

Re: /27 the new /24

[Mike Hammett]

How many routers out there have this limitation? A $100 router I bought ten 
years ago could manage many full tables. If someone's network can't match that 
today, should I really have any pity for them? 




- 
Mike Hammett 
Intelligent Computing Solutions 
http://www.ics-il.com 



Midwest Internet Exchange 
http://www.midwest-ix.com 


- Original Message -

From: "Matthew Kaufman" <matt...@matthew.at> 
To: "Mike Hammett" <na...@ics-il.net> 
Cc: "NANOG" <nanog@nanog.org> 
Sent: Friday, October 2, 2015 10:48:29 AM 
Subject: Re: /27 the new /24 

Cheaper than buying everyone TCAM 

Matthew Kaufman 

(Sent from my iPhone) 

> On Oct 2, 2015, at 8:32 AM, Mike Hammett <na...@ics-il.net> wrote: 
> 
> Much m ore than I'm willing to spend. ;-) 
> 
> 
> 
> 
> - 
> Mike Hammett 
> Intelligent Computing Solutions 
> http://www.ics-il.com 
> 
> 
> 
> Midwest Internet Exchange 
> http://www.midwest-ix.com 
> 
> 
> - Original Message - 
> 
> From: "Matthew Kaufman" <matt...@matthew.at> 
> To: "Justin Wilson - MTIN" <li...@mtin.net> 
> Cc: "NANOG" <nanog@nanog.org> 
> Sent: Friday, October 2, 2015 9:48:33 AM 
> Subject: Re: /27 the new /24 
> 
> A /24 isn't that expensive yet... 
> 
> Matthew Kaufman 
> 
> (Sent from my iPhone) 
> 
>> On Oct 2, 2015, at 7:32 AM, Justin Wilson - MTIN <li...@mtin.net> wrote: 
>> 
>> I was in a discussion the other day and several Tier2 providers were talking 
>> about the idea of adjusting their BGP filters to accept prefixes smaller 
>> than a /24. A few were saying they thought about going down to as small as a 
>> /27. This was mainly due to more networks coming online and not having even 
>> a /24 of IPv4 space. The first argument is against this is the potential 
>> bloat the global routing table could have. Many folks have worked hard for 
>> years to summarize and such. others were saying they would do a /26 or 
>> bigger. 
>> 
>> However, what do we do about the new networks which want to do BGP but only 
>> can get small allocations from someone (either a RIR or one of their 
>> upstreams)? 
>> 
>> Just throwing that out there. Seems like an interesting discussion. 
>> 
>> 
>> Justin Wilson 
>> j...@mtin.net 
>> 
>> --- 
>> http://www.mtin.net Owner/CEO 
>> xISP Solutions- Consulting – Data Centers - Bandwidth 
>> 
>> http://www.midwest-ix.com COO/Chairman 
>> Internet Exchange - Peering - Distributed Fabric 
> 

Re: /27 the new /24

[Matthew Kaufman]

Cheaper than buying everyone TCAM

Matthew Kaufman

(Sent from my iPhone)

> On Oct 2, 2015, at 8:32 AM, Mike Hammett <na...@ics-il.net> wrote:
> 
> Much m ore than I'm willing to spend. ;-) 
> 
> 
> 
> 
> - 
> Mike Hammett 
> Intelligent Computing Solutions 
> http://www.ics-il.com 
> 
> 
> 
> Midwest Internet Exchange 
> http://www.midwest-ix.com 
> 
> 
> - Original Message -
> 
> From: "Matthew Kaufman" <matt...@matthew.at> 
> To: "Justin Wilson - MTIN" <li...@mtin.net> 
> Cc: "NANOG" <nanog@nanog.org> 
> Sent: Friday, October 2, 2015 9:48:33 AM 
> Subject: Re: /27 the new /24 
> 
> A /24 isn't that expensive yet... 
> 
> Matthew Kaufman 
> 
> (Sent from my iPhone) 
> 
>> On Oct 2, 2015, at 7:32 AM, Justin Wilson - MTIN <li...@mtin.net> wrote: 
>> 
>> I was in a discussion the other day and several Tier2 providers were talking 
>> about the idea of adjusting their BGP filters to accept prefixes smaller 
>> than a /24. A few were saying they thought about going down to as small as a 
>> /27. This was mainly due to more networks coming online and not having even 
>> a /24 of IPv4 space. The first argument is against this is the potential 
>> bloat the global routing table could have. Many folks have worked hard for 
>> years to summarize and such. others were saying they would do a /26 or 
>> bigger. 
>> 
>> However, what do we do about the new networks which want to do BGP but only 
>> can get small allocations from someone (either a RIR or one of their 
>> upstreams)? 
>> 
>> Just throwing that out there. Seems like an interesting discussion. 
>> 
>> 
>> Justin Wilson 
>> j...@mtin.net 
>> 
>> --- 
>> http://www.mtin.net Owner/CEO 
>> xISP Solutions- Consulting – Data Centers - Bandwidth 
>> 
>> http://www.midwest-ix.com COO/Chairman 
>> Internet Exchange - Peering - Distributed Fabric
>