Re: [ipv6-wg] [official] What Shall This WG Do

[Lee Howard]

On 10/8/19 12:20 PM, Wolfgang Zenker wrote:

Hi Jen and group,

* Jen Linkova  [191005 04:20]:
to hurt. The working group should provide group b) with guidance how
to make at least their outward facing services available on dual stack.


I've started a Google sheet with links to hosting providers' 
instructions for turning on IPv6:


https://docs.google.com/spreadsheets/d/1gpHAfvsE_1mNcdHbrj1Uc4YFr0XspX76dfXGRA6eDFc/edit#gid=0

Make a comment (or otherwise contact me) with additional providers and 
links. I didn't make it globally writeable to avoid vandalism.


This is based on my blog post, which includes editorializing on some 
providers: https://www.retevia.net/ipv6-on/


I'm sure there are tutorials on enabling IPv6 on Apache, nginx, 
sendmail, Exchange, etc., firewalls, load balancers, etc., not to 
mention NAT64, SIIT-DC.


Lee



Wolfgang

Re: [ipv6-wg] Have we failed as IPv6 Working Group?

[Lee Howard]

On 10/5/19 12:06 PM, Dave Taht wrote:

Lee Howard  writes:



IPv6 deployment in your network means cutting your NAT expense in
half. More, as more sites deploy.

There's no fungable or measureable "NAT expense". NAT generally saves
money. Look at the container market for one recent example.


If you have a dedicated box or service card for NAT, you have a specific 
NAT expense. If you can buy a smaller box for half the cost, you've 
saved a specific amount of money. I know of mobile carriers who NATted 
every Internet connection in IPv4, and have halved their capital expense 
on NAT.




IPv6 deployment in your network might mean you can sell some of your
IPv4 addresses, a clever way I've seen to fund the transition.

That I agree with. But I see no sign anyone is actually doing that. It's
saner to horde at the moment.


I've had inquiries along these lines, and written proposals.



IPv6 deployment on your web site means improving your page load time,
and therefore SEO, and therefore revenue. At NANOG I showed quotes
that IPv6 increases revenue by 0.2%-7%.[1]

BS. Happy eyeballs costs time.


Here's my assessment: https://www.retevia.net/why-is-ipv6-faster/


The cost to deploy IPv6 is not high: it's mostly labor, and people who

BS. It needs to be implemented first in a deployable state.


"Deployable"? It's been deployed by the millions. Everyone who's done it 
says it was mostly labor.



complain that there's no training are ignoring the hundreds of
tutorials, books, articles, videos, and web sites available to them
for free, not to mention the thousands of friendly engineers.

To everyone who sees a high cost, I ask whether you know the value of
NAT reduction and web site speed (and avoiding buying addresses, or

I note that port exaustion is a real thing on ipv4 networks today that
more should measure. In one recent set of "coffee shop tests", I had an
over 30% initial syn failure rate. I don't know why (we were also
testing ecn) at the moment, but that was a shocking number.

ipv4 dns with udp was already using up a lot of udp port space.
with quic eating up a lot of udp more, I'm not happy.

JUST deploying dns over IPv6 as I did


Interesting!





selling addresses), in $LOCAL_CURRENCY, so you can evaluate every
obstacle you might encounter. For instance, "Our web conferencing
doesn't support IPv6, and it'll cost us $9,000 a year to change. But
IPv6 will save us $30,000." The decision is easy.

That last number is pure BS. It's not a single cost. It's that last
dangling set of apps that can't be converted to ipv6 that's the infinite
cost.


I'm not sure what you're calling "BS." Obviously $30,000 was a made-up 
number for the example. I don't know what "dangling set of apps" you 
mean that can't be converted, especially in the context of "How much 
would it cost to add IPv6 support?"



In another message on this thread I noted that small ISPs are squeezed
between CPE and IPv4 purchases. They can't get CPE that supports IPv6,
or that supports MAP or 464xlat, because they don't buy enough, so
they have to pay to buy addresses.

They can't get cheap CPE that has those features. ALL that code runs
great in openwrt.

And ipv4 addresses are needed until ipv6 hits 100% deployment.


Fewer IPv4 addresses are needed in the context of MAP or 464xlat.


That's easily solved by collective
action: 100 small ISPs can get the features they want (at a better
discount) than one acting alone.

Great. Is there an ISP association trying to do that already?


Not that I know of. If there's interest, I'll support, maybe organize.

Lee

Re: [ipv6-wg] Have we failed as IPv6 Working Group?

[Lee Howard]

On 10/5/19 11:53 AM, Dave Taht wrote:

Lee Howard  writes:


On 10/4/19 4:55 PM, Dave Taht wrote:

not being able to get a
static IPv6 address out of comcast, my hurricane tunnel getting blocked
by netflix, the still-huge prefix sub-distribution problem. The idea of
dynamic 2 week prefixes in part of the world prone to earthquakes
doesn't work for me...

I can think of several programmatic ways to deal with that.

...

for real use a static ipv6/48 to distribute is needed. Dynamic ipv6
assignments are fine if you are doing trivial stuff but if ipv6 is ever
to even start to supplant ipv4 it's got to become more static.


Or more resilient to renumbering. I didn't mean to suggest that you as a 
user should have to code your home network. I meant that we (maybe IETF) 
have not done well at handling the case of "I received a new prefix, and 
I am the edge router; I need to tell everything else what changed." Thus 
kicking off RAs, DHCPv6 updates, dDNS updates, and a firewall that 
understands dynamic prefixing.




I was thinking about some Hackathon projects to add IPv6 capability to
open source projects. Seems to me the hardest part is making sure
there's an adequate test environment.

Hackathons ARE useful tools for getting a short burst of focused work
out of people sharing the same space and time, but too many are thinking
hackathons alone will solve more detailed design, coding and iteration
problems; it's one of those ideas trivializing the costs of "Real
Programming(tm)" that really bugs me nowadays.

I could share here the detailed project management stuff that went into
cerowrt's run (3 years), or the outline of work we did for
make-wifi-fast - which we've now been at for over 5 years now - 3+ to
get fq_codel to work right on wifi, 2 to rework the API to work for more
devices, and a pointer to the latest work which has been going for 3+
months now - and for all that we've only accomplished about 1/10th what
we wanted to do, and only on 4 chipsets (most recently intel's ax200
chips) out of the hundreds.


This might make for an interesting WG discussion, getting actual work 
organized and done.




But Mr.Rey's reference about IPv6 deployment rates also makes a good point!

Nobody cares about deployment rates. What good does it do, if people don't use 
it ?
This is more realistic : https://www.google.com/intl/en/ipv6/statistics.html
During the week, we are below 25%.

(Replying to an item upthread)

APNIC's statistics show that in almost every network that has IPv6, it
is almost always used.

I pointed to coffee shops as one counter example.

If they don't have IPv6, they're not using it.

  To the lack of
DHCPv6-PD on android (and I think, IOS) for tethering as another.


Based on discussions at IETF, I guess Android is expecting to get 
multiple IPv6 addresses and reassign as needed. Don't know about IOS.




Another thought I've had:

One of the reasons small ISPs can't deploy IPv6 is that they don't
control the features in the CPE, because they don't buy enough.

I know a couple CPE vendors who would be happy to provide a specific
feature set for a guaranteed purchase of a couple thousand units a
month. This sounds like a good business to me: if a bunch of small
ISPs each contract for a specific number of units, but require
RIPE-554, RFC7084, and RFC8085, we could both get the needed features,
and get a larger volume discount than they get now.

Yes, the smaller ISPs should join together in a buying
club like that. Tried to get that going in NZ once. Failed.

tried harder to make the aftermarket do the right thing - the eeros and
google wifi's of the world are doing ok, the bottom part of the market
just copy/pastes whatever's in openwrt at that moment, slaps a label
on it and ships it. So we focused on making the openwrt base as good
as possible.

Might be another part of "What the WG should do" discussion.

Saving $1 per CPE is better than spending $20 for an IPv4 address for
every new user. Please confirm my math. :)

I always thought that ISPs would invest in their CPE far more than they
have. Free.fr being a shining example! ISPs get paid for modem rentals
and have customer support costs that could be reduced - that should have
been a great ongoing funding source and motivation, alone.

but I know a few vendors, like evenroute, doing bufferbloat AND ipv6
right, that have totally failed to crack ISP market thus far.

and for no reason I can think of, the rental folk don't push out
new hardware OR new software to their users - I think charter made
an effort to get docsis 3.1 stuff out there and retire all the docsis
2.0 gear in place, but not comcast.

Secondly none of those ipv6 standards help when you still really need a
real IPv4 address, so yer still out the $20, IF you can buy the /24s you
need. And there's more ipv6 RFCs left without runn

Re: [ipv6-wg] Have we failed as IPv6 Working Group?

[Lee Howard]

Replying to the Subject question. . .

The WG has done good work. RIPE-554 in particular I think is good work.

This WG isn't a marketing organization. That's appropriate.

One way to look at the problem is that IPv4 exhaustion is a problem of 
externalities: my network is growing, and it costs me more now because 
you don't support IPv6. As an economic problem, then, think about how to 
shift your costs to those who have only IPv4.


IPv6 deployment in your network means cutting your NAT expense in half. 
More, as more sites deploy.


IPv6 deployment in your network might mean you can sell some of your 
IPv4 addresses, a clever way I've seen to fund the transition.


IPv6 deployment on your web site means improving your page load time, 
and therefore SEO, and therefore revenue. At NANOG I showed quotes that 
IPv6 increases revenue by 0.2%-7%.[1]


The cost to deploy IPv6 is not high: it's mostly labor, and people who 
complain that there's no training are ignoring the hundreds of 
tutorials, books, articles, videos, and web sites available to them for 
free, not to mention the thousands of friendly engineers.


To everyone who sees a high cost, I ask whether you know the value of 
NAT reduction and web site speed (and avoiding buying addresses, or 
selling addresses), in $LOCAL_CURRENCY, so you can evaluate every 
obstacle you might encounter. For instance, "Our web conferencing 
doesn't support IPv6, and it'll cost us $9,000 a year to change. But 
IPv6 will save us $30,000." The decision is easy.


In another message on this thread I noted that small ISPs are squeezed 
between CPE and IPv4 purchases. They can't get CPE that supports IPv6, 
or that supports MAP or 464xlat, because they don't buy enough, so they 
have to pay to buy addresses. That's easily solved by collective action: 
100 small ISPs can get the features they want (at a better discount) 
than one acting alone.


In the mean time, this WG keeps having fascinating presentations, which 
I keep using when talking about IPv6 to enterprise IT departments. Keep 
it up.



Lee


[1] https://youtu.be/aTi4fia5s-k?t=4386

Re: [ipv6-wg] Have we failed as IPv6 Working Group?

[Lee Howard]

On 10/4/19 4:55 PM, Dave Taht wrote:

not being able to get a
static IPv6 address out of comcast, my hurricane tunnel getting blocked
by netflix, the still-huge prefix sub-distribution problem. The idea of
dynamic 2 week prefixes in part of the world prone to earthquakes
doesn't work for me...


I can think of several programmatic ways to deal with that. Or you can 
just buy Comcast's business service, which I think includes one IPv4 
address.





that said, we need more running code, still, which only then can
get into a deployment, and nobody's funding that.


Do you mean CeroWRT specifically, or code in general?

I was thinking about some Hackathon projects to add IPv6 capability to 
open source projects. Seems to me the hardest part is making sure 
there's an adequate test environment.





But Mr.Rey's reference about IPv6 deployment rates also makes a good point!

Nobody cares about deployment rates. What good does it do, if people don't use 
it ?
This is more realistic : https://www.google.com/intl/en/ipv6/statistics.html
During the week, we are below 25%.


(Replying to an item upthread)

APNIC's statistics show that in almost every network that has IPv6, it 
is almost always used.




One entertaining thing I've been up to is checking the state of multiple
kinds of deployment in the coffee shops of the world with a string
of simple tests anyone can do (after we package them up better)


Yeah, we need the GoGo and ATTWifi and such of the world to deploy.



Since there was demand for more IPv4, perhaps that would also fuel
more updates to ipv6, as
both require middlebox updates...

As for money to make middleboxes better in *any* way, don't make me
laugh. During the cerowrt project we approached everybody making money
from the internet and multiple non-profits and got nowhere. I spent
my own fortune on it, and got a lot of volunteers onboard, especially
in the openwrt universe... and made things better, but I got nothing left.

We need a new kame-like project to jointly handle the cracks in the
ipv6 network architecture, standards and code, at the very least.

The costs of "mo ipv4" are trivial in comparison.


Another thought I've had:

One of the reasons small ISPs can't deploy IPv6 is that they don't 
control the features in the CPE, because they don't buy enough.


I know a couple CPE vendors who would be happy to provide a specific 
feature set for a guaranteed purchase of a couple thousand units a 
month. This sounds like a good business to me: if a bunch of small ISPs 
each contract for a specific number of units, but require RIPE-554, 
RFC7084, and RFC8085, we could both get the needed features, and get a 
larger volume discount than they get now.


Saving $1 per CPE is better than spending $20 for an IPv4 address for 
every new user. Please confirm my math. :)






3 months ago, I turned DECNET off on my network. It was actually not
even an IT/network decision; customer decided they were done with a
product, and we de-commissioned the tools with DECNET. Business
decision. We run OS/2 Warp, MS-DOS, Windows 95, HPUX, Solaris, Windows
2000, and I probably forget some.

Please note the ipv4 extensions stuff won't work with most that
"legacy" ipv4 stuff.
It can, however, enable new applications and services to exist. Most of
the IOT and SDN stacks already do work. Most don't have decent ipv6 support
due to resource constraints.

Perversely I kind of like the idea of a portion of the internet immune from
legacy windows worms and viruses


DECNET isn't on the Internet. I don't care if some crusty old boxes in 
dark corners of data centers whisper IPv4 among themselves. How would I 
even know?






In 20 years, I will still need IPv4.

And it seems possible we can make more.


And I have enough IPv4 on my hands for the foreseeable future. I bought some 
recently, just in case.


I encourage the WG group to read this :
https://www.internetgovernance.org/2019/02/20/report-on-ipv6-get-ready-for-a-mixed-internet-world/
And the full text :
https://www.internetgovernance.org/wp-content/uploads/IPv6-Migration-Study-final-report.pdf
Serious work, paid by ICANN.

We cited that work in our presos on this subject as that was also key
on gilmore, paul wouters and myself to start looking hard at what it
would take to make ipv4 better in multiple ways. Please look it over!?

The ipv4 unicast extensions project is one outgrowth of that: A string
of trivial patches to a couple OSes and routing daemons and we're well
on our way to being able to add 420m new addresses to the internet,
within a 10 year time horizon.


You just mentioned your un-upgradable "OS/2 Warp, MS-DOS, Windows 95, 
HPUX, Solaris, Windows 2000," and now you say it's easy to upgrade.


Lee

Re: [ipv6-wg] IPv6 Implementation for an ISP

[Lee Howard]

If you have enough IPv4 address space, the easiest way to do IPv6 is to 
go dual-stack (both IPv4 and IPv6 to every customer), and then work on 
the transition technologies.


Here's my "Learn IPv6 before lunch" reading list: 
https://www.retevia.net/recommended-reading-on-ipv6/


Here's a high-level guide for ISPs: 
https://datatracker.ietf.org/doc/rfc6782/


You may want to think about this as if your goal is run an IPv6-only 
access network, with translation to IPv4 at the edges where necessary. 
Let customers run dual-stack inside the home; so if a customer has 
devices in their home (alarm systems, smart TVs) that require IPv4, the 
home gateway can translate from IPv4 to IPv6, and if the destination is 
IPv4-only, you have a translator in your data center or peering point 
translating from IPv6 to IPv4.


Jordi like 464xlat, but I like MAP-T (or MAP-E). Dual-stack Lite is also 
an option. In all of those cases, the hardest part seems to be getting 
support in the CPE. Here's a very short description of each: 
https://www.retevia.net/what-we-offer/


My experience is that most of your time will be spent working on your 
provisioning, monitoring, and support systems. Equipment generally 
supports IPv6, and you'll just have to read the manuals to find the 
right commands.



Lee


On 3/11/19 12:41 PM, Alessandro Valenza wrote:


Hello,

this is the first time I write in this mailing list.

I have a question for you.

I’m working for an ISP that obtained IPv4 (/22) and IPv6 (/29). I 
would like to use IPv6. Could you suggest me some documents and some 
manuals useful to implement IPv6, but avoiding problems for our 
customers?


In particular, I would like to avoid compatibility problems with VoIP 
PBXs, network printers, alarm systems, etc ...


In Italy only few ISPs provide IPv6 addresses and we would like to try 
to reverse this negative trend compared to the rest of Europe.


I hope you can help me understanding the advantages and opportunities 
in investing in the direction of IPv6.


Kind regards

Alessandro

Alessandro Valenza​
Ufficio TLC



Tel:*0571 1738257* 
Fax:*0571542536* 

*www.enegan.it* 

CONFIDENZIALITÀ
​Questo messaggio, inclusi gli eventuali allegati, è indirizzato solo 
ai destinatari e può contenere informazioni riservate
​e confidenziali.​    Se avete ricevuto il messaggio senza esserne un 
destinatario, o un suo rappresentante autorizzato, la
​presente comunicazione vale come formale divieto di diffusione del 
contenuto del messaggio.​Se avete ricevuto il ​messaggio
​per errore,  siete pregati di cancellarlo, assieme a tutti gli 
allegati, dal vostro sistema e di informare

​immediatamente il mittente.

[ipv6-wg] IPv6 operations discussions

[Lee Howard]

The IETF v6ops working group is chartered to improve the operation of IPv6.
We have several active documents right now that would benefit from broader
operator feedback. For instance, there is current active discussion on:

Requirements for IPv6 Routers

This document provides a list of requirements for operators’ routers. Is it
clear and exhaustive?

Basic Requirements for IPv6 Customer Edge Routers

What transition technologies should all CPE vendors put in their devices?
The current version proposes all of: 464xlat, DS-Lite, lw4o6, MAP-E, MAP-T,
6in4, and 6rd, as well as IPv4 multicast over IPv6.
There are related documents that split out some requirements for further
discussion.

Happy Eyeballs Version 2: Better Connectivity Using Concurrency

Is this a useful update to the existing Happy Eyeballs specification
(rfc6555)? Should we update Happy Eyeballs?
  
Considerations For Using Unique Local Addresses

Are ULAs useful, or are they a natural and risky predecessor to IPv6 NAT?


It would help future IPv6 operations if current operators would read these
documents and comment on them on the mailing list:
https://www.ietf.org/mailman/listinfo/v6ops
Note Well that comments become part of the IETF record, and thank you for
them.


Lee
v6ops co-chair