Re: SixXS shutting down 2017-06-06

[Jeroen Massar]

On 2017-03-23 18:28, David Farmer wrote:
> 
> 
> On Thu, Mar 23, 2017 at 10:46 AM, Brian E Carpenter
> > wrote
>  
> 
> One detail not mentioned in the /sunset page. Will you leave the
> ULA tool and registry in place (https://www.sixxs.net/tools/grh/ula/
> ) ?
> I believe some people like that.
> 
>Brian
> 
> 
> I'd be interested in some data on the use of the ULA tool and registry,
> I'd especially be interested in use over time.  Is use of the ULA
> registry increasing or decreasing over the last few years?

That is an easy question to answer:

$ SELECT YEAR(ula_date) AS Year, COUNT(*) AS Count FROM grh_ulas GROUP
BY YEAR(ula_date);
+--+---+
| Year | Count |
+--+---+
| 2007 |63 |
| 2008 |   140 |
| 2009 |   321 |
| 2010 |   611 |
| 2011 |   835 |
| 2012 |   742 |
| 2013 |   724 |
| 2014 |  1096 |
| 2015 |  1303 |
| 2016 |   640 |
| 2017 |   143 |
+--+---+
11 rows in set (0.00 sec)

I would say that it is going down if we look at that count ;)

> Basically,
> is there an argument for further or new work within the IPv6 community
> on this front?

>From my POV not really. It is extremely simple to get a prefix from one
of the RIRs. Yes, it costs some money, which is something that should be
addressed IMHO. (routing gear etc costs money too though).

Also, more importantly, ULA is random per definition, and the chance of
collisions is extremely low. (unless one does not use randomness).

> Or, should this service just be sustained as-is, maybe
> finding a new home or new support over the long-term?  Or, should this
> service also be sunset, maybe not on the same timeframe as the other
> SixXS services?
> 
> Personally, if anything, I like to see some new work here, but I'd like
> to drive what that is or should be with some data. 

Like with many things, I first would ask: what are the
requirements/usecases/etc.

> Finally, many thanks to SixXS for their years of service to the IPv6
> community!  And, kudos for planning an orderly sunset, rather than
> decaying into oblivion.   

We have been warning people since December 2015. Hopefully since then
they actually called their ISP or changed to ones that support IPv6.

Greets,
 Jeroen

Re: high v6 user-base, less atlas probes

[Jeroen Massar]

On 2017-01-28 13:51, Bajpai, Vaibhav wrote:
> Hello,
> 
> Here is a list of top 20 countries that have a large v6 user-base [*] but are
> underrepresented by RIPE Atlas due to less number of deployed dual-stacked
> probes. Consider hosting a dual-stacked probe in these regions to help remove
> the region-based bias that v6 studies using RIPE Atlas may otherwise have:
> 
> Full paper available here: http://goo.gl/hWGveA (§4)
> 
>   +++---|
>   | CC | v6 Users   | v6 Probes |
> |-+++---|
> |  01 | BE | 56.48 %| 2.82  %   |
> |  02 | LU | 35.23 %| 0.61  %   |
> |  03 | GR | 33.66 %| 1.39  %   |
> |  04 | CH | 33.48 %| 3.82  %   |

Would be useful to detail which ASNs these users are supposed to live in
and state if that ASN has probes or not...

Or otherwise said: instead of doing per-country, you might just want to
single out the ASNs where you are "missing" probes...

One approach, especially in Europe and the rest of RIPE NCC country: ask
every LIR / ASN to run at least a probe (and not from their datacenter,
but from a enduser connection ;) )

Greets,
 Jeroen

Re: macos Sierra with CGA address?

[Jeroen Massar]

On 2016-12-14 13:26, Jeroen Massar wrote:
> On 2016-12-14 13:10, Jeroen Massar wrote:
>> [..]
>>> Actually, it is not a stable address as some have found out (read:
>>> anecdotal), they also change at re-install and there are a couple of
>>> other possibilities from what I recall.
>>
>> From xnu-3248.60.10/bsd/netinet6/in6_ifattach.c:
> 
> And after finding 10.12.1 sources (note that is not 10.12.2 which is out):
> 
> There is a lot more SEND/CGA/"LLCGA" related stuff in here:
> 
> https://opensource.apple.com/source/xnu/xnu-3789.21.4/bsd/netinet6/in6_ifattach.c.auto.html
> 
> eg:
> 
> in6_ifattach_llcgareq has:
> 
>   if (nd6_send_opstate == ND6_SEND_OPMODE_DISABLED)
>   return (ENXIO);
> 
> Thus maybe one can disable it, if sysctl.conf has it disabled at
> boot.. lets find out :)

$ sysctl -a |grep opmode
net.inet6.send.opmode: 0
$ sysctl -a |grep opstate
net.inet6.send.opstate: 0
$ sysctl -a |grep maxif
net.inet6.ip6.maxifprefixes: 16
net.inet6.ip6.maxifdefrouters: 16

and have nice static or DHCP-assigned addresses without the silly
randomness...

Thus summary:

$ cat /etc/sysctl.conf
net.inet6.ip6.prefer_tempaddr=0
net.inet6.ip6.use_tempaddr=0
net.inet6.send.opmode=0

First too disable silly random addresses (same /48, thus how many people
can that be);

net.inet6.send.opmode=0 disables the CGA-Link-Local-Address and thus
also the global random address.

Noting that this works on 10.12.2 (
root:xnu-3789.31.2~1/RELEASE_X86_64), but as per code likely also in
10.12 aka MacOS Sierra.

Iljitsch, maybe a good thing to update on:
 
http://arstechnica.com/apple/2016/09/macos-10-12-sierra-the-ars-technica-review/6/#h3
 ?

that it can be disabled this way :)

Greets,
 Jeroen
  (now pushing out new sysctl.conf to all other boxes ;)

Re: macos Sierra with CGA address?

[Jeroen Massar]

On 2016-12-14 13:10, Jeroen Massar wrote:
> [..]
>> Actually, it is not a stable address as some have found out (read:
>> anecdotal), they also change at re-install and there are a couple of
>> other possibilities from what I recall.
> 
> From xnu-3248.60.10/bsd/netinet6/in6_ifattach.c:

And after finding 10.12.1 sources (note that is not 10.12.2 which is out):

There is a lot more SEND/CGA/"LLCGA" related stuff in here:

https://opensource.apple.com/source/xnu/xnu-3789.21.4/bsd/netinet6/in6_ifattach.c.auto.html

eg:

in6_ifattach_llcgareq has:

if (nd6_send_opstate == ND6_SEND_OPMODE_DISABLED)
return (ENXIO);

Thus maybe one can disable it, if sysctl.conf has it disabled at
boot.. lets find out :)

Greets,
 Jeroen

Re: macos Sierra with CGA address?

[Jeroen Massar]

On 2016-12-14 12:40, Jeroen Massar wrote:
[..]
>> There is another sysctl parameter (opmode) but unclear what 1 (or 0) means:
>> $ sysctl net.inet6.send
>> net.inet6.send.opstate: 1
>> net.inet6.send.opmode: 1
> 
> There is no documentation at all about these things, hence, nothing one
> can say about it, except begging Apple to finally document stuff.

Maybe this is the 'closest' to documentation: the source:

https://github.com/opensource-apple/xnu/blob/27ffc00f33925b582391b1ef318b78b8bd3939d1/bsd/netinet6/nd6_send.c

This is an older dump though.

Not a single mention of a RFC in there, but they are talking about
"SEND" (Not SeND") and there are CGA parameters in there which are not
exposed.

Apparently one needs "#if CONFIG_MACF" which we do not have...

Hence, I can only guess that that code is not active yet, at least in
that ancient version.

Thus lets peek at:
https://opensource.apple.com/tarballs/xnu/xnu-3248.60.10.tar.gz

mmm while I have:
 root:xnu-3789.21.3~60/RELEASE_X86_64 x86_64

Thus 500 something behind

In that tarball, look at bsd/netinet6/

Same file still there. Thus: nothing changed in that timeframe
concerning send.

Though, as various of the dumps show, opmode/state has changed to '1'
somewhere.

Maybe this is thus part of those last 500 changes?


in6.c has:
8<---
if (optdad) {
if ((optdad & ND6_OPTIMISTIC_DAD_LINKLOCAL) &&
IN6_IS_ADDR_LINKLOCAL(>ia_addr.sin6_addr))
flags = IN6_IFF_OPTIMISTIC;
else if ((optdad & ND6_OPTIMISTIC_DAD_AUTOCONF) &&
(ia->ia6_flags & IN6_IFF_AUTOCONF)) {
if (ia->ia6_flags & IN6_IFF_TEMPORARY) {
if (optdad & ND6_OPTIMISTIC_DAD_TEMPORARY)
flags = IN6_IFF_OPTIMISTIC;
} else if (ia->ia6_flags & IN6_IFF_SECURED) {
if (optdad & ND6_OPTIMISTIC_DAD_SECURED)
flags = IN6_IFF_OPTIMISTIC;
} else {
/*
 * Keeping the behavior for temp and CGA
 * SLAAC addresses to have a knob for
optimistic
 * DAD.
 * Other than that if
ND6_OPTIMISTIC_DAD_AUTOCONF
 * is set, we should default to optimistic
 * DAD.
 * For now this means SLAAC addresses
with interface
 * identifier derived from modified
EUI-64 bit
 * identifiers.
 */
flags = IN6_IFF_OPTIMISTIC;
}
>8

Which is likely related but that was also in the old github edition
already...

Hence, we are still none the wiser here.

Greets,
 Jeroen

Re: macos Sierra with CGA address?

[Jeroen Massar]

On 2016-12-14 12:25, Holger Zuleger wrote:
> Hi Jeroen,
> 
>>> I found two or three posts in the internet, all mentioning (or hoping)
>>> that this is related to a change to RFC7217 as default IID mechanism.
>>>
>>> But one guy sad, that the source code (of 10.11) shows, that this is a
>>> cryptographic generated interface identifier for SeND (RFC3971).
>>>
>>> I tend to believe that the latter is true.
>>
>> Seeing how Apple implemented things like "Happy Eyeballs" it likely is
>> neither. And in the case of "Happy Eyeballs" there is no way to turn it
>> off either. Filing radar bugs clearly does not help as they never get
>> addressed or marked as 'dupe' at which point you do not know the status
>> of the 'original' problem and well, nothing happens...
> 
> 
>>> Has anyone more information about this? Especially how to configure it?
>>
>> The only trick I found out was:
>>
>> https://twitter.com/tweetsix/status/77861562571649
>> 8<---
>> Also who has typed: "sudo sysctl -w net.inet6.ip6.maxifprefixes=1" (or
>> stored the setting in /etc/sysctl.conf) recently? ;)
>> ->8
> To be honest, that's definitively is not the way I like to go.
> 
>> As then you only get the DHCPd address (requires DHCPv6 server) on
>> your interface and not all the other magic ones that change all the time
>> and are extremely useless if you want to ADDRESS a host...
>> (yes, I love VNC'ing, SSH'ing and doing SSH-backups of my boxes...)
> Oh no, DHCPv6 is not needed here.

Until Sierra, I didn't have any DHCPv6 either... but now I do because I
really love my static and known addresses. People know I have a Mac
anyway, thus what info am I losing there?

> The problem is *not* that this IID is changing. It is a stable one. And
> yes, I vote not against temporary addresses.

Actually, it is not a stable address as some have found out (read:
anecdotal), they also change at re-install and there are a couple of
other possibilities from what I recall.

Unfortunately, documentation about all of this is completely lacking...

The maxifprefixes does what it needs to do for me: only allow using of
DHCPv6'd addresses.

But indeed, that was a pure guess to enable that.

>> There are claimed 'good' properties of a changing address but mostly
>> they are useless: "it works against tracking" which is useless if your
>> /48 is static and there are only ~10 hosts in that prefix that call
>> outbound. Also, something with HTTP Cookies for 99% of the other things.
>> And I am really not lugging my 27" iMac around to get it in another
>> network
>>
>> Hence, a switch to turn if off would be amazing.
>> The above trick kinda does that though and it mostly seem to work.
> My info is, to set
>   sysctl -w net.inet6.send.opstate=0
> to go back to mac address based eui64, but didn't checked it.
> 
> There is another sysctl parameter (opmode) but unclear what 1 (or 0) means:
> $ sysctl net.inet6.send
> net.inet6.send.opstate: 1
> net.inet6.send.opmode: 1

There is no documentation at all about these things, hence, nothing one
can say about it, except begging Apple to finally document stuff.

google("net.inet6.send.opstate") has 76 hits, going back to 2014-ish,
and those switches where in El Capitan (10.9) already.

The change with random addresses came with Sierra though as Iljitsch
wrote here:

http://arstechnica.com/apple/2016/09/macos-10-12-sierra-the-ars-technica-review/6/#h3

Greets,
 Jeroen

Re: macos Sierra with CGA address?

[Jeroen Massar]

On 2016-12-14 11:55, Holger Zuleger wrote:
> Hi,
> 
> I just realized that the permanent interface identifier of my MAC has
> changed after upgrading to OS 10.12 (I guess).
> 
> The output of ifconfig shows a new "secured" flag at the permanent address.
> $ ifconfig en0 | grep inet6 | \
>>   sed "s/2[^:]*:[^:]*:[^:]*:[^:]*:/:/"
> inet6 fe80::c54:6333:ac12:c67b%en0 prefixlen 64 secured scopeid 0x4
> inet6 :20e3:84f6:6794:5ace prefixlen 64 autoconf secured
> inet6 :8822:a8a3:b6ec:a79b prefixlen 64 autoconf temporary
> 
> I found two or three posts in the internet, all mentioning (or hoping)
> that this is related to a change to RFC7217 as default IID mechanism.
> 
> But one guy sad, that the source code (of 10.11) shows, that this is a
> cryptographic generated interface identifier for SeND (RFC3971).
> 
> I tend to believe that the latter is true.

Seeing how Apple implemented things like "Happy Eyeballs" it likely is
neither. And in the case of "Happy Eyeballs" there is no way to turn it
off either. Filing radar bugs clearly does not help as they never get
addressed or marked as 'dupe' at which point you do not know the status
of the 'original' problem and well, nothing happens...

> Has anyone more information about this? Especially how to configure it?

The only trick I found out was:

https://twitter.com/tweetsix/status/77861562571649
8<---
Also who has typed: "sudo sysctl -w net.inet6.ip6.maxifprefixes=1" (or
stored the setting in /etc/sysctl.conf) recently? ;)
->8

As then you only get the DHCPd address (requires DHCPv6 server) on
your interface and not all the other magic ones that change all the time
and are extremely useless if you want to ADDRESS a host...
(yes, I love VNC'ing, SSH'ing and doing SSH-backups of my boxes...)


There are claimed 'good' properties of a changing address but mostly
they are useless: "it works against tracking" which is useless if your
/48 is static and there are only ~10 hosts in that prefix that call
outbound. Also, something with HTTP Cookies for 99% of the other things.
And I am really not lugging my 27" iMac around to get it in another
network


Hence, a switch to turn if off would be amazing.
The above trick kinda does that though and it mostly seem to work.


With the trend that Apple is taking with their hardware line that
actually runs OSX ehmm MacOS, it seems that swapping platforms is a
likely next step for a lot of people...

2016 was supposed to be the year of IPv6 (201_IPv_6), which indeed
worked a wee bit, but maybe 2026 is the actual goal of many companies;
thus maybe 2017 is going to be the year of Linux on the Desktop? :)


Anybody already coded up a Little Snitch equiv for OpenBSD? :)

Greets,
 Jeroen

Re: Linux and ULA support and default route

[Jeroen Massar]

On 2016-10-13 09:36, Tore Anderson wrote:
> * Jeroen Massar <jer...@massar.ch>
> 
>> RA's only install the /64 and when default announced a default.
>>
>> Thus 'the rest of the ULA /48' would require a default route to be
>> installed to reach that...
>>
>> When the device does not install a default route, there won't be an
>> entry for anything in that /48, just the /64 and thus that space won't
>> be reachable.
> 
> Not if you set the accept_ra_rt_info_max_plen sysctl to >= 48 (and the
> router implements RFC 7084 L-3).

Which is why looking at the exact RA is important.

> As far as I know, this sysctl is 0 by
> default which causes the kernel to ignore RIOs.

Correct. See among st others here for reasoning:
https://patchwork.ozlabs.org/patch/331802/

and as it is default 0, unless one does already salt/puppet/etc to
change that default, it won't easily get deployed; and if one does
salt/puppet/etc then adding static routes can also work.

Much easier and better to let the actual routers decide on routing
though and not end-hosts.

> I believe that Windows do accept RIOs by default so that's probably why
> it worked for Brian. NetworkManager user-space RA processing will also
> respect RIOs by default.

Only silly people run broken software like "NetworkManager" ;)

Greets,
 Jeroen

Re: Linux and ULA support and default route

[Jeroen Massar]

On 2016-10-13 02:30, Brian E Carpenter wrote:
> Hi Jeroen,
> On 13/10/2016 12:16, Jeroen Massar wrote:
>> On 2016-10-13 00:51, Brian E Carpenter wrote:
>> [..]
>>> Kernel IPv6 routing table
>>> DestinationNext Hop   Flag Met Ref Use 
>>> If
>>> fd00::/64  fe80::be05:43ff:fe8e:ce39  UG   600 112 
>>> wlp2s0
>>> fe80::/64  :: U256 0 0 
>>> wlp2s0
>>> ::/0   :: !n   -1  1   137 
>>> lo
>>> ::1/128:: Un   0   3 7 
>>> lo
>>> fd00::c5bb:40f2:f3d5:94e4/128  :: Un   0   319 
>>> lo
>>> fe80::9051:543a:4c9e:e93e/128  :: Un   0   211 
>>> lo
>>> ff00::/8   :: U256 2  1763 
>>> wlp2s0
>>> ::/0   :: !n   -1  1   137 
>>> lo
>>
>> Do you receive those prefixes over RA or manual config?
> 
> RA of course
> 
>> Is forwarding enabled? 
> 
> No
> 
>> What does the ra_accept sysctl say?
> 
> accept_ra = 1
> 
>>
>> Also 'ip -6 ro get ' can be very useful to check where the
>> routing table thinks packets are supposed to go.
> 
> Well, once I create the default route it tells me exactly what it should,
> for any global-scope address. But after reboot it says "unreachable"
> for any address outside the ULA /64 (i.e. even the rest of the ULA /48
> is unreachable).

RA's only install the /64 and when default announced a default.

Thus 'the rest of the ULA /48' would require a default route to be
installed to reach that...

When the device does not install a default route, there won't be an
entry for anything in that /48, just the /64 and thus that space won't
be reachable.


Btw: IMHO ULAs are in 99% of the cases the wrong thing to use anyway.
But note, this is not specific to ULA at all. (Except maybe that your
device chose to not push a default route, as there is no default route
to the Internet).

You might want to check with tcpdump with the exact details of the RA are.

Greets,
 Jeroen

Re: Linux and ULA support and default route

[Jeroen Massar]

On 2016-10-13 02:05, Lorenzo Colitti wrote:
> On Wed, Oct 12, 2016 at 3:51 PM, Brian E Carpenter
> > wrote:
> 
> ::/0   :: !n   -1 
> 1   137 lo
> 
> 
> I think !n means network unreachable. Please provide the output of "ip
> -6 route".

That is indeed the default unreachable route, basically the root node of
the prefix tree ;)

Hence indeed why one should be using 'ip -6 ro sho' to check for the
actual routes, netstat output is just odd.

If only the system was using DHCPv6 eh ;)

Greets,
 Jeroen

Re: Linux and ULA support and default route

[Jeroen Massar]

On 2016-10-13 00:51, Brian E Carpenter wrote:
[..]
> Kernel IPv6 routing table
> DestinationNext Hop   Flag Met Ref Use If
> fd00::/64  fe80::be05:43ff:fe8e:ce39  UG   600 112 
> wlp2s0
> fe80::/64  :: U256 0 0 
> wlp2s0
> ::/0   :: !n   -1  1   137 lo
> ::1/128:: Un   0   3 7 lo
> fd00::c5bb:40f2:f3d5:94e4/128  :: Un   0   319 lo
> fe80::9051:543a:4c9e:e93e/128  :: Un   0   211 lo
> ff00::/8   :: U256 2  1763 
> wlp2s0
> ::/0   :: !n   -1  1   137 lo

Do you receive those prefixes over RA or manual config?
Is forwarding enabled? What does the ra_accept sysctl say?

Also 'ip -6 ro get ' can be very useful to check where the
routing table thinks packets are supposed to go.

In general on a Linux install from the last decade or so, avoid
'netstat' and 'ifconfig' and use iproute: 'ip ro sho' or 'ip -6 ro sho',
'ip -6 addr show'

Greets,
 Jeroen

Re: SV: CPE Residential IPv6 Security Poll

[Jeroen Massar]

On 2016-09-21 13:49, Benedikt Stockebrand wrote:
[..]
> There's a fairly large SIP operator (sipgate) here in Germany who for
> quite some time has told people that their service not working over
> DS-Lite was entirely a problem between the customer and their ISP,
> giving technical reasons you can quite likely figure out yourself.  With
> DS-Lite gaining more and more of a foothold here---and at least one
> major ISP slipstreaming that on existing lines without notifying the
> customers---technical explanations are exactly not what to tell people
> whose phones suddenly stopped working.
> 
> Once you screw your customer relation up with this sort of stunt it
> takes a lot of time (and marketing) to fix that up again.

sipgate messed up by not upgrading to IPv6 (though, yeah find an IPv6
SIP capable device, they are quite rare ;) which they knew was coming
and could have solved, as that clearly is a business case
(still waiting for Gigaset to make IPv6 upgrades...)

... this while Liberty Global is abusing their monopoly in all of Europe
by forcing people (without notification or contract change; well they
did remove the word "IPv4" from their new contracts at one point) onto
DS-Lite because "we are out of IPv4" while their business customers, who
are paying significantly more, cannot even get IPv6 even though they are
asking for it.

Oh, and yes, poor people who don't get proper IPv4 anymore and are still
waiting for Sony to make a move to IPv6; though apparently at least IPv6
addresses are now being configured since 4.00; that quite breaks
multiplayer games though...

It is sad that people didn't bother to listen and that literally
thousands are now noticing how crappy the industry handled this
"transition" to IPv6, as many ISPs seem to make it a flag day: one day
you have IPv4, the other you have broken IPv4 + 'working' IPv6...


The major mistake that ISPs are making here btw is marketing:
  they are not informing their users

nor did they ask (or look with netflow) who are using IPv4 in a way that
would not work with the AFTR stuff they just push onto them.

I guess the loss of customers (for the few who have the choice to
change, many are stuck in monopoly situations) or the amount of support
desk calls is less cost than the money expected to be made by selling
IPv4 service to other parts of the company.

Sad that the Internet is so commercial and not about letting people
communicate... :(

Greets,
 Jeroen

Re: pxe boot for ipv6 only machine not working on vmware

[Jeroen Massar]

On 2016-07-01 13:41, Benedikt Stockebrand wrote:
> Hi Shilpa,
> 
> "S, Shilpashree 1. (Nokia - IN/Bangalore)" 
> writes:
> 
>> We have a case to install machine with ipv6 only using pxe boot.
> 
> bad news for you: PXE as per the Intel specs is IPv4 only.  Intel wants
> people to move from BIOS (and as such, PXE as a BIOS extension) to UEFI
> and has shown absolutely no intention to add IPv4 support to PXE.
> 
> Your options are
> 
>   - to use UEFI with a suitable module to boot via IPv6, which pretty
> much defeats your use case for PXE to install the machine,

https://wiki.kubuntu.org/UEFI/SecureBoot-PXE-IPv6

Similar instructions have worked as a charm for years already

Greets,
 Jeroen

Re: v6 naming and shaming - *.europa.eu

[Jeroen Massar]

On 2016-05-18 16:10, Phil Mayers wrote:
> On 18/05/16 15:03, Jeroen Massar wrote:
> 
>> The best advice for getting IPv6 fixed is for a large well used network
>> (google, facebook) to stop providing IPv4. Then suddenly people will fix
>> things as they won't have working "Internet" and their users will
>> complain really really loud.
> 
> Ok so basically, if more/most access networks were IPv6-enabled (because
> big or vital providers are IPv6 only) then all service networks would
> have to get it working?

Then they have a REAL reason: complaining users who cannot Google/Facebook.

And that is for many ISPs what it will take for them to even remotely
think of IPv6, see again:
 https://www.sixxs.net/wiki/?title=Call_Your_ISP_for_IPv6

> Not unreasonable, but that's a very long term prospect I guess.

It *IS* unreasonable.

As when such an event happens, it will have to be done in 1 day instead
of the 20 years that they already knew this was going to happen.

Also, likely such an event will not happen, as the establish "big"
players have more than enough IPv4 to last their lifetimes putting up
big load balancer farms.

The BIG problem there is newcomers to the market, they won't be able to
enter as they won't get any IPv4.

> I'd be curious to know if people have suggestions that work shorter term.

Please read through every IPv6-related list from the last 20 years.

People have been trying to convince folks for a while already...

> I'm in agreement that shaming is not effective; but I'm frustrated and
> it just seemed so ironic that their public claims were so pro-v6.

I've never been frustrated about this. I only see that folks doing
consulting on the subject will have a lot of work at one point.

> Question for any access network providers: if/when you run into these
> issues, how do you plan to proceed? Leave the site broken and force the
> site owner to fix, or work around at your end and hide the problem?

They will likely set up a big HTTP proxy and then tunnel IPv6 from
somewhere...

Please note that this list is the wrong audience, and so is any other
list where folks know about IPv6. These folks already have IPv6.

If they do not have IPv6 it is because of some "C-level" "business
decision" to not look into it.

You cannot fix those folks unfortunately. The only thing one can do is
"I told you so" and laugh very very hard when they turn over their cash
to the consultants and companies taking over their networks...

Greets,
 Jeroen

Re: v6 naming and shaming - *.europa.eu

[Jeroen Massar]

On 2016-05-18 15:52, Phil Mayers wrote:
> On 18/05/16 14:29, Jeroen Massar wrote:
> 
>> Really, you cannot keep on telling people to finally deploy IPv6, it
>> does not have any effect whatsoever, only their pocket books care and
>> those will only notice when it is too late...
> 
> So it's hopeless and we should just give up?

You can keep on trying to fix OTHER people's networks but you'll end
up in an abyss at one point...

> That doesn't seem like the most encouraging advice ever, but thanks for
> the reply.

The best advice for getting IPv6 fixed is for a large well used network
(google, facebook) to stop providing IPv4. Then suddenly people will fix
things as they won't have working "Internet" and their users will
complain really really loud.

Till that happens do not hold your breath.

> Anyone else got thoughts on how to discourage half-working/half-broken
> setups which create negative externalities?

Public shaming does not work, that has been tried for a long long time
already.

Contacting people who do not care about their own network does not work
either.

> I'm specifically not asking about encouraging people who haven't
> deployed; rather people who have and who have broken or abandoned their
> efforts.

Understand it this way: they officially claimed 12 years ago to be
launching IPv6 and they have not noticed their own network to be broken...

Technical contacts are badly published and likely won't reply.

Thus... little chance to fix a network that does not want to be reached.

Yes, that is unfortunate, but that is the way it seems to be.


I'll add to that that in the cases of Viruses/Bots and Spam many
networks are already big blackholes for getting these resolved. Either
you do not find a contact or they won't fix it even when they have read
the message. IPv6 is not on these network's priority lists at all...

Greets,
 Jeroen

Re: v6 naming and shaming - *.europa.eu

[Jeroen Massar]

On 2016-05-18 15:23, Phil Mayers wrote:
> Broken over IPv6:
> 
> https://webcast.ec.europa.eu/281715cafa675bf359ebaa42cb44fa17
> 
> (Webserver has , returns 404 over v6, fine over v4)
> 
> And yet:
> 
> https://ec.europa.eu/digital-single-market/en/blog/ipv6-more-than-a-reality-a-necessity
>

You are aware that the EU "launched" IPv6 in 2004 right:

 http://www.global-ipv6.org/index.htm

Not that many of their sites/locations actually have even remotely heard
of IPv6.

Problems like this are simply still existent because they do not care,
and they will not.

See also the comments in this little wiki page:
https://www.sixxs.net/wiki/?title=Call_Your_ISP_for_IPv6

and unfortunately there are many more of even employees at ISPs who are
unable to convince their ISP that they really should be doing IPv6.

I wonder when the first large companies are going to ask for some kind
of 'fund' for getting IPv6 deployed


Really, you cannot keep on telling people to finally deploy IPv6, it
does not have any effect whatsoever, only their pocket books care and
those will only notice when it is too late...

Greets,
 Jeroen

Re: push apps failing in Android until you disable IPv6

[Jeroen Massar]

On 2016-05-09 19:28, Lorenzo Colitti wrote:
> On Mon, May 9, 2016 at 6:17 PM, Tore Anderson  > wrote:
> 
> Even a ULA PIO could be problematic if Android's source
> address selection algorithm isn't updated to RFC6724 defaults. RFC3484
> predates ULAs, so it treats them the same as other globally scoped
> addresses.
> 
> 
> Android's source address selection was updated to RFC 6724 in early 2013
> [commit
> ].
> I think that went into 4.3.

And how many devices will thus actually receive such an update? 3%?

Oh, not too much off:

http://www.statista.com/statistics/271774/share-of-android-platforms-on-mobile-devices-with-android-os/

4.3 has 2.9% of the market, same amount as people still running 2.3.x...
but fortunately apparently some people have been replacing devices with
4.4, at 32%, not as bad as for actual security issues that are still
available on all those platforms though. And that is all assuming it
really landed in 4.3 and not in a 5.x branch

Now to hope that one day DHCPv6 gets added and then the long long
tail of having all those ancient versions going extinct.

Would be so amazing if Google forced vendors to actually support their
devices for a long time and otherwise not allow those devices to use
Google services, would be a great first step... though more important
for security than for IPv6...

Greets,
 Jeroen

Re: push apps failing in Android until you disable IPv6

[Jeroen Massar]

[yep, this is an off-topic comment, but partially it isn't as it does
affect deployability of IPv6 on a device that was build even as shortly
as a year ago...]

On 2016-05-01 03:20, Erik Kline wrote:
> A bug report, especially from a Marshmallow (or even N Developer
> Preview) device could be helpful.

Now if only all Android devices in the field could be actually upgraded
to the newest editions ;)

Indeed, not much Google can do about unless they start requiring that
(guaranteed updates in under a month for a device that has a max age of
say 5 years) from the vendors that use Android... that would be an
amazing step for Internet stability and more importantly security...

Don't think that will ever happen though as there is no money in that,
selling a new device is a great step though...

Greets,
 Jeroen

Re: Slow WiFi with Android Marshmallow & IPv6?

[Jeroen Massar]

On 2016-04-27 09:59, Mikael Abrahamsson wrote:
> On Tue, 26 Apr 2016, Jeroen Massar wrote:
> 
>> specifically a 'router' but a CPE (that for IPv4 NATs and for IPv6 sorta
> 
> Let me just say that when I used "CPE" in a document, I was reminded by
> BBF (BroadBand Forum) people that they use "CPE" as "any device in the
> home". Their terminology for the first router in the home is "RG" as in
> "Residential Gateway".

That is a better definition indeed.

CPE is Customer Premises Equipment, thus that indeed would cover everything.

The problem with "Residential" though is that not every such device is
in a residence, "Customer Gateway" would be better IMHO... then again
those would be quite new terms.

There should be an IETF RFC for those things somewhere just like
SHOULD/MUST ;)

Greets,
 Jeroen

Re: Slow WiFi with Android Marshmallow & IPv6?

[Jeroen Massar]

On 2016-04-26 10:52, Thomas Schäfer wrote:
> @Jeroen
> Thanks for the definition of router and cpe.
> 
> 
> But the chain of problems must be long:
> 
> dead DNS-resolvers by isp
> 
> dead DNS-resolvers not recognized by the "cpe" as a cache resolver

Dead-detection is a standard thing. Only apparently Android does not do
this.

> slow/no switch to alternative DNS-resolvers at the enduser device.

The enduser device uses the CPE, if that does not work for whatever
reason the user will just consider this as "broken internet" (as they do
not know better, and they should not).

Their first thing is to power off then power on the device, if that does
not work, they will call their ISP for support who will then do whatever
magic to fix things.

> All three instances with bad software / without monitoring?

When all ISP resolvers are dead, the ISP will have a *LOT* of support
calls (which is a costly thing).

That next to hoping the monitoring they have in place.

If they do not have functioning monitoring in place, and also actually
check those alerts, I don't think one should consider calling that "ISP"
a ISP. But yes, those places do exist. Likely this does not happen
though when they dogfood their own servers, which is likely, as then
their own stream of cat pictures stop functioning...

Best Internet monitoring well over a decade ago: Gamers playing Quake...

Greets,
 Jeroen

Re: Slow WiFi with Android Marshmallow & IPv6?

[Jeroen Massar]

On 2016-04-26 10:09, Thomas Schäfer wrote:
> Am 25.04.2016 um 17:40 schrieb Eric Vyncke (evyncke):
>> Thanks to all people pointing me towards a DNS issue.
>>
> 
> I read this thread with great interest. I have a Marshmallow dualstacked
> via Wifi (and via mobile), and I have no issues so far.
> One thing troubles me.
> In my configuration the router gets at least four dns-resolvers two ipv4
> and two ipv6 from the isp. The router itself checks the reliability and
> announces its own ip-address as resolver to the clients. A dead resolver
> would theoretically never reach the client.
> 
> Is this a special feature of avm?

With "avm" you likely mean "AVM Fritz!Box", which btw is not
specifically a 'router' but a CPE (that for IPv4 NATs and for IPv6 sorta
'routes'; on a list like this, please don't call those "routers" though
that is an insult to the real boxes).

Like any other CPE in the world, it will receive DNS servers from the
upstream connection and then offer a local recursive DNS server to the
network behind the device.

And that recursive DNS server will forward DNS queries to the upstream
DNS servers, and indeed, in most implementations (unlike Android which
is a enduser device and does not do what every other platform in the
world does) it will use the fastest responding upstream DNS and keep on
using that till that one fails and then retry the next one till it does
not have any left.

Greets,
 Jeroen

Re: Slow WiFi with Android Marshmallow & IPv6?

[Jeroen Massar]

On 2016-04-24 16:17, Eric Vyncke (evyncke) wrote:
> Jeroen, Erik and John,
> 
> Thanks for the hint. I will advise the ISP to investigate any DNS issue
> (such as not returning an error message when requesting a non-existing
> ) but I wonder why it is linked to that specific Android Marshmallow
> version.

Something-Something-Happy-Eyeballs... HE also exists because of broken
DNS, not only broken connectivity.

Other versions might thus have better HE, and of course, as it is
Android, newer versions that have that might not be available yet for
the devices that you have...

No idea if Android actually does HE though and if which version, and
even if the "OS" itself does it, it then still depends on the
application if it uses it, which unfortunately as we get a zillion
methods & libraries to actually do connects and every application does
it by-themselves more or less, it is all magic anyway...

Greets,
 Jeroen

Re: Slow WiFi with Android Marshmallow & IPv6?

[Jeroen Massar]

On 2016-04-24 11:51, Eric Vyncke (evyncke) wrote:
> One of the first Belgian ISP to deploy IPv6 (VOO) is now recommending to
> its Android Marshmallow (6.0.1) users to deactivate IPv6 on their
> residential WiFi CPE... :-(
> 
> It appears that the issue is about IPv6 web sites/apps being really
> slower when using IPv6.

Is it a DNS issue maybe?

https://www.sixxs.net/faq/dns/?faq=ipv6slowconnect

As that has been the general cause of "Disable IPv6!" around the
world for many years already.

Of course, without more details, little one really can say. Bug number
maybe?

Greets,
 Jeroen

Re: Ubuntu 16.04

[Jeroen Massar]

On 2016-04-22 14:37, Mikael Abrahamsson wrote:
> On Fri, 22 Apr 2016, Jeroen Massar wrote:
> 
>> Isn't it awesome that Ubuntu wants dynamic addresses on servers? :)
> 
> Well, this wasn't a server, this is installed as a desktop.

Then you should expect all kinds of magic...

>> They have been told about that problem by many people already,
>> unfortunately, they claim to know better...
> 
> Who are "they"?

The people who decide what does and what does not go into Ubuntu:

https://bugs.launchpad.net/ubuntu/+source/procps/+bug/1068756

Oh, indeed, toggling the sysctl back will lose your default route, thus
you can't just put it in an extra sysctl statement that runs after it.

Putting an empty file into /etc/sysctl.d/10-ipv6-privacy.conf solves
this partially though.

[..]
> $ sudo sysctl -a | grep net.ipv6.conf | grep -i eth0
[..]
> net.ipv6.conf.eth0.use_tempaddr = 2

Recognize what that knob does?

It has to do with the above mentioned file.

Still, all of that can be ignored by systemd, network-manager and other
such things running on a host.

Greets,
 Jeroen

Re: Ubuntu 16.04

[Jeroen Massar]

On 2016-04-22 13:39, Mikael Abrahamsson wrote:
> 
> Hi,
> 
> I have a pretty standard Ubuntu 14.04 machine I just upgraded to 16.04,
> which means I get a "4.4.0-21-generic" kernel.
> 
> I guess I'm using straight up network manager, because my
> /etc/network/interfaces doesn't mention anything about eth0 or wlan0,
> only lo.
> 
> In the GUI, it came default with "privacy extensions disabled".
> 
> I used to administrate this device using its EUI64 based SLAAC address,
> which was stable across reboots. Now with 16.04, I get two addresses,
> none of them stable across reboots.
> 
> Anyone know what the thought is behind this? I want to continue using
> SLAAC and I'm fine with privacy extension addresses over time, but I
> want a single stable address across reboots.

Isn't it awesome that Ubuntu wants dynamic addresses on servers? :)

They have been told about that problem by many people already,
unfortunately, they claim to know better...


But, check your 'sysctl -a | net.ipv6.conf' you might find some knobs
there. Next to that, check systemd settings as that thing wants to take
over the kernel and thus ignores those settings and comes up with it's
own...

Greets,
 Jeroen

Re: IPv6 Dynamic Prefix Problems

[Jeroen Massar]

On 2015-12-16 10:40, Jens Link wrote:
> Johannes Weber  writes:
[..]
> 5) Use a SIXXS / HE Tunnel 

Tunnel brokers (RFC3053) are transition technologies, they won't be here
forever. You likely wanted to point out commercial VPN solutions that
can provide these services just like the normal ISP who is apparently
providing insufficiently configured connectivity.

With IPv6 being 20 years old (RF1883) that transition has to end
somewhere...

Note that SixXS will be having a nice "Call your ISP for IPv6" action[1]
starting somewhere next week.

This hopefully will get people finally calling up to their ISPs and
asking for IPv6 instead of just easily bypassing IPv6 deployment with
easier means.

There is no reason anymore (missing CPE/PE device support, missing OS
support, missing software support) for 'testing IPv6', various locations
are running it natively, many are even forcing DS-Lite/CGN to make sure
they can keep the IPv4 addresses for 'business' customers. Hence, if an
ISP did not take care in the last say 10 years of getting ready for
IPv6, then they won't do that in the next few years either, thus better
to abandon hope and chose wisely with your money.


As for the dynamic issue, everybody seems to forget the great idea that
Microsoft provided: Direct Access[2] or using the 'IPv6 security
feature': IPSEC.

Sign your packets, and check that the signature is valid & known on the
receiving side, presto, does not matter what the prefix is anymore.

Indeed, that does not work like PI, but all/most-of the work on
alternative models have been abandoned, which is why there are so many
/48s PI prefix and sub-prefixes out of PA (Provider Aggregated, remember
that ;) in your routing tables

Routing scaling research will be fun, but in the end, that is the only
real way to handle that situation.

Greets,
 Jeroen

[1] https://www.sixxs.net/news/2015/#happybirthdayipv6ipv6turns20ye-1201
[2] https://en.wikipedia.org/wiki/DirectAccess

Re: New operational tool

[Jeroen Massar]

On 2014-12-16 20:54, Bjørn Mork wrote:
 Brian E Carpenter brian.e.carpen...@gmail.com writes:
 
 http://gabrielmartin.net/projects/hipku/
 
 Something's wrong here
 
 bjorn@canardo:~$ dig  gabrielmartin.net

Yeah, always silly folks who still don't have  on their sites...

Seem it uses the javascript from:
http://www.telize.com/jsonip?callback=init

Great way to have that site inject arbitrary javascript into your page...

That one does have IPv6:
$ dig +short www.telize.com 
2a02:2770:9:0:21a:4aff:feed:aa95

and that is where the 'your ip' gets your IP from, which might be IPv6,
or on the next reload IPv4 or on the next reload something else
depending on the location of the moon and how much air you puff into the
sky... long live happy eyeballs..

Greets,
 Jeroen

Re: IPv6-only residential service (MAP, lw4o6)

[Jeroen Massar]

On 2014-12-06 17:45, Daniel Roesen wrote:
 On Sat, Dec 06, 2014 at 08:08:26PM +0900, Lorenzo Colitti wrote:
 It won't be easy to prove that DS-Lite is not being deployed, because there
 are some fairly large deployments in Germany (Kabel Deutschland and
 Unitymedia, both owned by Liberty Global).
 
 nitpick
 Kabel Deutschland (owned by Vodafone), Unitymedia and Kabel BW (both
 owned by Liberty Global).
 /nitpick
 
 That's almost all of the german cable MSO market.

Add the Austrian market for Liberty Global where they are also 'testing'
it. The Dutch variant (upc.nl + ziggo.nl) has mentions about it and so
does the Swiss edition (UPC/cablecom.ch).

Thus that is most of the German speaking cable world actually...

The UK portion of LG will likely follow to, all in the name of
liberating IPv4 space for business customers^Wthe fact that those
customers did not have typically 4 IPs in use before (modem management,
NAT-IP, phone, mediabox and optionally 3 extra NAT IPs).

And they own a whole lot more of cable around the world:
https://en.wikipedia.org/wiki/Liberty_Global#Markets

who are all going to likely follow too. See it as the Cablecom of
Europe, same business model based on DOCSIS.

According to the top of that page:
Its cable services pass 47 million homes, with 24.5 million customers
or 48.3 million RGUs (video, internet, and voice subscribers)[3]

That is quite a big market bigger than Comcast according to:
 
http://dealbook.nytimes.com/2014/02/13/the-comcast-time-warner-deal-by-the-numbers/

Greets,
 Jeroen

Re: IPv6-only residential service (MAP, lw4o6)

[Jeroen Massar]

On 2014-12-05 14:30, Yannis Nikolopoulos wrote:
 Hello all,
 
 I'm wondering, have people deployed IPv6-only residential services? I
 know of a couple of DS-lite implementations, but we'd be more interested
 to hear about network operators deploying either MAP or lightweight
 4over6 (not just trials though, but actual commercial services)

There is a short question: does user user base have gamers?

If yes, then you won't be happy with the amount of helpdesk calls when
going that route. (and that is ignoring all the P2P users out there)

Quake used to be a great alert system for ISPs there is latency ;)

Greets,
 Jeroen

Who is stilll running 6to4 relays (Was: I-D Action: draft-ietf-v6ops-6to4-to-historic-08.txt)

[Jeroen Massar]

BCC'ing ipv6-ops@lists.cluenet.de thus a bit of background info:

draft-ietf-v6ops-6to4-to-historic-08.txt currently contains:

Section 4 Deprecation
8
   Current operators of an anycast 6to4 relay with the IPv4 address
   192.88.99.1 SHOULD review the information in [RFC6343] and the
   present document, and then consider carefully when the anycast relay
   can be discontinued as traffic diminishes.  Internet service
   providers SHOULD filter out routes to 192.88.99.1.  However, networks

   SHOULD NOT filter out packets whose source address is 192.88.99.1,
   because this is normal 6to4 traffic from a 6to4 return relay
   somewhere in the Internet.
8

Hence this BCC to poll what the operators of these current relays think
about this and what they think they will do.

Note that the discussion is really taking place on v6...@ietf.org hence
to post you either have to be (temporarily) subscribed and/or post
anyway and wait for one of the listadmins to approve your messages.

Below the view that RIPEs RIS thinks are the current 6to4 anycast relays.

On 2014-11-19 08:35, Tim Chown wrote:
[..]
 I wonder what the first publicly announced IPv4 6to4 relay was.
 Perhaps SWITCH (via Simon Leinen) or FUNET (Pekka Savola)?
 I remember the days of SWITCH’s relay being a world 6to4 magnet.

 Would be poetic to let them be the last to switch off too, if they’re still 
 running :)

Afaik the SWITCH one is gone for a long long time already.
FUNET is still up, but only limited in BGP, only RRC13 in Moscow sees
it, which is funny as for instance RRC07 in Stockholm does not.

The list:
https://stat.ripe.net/widget/looking-glass#w.resource=192.88.99.1

 8903 = ES BT Espana
 6939 = US Hurricane Electric*
 7575 = AU AARnet
16150 = SE Availo (Port80)*
12779 = IT ItGate*
 1103 = NL Surfnet*
 8954 = NL Intouch*
15598 = DE QSC AG
28917 = RU FIORD AS
21416 = RU TCINET
 1741 = FI FUNET*
 8359 = RU MTS
44581 = SE AllTele


Note that only 6939/7575/8903 are 'globally' visible, others seem to
have very limited announcement.

* = person who operates it is well known, all of which will be on
ipv6-ops@ hence, BCCd them that way to get them into the loop on this
discussion as they will be the folks disabling those boxes or not.

Greets,
 Jeroen

Re: Teredo sunset - did it happen?

[Jeroen Massar]

On 2014-11-17 17:38, Phil Mayers wrote:
 On 17/11/2014 16:23, Jeroen Massar wrote:
 
 What are you trying to achieve by blocking that port?
 
 I honestly don't know why you want to talk about other things, but I've
 no interest in discussing them with you.

Then don't make statements that you are blocking them...

Greets,
 Jeroen

Re: Some very nice broken IPv6 networks at Google and Akamai

[Jeroen Massar]

On 2014-11-11 16:00, Emanuel Popa wrote:
 Hi,
 
 Is there anyway to intentionally and immediately get on Google's DNS
 blacklist in order to avoid similar outages in the future affecting
 only IPv6 traffic?
 http://www.google.com/intl/en_ALL/ipv6/statistics/data/no_.txt
 
 Or maybe the smart thing to do is building another ISP controllable
 blacklist of broken domains and tell BIND on the caches to return only
 A records for blacklisted domains. Or the other way around: only 
 records for IPv4 broken/blacklisted domains...

As most modern clients do Happy Eyeballs, you could just null route the
destination prefixes and see all clients fall back to IPv6.

But it is rather evil to do that especially at an ISP level. Could have
done that for SixXS and give people working stuff that way, but that
would not have actually resolved the problem, just hidden it.

If you expect that they have outages that they cannot quickly see or
not, then you should also expect a blacklist like to be broken or not
properly update. Hence, better to see the problems and to alert the
folks so that they can fix these issues properly (though Google is now
just hacking around with MSS clamping...).


They typically do not have these issues, they just did not notice it
this time around and thus it took a while for them to wake up (timezones
:) figure out what it is and fix the issue.

I am fairly confident though that Google is now monitoring their stuff
correctly. Lots of good folks there, stuff breaks, they fix it.

Greets,
 Jeroen

How to unsubscribe (Was: Opt Out (UNCLASSIFIED))

[Jeroen Massar]

On 2014-11-11 17:25, Deal, Victor T MSG RET wrote:
 UNCLASSIFIED
 How do I opt out of this list?

By looking at the standardized List headers which show:

List-Unsubscribe: http://lists.cluenet.de/mailman/listinfo/ipv6-ops,
mailto:ipv6-ops-requ...@lists.cluenet.de?subject=unsubscribe

Either go that webpage or mail the address listed.

That is btw also the location you signed up...

Greets,
 Jeroen

Re: Some very nice broken IPv6 networks at Google and Akamai

[Jeroen Massar]

On 2014-11-11 19:09, Andras Toth wrote:
 On Tue, Nov 11, 2014 at 3:36 PM, Jeroen Massar jer...@massar.ch
 mailto:jer...@massar.ch wrote:
 
 If you expect that they have outages that they cannot quickly see or
 not, then you should also expect a blacklist like to be broken or not
 properly update. Hence, better to see the problems and to alert the
 folks so that they can fix these issues properly (though Google is now
 just hacking around with MSS clamping...).
 
 
 [de-cloak]
 
 Google has been doing MSS clamping for a long time, I've seen this
 myself in packet captures and Lorenzo also confirmed in his email:
 ...some Google servers temporarily stopped doing MSS clamping.
 
 They do it for a good reason: to prevent PMTUD as it introduces delay
 and their customers (eyeballs) wouldn't like it. Lorenzo and others
 explained this too several times.

That explanation was seen, at least for me, the first time in this thread.

As stated, the MSS clamping is just hiding the real problems. It does
not properly resolve anything.

 The world is not spinning around sixxs and your design ideas. Please
 turn off write-only.

Wow, here come the ad-hominem attacks, stay in lurk mode if you can't
handle people raising issues. If I had not commented about this problem,
it would never have come to light... maybe in several years when nothing
could have been done anymore. But today, we still can fix things.

Please realize that the world does have a lot more users than SixXS.

Noting problems and properly fixing them are important.

Greets,
 Jeroen

Fwd: [v6ops] IPv6 MTU Flow-label.... (related to draft-v6ops-pmtud-ecmp-problem-01)

[Jeroen Massar]

Forwarding this so that everybody can comment on this nasty proposal ;)

Forcing replies to v6...@ietf.org where they likely should be taking
place as that is where recently the mentioned draft was accepted as a WG
item.

Greets,
 Jeroen

 Forwarded Message 
Subject: [v6ops] IPv6 MTU Flow-label (related to
draft-v6ops-pmtud-ecmp-problem-01)
Date: Mon, 10 Nov 2014 11:31:52 +0100
From: Jeroen Massar jer...@massar.ch
Organization: Massar
To: i...@ietf.org, v6...@ietf.org

Hola folks (and folks in BCC ;),

With the recent Google and Akamai outages (latter still ongoing afaik),
it came to light that the cause is likely the model and problem
described here:

 https://tools.ietf.org/html/draft-v6ops-pmtud-ecmp-problem-01
which previously was:
 https://tools.ietf.org/html/draft-v6ops-jaeggli-pmtud-ecmp-problem-01

Or shortly described: terminating an IP address at different hosts and
having the balancer box not knowing where to deliver the ICMP PTBs that
get send for large packets.

One of the suggestions there is to lower the MSS for every connection by
forcing it (either on the loadbalancer or on the final host) to a value
that works everywhere: the one for an MTU of 1280.

MSS only applies to TCP, and people like Google are coming out with QUIC
and other schemes.

As we really do not want an Internet at an MTU of 1280, why don't we
indicate in the packet what the MTU is when it is diverting from the norm?

What if we instead let a router that sources a packet from a link or is
going to transmit a packet over a link  1500 indicate with that packet
that that packet came from/is going to is a link with a MTU  1500.

We can't use an additional extension header, as adding anything would
mean we might hit the MTU of the packet and we have other issues.

As our least-known-used field is the FlowLabel field, we could abuse
that and have enough bits there to stuff our data.

What if we define that when the first 4 bits are set to 0xF (all one)
that the rest (16bits) defines the MTU of the link (MTU 0 - 65k)?
(We could even use a 'base of 1280' and thus 0xf = 1280 MTU, but
possibly it is better to state value of  0xf0500 is invalid)

Thus allowing when the first 4 bits are not set to all-1 that the
flowlabel field is a normal flowlabel field ala RFC6437. We could even
state Only set this MTU option when the FlowLabel field == 0 to avoid
incompatibility (though I do not expect any as I rarely see packets with
the field non-0...)

Thus given a network like:

  [H1]
2001:db8:1500::1/64
   | mtu = 1500
2001:db8:1500::a/64
  [RA]
2001:db8:1501::a/64
   | mtu = 1500
2001:db8:1501::b/64
  [RB]
2001:db8:1480::b/64
   | mtu = 1480
2001:db8:1480::c/64
  [RC]
2001:db8:1280::c/64
   | mtu = 1280
2001:db8:1280::d/64
  [RD]
2001:db8:9000::d/64
   | mtu = 9000
2001:db8:9000::2/64
  [H2]

RA receives packet, src+dst interface are MTU=1500, thus does nothing
RB receives packet, src = 1500, dst = 1480, thus sets FL = 0xf05c8
RC receives packet, src = 1480, dst = 1280, thus sets FL = 0xf0500
RD receives packet, src = 1280, dst = 9000, thus sets FL = 0xf0500
 (again, just set is quicker than checking)

Now even if H2 is a loadbalancer, if the flow is just forwarded (without
TTL change btw...) the destination receives it correctly.

The disadvantage is of course that you lose the ability to balance based
on the FlowLabel, but if we go with only change when not 0 then there
was one anyway. Also you got src+dst which is 256bits, which should be
pretty good already and optionally next-header + the contents of the
header if you want that.

Note that as we have no checksum in IPv6, there is little overhead to do
this kind of forwarding, HopLimit already needs updating, this is just
another field to update.


In another model from the above, we could even just let every hop set
the known lowest MTU.

In that case, H1 would set 0xf05dc in the packet, and then it gets
lowered automatically. Which would also mean that a pure 9000 path would
nicely work suddenly as everybody knows that 9000 will fit :)

Greets,
 Jeroen

___
v6ops mailing list
v6...@ietf.org
https://www.ietf.org/mailman/listinfo/v6ops

Re: RING measurements don't match access-networks (Was: Some very nice broken IPv6 networks...)

[Jeroen Massar]

On 2014-11-10 11:48, Job Snijders wrote:
 On Mon, Nov 10, 2014 at 08:46:50AM +0100, Jeroen Massar wrote:
 Some hosts are behind exotic 6to4 NATted tunnels,

 I am a bit surprised by such a statement, or the need for it
 
 Because that's what it looks like? I don't know what else i'd call this:
 
 job@amazon01:~$ ifconfig tun6to4
 tun6to4   Link encap:IPv6-in-IPv4
   inet6 addr: 2406:da00:ff00::1715:b057/128 Scope:Global
   inet6 addr: ::10.28.34.28/128 Scope:Compat

6in4 with a wrong name. Lots of people make that mistake ;)

It still does not make it 6to4 though. And I try as much as possible to
point out these wrong labels as well, they confuse the heck out of
everybody. (indeed, 6in4,6to4,6over4,etc all don't make it easy).

Technically 6to4 is correct as they are tunneling IPv6 to IPv4, hence
6to4. It is just a wrong name because 6to4 is an actual technology.

hence why those links used to be called SIT on Linux. or just tunnel.

 Also making a claim like that they are 6to4 that is known to be false,
 is a bit weird as there is no 2002::/16 address in the participants
 hosts:
 
 All I wanted to point out is that there is a large diversity in how RING
 nodes' IPv6 connectivity is arranged. I'll rephrase it as Some hosts
 are behind exotic setups. 

Quite likely. But most will be properly connected and managed and in a
nice coloration. Not on a flaky dsl link.

 snip
 
 Actually, what that demonstrates is that out of a set of well-managed
 nodes, there are issues with PMTU. Hence, please contact these
 networks (they are part of the ring, thus that should be easy) and
 make them behave properly. That solves another set of problems.
 
 Do you want to help engage with these networks? 

Have been doing so for over 10 years... thus if help is needed, I'll try
to scrounge up some cycles from somewhere. A better IPv6 is the goal of
it all, not bandaiding it.

But as you have the contacts at those nodes, contacting these folks
should be very easy to get these issues resolved.

 You might want to run a similar one from all the nodes, thus making a
 cross-RING MTU determination to get even better results, as I did it
 only from one vantage point. Maybe time for ring-mtu? :)
 
 That sounds like a great idea, maybe it can be added to RING SQA as a
 secondary type of alert? MTU suddenly decreased or increased towards a
 significant set of RING nodes? https://github.com/NLNOG/ring-sqa

That sounds something very useful.

And actually, having that tool might be a big reason for a lot of ISPs
to contribute a node to the RING as they get free monitoring.

Note that there is on problem with MTU, the paths might just be
assymetric and thus wrong indicators might be received. But as you have
access to all the nodes, should be easy to do a traceroute from both
sides to see if the route is assymetric or not.

 For NLNOG RING applications we mandate that there is 1 globally unique
 IPv6 address on the host, we do not specify how this should be
 accomplished. This leads to some variety, not all of those
 implementations I would describe as well behaved.

 While that is absolutely true, most of those boxes ARE well behaved, as
 the providers involved will take sure that there are no problems.

 And they definitely are not located in an access network on a dingy home
 DSL line...
 
 I've asked some incumbents to provide this, but so far no dice.

It basically means they need to allocate a DSL link or similar to this.

 Thanks
 for taking a deep dive. Randy Bush presented at RIPE69 comparing the
 RING with RIPE Atlas, I thought it interesting:
 
 
 https://ripe69.ripe.net/presentations/92-141105.ripe-atlas-nlnog-compare.pdf

Atlas is likely the best we have for this.

Would be great to have some MTU tests from those boxes to check for
PMTUD blackholes.

Greets,
 Jeroen

Re: Some very nice broken IPv6 networks at Google and Akamai

[Jeroen Massar]

On 2014-11-09 10:42, Daniel Austin wrote:
 Hi,
 
 I've been having terrible connectivity to Google via IPv6 the last few
 days (i'd even resorted to using Bing!), but can confirm it is working
 fine for me today.

It indeed is looking fine now for the Google problem. Now for Akamai to
use their weekend to debug their issue ;)

 Google did have some issues, look at the outage list.  They are resolved
 now:

   Damian Menscher dam...@google.com mailto:dam...@google.com 
 6:44 PM (3 hours ago)

 The issue with IPv6 access to Google should now be resolved.  Please let
 us know if you're still having problems.

The fun question of course is: what/why/how the heck happened?

Greets,
 Jeroen

Re: Some very nice broken IPv6 networks at Google and Akamai (Was: Some very nice IPv6 growth as measured by Google)

[Jeroen Massar]

On 2014-11-09 12:00, Tore Anderson wrote:
 * Jeroen Massar
 
 On 2014-11-08 18:38, Tore Anderson wrote:
 Yannis: «We're enabling IPv6 on our CPEs»
 Jeroen: «And then getting broken connectivity to Google»

 I'm not a native speaker of English, but I struggle to understand it
 any other way than you're saying there's something broken about
 Yannis' deployment. I mean, your reply wasn't even a standalone
 statement, but a continuation of Yannis' sentence. :-P

 That statement is correct though. As Google and Akamai IPv6 are
 currently broken, enabling IPv6 thus breaks connectivity to those
 sites.
 
 Only if Google and Akamai are universally broken, which does not seem
 to have been the case. I tested Google from the RING at 23:20 UTC
 yesterday:

And Google confirmed that they fixed something, we'll never really
know what they fixed though.

Your test was done from colocated hosts. While real people use access
networks.

Thus while such a test gives insight that some of it works, it does not
cover corner cases.


Also note that the Akamai problem (which still persists) is a random
one. Hence fetching one URL is just a pure luck thing if it works or
not. As a generic page has multiple objects though, you'll hit it much
quicker.


 No, PMTUD is fine in both IPv4 and IPv6.

 What is broken is people wrongly recommending to break and/or
 filtering ICMP and thus indeed breaking PMTUD.
 
 There's a critical mass of broken PMTUD on the internet (for whatever
 reasons). It does not matter who's fault it is, the end result is the
 same - the mechanism cannot be relied upon if you actually care about
 service quality.
 
 From where I'm sitting, Google is advertising me an IPv6 TCP MSS of
 1386. That speaks volumes. I don't believe for a second that my local
 Google cluster is on links with an MTU of 1434; the clamped TCP MSS must
 have intentionally have been configured, and the only reason I can
 think of to do so is to avoid PMTUD.
 
 What works fine in theory sometimes fail operationally (cf. 6to4).
 Insisting that there exists no problem because it's just everyone else
 who keeps screwing it up doesn't change operational realities.

I am not 'insisting' that there is no problem with PMTUD.

I am stating that the problem has to be fixed at the source, not hidden
in the network.


 I also have to note that in the 10+ years of having IPv6 we rarely saw
 PMTU issues, and if we did, contacting the site that was filtering
 fixed the issue.
 
 Looking at it from the content side, users using IPv6 tunnels are in a
 tiny, tiny minority, while still managing to be responsible for a
 majority of trouble reports.

Maybe as those users are more technically experienced and are able to
get their message out, while non-techie users just disable IPv6 as is
advised in a LOT of places? :)

[..]
 Native users are immune against these problems, because they do not have
 to use PMTUD.

You are forgetting the little fact that native is a really strange
word. Quite a few DSL deployments use PPPoE etc.

There are also a lot of native deployments out there that use 6rd.


Instead of just coming with TUNNELS SUCK@$!@#$%^!*@%! actually
Contact the networks that are broken and try to get them to fix the
problem. You might not want to fix those as it is not your problem, but
it is a problem for access networks.

Note btw that Google is not stating anything about the problem they had.
And Akamai, well, they are still digging.

Thus PMTUD might be an issue, might also be something else completely.

Without insight into those systems, one just has to guess.




 The two 'workarounds' you mention are all on the *USER* side (RA MTU)
 or in-network, where you do not know if the *USER* has a smaller MTU.
 
 LAN RA MTU, yes. TCP MSS, no - it can be done in the ISP's tunnel
 router.

Do you really suggest making the Internet have an MTU of 1280? :)

 Hence touching it in the network is a no-no.
 
 It appears to me that the ISPs that are deploying tunnels (6RD) for
 their users consider these a yes-yes. Presumably because they've
 realised that reducing reliance on PMTUD is in their customer's best
 interest, as it gives the best user experience.
 
 Is there *any* ISP in the world that does 6RD that does *not* do TCP MSS
 clamping and/or reduced LAN RA MTUs? (Or, for that matter, does IPv4
 through PPPoE and does not do TCP MSS clamping?)
 
 For what it's worth, the vast majority of tunneled IPv6 traffic we see
 comes from ISPs with 6RD, which generally works fine due to these
 workarounds. Thankfully.

Till people start using non-TCP protocols, and everything breaks.

Hence, don't hide the fact, instead fix it.

[..]
 That is indeed an assumption, as we can't see the Google/Akamai end of
 the connection.
 
 If you see failures on MTU=1500 links, I think there must be at least
 two distinct problems at play. When users report «MTU 1480 MSS 1220 =
 fix», then that is extremely indicative of a PMTUD problem.

For the Google case that was reported

Re: Some very nice broken IPv6 networks at Google and Akamai

[Jeroen Massar]

On 2014-11-09 22:10, Lorenzo Colitti wrote:
 On Sat, Nov 8, 2014 at 11:48 PM, Jeroen Massar jer...@massar.ch
 mailto:jer...@massar.ch wrote:
 
  The issue with IPv6 access to Google should now be resolved.  Please 
 let
  us know if you're still having problems.
 
 The fun question of course is: what/why/how the heck happened?
 
 
 Another fun question is why folks are relying on PMTUD instead of
 adjusting their MTU settings (e.g., via RAs).

Because why would anybody want to penalize their INTERNAL network?

Does Google run non-1500 MTU internally? I hope you are running
JumboPackets at least internally (the ~9000 ones, not the 65k+ ones in
IPv6 thought ;)

Also, nobody knows if a packet somewhere in the middle of the path to
their destination will have a non-1500 MTU.

 But relying on PMTUD still
 imposes a 1-RTT penalty on every TCP connection to an IPv6 address you
 haven't talked to in the last few minutes. Why would you do that to your
 connection?

Because you can't know if that is always the case.

 As to what happened: what happened here is that some Google servers
 temporarily stopped doing MSS clamping. That was an outage, and AIUI it
 has since been fixed.

Thanks for admitting AND explaining what the problem is.

As you work at Google, ever heard of this QUIC protocol that does not
use TCP?

Maybe you want to ask your colleagues about that :)

 (Some parts of) Google infrastructure do not do
 PMTUD for the latency reasons above and for reasons similar to those
 listed
 in https://tools.ietf.org/html/draft-v6ops-jaeggli-pmtud-ecmp-problem-00 .

As such, you are ON PURPOSE breaking PMTUD, instead trying to fix it
with some other bandaid.

And thus you are hiding problems that will happen when QUIC actually
starts to get used?

Or are you going to just reset the Internet to 1280? :)

Greets,
 Jeroen

Re: Some very nice broken IPv6 networks at Google and Akamai (Was: Some very nice IPv6 growth as measured by Google)

[Jeroen Massar]

On 2014-11-09 21:27, Tore Anderson wrote:
 * Jeroen Massar
 
 Testing from colod boxes on well behaved networks (otherwise they
 would not know or be part of the RING), while the problem lies with
 actual home users is quite a difference.
 
 So far you've been claiming that the problem lies with Google or
 Akamai.

Google has acknowledged it, apparently they are doing MSS clamping on
*THEIR* side (while they don't know what your network looks like ;)

Akamai is still peeking.

 If true - and I don't dispute that it is - then testing from
 the RING should work just as well as from any home network.

Completely different environment.

 And, as Job has pointed out, the RING nodes are not all «well behaved».

You had 10 nodes that failed which demonstrates that.

As you have contact to these folks, ask them to fix that situation.

 Also the statement universally broken comes from you.
 
 I refer to this blanket statement of yours, responding to my
 paraphrasing you and Yannis:
 
 «Yannis: «We're enabling IPv6 on our CPEs
 Jeroen: «And then getting broken connectivity to Google»
 
 You: «That statement is correct though. As Google and Akamai IPv6 are
 currently broken, enabling IPv6 thus breaks connectivity to those
 sites. Not enabling IPv6 thus is a better option in such a situation.»
 
 In order for this to be correct, Google and Akamai must necessarily be
 universally broken over IPv6.

Why are you so hang up on words? While in your own admission you are not
a native speaker?

Can you please stop bickering over those words?

Google has admitted they broke something and fixed it.

Stop hanging yourself up.

Greets,
 Jeroen

Re: Some very nice broken IPv6 networks at Google and Akamai (Was: Some very nice IPv6 growth as measured by Google)

[Jeroen Massar]

On 2014-11-09 22:10, Tore Anderson wrote:
 * Jeroen Massar
 
 Also note that the Akamai problem (which still persists) is a random
 one. Hence fetching one URL is just a pure luck thing if it works or
 not. As a generic page has multiple objects though, you'll hit it much
 quicker.
 
 Hm. As I've said before - WFM. Any more information you could provide
 to help me try to reproduce it?

Try reading the links provided.

They contain the details that users have provided.

Note again: Google problem has been fixed (spoofing MSS is not
fixing the problem).

The Akamai things seems to still be in progress.


 I am not 'insisting' that there is no problem with PMTUD.
 
 «No, PMTUD is fine in both IPv4 and IPv6», you said...

Fine is not perfect.

Also, taking single sentences out of somebodies comment does make the
whole sentence.

I have stated several times that there ARE issues with PMTUD and that
people need to fix them instead of hide them.

Again, please stop getting hung up on words.

 I am stating that the problem has to be fixed at the source, not
 hidden in the network.
 
 In an ideal world, perhaps. It's like with 6to4; if all relay operators
 did a wonderful job, and no-one filtered proto-41, and nobody did
 NAT44, then 6to4 would just be hunky-dory. But it's just too much
 brokenness out there.
 
 Same with PMTUD. It's beyond repair, IMHO. The pragmatic thing is to
 accept that and move on.

What you are saying is to just stick to an MTU of 1280 and TCP
everything forgetting about ever being able to move to anything else
than using TCP.

As QUIC is deployed and HTTP/2 is coming, forget about that.

You will need to address these concerns properly.


[..]
 Or that the tunnel ingress routers rate-limit ICMPv6 error generation.

sixxsd does not have this problem. There are no rate limits.

Thus at least everybody behind SixXS tunnels will not have that issue.

Contact your vendor to resolve your problems.

 You are forgetting the little fact that native is a really strange
 word. Quite a few DSL deployments use PPPoE etc.

 There are also a lot of native deployments out there that use 6rd.
 
 In my experience, these ISPs deploy workarounds to avoid PMTUD. TCP MSS
 clamping, and LAN RA MTUs (for IPv6). That helps.

For TCP, not for anything else.

Chrome speaks QUIC to various Google properties.

 Instead of just coming with TUNNELS SUCK@$!@#$%^!*@%! actually
 Contact the networks that are broken and try to get them to fix the
 problem. You might not want to fix those as it is not your problem,
 but it is a problem for access networks.
 
 I think PMTUD on the internet is broken beyond salvation

Then please give up on it and let the rest of the world care about it
and notify folks and let them fix the problem properly.

Greets,
 Jeroen

Re: Some very nice IPv6 growth as measured by Google

[Jeroen Massar]

On 2014-11-08 16:16, Tore Anderson wrote:
 * Jeroen Massar
 
 On 2014-11-08 11:34, Tore Anderson wrote:
 * Jeroen Massar

 On 2014-11-08 10:27, Yannis Nikolopoulos wrote:
 [..]
 the short story here is that we're (finally) enabling IPv6 on our
 (already capable) CPEs :)

 And then getting broken connectivity to Google:

 https://www.sixxs.net/forum/?msg=general-12626989
 https://forums.he.net/index.php?topic=3281.0

 Non sequitur. I'd be extremely interesting in understanding how
 Yannis' IPv6 deployment in OTE (kudos!) could possibly impact the
 SixXS/HE tunnel users' ability to contact Google.

 That is not what I wrote or intended.

 Something unrelated to their deployment broke. But doing the
 deployment does mean that you are now providing connectivity that
 breaks to two major providers: Google and Akamai.
 
 I still do not follow. Is (or were) OTE's deployment broken?

No.

But their deployment, just like that of anybody else, will cause them to
make Google/Akamai unreachable or at least lag in the browser of the
people that any provider

 If
 yes, is the reason for the brokenness the same as for the SixXS/HE
 users? (Is OTE using 6RD?) If no, what is the connection between OTE's
 deployment and the SixXS/HE problems you linked to?

The only link: they are all using IPv6.

You are trying to make this OTE link. I have never stated anything like
that. Though, you likely take that from the fact that the reply followed
in that thread.

 Tunnels do not suck, people who have broken clusters that randomly
 drop packets suck.
 
 Let me rephrase: PMTUD sucks. Tunnels suck by association, because
 they rely on PMTUD not sucking.

No. PMTUD is fine.

What sucks is 'consultants' advising blocking ICMPv6 because that is
what we do in IPv4 and that some hardware/software gets broken once in
a while.

 (Except where the tunnel can accomodate an inner MTU of 1500.)

That is irrelevant.

And likely in the case of Akamai it has little to do with MTU, just
nodes that hang and never reply as Wireshark shows an ongoing TCP
conversation, but that is just a guess.

 Note that even with a full 1500 MTU you will have
 broken connectivity to Google at the moment, lots of fun thus for
 those native deployments like Unitymedia who forcefully stuff folks
 in DSlite land.
 
 This is news to me, both Google and Akamai works just fine from here
 in Norway. Could you elaborate on what broke and how I could try to
 reproduce it?

See the threads I referenced, they are still in the above quoted text.

Note that the Google case is consistent: (as good as) every IPv6
connection breaks.

The Akamai case is random: sometimes it just works as you hit good nodes
in the cluster, sometimes it breaks.

In both cases, it is hard to say what exactly breaks as only the people
in those networks/companies have access to their side of the view.

As such... here is for hoping they debug and resolve the problem.

Greets,
 Jeroen

Some very nice broken IPv6 networks at Google and Akamai (Was: Some very nice IPv6 growth as measured by Google)

[Jeroen Massar]

On 2014-11-08 18:38, Tore Anderson wrote:
 * Jeroen Massar
 
 The only link: they are all using IPv6.

 You are trying to make this OTE link. I have never stated anything
 like that. Though, you likely take that from the fact that the reply
 followed in that thread.
 
 Yannis: «We're enabling IPv6 on our CPEs»
 Jeroen: «And then getting broken connectivity to Google»
 
 I'm not a native speaker of English, but I struggle to understand it
 any other way than you're saying there's something broken about
 Yannis' deployment. I mean, your reply wasn't even a standalone
 statement, but a continuation of Yannis' sentence. :-P

That statement is correct though. As Google and Akamai IPv6 are
currently broken, enabling IPv6 thus breaks connectivity to those sites.

Not enabling IPv6 thus is a better option in such a situation.

But it was just a hook into it. Don't further worry about it.

 Anyway, I'm relieved to hear that there's no reason to supect IPv6
 breakage in OTE. As we host a couple of the top-10 Greek sites, one of
 which has IPv6, we're dependent on the big Greek eyeball network like
 OTE to not screw up their IPv6 deployment - it is *I* who get in trouble
 if they do. :-)

But your network was not involved in the above statement.

And if you monitor your sites correctly, also from non-native setups.
Then you should be fine.

 PMTUD is fine.

 What sucks is 'consultants' advising blocking ICMPv6 because that is
 what we do in IPv4 and that some hardware/software gets broken once
 in a while.
 
 PMTUD is just as broken in IPv4, too.

No, PMTUD is fine in both IPv4 and IPv6.

What is broken is people wrongly recommending to break and/or filtering
ICMP and thus indeed breaking PMTUD.

 PMTUD has *never* been «fine»,
 neither for IPv4 nor for IPv6. That's why everyone who provides links
 with MTUs  1500 resorts to workarounds such as TCP MSS clamping and

I am one of the people on this planet providing a LOT of links with
MTUs  1500 and we really will never resort to clamping MSS.

It does not fix anything. It only hides the problem and makes diagnosing
issues problematic as one does not know if that trick is being applied
or not.

I also have to note that in the 10+ years of having IPv6 we rarely saw
PMTU issues, and if we did, contacting the site that was filtering fixed
the issue.

 reducing MTU values in LAN-side RAs,

That is an even worse offender than MSS. Though at least visible in
tracepath6. Note that you are limiting packet sizes on your local
network because somewhere some person is filtering ICMP.

 so that reliance on PMTUD
 working is limited as much as possible. If you want to deliver an
 acceptable service (either as an ISP or as a content hoster), you just
 *can't* depend on PMTUD.

The two 'workarounds' you mention are all on the *USER* side (RA MTU) or
in-network, where you do not know if the *USER* has a smaller MTU.

Hence touching it in the network is a no-no.

 Even when PMTUD actually works as designed it sucks, as it causes
 latency before data may be successfully transmitted.

I fully agree with that statement. But this Internet thing is global and
one cannot enforce small or large packets on the world just because some
technologies do not support them.

Note that PMTUD is typically cached per destination. Unfortunately there
is no way for for a router to securely say this MTU is for the whole
/64 or /32 etc that would have been beneficial.

 See the threads I referenced, they are still in the above quoted text.

 Note that the Google case is consistent: (as good as) every IPv6
 connection breaks.

 The Akamai case is random: sometimes it just works as you hit good
 nodes in the cluster, sometimes it breaks.
 
 I see in the threads referenced things statements such as:
 
 «this must be a major issue for everybody using IPv6 tunnels»
 «MTU 1480 MSS 1220 = fix»
 «the 1480MTU and 1220MSS numbers worked for my pfsense firewall»
 «The only thing that worked here is 1280 MTU / 1220 MSS»
 «clamping the MSS to 1220 seems to have fixed the problem for me»
 «I changed the MSS setting [...] for the moment Google pages are
 loading much better»
 
 This is all perfectly consistent with common PMTUD mailfunctioning /
 tunnel suckage.

NOTHING to do with tunnels, everything to do with somebody not
understanding PMTUD and breaking it, be that on purpose or not.

Note that both Google and Akamai very well know about PMTUD and up to
about a week ago both where running perfectly fine.

 I'm therefore very sceptical that this problem would
 also be experienced by users with 1500 byte MTU links.

Tested failing also on MTU=1500 links

 (Assuming there's only a single problem at play here.)

That is indeed an assumption, as we can't see the Google/Akamai end of
the connection.

Note that in the Akamai case it is a random thing, it happens to random
nodes inside the cluster (at least, that is my assumption...)

 In both cases, it is hard to say what exactly breaks as only the
 people in those networks

Re: 6to4 in Internet aaaa records

[Jeroen Massar]

On 2014-10-04 12:56, Gert Doering wrote:
 Hi,
 
 On Sat, Oct 04, 2014 at 12:49:00PM +0200, Gert Doering wrote:
 On Thu, Oct 02, 2014 at 10:31:25PM -0400, Jeroen Massar wrote:
 http://www.azdes.gov)... 2002::cf6c:8846

 That is an invalid 6to4 address as it would have a 6to4 gateway of 0.0.0.0.

 Uh, what?

 Who are you and what happens to the Jeroen I know who understands IPv6,
 [..]
 
 Aliens stole the body of Gert Doering, and now he can't find the button
 withdraw this e-mail in his Outlook mail client anymore...

Hmmm, yeah a Gert using Outlook would be an excuse, but it seems more
that an alien stole his dog: User-Agent: Mutt/1.5.23 (2014-03-12) :)

Dear Aliens: please return our Gert!

 Sorry, of course Jeroen is right here, and I need to sleep more before
 typing.  While anycast *is* the problem, of course the exit side of
 the tunnel needs an address, aka gateway...

Can anybody local to Munich deliver some fresh coffee to Gert? :)

Greets,
 Jeroen

Re: 6to4 in Internet aaaa records

[Jeroen Massar]

On 2014-10-02 22:24, Ca By wrote:
 Folks,
 
 What is the general impression of 6to4 addresses in  records?
 
 I recently had a customer complain about this situation, and i am not
 sure, as a service provider, how to deal with it.
 
 From my home comcast connection with real full dual-stack, i get this 
 
 
 
 cbyrne@ ~ $ wget -6 www.azdes.gov http://www.azdes.gov  
 --2014-10-02 19:19:48--  http://www.azdes.gov/
 Resolving www.azdes.gov http://www.azdes.gov (www.azdes.gov
 http://www.azdes.gov)... 2002::cf6c:8846

That is an invalid 6to4 address as it would have a 6to4 gateway of 0.0.0.0.

One would think with all the IPv6 consultants in the US, that .gov
agencies would be able to get that part right...

Though, better point them out that 6to4 is a bad idea in general anyway.

I would not be surprised if the DNS solution generated that broken
address though as cf6c:8846 does map to 207.108.136.70 which matches the
A record.

Greets,
 Jeroen

Re: Something with filters

[Jeroen Massar]

On 2014-08-28 07:57, Tassos Chatzithomaoglou wrote:
 Jen had presented some similar stats a year ago.
 
 https://ripe67.ripe.net/presentations/288-Jen_RIPE67.pdf

These kind of issues have been demonstrated for as long as IPv6 has
existed, and people have been complaining to their account managers and
technical contacts too.

Vendors just do not see what the problem is it seems...

On 2014-08-28 07:46, Lorenzo Colitti wrote:
 On Wed, Aug 27, 2014 at 9:01 AM, Jeroen Massar jer...@massar.ch
 mailto:jer...@massar.ch wrote:

  9  2001:5a0:a00::2e (2001:5a0:a00::2e)  79.018 ms  79.910 ms
79.960 ms
 10  :: (::)  101.893 ms  102.004 ms  103.574 ms
 11  rar3.chicago-il.us.xo.net http://rar3.chicago-il.us.xo.net
 (:::65.106.1.155)  104.732 ms

 Yeah baby, we can use the unspecified address in ICMP replies!


 The mapped IPv4 address in there is pretty cool, too...

Wow, I honestly had not even noticed that one; but indeed, another one
that never should exist on the wire. At least for that one we can easily
point at XO. Likely they are the problem for hop 10 too.
Lets see if we can get them to resolve that mess...

(even though I rather had proper source filtering installed worldwide...)

Greets,
 Jeroen

Something with filters

[Jeroen Massar]

I was doing some traceroutes to determine some weird claim of a transit
(not shown in the below trace) being tier1 while another transit
actually popped up in their network and then noticed this beauty:

 9  2001:5a0:a00::2e (2001:5a0:a00::2e)  79.018 ms  79.910 ms  79.960 ms
10  :: (::)  101.893 ms  102.004 ms  103.574 ms
11  rar3.chicago-il.us.xo.net (:::65.106.1.155)  104.732 ms

Yeah baby, we can use the unspecified address in ICMP replies!

Why oh why is that packet even allowed to come back to me, let alone
travel all those hops...

Oh, yeah, something with uRPF and other such awesome standards.

Greets,
 Jeroen

Re: Something with filters

[Jeroen Massar]

On 2014-08-27 19:52, Jared Mauch wrote:
 
 On Aug 27, 2014, at 12:01 PM, Jeroen Massar jer...@massar.ch wrote:

 I was doing some traceroutes to determine some weird claim of a transit
 (not shown in the below trace) being tier1 while another transit
 actually popped up in their network and then noticed this beauty:

 9  2001:5a0:a00::2e (2001:5a0:a00::2e)  79.018 ms  79.910 ms  79.960 ms
 10  :: (::)  101.893 ms  102.004 ms  103.574 ms
 11  rar3.chicago-il.us.xo.net (:::65.106.1.155)  104.732 ms

 Yeah baby, we can use the unspecified address in ICMP replies!

 Why oh why is that packet even allowed to come back to me, let alone
 travel all those hops...

 Oh, yeah, something with uRPF and other such awesome standards.
 
 uRPF is an expensive feature in hardware that most people don’t
 ask their vendors for.  uRPF for IPv6 is even harder because of
 things like hop #11 seen above.
 
 We keep asking the vendors but apparently we are in the minority.

I know that the majority of the list here wants it; but the vendors
don't it seems... one has to wonder why...

Especially a check for a zero'd address is really not that hard; it is
just crazyness that that is not checked for.

If possible, please file this problem with your relevant technical
contacts and account managers, as it is just nonsense that that packet
is allowed to travel over the Internet.

Greets,
 Jeroen

Re: IPv6-related (?) Bind issue

[Jeroen Massar]

On 2014-03-06 10:00 , Yannis Nikolopoulos wrote:
[..]
 https://lists.isc.org/pipermail/bind-users/2014-March/092706.html
 https://lists.dns-oarc.net/pipermail/dns-operations/2014-March/011385.html

And asking on yet-another list, did you do the /64 change:

https://lists.dns-oarc.net/pipermail/dns-operations/2014-March/011387.html

?

Greets,
 Jeroen

Re: IPv6-related (?) Bind issue

[Jeroen Massar]

On 2014-03-06 10:53 , Yannis Nikolopoulos wrote:
 On 03/06/2014 11:07 AM, Jeroen Massar wrote:
 On 2014-03-06 10:00 , Yannis Nikolopoulos wrote:
 [..]
 https://lists.isc.org/pipermail/bind-users/2014-March/092706.html
 https://lists.dns-oarc.net/pipermail/dns-operations/2014-March/011385.html

 And asking on yet-another list, did you do the /64 change:
 
 yes we did (although we didn't expect much) and no, it didn't solve the
 problem. Actually, it seemed like the problem worsened (more SERVFAILs)
 when we switched from /127 to /64 (using ::a and ::b) and that, frankly,
 made no sense to me

Then next step is monitoring if your addresses are still on the
interface (ip mon ...)

Greets,
 Jeroen

Re: Reaching google.com using Chrome

[Jeroen Massar]

On 2014-01-13 19:02 , Sammer Mati wrote:
[..]
 We ran wireshark and found out that the IPv6 address is different for
 Google.com when using IE or Chrome! I haven't tested yet with Windows7 

That is just pure DNS selection luck...

Note that a lot of properties on this massive Internet are using
Geo-DNS, load-balancing, BGP-based routing tricks/anycasting and a lot
of other nasty funny tricks.

Hence, as you did not include any traceroute or other data, little else
anybody can say...

Greets,
 Jeroen

Re: IPv6 broken on Fedora 20?

[Jeroen Massar]

On 2013-12-19 17:09 , Simon Perreault wrote:
 Is there any other Fedora user on this list that could confirm this?
 
 I filed a bug here:
 https://bugzilla.redhat.com/show_bug.cgi?id=1045118

net.ipv6.conf.em1.accept_ra = 0

How do you expect that to work?

Change to either 1 or 2 (in case you want forwarding enabled but accept
RA nevertheless).

Greets,
 Jeroen

How to unsubscribe from ipv6-ops (Was: IPv6 broken on Fedora 20?)

[Jeroen Massar]

On 2013-12-19 17:30 , McKnight, Joe wrote:
 Hi,  
 
 I ended up on this listserve by mistake. Will someone please remove me? 

If you don't know how to unsubscribe from mailinglists you indeed do not
belong here.

From the email-headers:

List-Id: IPv6 operators forum ipv6-ops.lists.cluenet.de
List-Unsubscribe: http://lists.cluenet.de/mailman/listinfo/ipv6-ops,
mailto:ipv6-ops-requ...@lists.cluenet.de?subject=unsubscribe
List-Archive: http://lists.cluenet.de/pipermail/ipv6-ops
List-Post: mailto:ipv6-ops@lists.cluenet.de
List-Help: mailto:ipv6-ops-requ...@lists.cluenet.de?subject=help
List-Subscribe: http://lists.cluenet.de/mailman/listinfo/ipv6-ops,
mailto:ipv6-ops-requ...@lists.cluenet.de?subject=subscribe


That is standardized in RFC2369 btw.

Greets,
 Jeroen

Re: Microsoft: Give Xbox One users IPv6 connectivity

[Jeroen Massar]

On 2013-10-10 00:02, Christopher Palmer wrote:
 John and Lorenzo beat me to it J.
 
  
 
 Example:
 
 Samantha has native IPv6 and Teredo.
 
 Albert has Teredo only.

But what do you do with the more and more common case[1] where one gets
native IPv6 and IPv4-over-DSlite; especially considering the high rate
of connection problem over that IPv4 path? This as the dslite gateways
are heavily overloaded as most destinations (read: http/bittorrent) are
IPv4 only. Will then Teredo be used which is broken or the perfectly
working IPv6 native path?

Getting out over native IPv6 in that specific scenario will be the
better thing to do.

From that perspective, applying the Apple-variant of Happy Eyeballs will
be beneficial. It will mean that one will have to expose all the
possible IPv4 and IPv6 addresses amongst peers so that they can try out
the variant combinations. SCTP or MP-TCP might be a good fit there too.


[1] German ISPs like Unitymedia, which is part of UPC/LibertyGlobal and
thus it is expected when that trial pans out that all other countries
where UPC is located will be following down that rabbit hole too

Re: IPV6 Minimom alocation for recidential customers

[Jeroen Massar]

On 2013-08-20 16:33 , Arturo Servin wrote:
 
   I wouldn't say that it is dependent in the RIR, it is about an ISP
 decision, not about a regional organization. (note, I work for one).

Working for a RIR just means that you are implementing the rules that
are set by that RIRs membership. Thus working for one says little...

Every RIR has different rules and regulations, and on top of that they
keep on changing all the time too.

   It may be some bias from some organizations or individuals in those
 regions, but at the end the decision of using /64, /60, /56 or /48
 depends on the ISP alone.

As prefixes are allocated based on the amount of address space one
needs, the ISP receives that allocation from RIR based on the intended
usage. As such, it is also expected that the space is actually used for
those purposes.

Next to that there is a very nice IETF recommendation too...

If ISPs are just going to give a single /64 to end-sites, then they
could just as well just stick with IPv4.

Greets,
 Jeroen

Re: IPV6 Minimom alocation for recidential customers

[Jeroen Massar]

On 2013-08-20 16:40 , Arturo Servin wrote:
 
   So it seems that we agree.

No, we do not agree as your statement is wrong.

I suggest you read up on:
 http://www.ripe.net/ripe/docs/ripe-589

and as you claim to work for LACNIC:
 http://lacnic.net/en/politicas/manual5.html

Greets,
 Jeroen

 On 8/20/13 11:36 AM, Jeroen Massar wrote:
It may be some bias from some organizations or individuals in those
 regions, but at the end the decision of using /64, /60, /56 or /48
 depends on the ISP alone.
 As prefixes are allocated based on the amount of address space one
 needs, the ISP receives that allocation from RIR based on the intended
 usage. As such, it is also expected that the space is actually used for
 those purposes.

 Next to that there is a very nice IETF recommendation too...

 If ISPs are just going to give a single /64 to end-sites, then they
 could just as well just stick with IPv4.

Re: l.yimg.com (Yahoo!) over IPv6

[Jeroen Massar]

On 2013-08-19 23:10 , Phil Pennock wrote:
 On 2013-08-19 at 15:10 -0500, Frank Bulk wrote:
 It works for me, too.

 Can you replicate this issue on other machines and vantage points?
 
 I did, last week when I first saw it, to initially confirm it wasn't my
 home connection; at which point, I shrugged and left the issue.
 
 At home, SixXS tunnel in USA, correctly configured.

While your end might be correctly configured, the path between you and
them might not be. Especially in the case of tunnels where a non-1500
byte MTU is likely to exist and silly admins still exist who think that
ICMP is evil and thus are stupid and drop it, especially that all
important ICMPv6-Packet-Too-Big packet.

 Replicated behaviour on my colo box in NL, which has native v6 in
 continuous operation for about seven years now.  AS8283.

 When I check now, the site has been fixed.  No idea if my questioning
 the status influenced that or not ...

Please note that the 'l' in l.yimg.com stands for load balancer, a
geographic one in this case.

See below, three different ASNs, three different results; and that is
just on the DNS/IP level, what actually happens at these sites is likely
another level of load-balancers with a bunch of machines behind it.
Stuff can be broken at any of those levels.

As such, a traceroute/tracepath is very useful when making any
statements like this, next to that all important wireshark dump which
will likely tell you where/why your TCP session hangs.

Greets,
 Jeroen

--

$ host l.yimg.com
l.yimg.com is an alias for fd-geoycs-l.gy1.b.yahoodns.net.
fd-geoycs-l.gy1.b.yahoodns.net is an alias for
ds-fo-anyycs-l.ay1.b.yahoodns.net.
ds-fo-anyycs-l.ay1.b.yahoodns.net has address 206.190.60.138
ds-fo-anyycs-l.ay1.b.yahoodns.net has address 206.190.60.139
ds-fo-anyycs-l.ay1.b.yahoodns.net has IPv6 address
2001:4998:f011:1fb::c:1101
ds-fo-anyycs-l.ay1.b.yahoodns.net has IPv6 address
2001:4998:f007:1fb::c:1103
ds-fo-anyycs-l.ay1.b.yahoodns.net has IPv6 address 2001:4998:8:1::2000
ds-fo-anyycs-l.ay1.b.yahoodns.net has IPv6 address
2001:4998:f007:1fb::c:1101
ds-fo-anyycs-l.ay1.b.yahoodns.net has IPv6 address 2001:4998:8:1::2001

$ host l.yimg.com
l.yimg.com is an alias for fd-geoycs-l.gy1.b.yahoodns.net.
fd-geoycs-l.gy1.b.yahoodns.net is an alias for
ds-fo-anyycs-l.ay1.b.yahoodns.net.
ds-fo-anyycs-l.ay1.b.yahoodns.net has address 66.196.65.174
ds-fo-anyycs-l.ay1.b.yahoodns.net has address 66.196.65.111
ds-fo-anyycs-l.ay1.b.yahoodns.net has address 66.196.65.112
ds-fo-anyycs-l.ay1.b.yahoodns.net has address 66.196.65.188
ds-fo-anyycs-l.ay1.b.yahoodns.net has IPv6 address
2a00:1288:f00e:1fc::c:1101
ds-fo-anyycs-l.ay1.b.yahoodns.net has IPv6 address
2a00:1288:f00e:1fc::c:1103

$ host l.yimg.com
l.yimg.com is an alias for fd-geoycs-l.gy1.b.yahoodns.net.
fd-geoycs-l.gy1.b.yahoodns.net is an alias for
ds-geoycs-l.gy1.b.yahoodns.net.
ds-geoycs-l.gy1.b.yahoodns.net is an alias for
ds-fo-vnycs-l.gy1.b.yahoodns.net.
ds-fo-vnycs-l.gy1.b.yahoodns.net is an alias for
ds-pxe-vnycs-p.gy1.b.yahoodns.net.
ds-pxe-vnycs-p.gy1.b.yahoodns.net has address 210.211.100.99
ds-pxe-vnycs-p.gy1.b.yahoodns.net has address 106.10.199.78
ds-pxe-vnycs-p.gy1.b.yahoodns.net has address 210.211.100.100
ds-pxe-vnycs-p.gy1.b.yahoodns.net has address 210.211.107.99
ds-pxe-vnycs-p.gy1.b.yahoodns.net has IPv6 address
2406:2000:f008:1fd::c:1101
ds-pxe-vnycs-p.gy1.b.yahoodns.net has IPv6 address 2406:2000:ac:5::c:1101
ds-pxe-vnycs-p.gy1.b.yahoodns.net has IPv6 address
2406:2000:f008:1fd::c:1103
ds-pxe-vnycs-p.gy1.b.yahoodns.net has IPv6 address 2406:2000:fc:402::c:2203

Re: Linux IPv6 routing strange behaviour

[Jeroen Massar]

On 2013-08-15 13:26, Phil Mayers wrote:
 On 15/08/13 12:14, Pim van Pelt wrote:
 Just ad a datapoint to Max' last remark, at sixxs we moved away from
 kernel based routing by implementing ipv6 routing in userspace (taking
 tap input and raw socket output) largely because of neighbor cache
 
 Interesting. Was this custom/proprietary software or is it available
 somewhere?

To add to Pim's comments:

It is quite specific to the problems that SixXS PoPs have:
 Large amount of tunnels and routes

Also note that these tunnels are dynamic and thus endpoints change all the time.

The Linux kernel (nor likely any other kernel) is just not (and likely will 
never) be designed for what the SixXS PoPs do. We saw random 'forgetting' of 
_static_ routing entries, and even tunnel interfaces going missing and other 
weird effects without any error/warnings whatsoever; thus what really happened 
is a mystery.

The routing logic along with the caching/neighbor lookups etc on top of those 
issues did not help at all either. Note that the same goes for 
FreeBSD/NetBSD/OpenBSD/OSX from our testing (yes, we checked if OSX was smarter 
about it, it is not ;)

From our testing, performance characteristics are mostly the same when running 
sixxsd on the above platforms: it fills about 10G of tunneled traffic on a 
virtual interface on a i7 3.4Ghz. (Simulated traffic, but as everything is a 
static non-locking lookup that should be quite okay ;) If we ever hit the 
limits of that setup, we can always think about adding some threads or so to 
use the other cpus (hence why I don't mention quad-core above)...

Since deploying it we then also have not had any issues with the PoPs 
themselves anymore except for hardware outages or routing issues outside on the 
network itself. (code can't solve those... yet ;)

sixxsd is available for use solely by SixXS PoPs, but as said, it is solving a 
very specific problem that one likely does not have outside the scope of this. 
Thus it likely won't solve any problem you are having: as always, actually 
defining the problem one has might lead to a solution.

Some more details are available here:
 http://www.sixxs.net/faq/sixxs/?faq=sixxsd

As a bonus, this is how the routing table of deham01 looks like:
8--
root@deham01:~# ip -6 ro show
2001:6f8:862:1::/64 dev eth0  proto kernel  metric 256  mtu 1500 advmss 1440 
hoplimit 4294967295
2001:6f8:900:::1 dev sixxs  metric 1024  mtu 1500 advmss 1440 hoplimit 
4294967295
2001:6f8:900::/48 via 2001:6f8:900:::1 dev sixxs  metric 1024  mtu 1500 
advmss 1440 hoplimit 4294967295
2001:6f8:900::/40 via 2001:6f8:900:::1 dev sixxs  metric 1024  mtu 1500 
advmss 1440 hoplimit 4294967295
2001:6f8:1000::/40 via 2001:6f8:900:::1 dev sixxs  metric 1024  mtu 1500 
advmss 1440 hoplimit 4294967295
2001:6f8:1100::/40 via 2001:6f8:900:::1 dev sixxs  metric 1024  mtu 1500 
advmss 1440 hoplimit 4294967295
2001:6f8:1200::/40 via 2001:6f8:900:::1 dev sixxs  metric 1024  mtu 1500 
advmss 1440 hoplimit 4294967295
2001:6f8:1300::/40 via 2001:6f8:900:::1 dev sixxs  metric 1024  mtu 1500 
advmss 1440 hoplimit 4294967295
fe80::/64 dev eth0  proto kernel  metric 256  mtu 1500 advmss 1440 hoplimit 
4294967295
default via fe80::5:73ff:fea0:1 dev eth0  metric 1024  mtu 1500 advmss 1440 
hoplimit 4294967295
--8

Yes, that is 5 /40s worth of address space and everything is piped into the 
sixxs interface to a single neighbor that lives on the tapped interface. We 
thus indeed hit the Linux routing logic a bit, but as the table is small and it 
is a single neighbor nothing much dynamic happens there. ip -6 monitor route 
is thus nice an silent.

Greets,
 Jeroen

Re: Linux IPv6 routing strange behaviour

[Jeroen Massar]

On 2013-08-15 14:41, Mikael Abrahamsson wrote:
 On Thu, 15 Aug 2013, Jeroen Massar wrote:
 
 Yes, that is 5 /40s worth of address space and everything is piped
 into the sixxs interface to a single neighbor that lives on the tapped
 interface. We thus indeed hit the Linux routing logic a bit, but as
 the table is small and it is a single neighbor nothing much dynamic
 happens there. ip -6 monitor route is thus nice an silent.
 
 So you're actually not seeing any flow based routing here?
 
 cat /proc/net/ipv6_route contains just those routes you see in ip -6
 r show?
 
 Because in my linux kernel 3.2 based machines I have a lot more entries
 in cat /proc/net/ipv6_route than I have routes.

That is correct. Though on 2.6 you won't see those there from what I
recall, on 3.2 you will indeed see them.

In our case that means that the tunnels are not amongst them (and that
is where the majority of endpoints for us are, hence at minimum half the
table entries), while the uplink (which is a default route) will cause a
the packet to go through Linux's kernel and create the same entry over
and over.

We could likely avoid that if we wanted to, by sending the packet
ourselves to gateway and thus skipping the kernel's routing completely.
As the scaling[2] and performance is already much better (and we do not
have the randomly dropping interfaces[1] , and overhead is already
minimal enough, we did not bother doing that yet.

Greets,
 Jeroen

[1] Linux kernel uses a hashtable that can collide when there are lots
of tunnels; but as we know the address space layout anyway, we do not
have to bother with that.
[2] I recall that the interface table used to/is a linked list...

Re: Linux IPv6 routing strange behaviour

[Jeroen Massar]

On 2013-08-14 12:58 , Max Tulyev wrote:
 On 14.08.13 13:39, Jeroen Massar wrote:
 I see the strange behaviour of my Linux routers. There are quagga and
 bird with IPv6 BGP full view.

 On the same box? Are they using the same routing table? I am fairly
 confident that will end up in a fight.
 
 No! Some boxes have Quagga, some - Bird, not together of course.

And are these boxes interconnected and are they thus possibly forwarding
the information to each other?

 Quagga/bird reports about 13500 prefixes,
 but route table constantly grow up to 10 routes and more.

 Which prefixes, and who/what originates them?
 
 The prefixes from the global IPv6 routing table. It should be one
 routing entry per route, but have the lot of totally same route strings.

Your view on 'global' will be different than that from any other person
depending on the peers you have, configuration settings and lots lots more.

Without details, little that can be stated though.

Note that my teeny Quagga box receives multiple full feeds, though does
not install them in a FIB, and has been running fine for over 10 years
(except for some strange lockup bug once in a while, likely caused by
multiple connections appearing/disappearing at the same time or some
other odd and non-easily-reproduceable race condition)

 Are you maybe causing a routing loop towards yourself?
 
 I checked - no, but why a lot of routing entries, even if loop?

People cause bugs? And if the path does not contain repeated ASNs it
will nicely loop till something stops it.

 Did anyone see something like that? How I can fix it?

 Providing outputs/log files would be a good start for people to look at it.
 
 Which logs can explain something?

The ones on your hosts? The ones generated by the tools that apparently
break.

Of course it depends all on how you configure things, nobody can guess
what your setup looks like or how it is acting.

 P.S. It seems if there is more IPv6 traffic - there is faster growth of
 routing table.

One would hope so. Though at one point it will slow down, till
companies/end-sites start getting PI prefixes...

Greets,
 Jeroen

Re: teredo.ipv6.microsoft.com off?

[Jeroen Massar]

On 2013-07-17 15:09 , Ron Broersma wrote:
 
 On Jul 16, 2013, at 10:40 PM, Mikael Abrahamsson wrote:
 
 On Tue, 16 Jul 2013, Christopher Palmer wrote:
 
 If there is feedback on the ongoing experiment or our
 consideration of sunsetting Teredo, do let me know.
 
 So far people have been quite enthusiastic.
 
 I am too. I would really like to see 6to4 and teredo be default off
 everywhere, and people who want it can manually turn it on. If
 teredo went away completely, that would also be a good thing.
 
 Strongly concur here as well.  One less thing I have to disable on
 all my systems in enterprise nets.

Windows boxes that are in an Active Domain (which should match your
'enterprise net') have Teredo and 6to4 disabled per default.

Next to that one can enforce that of course through AD policies.

Greets,
 Jeroen

Re: Linux 3.9 routing oddity

[Jeroen Massar]

On 2013-07-02 21:27, Pierre Emeriaud wrote:
[..]
 $ ip -6 nei show
 fe80::f6ca:e5ff:fe43:d114 dev eth0 lladdr f4:ca:e5:43:d1:14 router REACHABLE

The neighbor tables are caches, thus indeed, until they are used they
won't appear there.

 $ ip -6 route get 2001:db8:400c:c03::be
 2001:db8:400c:c03::be from :: via fe80::f6ca:e5ff:fe43:d114 dev eth0
 src 2001:db8:ee8c:180:216:d3ff:feb6:d908  metric 0

That is quite wrong indeed. A specific route should always be used, even
if it cannot be used because the next-hop is not available. Are you sure
that the route entry for your 2000::/3 route is fully active and that
forwarding etc is enabled on that interface?

Your metrics seem to be '0' which is quite odd.

You might want to check with 'ip -6 ro show table all' to reveal some
more routing alternatives and check if you are not accidentally using
multiple routing tables.

You might also want to try this out with something that is not a tun/tap
interface, thus a default ethernet interface, as tun/tap might have all
kinds of odd behavior, eg no or hacked neighbor discovery depending on
the tool being used by the tun device. (and in case you use openvpn,
kick Gert ;)

Sounds rather something to raise on the netdev list
(http://www.spinics.net/lists/netdev/)

FTR: From my testing with 3.9.x and 3.10-rc7 I have not noticed that
kind of behavior on boxes with normal ethernet interfaces.

Greets,
 Jeroen

Re: Point-to-point /64

[Jeroen Massar]

On 2013-06-01 13:38, Arturo Servin wrote: Ole,

   I know!

   Basically I want to have the whole picture before recommend or not
 recommend to use /64s in p2p links (or use them myself)

   /64s in p2p looks very appealing for many reasons, but they have a
 counter argument in security. Is it possible to overcome?

   Perhaps the only solution is to avoid /64s in p2p links.

It really completely depends on what the security threat you are trying
to 'protect' against. Except for using more space and giving the option
of using that space later on there is no real benefit between using a
/64 and configuring only 2x/128s or just configuring 2x/128.

One thing to keep in mind though is that quite some gear is optimized
upto the first /64 bits, and might use slower paths for longer prefixes,
thus if one is going to put a lot of /128s in a single /64, thus when
really stuffing all p2p links in a single /64 or so, it might hurt
performance on the gear being used. As such, do ask your vendor about
their limitations.

On 2013-06-01 13:49, cb.list6 wrote:
[..]
 I do /127 p2p
 
 Subnet anycast is not a supported feature or requirment in my network.

As subnet anycast is a integral part of IPv6, you might not want to
support or require it, but the gear will implement it and thus one point
it will bite you as suddenly it won't work.

Likely though you are just putting the PtP links as 2x /128 and not as a
/127 on the link. The former works, the latter breaks.

Greets,
 Jeroen

Re: Point-to-point /64

[Jeroen Massar]

On 2013-06-01 15:03, Ole Troan wrote:
[..]
 As the subject was about 'security', more in the rule of DoS/DDoS, the
 problem becomes that some miscreants target exactly those addresses
 because they are expected to not forward much

 Indeed for normal operation it should be okay, but miscreants are
 getting smarter too...
 
 In the case of /127 or /128 you'd always  hit the router's host stack. 

PtP links have two sides, the remote end would thus cause it to hit the
final router's host stack indeed, but due to some address lookup
implementation also on the intermediate hop. Just something to keep in mind.

Greets,
 Jeroen

6PE mapped addresses used for ICMPv6 responses - knob to fix that?

[Jeroen Massar]

In relation to the fd00::/8 thread, it seems the bug with respect to 6PE
is not resolved yet.

That is, if you have 6PE (IPv4 LSP) in your network routers might send
an ICMPv6 message from the IPv6-Mapped-IPv4 address.

And as ::::0.0.0.0/96 should not be in anybody's BGP table, it will
fail uRPF.

Is anybody aware of a knob that can force for instance the loopback
address to be used on these boxes?

Greets,
 Jeroen

Re: http://www.6assist.net/ - call for test

[Jeroen Massar]

On 2013-05-10 11:17, Max Tulyev wrote:
 Hi All,
 
 We are developping a new kind of IPv6 tunnel broker. The main idea is
 let the traffic flow directly between peers, not through the tunnel
 broker server.
 
 The website of that project is http://www.6assist.net/
 
 It acts like a virtual media IXP, and anybody have own ASN can join it.

For SixXS we have thought about allowing people, FOR EXPERIMENTATION, to
use a private ASN, and then do something similar, this so that people
can play a bit with BGP (time and other things has caused it not to be
there though).

There are a number of IX/VPN platforms out there that allow this
already though, http://www.virt-ix.net/ for instance and the numerous
hackerspace VPN networks.

For your problem statement from the page:

The main disadvantage of using a tunnel broker is traffic path not
optimal. Imagine you are in Germany, use the IPv6 tunnel broker in USA
and connecting via IPv6 to some server in Italy

The answer is simple: get an ISP in that local country to set up a
tunnel broker PoP. Traffic will then be properly routed.

Tunnels should go to endsites, not between networks.

Greets,
 Jeroen