[Bug 1878682] Re: IPv6 not enabled by default
** Changed in: cloud-init (Ubuntu) Importance: Undecided => High ** Changed in: cloud-init (Ubuntu) Status: New => Triaged -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1878682 Title: IPv6 not enabled by default To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/cloud-init/+bug/1878682/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1878682] Re: IPv6 not enabled by default
> Ryan, > > Thank you so much for taking the time to go over the constraints and > competing goals. It's always helpful to know what constituencies are > involved. Sure. > > > That said, I do think that cloud-init can make it easier to enable ipv6 > > where > > it's desired (but without having to write your own network config). We're > > looking into supporting a kernel parameter or cloud-config option to enable > > the default config to block on ipv6 (note, we cannot use user-data to > > indicate > > this preference as user-data is processed after networking is up as > > user-data > > can be fetched via network URL). > > I think this would be a good compromise to get things moving. In my case of > downloading the cloud image and specifying NoCloudNet via the SMBIOS serial > number option, another key might be the simplest path. Something like: > > n=4 : configure IPv4 only (default now) > n=6 : configure IPv6 only > n=64 : configure both stacks > > I might then pass in: > ds=nocloud-net;n=6;h=cloudtest;i=iid-local01;s=http://gemini.home.nivex.net/cidata/ I like this a lot. This example would only apply to NoCloud datasource shorthand, which is fine, but we should expand this into a general form where users can control the policy for any datasource. I suspect we might bike-shed a bit on the long and short form of the config. We internally refer to 'fallback' network config, and to express what the current default configuration in long form. I've started a hackmd document. https://hackmd.io/hoAMkuQ8S_uj62ot3St9fw > > Of course it's easy for me to throw out such a suggestion since I don't know > how much is involved in the codebase. Is this something that might be doable > in the near future? Definitely! Would you be interested in contributing? We can help shepard the work along the way. -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1878682 Title: IPv6 not enabled by default To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/cloud-init/+bug/1878682/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1878682] Re: IPv6 not enabled by default
Ryan, Thank you so much for taking the time to go over the constraints and competing goals. It's always helpful to know what constituencies are involved. > That said, I do think that cloud-init can make it easier to enable ipv6 where > it's desired (but without having to write your own network config). We're > looking into supporting a kernel parameter or cloud-config option to enable > the default config to block on ipv6 (note, we cannot use user-data to indicate > this preference as user-data is processed after networking is up as user-data > can be fetched via network URL). I think this would be a good compromise to get things moving. In my case of downloading the cloud image and specifying NoCloudNet via the SMBIOS serial number option, another key might be the simplest path. Something like: n=4 : configure IPv4 only (default now) n=6 : configure IPv6 only n=64 : configure both stacks I might then pass in: ds=nocloud-net;n=6;h=cloudtest;i=iid-local01;s=http://gemini.home.nivex.net/cidata/ Of course it's easy for me to throw out such a suggestion since I don't know how much is involved in the codebase. Is this something that might be doable in the near future? -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1878682 Title: IPv6 not enabled by default To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/cloud-init/+bug/1878682/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1878682] Re: IPv6 not enabled by default
Hi Kevin, > In the year 2020, IPv6 should be a first class citizen and I should not need > to provide additional config for it to work right. The issue here is not > that I can't find a workaround to make my scenarios work. The issue here is > that I should not need any such workarounds on a modern network. Thanks. I understand, and in general, agree with the goal. Our challenge is how to balance default configurations with competing goals. We can enable lots of default features for maximum compatibility; however there is almost always a trade-off. The trade off for IPv6 is significant delay during boot if IPv6 is not present. Cloud-init blocks during boot to ensure that networking is configured (as expected) before continuing to boot; later stages of boot can expect that networking is available for fetching data from the network. In the case of IPv6, and *blocking* for IPv6 during boot there is a very high cost in networks where IPv6 is not available. The Ubuntu kernel already accepts IPV6 Router Advertisements by default, but asynchronously from the systems network configuration. As mentioned, we can enable blocking on RAs by default but there is a very large boot time delay for networks without RAs. As an example, I've two instance differing only in one where I've enabled blocking for RAs and the boot time is 12 seconds slower due to waiting on RAs. # default fallback networking (dhcp4 only) root@g3:~# systemd-analyze Startup finished in 8.132s (userspace) graphical.target reached after 7.603s in userspace # dhcp4 and accept-ra: true root@g4:~# systemd-analyze Startup finished in 19.988s (userspace) graphical.target reached after 19.472s in userspace # using systemd-analyze critical-chain we can see where the time went root@g3:~# systemd-analyze critical-chain The time when unit became active or started is printed after the "@" character. The time the unit took to start is printed after the "+" character. graphical.target @7.603s └─multi-user.target @7.602s └─rsyslog.service @2.524s +5.074s └─basic.target @2.491s └─sockets.target @2.491s └─snapd.socket @2.489s +2ms └─sysinit.target @2.485s └─cloud-init.service @1.853s +631ms └─systemd-networkd-wait-online.service @1.608s +243ms └─systemd-networkd.service @1.326s +279ms └─network-pre.target @1.325s └─cloud-init-local.service @747ms +578ms └─systemd-journald.socket @165ms └─system.slice @162ms └─-.slice @162ms root@g4:~# systemd-analyze Startup finished in 19.988s (userspace) graphical.target reached after 19.472s in userspace root@g4:~# systemd-analyze critical-chain The time when unit became active or started is printed after the "@" character. The time the unit took to start is printed after the "+" character. graphical.target @19.472s └─multi-user.target @19.472s └─rsyslog.service @14.378s +5.092s └─basic.target @14.338s └─sockets.target @14.338s └─snapd.socket @14.333s +4ms └─sysinit.target @14.329s └─cloud-init.service @13.686s +642ms └─systemd-networkd-wait-online.service @1.672s +12.010s └─systemd-networkd.service @1.376s +290ms └─network-pre.target @1.375s └─cloud-init-local.service @781ms +593ms └─systemd-journald.socket @159ms └─system.slice @156ms └─-.slice @156ms And with debugging on we can see where networkd is sending Router Solicitations and waiting for a response. [205653.809964] g4 systemd[1]: Starting Network Service... ... [205654.706539] g4 systemd-networkd[191]: NDISC: Sent Router Solicitation, next solicitation in 4s [205658.783325] g4 systemd-networkd[191]: NDISC: Sent Router Solicitation, next solicitation in 7s [205659.231119] g4 systemd-networkd[191]: rtnl: received non-static neighbor, ignoring. [205666.110936] g4 systemd-networkd[191]: NDISC: No RA received before link confirmation timeout [205666.112556] g4 systemd-networkd[191]: NDISC: Invoking callback for 'timeout' event. [205666.112822] g4 systemd-networkd[191]: eth0: State changed: configuring -> configured [205666.113225] g4 systemd-networkd-wait-online[192]: managing: eth0 [205666.116688] g4 systemd[1]: Finished Wait for Network to be Configured. If we were to change the default to block for IPv6, and every cloud image boot were to block 12 seconds waiting for an RA, I'm positive the opposite bug would be filed requesting that we don't enable ipv6 when it's not widely deployed. On clouds where ipv6 is available, those platforms provide cloud-init with networking configuration indicating that cloud-init should turn on ipv6 and expect a response. This prevents instances from waiting for ipv6 responses which will never come and delaying boot. That said, I do think that cloud-init
[Bug 1878682] Re: IPv6 not enabled by default
> If you are not providing any network config to cloud-init, current the default fallback is to DHCP (ipv4) only This is precisely the bug I am reporting. > And for ipv6 config, I suspect you'll want to provide an netplan config with: In the year 2020, IPv6 should be a first class citizen and I should not need to provide additional config for it to work right. The issue here is not that I can't find a workaround to make my scenarios work. The issue here is that I should not need any such workarounds on a modern network. ** Changed in: cloud-init (Ubuntu) Status: Incomplete => New -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1878682 Title: IPv6 not enabled by default To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/cloud-init/+bug/1878682/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1878682] Re: IPv6 not enabled by default
In your dual stack example, can you run cloud-init collect-logs and attach the created tarball? In particular the cloud-init.log and what, if any network configuration you're supplying in your NoCloud datasource. > This tells me that someone enabled dhcp4 but not dhcp6 for cloud-init. We > need to make sure we have feature parity across the protocols. If you are not providing any network config to cloud-init, current the default fallback is to DHCP (ipv4) only; Ubuntu itself defaults to accepting IPv6 RAs, but asynchronously from boot. And for ipv6 config, I suspect you'll want to provide an netplan config with: network: version: 2 ethernets: primary: match: macaddress: 52:54:00:4d:27:e9 dhcp4: true dhcp6: true accept-ra: yes Which ensures that systemd-networkd-wait-online.service will wait the required time for querying and accepting any RA. ** Changed in: cloud-init (Ubuntu) Status: New => Incomplete -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1878682 Title: IPv6 not enabled by default To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/cloud-init/+bug/1878682/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1878682] Re: IPv6 not enabled by default
Executive summary: I expected this test to go much differently. This bug
should probably be "cloud-init does not wait for IPv6 config in dual-
stack network"
--
One of my initial tests was in my primary dual-stack network, and it
also worked similar to your example.
Excerpt from my deploy script:
iid="iid-$(printf "%x" $(date +%s))"
virt-install --name ${vm_name} --memory ${mem_size} \
--sysinfo
system_serial="ds=nocloud-net;h=${vm_name};i=${iid};s=http://gemini.home.nivex.net/cidata/;
\
--controller=scsi,model=virtio-scsi \
--disk "path=${vdisk},bus=scsi,discard=unmap" \
--import \
--os-variant ubuntu18.04 \
--machine q35 \
--network network=ovs-net,target=ve0-${vm_name} \
--graphics none \
--noautoconsole
cloud-init[321]: Cloud-init v. 20.1-10-g71af48df-0ubuntu5 running 'init' at
Tue, 19 May 2020 23:03:21 +. Up 6.19 seconds.
cloud-init[321]: ci-info: ++Net device
info+++
cloud-init[321]: ci-info:
++---++---++---+
cloud-init[321]: ci-info: | Device | Up | Address |
Mask | Scope | Hw-Address|
cloud-init[321]: ci-info:
++---++---++---+
cloud-init[321]: ci-info: | enp1s0 | True |172.31.3.134|
255.255.255.0 | global | 52:54:00:4d:27:e9 |
cloud-init[321]: ci-info: | enp1s0 | True | fe80::5054:ff:fe4d:27e9/64 |
. | link | 52:54:00:4d:27:e9 |
cloud-init[321]: ci-info: | lo | True | 127.0.0.1 |
255.0.0.0 | host | . |
cloud-init[321]: ci-info: | lo | True | ::1/128 |
. | host | . |
cloud-init[321]: ci-info: | sit0 | False | . |
. | .| . |
cloud-init[321]: ci-info: | tunl0 | False | . |
. | .| . |
cloud-init[321]: ci-info:
++---++---++---+
cloud-init[321]: ci-info: +Route IPv4
info++
cloud-init[321]: ci-info:
+---+-++-+---+---+
cloud-init[321]: ci-info: | Route | Destination | Gateway | Genmask
| Interface | Flags |
cloud-init[321]: ci-info:
+---+-++-+---+---+
cloud-init[321]: ci-info: | 0 | 0.0.0.0 | 172.31.3.1 | 0.0.0.0
| enp1s0 | UG |
cloud-init[321]: ci-info: | 1 | 172.31.3.0 | 0.0.0.0 | 255.255.255.0
| enp1s0 | U |
cloud-init[321]: ci-info: | 2 | 172.31.3.1 | 0.0.0.0 | 255.255.255.255
| enp1s0 | UH |
cloud-init[321]: ci-info:
+---+-++-+---+---+
cloud-init[321]: ci-info: +++Route IPv6 info+++
cloud-init[321]: ci-info: +---+-+-+---+---+
cloud-init[321]: ci-info: | Route | Destination | Gateway | Interface | Flags |
cloud-init[321]: ci-info: +---+-+-+---+---+
cloud-init[321]: ci-info: | 1 | fe80::/64 |:: | enp1s0 | U |
cloud-init[321]: ci-info: | 3 |local|:: | enp1s0 | U |
cloud-init[321]: ci-info: | 4 | ff00::/8 |:: | enp1s0 | U |
cloud-init[321]: ci-info: +---+-+-+---+---+
Note the absence of an IPv6 global address or default route. We end up
resolving the HTTP server via IPv4 DNS.
We also see in the access log on the webserver that the link-local address was
used:
fe80::5054:ff:fe4d:27e9 - - [19/May/2020:19:03:23 -0400] "GET /cidata/meta-data
HTTP/1.1" 200 256 "-" "Cloud-Init/20.1-10-g71af48df-0ubuntu5"
fe80::5054:ff:fe4d:27e9 - - [19/May/2020:19:03:23 -0400] "GET /cidata/user-data
HTTP/1.1" 200 2059 "-" "Cloud-Init/20.1-10-g71af48df-0ubuntu5"
Let's try out this example:
http router - cloud
server vlan1 | vlan101 device
|
| vlan121
|
dns server
vlan101 has an IPv6 router that announces a prefix and RDNSS
information. It has no IPv4 available.
We run an almost identical command to above, but putting the test VM in the new
VLAN with a portgroup parameter:
virt-install --name ${vm_name} --memory ${mem_size} \
--sysinfo
system_serial="ds=nocloud-net;h=${vm_name};i=${iid};s=http://gemini.home.nivex.net/cidata/;
\
--controller=scsi,model=virtio-scsi \
--disk "path=${vdisk},bus=scsi,discard=unmap" \
--import \
--os-variant ubuntu18.04 \
--machine q35 \
--network network=ovs-net,portgroup=v6only-slaac,target=ve0-${vm_name} \
--graphics none \
--noautoconsole
And here's where it got interesting.
[Bug 1878682] Re: IPv6 not enabled by default
Hi Kevin,
I did some testing and ipv6 seems to work as expected for me. I did the
following:
1. `virsh net-edit default` and added:
to have IPv6 configured on virbr0. As the prefix is a /64 dnsmasq will
automatically send router advertisements on the interface, allowing
SLAAC configuration.
2. `virsh net-destroy default`
`virsh net-start default`
`ip -c addr show virbr0` <- to check the v6 addr is there
3. Downloaded ubuntu-20.04-server-cloudimg-amd64.img from
cloud-images.ubuntu.com
4. qemu-img create -f qcow2 -F qcow2 -o backing_file=ubuntu-20.04
-server-cloudimg-amd64.img focal-v6test.qcow2
6. Created the following meta-data and user-data files:
$ cat meta-data
instance-id: iid-local01
local-hostname: cloudimg
$ cat user-data
#cloud-config
password: passw0rd
chpasswd: { expire: False }
ssh_pwauth: True
6. Served them over IPv6 with: `python3 -m http.server --bind ::`
7. Deployed the VM with:
virt-install --name focal-v6test --os-variant=ubuntu20.04 --disk path
=focal-v6test.qcow2 --import --sysinfo 'system.serial=ds=nocloud-
net;s=http://[2001:db8::1]:8000/'
Everything worked as expected: the cloud-init picked up user-data and
meta-data via ipv6 and configured the VM accordingly.
If something is not working for you I doubt the cause is IPv6 not being
enabled. How are IPv6 addresses assigned on your network?
I'm marking this report as Incomplete for the moment, please set it back
to New after commenting back, and we'll look at it again. Thanks!
** Changed in: cloud-init (Ubuntu)
Status: New => Incomplete
--
You received this bug notification because you are a member of Ubuntu
Bugs, which is subscribed to Ubuntu.
https://bugs.launchpad.net/bugs/1878682
Title:
IPv6 not enabled by default
To manage notifications about this bug go to:
https://bugs.launchpad.net/ubuntu/+source/cloud-init/+bug/1878682/+subscriptions
--
ubuntu-bugs mailing list
ubuntu-bugs@lists.ubuntu.com
https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
