Public bug reported: On Google Cloud VMs with multiple NICs, the MDS (`169.254.169.254`) should only be reachable via the primary interface, and any secondary interfaces (e.g. `ens5`) should not have access. Google’s `google-guest- agent` package writes a `netplan` config and a `systemd-networkd` drop in that should disable DNS routing via `ens5`. FWIW this configuration does behave as expected on Noble 24.04 (`netplan.io` version `1.1.2-2~ubuntu24.04.2`), but on Jammy 22.04 (`netplan.io` version `0.106.1-7ubuntu0.22.04.4`) the metadata server remains reachable via `ens5`.
I confirmed this behaviour on jammy VM: ``` ubuntu@jammy-multi-nic-check-v3:~$ curl -H "Metadata-Flavor: Google" http://metadata.google.internal/computeMetadata/v1/project/project-id --interface ens5 ubuntu-kajiya ``` whereas on a Noble VM the same command just times out. ### Configs written by the `google-guest-agent` pkg On both Jammy and Noble, the `google-guest-agent` pkg writes these files: 1. `netplan` yaml @ `/run/netplan/20-google-guest-agent-ethernet.yaml`: ``` network: version: 2 ethernets: ens5: match: name: ens5 mtu: 1460 dhcp4: true dhcp4-overrides: use-domains: false ``` 2. `systemd-networkd` config @ `/run/systemd/network/10-netplan- ens5.network.d/override.conf`: ``` [Match] Name = ens5 [Network] DHCP = ipv4 DNSDefaultRoute = false [DHCPv4] RoutesToDNS = false RoutesToNTP = false ``` The intent of both of these being that the metadata server/DNS IP addr `169.254.169.254` is only reachable via the primary NIC. ### Some additional netplan overrides I tried on Jammy I tried to brute force things on Jammy by adding this (more explicit) override to `/etc/netplan`: ``` network: version: 2 ethernets: ens5: dhcp4: true dhcp4-overrides: use-dns: false use-domains: false use-routes: false use-ntp: false dhcp6: false ``` but I could still access the MDS via `curl -H "Metadata-Flavor: Google" http://metadata.google.internal/computeMetadata/v1/project/project-id --interface ens5` unfortunately. ### Steps to reproduce 1. Create a 2, two-NIC instances on Google Cloud * One Jammy, the other Noble 2. Verify the files written by `google-guest-agent` - check that the following files exist and match the snippets above on both instances: ``` sudo cat /run/netplan/20-google-guest-agent-ethernet.yaml ``` ``` sudo cat /run/systemd/network/10-netplan-ens5.network.d/override.conf ``` 3. Move the `systemd` config to `/etc/` (to exclude any issues that may have been introduced by it being in `/run/`) on the Jammy instance: ``` sudo mkdir --parents /etc/systemd/network/10-netplan-ens5.network.d sudo cp /run/systemd/network/10-netplan-ens5.network.d/override.conf \ /etc/systemd/network/10-netplan-ens5.network.d/override.conf sudo systemctl daemon-reload sudo systemctl restart systemd-networkd ``` 4. [Optional] Also add the more explicit `netplan` override from above on the Jammy instance 5. Compare behaviour on Jammy vs Noble * On both instances, run: ``` curl -H "Metadata-Flavor: Google" http://metadata.google.internal/computeMetadata/v1/project/project-id --interface ens5 ``` ### Expected behaviour * On both Jammy and Noble: * `ens5` should not have any route to the MDS; `curl -H "Metadata-Flavor: Google" http://metadata.google.internal/computeMetadata/v1/project/project-id --interface ens5` should fail (no route, timeout, etc.) * The primary interface (e.g. `ens4`) should still reach metadata as expected ### Actual behaviour * On Jammy, `curl -H "Metadata-Flavor: Google" http://metadata.google.internal/computeMetadata/v1/project/project-id --interface ens5` indeed reports back the project-id * Noble behaves better and that command times out ### Misc questions * Is this a known limitation or bug in netplan on Jammy when used with systemd-networkd’s `RoutesToDNS`/`DNSDefaultRoute` and/or `dhcp4-overrides`? * Is there a recommended netplan way on Jammy to ensure that a secondary NIC still uses DHCP for addressing, but doesn't inherit any DNS/metadata routes? * If this is indeed a bug, is there a backportable fix or a configuration pattern you’d recommend for Jammy users? Thanks in advance :) ** Affects: netplan.io (Ubuntu) Importance: Undecided Status: New -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/2131966 Title: Netplan on Jammy doesn't prevent metadata/DNS routing on a secondary NIC despite any DHCP overrides given To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/netplan.io/+bug/2131966/+subscriptions -- ubuntu-bugs mailing list [email protected] https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
