[Bug 1724441] Re: No external network access for VMs
Speaking specifically of the libvirt task: in comment #12 Christian dug into this issue and concluded that this is not a libvirt bug, proposing to set it to Invalid and focus on LP: #1754871, which is now Fix Released. I see no elements here pointing to a libvirt bug, but as I can't fully rule it out I'm setting the libvirt task status to Incomplete for the moment. If you think we do have a libvirt bug here please share your findings/reasoning, change the task status back to New and we'll look at it again. Thanks! ** Changed in: libvirt (Ubuntu) Status: Confirmed => Incomplete -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1724441 Title: No external network access for VMs To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/gnome-boxes/+bug/1724441/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1724441] Re: No external network access for VMs
Same problem again: guests can resolve internet addresses, but are unable to access them: # host google.com google.com has address 172.217.20.238 google.com has IPv6 address 2a00:1450:4016:801::200e google.com mail is handled by 30 alt2.aspmx.l.google.com. google.com mail is handled by 10 aspmx.l.google.com. google.com mail is handled by 20 alt1.aspmx.l.google.com. google.com mail is handled by 50 alt4.aspmx.l.google.com. google.com mail is handled by 40 alt3.aspmx.l.google.com. # ping google.com PING google.com (172.217.20.238): 56 data bytes ^C --- google.com ping statistics --- 4 packets transmitted, 0 packets received, 100.0% packet loss iptables is set as expected: # iptables-save # Generated by iptables-save v1.8.5 on Fri Mar 26 13:03:26 2021 *filter :INPUT ACCEPT [0:0] :FORWARD ACCEPT [0:0] :OUTPUT ACCEPT [0:0] :LIBVIRT_INP - [0:0] :LIBVIRT_OUT - [0:0] :LIBVIRT_FWO - [0:0] :LIBVIRT_FWI - [0:0] :LIBVIRT_FWX - [0:0] -A INPUT -j LIBVIRT_INP -A FORWARD -j LIBVIRT_FWX -A FORWARD -j LIBVIRT_FWI -A FORWARD -j LIBVIRT_FWO -A OUTPUT -j LIBVIRT_OUT -A LIBVIRT_INP -i virbr8 -p udp -m udp --dport 53 -j ACCEPT -A LIBVIRT_INP -i virbr8 -p tcp -m tcp --dport 53 -j ACCEPT -A LIBVIRT_INP -i virbr8 -p udp -m udp --dport 67 -j ACCEPT -A LIBVIRT_INP -i virbr8 -p tcp -m tcp --dport 67 -j ACCEPT -A LIBVIRT_INP -i virbr1 -p udp -m udp --dport 53 -j ACCEPT -A LIBVIRT_INP -i virbr1 -p tcp -m tcp --dport 53 -j ACCEPT -A LIBVIRT_INP -i virbr1 -p udp -m udp --dport 67 -j ACCEPT -A LIBVIRT_INP -i virbr1 -p tcp -m tcp --dport 67 -j ACCEPT -A LIBVIRT_INP -i virbr0 -p udp -m udp --dport 53 -j ACCEPT -A LIBVIRT_INP -i virbr0 -p tcp -m tcp --dport 53 -j ACCEPT -A LIBVIRT_INP -i virbr0 -p udp -m udp --dport 67 -j ACCEPT -A LIBVIRT_INP -i virbr0 -p tcp -m tcp --dport 67 -j ACCEPT -A LIBVIRT_OUT -o virbr8 -p udp -m udp --dport 53 -j ACCEPT -A LIBVIRT_OUT -o virbr8 -p tcp -m tcp --dport 53 -j ACCEPT -A LIBVIRT_OUT -o virbr8 -p udp -m udp --dport 68 -j ACCEPT -A LIBVIRT_OUT -o virbr8 -p tcp -m tcp --dport 68 -j ACCEPT -A LIBVIRT_OUT -o virbr1 -p udp -m udp --dport 53 -j ACCEPT -A LIBVIRT_OUT -o virbr1 -p tcp -m tcp --dport 53 -j ACCEPT -A LIBVIRT_OUT -o virbr1 -p udp -m udp --dport 68 -j ACCEPT -A LIBVIRT_OUT -o virbr1 -p tcp -m tcp --dport 68 -j ACCEPT -A LIBVIRT_OUT -o virbr0 -p udp -m udp --dport 53 -j ACCEPT -A LIBVIRT_OUT -o virbr0 -p tcp -m tcp --dport 53 -j ACCEPT -A LIBVIRT_OUT -o virbr0 -p udp -m udp --dport 68 -j ACCEPT -A LIBVIRT_OUT -o virbr0 -p tcp -m tcp --dport 68 -j ACCEPT -A LIBVIRT_FWO -s 172.19.18.0/24 -i virbr8 -j ACCEPT -A LIBVIRT_FWO -i virbr8 -j REJECT --reject-with icmp-port-unreachable -A LIBVIRT_FWO -i virbr1 -j REJECT --reject-with icmp-port-unreachable -A LIBVIRT_FWO -s 172.19.10.0/24 -i virbr0 -j ACCEPT -A LIBVIRT_FWO -i virbr0 -j REJECT --reject-with icmp-port-unreachable -A LIBVIRT_FWI -d 172.19.18.0/24 -o virbr8 -m conntrack --ctstate RELATED,ESTABLISHED -j ACCEPT -A LIBVIRT_FWI -o virbr8 -j REJECT --reject-with icmp-port-unreachable -A LIBVIRT_FWI -o virbr1 -j REJECT --reject-with icmp-port-unreachable -A LIBVIRT_FWI -d 172.19.10.0/24 -o virbr0 -j ACCEPT -A LIBVIRT_FWI -o virbr0 -j REJECT --reject-with icmp-port-unreachable -A LIBVIRT_FWX -i virbr8 -o virbr8 -j ACCEPT -A LIBVIRT_FWX -i virbr1 -o virbr1 -j ACCEPT -A LIBVIRT_FWX -i virbr0 -o virbr0 -j ACCEPT COMMIT # Completed on Fri Mar 26 13:03:26 2021 # Generated by iptables-save v1.8.5 on Fri Mar 26 13:03:26 2021 *nat :PREROUTING ACCEPT [0:0] :INPUT ACCEPT [0:0] :POSTROUTING ACCEPT [0:0] :OUTPUT ACCEPT [0:0] :LIBVIRT_PRT - [0:0] -A POSTROUTING -j LIBVIRT_PRT -A LIBVIRT_PRT -s 172.19.18.0/24 -d 224.0.0.0/24 -j RETURN -A LIBVIRT_PRT -s 172.19.18.0/24 -d 255.255.255.255/32 -j RETURN -A LIBVIRT_PRT -s 172.19.18.0/24 ! -d 172.19.18.0/24 -p tcp -j MASQUERADE --to-ports 1024-65535 -A LIBVIRT_PRT -s 172.19.18.0/24 ! -d 172.19.18.0/24 -p udp -j MASQUERADE --to-ports 1024-65535 -A LIBVIRT_PRT -s 172.19.18.0/24 ! -d 172.19.18.0/24 -j MASQUERADE COMMIT # Completed on Fri Mar 26 13:03:26 2021 # Generated by iptables-save v1.8.5 on Fri Mar 26 13:03:26 2021 *mangle :PREROUTING ACCEPT [0:0] :INPUT ACCEPT [0:0] :FORWARD ACCEPT [0:0] :OUTPUT ACCEPT [0:0] :POSTROUTING ACCEPT [0:0] :LIBVIRT_PRT - [0:0] -A POSTROUTING -j LIBVIRT_PRT -A LIBVIRT_PRT -o virbr8 -p udp -m udp --dport 68 -j CHECKSUM --checksum-fill -A LIBVIRT_PRT -o virbr1 -p udp -m udp --dport 68 -j CHECKSUM --checksum-fill -A LIBVIRT_PRT -o virbr0 -p udp -m udp --dport 68 -j CHECKSUM --checksum-fill COMMIT # Completed on Fri Mar 26 13:03:26 2021 IP-forwarding is enabled: # cat /proc/sys/net/ipv4/ip_forward 1 but guests do not receive packets send back to them from servers. I am not absolutely sure if this is the error described here, but I think it is the same. OS: # uname -a Linux ivory 5.8.0-48-generic #54-Ubuntu SMP Fri Mar 19 14:25:20 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux # cat /etc/lsb-release DISTRIB_ID=Ubuntu DISTRIB_RELEASE=20.10 DISTRIB_CODENAME=groovy DISTRIB_DESCRIPTION="Ubuntu 20.10"
[Bug 1724441] Re: No external network access for VMs
Status changed to 'Confirmed' because the bug affects multiple users. ** Changed in: gnome-boxes (Ubuntu) Status: New => Confirmed -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1724441 Title: No external network access for VMs To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/gnome-boxes/+bug/1724441/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1724441] Re: No external network access for VMs
Hi Michael, thank you for the discussion, no need to apologize - I can see your troubles and want to help. But lets dissect the different aspects of it. TL;DR: - path in qemu.conf should not matter - need to set-uid manually is wanted for security reasons - apparmor fixes exist, please chime in on bug 1754871 [2] if you think 18.04 should get that 1. apparmor to be able to use ubuntu-bridge-helper. I pushed a change upstream [3] for bug 1754871 already, that is similar to yours but more limited to what you actually need to get it working. Although this was done at a low prio and my intention was to only pick this up with Ubuntu 18.10 and I was not yet planning to backport the fix. The reason is that it is a config file change everybody can do and the number of use cases depending on it was considered very low. Although I think I could be convinced this is used more often than I thought to push the fix to the apparmor profile at least to all >= Ubuntu 18.04. I'd ask you to post on bug 1754871 if you really think we should make the apparmor change generally available. Since 18.04 is an LTS I think that would be fair. 2. set-uid qemu-bridge-helper is a bit of an ugly case, qemu.bridge-helper is considered "unwanted but sometimes needed" at best I feel. In bug 1754871 we outlined and referred to a bunch of reasons why it won'd be delivered as set-uid (not doing again here). Therefore you'd still need to add the setuid as it was considered to insecure to deliver it with setuid. 3. path in qemu.conf I think changing the path for qemu-bridge-helper in qemu.conf to be correct should be "ok", but I actually think it is not needed. At build time libvirt detects where qemu-bridge-helper is on a system (as it might differ per distribution) and it does: ... checking for sys/inotify.h... yes checking for qemu-bridge-helper... /usr/lib/qemu/qemu-bridge-helper checking for xen_vm_start in -lxenserver... no ... So if nothing is set in the config it should use this path. The path you see commented out in the config file is just an example and that way because of [1] is what it is upstream. Summary: - I think #1 is covered by the bug I referred. - #2 is intentionally staying the way it is. - And #3 should not be an issue - it would be great if you can double-check that. If you agree we can set this bug here back to invalid and you can chime in on [2] to push this to Bionic as well. [1]: https://libvirt.org/git/?p=libvirt.git;a=blob_plain;f=src/qemu/qemu.conf;hb=HEAD [2]: https://bugs.launchpad.net/ubuntu/+source/libvirt/+bug/1754871 [3]: https://libvirt.org/git/?p=libvirt.git;a=commit;h=6a9bdf3f25fb3941d587b3f2877b36e4412d6762 -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1724441 Title: No external network access for VMs To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/gnome-boxes/+bug/1724441/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1724441] Re: No external network access for VMs
The attachment "Changes needed to /etc/qemu.conf" seems to be a patch. If it isn't, please remove the "patch" flag from the attachment, remove the "patch" tag, and if you are a member of the ~ubuntu-reviewers, unsubscribe the team. [This is an automated message performed by a Launchpad user owned by ~brian-murray, for any issues please contact him.] ** Tags added: patch -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1724441 Title: No external network access for VMs To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/gnome-boxes/+bug/1724441/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1724441] Re: No external network access for VMs
** Patch added: "Changes needed to /etc/libvirt/usr.sbin.libvirtd" https://bugs.launchpad.net/ubuntu/+source/gnome-boxes/+bug/1724441/+attachment/5125661/+files/usr.sbin.libvirtd-fixed.diff -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1724441 Title: No external network access for VMs To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/gnome-boxes/+bug/1724441/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1724441] Re: No external network access for VMs
The changes to the apparmour profile aren't minimal, they could probably be tightened up, but I'm not very familiar with it. I also needed to "sudo chmod u+s /usr/lib/qemu/qemu-bridge-helper" again, as well. -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1724441 Title: No external network access for VMs To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/gnome-boxes/+bug/1724441/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1724441] Re: No external network access for VMs
** Patch added: "Changes needed to /etc/qemu.conf" https://bugs.launchpad.net/ubuntu/+source/gnome-boxes/+bug/1724441/+attachment/5125660/+files/qemu.conf-fixed.diff -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1724441 Title: No external network access for VMs To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/gnome-boxes/+bug/1724441/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1724441] Re: No external network access for VMs
Apologies to re-open this for libvirt again, but I just had to go through the same song and dance again after upgrading to 18.04 and fixing the problem this required making some changes to the apparmour profile for libvirt. Will attach a diff of the changes in a moment. -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1724441 Title: No external network access for VMs To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/gnome-boxes/+bug/1724441/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1724441] Re: No external network access for VMs
** Changed in: libvirt (Ubuntu) Status: Invalid => Confirmed -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1724441 Title: No external network access for VMs To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/gnome-boxes/+bug/1724441/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1724441] Re: No external network access for VMs
Hi, the dependency to libvirt atm is only -> libvirt-daemon (Depend) -> libvirt-daemon-system (suggest) And if that would be all that would be needed I'd expect that it could be changed in gnome-boxes. But I wonder about "whichever libvirt package is responsible for creating that network needs to ensure it bridges to all physical devices by default.". The default configuration is to have the default network as "nat". Which means the guests can reach out and since the host has a device to the guest network it can reach the guests. It will give you external network access - if that is all you need, fine - no change in libvirt will be needed. If you actually meant real "bridged to all physical devices" that is unlikely to be the default as it is a too invasive change to the networking on a package install. For now I assume you "just" need external network which you will have - with that the libvirt part of this bug is "invalid" as no change is needed - Let me know if I misunderstood this part of the request. ** Changed in: libvirt (Ubuntu) Status: New => Invalid -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1724441 Title: No external network access for VMs To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/gnome-boxes/+bug/1724441/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1724441] Re: No external network access for VMs
I managed to get this working in the end. Turns out that despite using the libvirt/qemu user session for the VMs, gnome-boxes actually uses the *system* libvirt network named "default" for networking so it works "out of the box". In reality, I had to delete that network and re-create it using virt-manager before it actually started bridging traffic to my physical connection. See this blog post and the comments: http://blog.wikichoon.com/2016/01 /qemusystem-vs-qemusession.html for more information. So to fix this, at a minimum gnome-boxes should depend on libvirt- daemon-system so that the "default" network is running and attendent virbr0 interface is created at startup, and whichever libvirt package is responsible for creating that network needs to ensure it bridges to all physical devices by default. ** Changed in: libvirt (Ubuntu) Status: Incomplete => New -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1724441 Title: No external network access for VMs To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/gnome-boxes/+bug/1724441/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1724441] Re: No external network access for VMs
Hey, thanks for the pointers, I've been looking into this a bit and performed the following: - Set `bridge_helper` correctly in `/etc/libvirt/qemu.conf` - it was commented out and pointing to the wrong binary - Made qemu-bridge-helper set-uid - Created an appropriate /etc/qemu/bridge.conf - Restarted all the services I'm getting a different error message now, at least. As far as I can tell, gnome-boxes is just a libvirt client, so it just expects a working installation. -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1724441 Title: No external network access for VMs To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/gnome-boxes/+bug/1724441/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1724441] Re: No external network access for VMs
Hi, thanks Michael for the report. Looking into this from the libvirt POV. This is - on the virtualization side - a known limitation of qemu:///session Here an example to show the basics. I first started with a test to confirm/debug the issue test xml describing a most basic nat network: # cat testnet.xml testnet That works fine in a normal lifecycle in qemu:///system. $ virsh -c qemu:///system net-destroy testnet $ virsh -c qemu:///system net-start testnet $ virsh -c qemu:///system net-destroy testnet $ virsh -c qemu:///system net-undefine testnet It also works fine if running on qemu:///session as root, but if being a normal user that fails on the "net-start" action. The reason for this is a lack of permissions of the normal user to e.g. define the bridge. In detail you see like: $ virsh -c qemu:///session net-start testnet error: Failed to start network testnet error: error creating bridge interface testnetbr0: Operation not permitted Now there is a way to resolve that usually in the form of "/usr/lib/qemu/qemu-bridge-helper". This is a special tool provided, but not further enabled/preconfigured mostly for: a) being an uncommon case b) security concerns So an admin (or another program using it) has to it set up as needed I wonder if gnome-boxes did something about/with it before which now doesn't work - but if so I don't know what - sorry. If this is an issue in libvirt I'd need more details what worked before and failed now. Maybe you or some gnome people can uncover that? Some references on the general topic: https://wiki.qemu.org/Features/HelperNetworking http://jonaspfannschmidt.com/libvirt_session.html http://isonprojects.com/qemu-bridge-network-in-ubuntu-14-04/ ** Changed in: libvirt (Ubuntu) Status: New => Incomplete -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1724441 Title: No external network access for VMs To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/gnome-boxes/+bug/1724441/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1724441] Re: No external network access for VMs
Added libvirt since I can't add a NAT network to the qemu:///session using virt-manager, either. ** Also affects: libvirt (Ubuntu) Importance: Undecided Status: New -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1724441 Title: No external network access for VMs To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/gnome-boxes/+bug/1724441/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
