Re: [libvirt] Fwd: libvirtd failing on MacOS in setgroups
On Mon, 30 Sep 2019 at 21:05, Bruno Haible wrote: > > Daniel P. Berrangé wrote: > > > For what purpose is libvirt or QEMU using setgroups()? What goes wrong if > > > setgroups() fails? On macOS, as far as I can see, everything works as expected without it. So not sure if it's actually needed? > > QEMU potentially needs access to files owned by a supplementary group. > > On Linux for example, /dev/kvm is often owned by 'kvm' group, but the > > 'qemu' user on Fedora has 'qemu' group as its primary group. So QEMU > > would be unable to open /dev/kvm without the setgroups call to set up > > supplementary groups. > > Ah, it's libvirt which calls setgroups and qemu which needs the groups. > Then my suggested workaround that consists of overriding setgroups() and > open() won't work. > > > > - Is using the first 16 groups and ignoring the extra ones an acceptable > > > solution? > > > > Certainly that's better than just ignoring groups entirely, as it will > > work for many more cases, even if not perfect. > > Hmm. If the group of /dev/kvm comes at 17th group, it will still not work. > I.e. it will be unreliable. > > Then, how about if libvirt collects the set of groups that qemu might need > for accessing devices (surely less than 16), then fills up the remaining > up to 16 slots with secondary groups? Admittedly it makes qemu less > self-contained. But given that setgroups() works only for root on macOS [1] > I see no better way. Note that /dev/kvm is for linux and does not exist on macOS. Unless we identify specific devices on macOS that qemu requires access to, then something like the following might work? https://github.com/furlongm/libvirt/commit/01a1d3d0e37c7f81a04da2e9707ac1c39f4642b9 Seems to work correctly for me (virsh capabilities now returns the correct output, and VMs run). -- Marcus Furlong -- libvir-list mailing list libvir-list@redhat.com https://www.redhat.com/mailman/listinfo/libvir-list
Re: [libvirt] Fwd: libvirtd failing on MacOS in setgroups
On Fri, 30 Aug 2019 at 21:33, Bruno Haible wrote: > > Hi Eric, > > > I wonder if gnulib could provide a workaround setgroups() that overcomes > > this issue > > I don't see how a workaround could look like. The problem is not the value > of NGROUPS_MAX in user-space, but the same value NGROUPS_MAX in the kernel. > More precisely, in the Darwin kernel file bsd/kern/kern_prot.c there is a > function 'setgroups1', that contains the common implementation of the > setgroups() and initgroups() system call, and this function fails with EINVAL > if the number of groups in the set is > NGROUPS. In the kernel sources, > NGROUPS is defined as NGROUPS_MAX, and NGROUPS_MAX is defined as 16. > > So, the situation on macOS has not changed since this page was written: > https://www.j3e.de/ngroups.html > > What kind of workaround are you imagining? That we override open(), > access(), eaccess() to call setgroups() first, in an intelligent way? > That would be quite gross. > > For what purpose is libvirt or QEMU using setgroups()? FWIW I compiled libvirt without the setgroups code on Mac and it worked as expected. Not sure what the implications of that are though? Marcus. -- Marcus Furlong -- libvir-list mailing list libvir-list@redhat.com https://www.redhat.com/mailman/listinfo/libvir-list
Re: [libvirt] Fwd: libvirtd failing on MacOS in setgroups
On Thu, 22 Aug 2019 at 05:34, Daniel P. Berrangé wrote:
>
> On Wed, Aug 21, 2019 at 09:34:05PM +0300, Roman Bolshakov wrote:
> > On Wed, Aug 21, 2019 at 05:55:51PM +0100, Daniel P. Berrangé wrote:
> > > On Wed, Aug 21, 2019 at 12:47:03PM -0400, Marcus Furlong wrote:
> > > > On Wed, 21 Aug 2019 at 08:23, Daniel P. Berrangé
> > > > wrote:
> > > > >
> > > > > On Tue, Aug 20, 2019 at 11:11:07AM -0400, Marcus Furlong wrote:
> > > > > > Resend to libvir-list in case that is more appropriate:
> > > > > >
> > > > > >
> > > > > > Hi,
> > > > > >
> > > > > > I get the following error when running libvirtd on MacOS as root:
> > > > > >
> > > > > > 2019-07-11 00:12:33.673+: 123145573953536: error :
> > > > > > qemuProcessQMPLaunch:8501 : internal error: Failed to start QEMU
> > > > > > binary /usr/local/bin/qemu-system-x86_64 for probing: libvirt:
> > > > > > error
> > > > > > : cannot set supplemental groups: Invalid argument
> > > > >
> > > > > Are you able to run 'strace' (or whatever MacOS eqiuv is) to see
> > > > > the values passed to setgroups when it fails ?
> > > >
> > > > I ran `dtruss -f -l -s /usr/local/sbin/libvirtd` but the setgroups
> > > > calls seem to be missing.
> > > >
> > > > Looking at other sources, it seems like some have special treatment of
> > > > setgroups on MacOS, e.g. samba:
> > > >
> > > >
> > > > https://github.com/samba-team/samba/blob/v4-11-stable/source3/smbd/sec_ctx.c#L261-#L305
> > > >
> > > > Perhaps this is needed for libvirt?
> > >
> > > The capping of ngroups to NGROUPS_MAX looks like a possibe reason.
> > >
> > > Adding this debug might show us if we're exceeding it:
> > >
> > > diff --git a/src/util/virutil.c b/src/util/virutil.c
> > > index 89d2cf011f..effc02b898 100644
> > > --- a/src/util/virutil.c
> > > +++ b/src/util/virutil.c
> > > @@ -1043,6 +1043,7 @@ virSetUIDGID(uid_t uid, gid_t gid, gid_t *groups
> > > ATTRIBUTE_UNUSED,
> > > }
> > >
> > > # if HAVE_SETGROUPS
> > > +VIR_DEBUG("setgroups %d max %d", ngroups, NGROUPS_MAX);
> > > if (gid != (gid_t)-1 && setgroups(ngroups, groups) < 0) {
> > > virReportSystemError(errno, "%s",
> > > _("cannot set supplemental groups"));
> > >
> > >
> >
> > Yes, there's an overflow:
> > 2019-08-21 18:25:37.943+: 123145413914624: debug : virSetUIDGID:1046 :
> > setgroups 23 max 16
> >
> > Related samba ticket (it also has references to the python and dovecot
> > issues):
> > https://bugzilla.samba.org/show_bug.cgi?id=8773
>
> The quick hack is to simply truncate groups to NGROUPS_MAX.
>
> Reading that bug, the proper fix looks quite alot more complex
Just to confirm that the root cause of the issue was the above block,
I bypassed the HAVE_SETGROUPS section completely on Darwin, and things
work as expected.
Where would be an appropriate place to track this bug?
Marcus.
--
Marcus Furlong
--
libvir-list mailing list
libvir-list@redhat.com
https://www.redhat.com/mailman/listinfo/libvir-list
Re: [libvirt] Fwd: libvirtd failing on MacOS in setgroups
On Wed, 21 Aug 2019 at 08:23, Daniel P. Berrangé wrote: > > On Tue, Aug 20, 2019 at 11:11:07AM -0400, Marcus Furlong wrote: > > Resend to libvir-list in case that is more appropriate: > > > > > > Hi, > > > > I get the following error when running libvirtd on MacOS as root: > > > > 2019-07-11 00:12:33.673+: 123145573953536: error : > > qemuProcessQMPLaunch:8501 : internal error: Failed to start QEMU > > binary /usr/local/bin/qemu-system-x86_64 for probing: libvirt: error > > : cannot set supplemental groups: Invalid argument > > Are you able to run 'strace' (or whatever MacOS eqiuv is) to see > the values passed to setgroups when it fails ? I ran `dtruss -f -l -s /usr/local/sbin/libvirtd` but the setgroups calls seem to be missing. Looking at other sources, it seems like some have special treatment of setgroups on MacOS, e.g. samba: https://github.com/samba-team/samba/blob/v4-11-stable/source3/smbd/sec_ctx.c#L261-#L305 Perhaps this is needed for libvirt? Marcus. -- Marcus Furlong -- libvir-list mailing list libvir-list@redhat.com https://www.redhat.com/mailman/listinfo/libvir-list
[libvirt] Fwd: libvirtd failing on MacOS in setgroups
: internal error: Failed to start QEMU binary /usr/local/bin/qemu-system-sparc for probing: libvirt: error : cannot set supplemental groups: Invalid argument 2019-07-11 00:12:33.619+: 123145573953536: error : qemuProcessQMPLaunch:8501 : internal error: Failed to start QEMU binary /usr/local/bin/qemu-system-sparc64 for probing: libvirt: error : cannot set supplemental groups: Invalid argument 2019-07-11 00:12:33.619+: 123145573953536: warning : virQEMUCapsLogProbeFailure:4578 : Failed to probe capabilities for /usr/local/bin/qemu-system-sparc64: internal error: Failed to start QEMU binary /usr/local/bin/qemu-system-sparc64 for probing: libvirt: error : cannot set supplemental groups: Invalid argument 2019-07-11 00:12:33.667+: 123145573953536: error : qemuProcessQMPLaunch:8501 : internal error: Failed to start QEMU binary /usr/local/bin/qemu-system-unicore32 for probing: libvirt: error : cannot set supplemental groups: Invalid argument 2019-07-11 00:12:33.667+: 123145573953536: warning : virQEMUCapsLogProbeFailure:4578 : Failed to probe capabilities for /usr/local/bin/qemu-system-unicore32: internal error: Failed to start QEMU binary /usr/local/bin/qemu-system-unicore32 for probing: libvirt: error : cannot set supplemental groups: Invalid argument 2019-07-11 00:12:33.673+: 123145573953536: error : qemuProcessQMPLaunch:8501 : internal error: Failed to start QEMU binary /usr/local/bin/qemu-system-x86_64 for probing: libvirt: error : cannot set supplemental groups: Invalid argument 2019-07-11 00:12:33.673+: 123145573953536: warning : virQEMUCapsLogProbeFailure:4578 : Failed to probe capabilities for /usr/local/bin/qemu-system-x86_64: internal error: Failed to start QEMU binary /usr/local/bin/qemu-system-x86_64 for probing: libvirt: error : cannot set supplemental groups: Invalid argument 2019-07-11 00:12:33.679+: 123145573953536: error : qemuProcessQMPLaunch:8501 : internal error: Failed to start QEMU binary /usr/local/bin/qemu-system-xtensa for probing: libvirt: error : cannot set supplemental groups: Invalid argument 2019-07-11 00:12:33.679+: 123145573953536: warning : virQEMUCapsLogProbeFailure:4578 : Failed to probe capabilities for /usr/local/bin/qemu-system-xtensa: internal error: Failed to start QEMU binary /usr/local/bin/qemu-system-xtensa for probing: libvirt: error : cannot set supplemental groups: Invalid argument 2019-07-11 00:12:33.685+: 123145573953536: error : qemuProcessQMPLaunch:8501 : internal error: Failed to start QEMU binary /usr/local/bin/qemu-system-xtensaeb for probing: libvirt: error : cannot set supplemental groups: Invalid argument 2019-07-11 00:12:33.685+: 123145573953536: warning : virQEMUCapsLogProbeFailure:4578 : Failed to probe capabilities for /usr/local/bin/qemu-system-xtensaeb: internal error: Failed to start QEMU binary /usr/local/bin/qemu-system-xtensaeb for probing: libvirt: error : cannot set supplemental groups: Invalid argument -- Marcus Furlong -- libvir-list mailing list libvir-list@redhat.com https://www.redhat.com/mailman/listinfo/libvir-list
