Re: SELinux labels change in libvirt
On Thu, Jul 16, 2020 at 11:22:26AM +0300, Ram Lavi wrote:
> On Tue, Jul 14, 2020 at 6:03 PM Daniel P. Berrangé
> wrote:
>
> I checked the dumpxml of the virt-launcher pod (that runs the qemu in
> kubevirt) - it has dynamic policy.
>
>
> +107:+107
> +107:+107
>
>
> Are you saying this a wrong configuration for a kubevirt vmi?
No, this is exactly what I'm expecting and matches the behaviour I
mentioned in my previous mail, where SELinux is not enabled in libvirtd.
This setting is completely unrelated to SELinux. This seclabel is for the
"dac" model, aka Discretionary Access Control, aka plain old POSIX users
and groups. It is used when libvirtd runs as root, and then launches QEMU
as a separate "qemu:qemu" user:group pair.
If selinux / sVirt was in use you would see a separate with
model="selinux".
If you don't see that, then libvirtd is *not* doing any SELinux labelling
at all.
This makes me wonder how you hit the relabelling AVC ? It feels very much
like your custom work to deal with passing in TAP devices has mistakenly
changed something else and thus incorrectly turned on SELinux/sVirt in
libvirtd.
> > If libvirtd is doing relabelling, I'm not sure how it works for anything,
> > let alone tap devices.
> >
> I want to emphasize that in this current situation the reason we fail is
> that we are trying to give libvirt in a non-privileged pod (i.e.
> virt-launcher) a tap device that was created externally (by a privileged
> pod - virt-handler).
> I think the reason it works so far is that libvirt was using only devices
> which were self created so there was no issue to relabel.
I believe the reason it works historically is that libvirtd is not using
SELinux, per your config above, and so was not attempting
todo any relabelling in the first place.
> Our thought was that if we know how libvirt is relabeling then we could
> also do it so that the externally created tap's label will match the
> virt-launcher's.
You almost certainly *will* need to relabel the tap device when passing
it from your privileged container into the unprivileged, but this is not
due to libvirtd. This is because podman is likely using a different
SELinux label for the two containers as they have different privilege
levels. The tap device needs to be given the SELinux label matching the
unprivileged container.
> Is this were libvirt does the relabeling
> https://github.com/libvirt/libvirt/blob/e71e13488dc1aa65456e54a4b41bc925821b4263/src/security/security_selinux.c#L1256
> ?
That's for setting contexts on files, for FDs you need to call
fsetfilecon_raw
> btw the error we get is (from audit)
>
> type=AVC msg=audit(1586956552.265:513): avc: denied { relabelfrom }
> for pid=27423 comm="libvirtd"
> scontext=system_u:system_r:container_t:s0:c143,c582
> tcontext=system_u:system_r:spc_t:s0 tclass=tun_socket permissive=0
This just suggests libvirtd has been misconfigured in some way that causes
SELinux to become enabled again.
You need to do relabelling of the TAP device, but that isn't going to
avoid this AVC - it will just change what you see as the tcontext in
the AVC. You still need to fix the configuration of libvirtd I believe.
If you run 'virsh capabilities' in your container that is showing this
problem, I expect it will say:
selinux
0
system_u:system_r:svirt_t:s0
system_u:system_r:svirt_tcg_t:s0
If you compare with the "virsh capabilities" from a correctly operating
existing KubeVirt install, then I expect selinux won't be reported at
all, which would match your example from earlier.
Regards,
Daniel
--
|: https://berrange.com -o-https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org -o-https://fstop138.berrange.com :|
|: https://entangle-photo.org-o-https://www.instagram.com/dberrange :|
Re: SELinux labels change in libvirt
On Tue, Jul 14, 2020 at 6:03 PM Daniel P. Berrangé
wrote:
> On Tue, Jul 14, 2020 at 04:02:17PM +0300, Ram Lavi wrote:
> > On Tue, Jul 14, 2020 at 3:33 PM Daniel P. Berrangé
> > wrote:
> >
> > > On Tue, Jul 14, 2020 at 03:21:17PM +0300, Ram Lavi wrote:
> > > > Hello all,
> > > >
> > > > tl;dr, can you point me to the point in the libvirt repo where it's
> > > trying
> > > > to change a tap-device's SELinux label?
> > > >
> > > > I am trying to create a tap device with libvirt on a
> > > > super-privileged container, and then use it on another,
> > > > unprivileged container with libvirt.
> > > > User wise, I know I need the super-privileged container to open the
> tap
> > > > device with the user of the unprivileged one - that I already did and
> > > it's
> > > > not the issue.
> > > > But I have a problem when I open the tap device in the
> > > > non-privileged container: the tap device currently has the spc_t
> label
> > > > since the tun_socket inherited the selinux context from the
> > > > super-privileged container who creates it. then libvirt is trying to
> > > change
> > > > the SELinux labels, and since it's not privileged then it fails.
> > > > But I didn't find where and how libvirt is trying to change the tap
> > > > device's label.
> > > >
> > > > Can you point me to that specific code on libvirt?
> > >
> > > If the SELinux policy that libvirtd is running under prevents it from
> > > re-labelling, then TAP devices label failure is just going to be one
> > > out of 100's of labelling failures.
> > >
> > IIUC normally libvirtd would use devices created by itself, so there
> > shouldn't be relabeling failures, right?
>
> Whether or not there are relabeling failures is determined by what
> SELinux policy libvirtd is running under.
>
> > >
> > > Either the SELinux policy needs to be changed to allow libvirtd to
> > > relabel stuff in the normal manner, or you will have to turn off
> > > SELinux support in libvirtd. in /etc/libvirt/qemu.conf via the
> > > param security_driver = "none". If you turn off SELinux in
> > > libvirt, then you no longer have separation of QEMU processes
> > > which may be a security flaw depending on your deplyoment
> > > scenario.
> > >
> > turning SELinux in libvirtd off or allowing libvirt to relabel are
> tempting
> > options but it is not an option I'm afraid due to security concerns.
> > Our plan (or at least this particular effort) is to try to relabel the
> > tun-socket after created in the super-privileged container to be the same
> > one as the one used in the unprivileged one, then it won't have an issue
> > consuming it.
> >
> > btw the change I (or rather Migule's my teammate) made is in this PR,
> where
> > I want to add a tap device in virt-handler (i.e. the super privileged
> > container) to be further uses in virt-launcher (i.e. the non-privileged
> > container): https://github.com/kubevirt/kubevirt/pull/3290
>
> In normal host OS deployment, libvirtd runs under virtd_t, and when
> it spawns QEMU, it will relabel files to svirt_image_t:s0:$MCS, and
> spawn QEMU as svirt_t:s0:$MCS.
>
> My understanding is what in kubevirt, things work differently. Docker
> (or podman), launch the container as container_t:s0:$MCS. libvirtd
> *and* QEMU thus both run as container_t:s0:$MCS. ie All the labelling
> is setup when the container is launched and libvirtd should not do
> anything.
>
> So I'm really not sure why you have libvirtd configured to do relabelling
> at all ? I'd be expecting it to have security_driver=none in the qemu.conf
> file so that libvirtd doesn't do anything.
>
I checked the dumpxml of the virt-launcher pod (that runs the qemu in
kubevirt) - it has dynamic policy.
+107:+107
+107:+107
Are you saying this a wrong configuration for a kubevirt vmi?
>
> If libvirtd is doing relabelling, I'm not sure how it works for anything,
> let alone tap devices.
>
I want to emphasize that in this current situation the reason we fail is
that we are trying to give libvirt in a non-privileged pod (i.e.
virt-launcher) a tap device that was created externally (by a privileged
pod - virt-handler).
I think the reason it works so far is that libvirt was using only devices
which were self created so there was no issue to relabel.
Our thought was that if we know how libvirt is relabeling then we could
also do it so that the externally created tap's label will match the
virt-launcher's.
Is this were libvirt does the relabeling
https://github.com/libvirt/libvirt/blob/e71e13488dc1aa65456e54a4b41bc925821b4263/src/security/security_selinux.c#L1256
?
btw the error we get is (from audit)
type=AVC msg=audit(1586956552.265:513): avc: denied { relabelfrom }
for pid=27423 comm="libvirtd"
scontext=system_u:system_r:container_t:s0:c143,c582
tcontext=system_u:system_r:spc_t:s0 tclass=tun_socket permissive=0
> Regards,
> Daniel
> --
> |: https://berrange.com -o-
> https://www.flickr.com/photos/dberrange :|
> |: https://libvirt.org -o-
>
Re: SELinux labels change in libvirt
On Tue, Jul 14, 2020 at 04:02:17PM +0300, Ram Lavi wrote: > On Tue, Jul 14, 2020 at 3:33 PM Daniel P. Berrangé > wrote: > > > On Tue, Jul 14, 2020 at 03:21:17PM +0300, Ram Lavi wrote: > > > Hello all, > > > > > > tl;dr, can you point me to the point in the libvirt repo where it's > > trying > > > to change a tap-device's SELinux label? > > > > > > I am trying to create a tap device with libvirt on a > > > super-privileged container, and then use it on another, > > > unprivileged container with libvirt. > > > User wise, I know I need the super-privileged container to open the tap > > > device with the user of the unprivileged one - that I already did and > > it's > > > not the issue. > > > But I have a problem when I open the tap device in the > > > non-privileged container: the tap device currently has the spc_t label > > > since the tun_socket inherited the selinux context from the > > > super-privileged container who creates it. then libvirt is trying to > > change > > > the SELinux labels, and since it's not privileged then it fails. > > > But I didn't find where and how libvirt is trying to change the tap > > > device's label. > > > > > > Can you point me to that specific code on libvirt? > > > > If the SELinux policy that libvirtd is running under prevents it from > > re-labelling, then TAP devices label failure is just going to be one > > out of 100's of labelling failures. > > > IIUC normally libvirtd would use devices created by itself, so there > shouldn't be relabeling failures, right? Whether or not there are relabeling failures is determined by what SELinux policy libvirtd is running under. > > > > Either the SELinux policy needs to be changed to allow libvirtd to > > relabel stuff in the normal manner, or you will have to turn off > > SELinux support in libvirtd. in /etc/libvirt/qemu.conf via the > > param security_driver = "none". If you turn off SELinux in > > libvirt, then you no longer have separation of QEMU processes > > which may be a security flaw depending on your deplyoment > > scenario. > > > turning SELinux in libvirtd off or allowing libvirt to relabel are tempting > options but it is not an option I'm afraid due to security concerns. > Our plan (or at least this particular effort) is to try to relabel the > tun-socket after created in the super-privileged container to be the same > one as the one used in the unprivileged one, then it won't have an issue > consuming it. > > btw the change I (or rather Migule's my teammate) made is in this PR, where > I want to add a tap device in virt-handler (i.e. the super privileged > container) to be further uses in virt-launcher (i.e. the non-privileged > container): https://github.com/kubevirt/kubevirt/pull/3290 In normal host OS deployment, libvirtd runs under virtd_t, and when it spawns QEMU, it will relabel files to svirt_image_t:s0:$MCS, and spawn QEMU as svirt_t:s0:$MCS. My understanding is what in kubevirt, things work differently. Docker (or podman), launch the container as container_t:s0:$MCS. libvirtd *and* QEMU thus both run as container_t:s0:$MCS. ie All the labelling is setup when the container is launched and libvirtd should not do anything. So I'm really not sure why you have libvirtd configured to do relabelling at all ? I'd be expecting it to have security_driver=none in the qemu.conf file so that libvirtd doesn't do anything. If libvirtd is doing relabelling, I'm not sure how it works for anything, let alone tap devices. Regards, Daniel -- |: https://berrange.com -o-https://www.flickr.com/photos/dberrange :| |: https://libvirt.org -o-https://fstop138.berrange.com :| |: https://entangle-photo.org-o-https://www.instagram.com/dberrange :|
Re: SELinux labels change in libvirt
On Tue, Jul 14, 2020 at 3:33 PM Daniel P. Berrangé wrote: > On Tue, Jul 14, 2020 at 03:21:17PM +0300, Ram Lavi wrote: > > Hello all, > > > > tl;dr, can you point me to the point in the libvirt repo where it's > trying > > to change a tap-device's SELinux label? > > > > I am trying to create a tap device with libvirt on a > > super-privileged container, and then use it on another, > > unprivileged container with libvirt. > > User wise, I know I need the super-privileged container to open the tap > > device with the user of the unprivileged one - that I already did and > it's > > not the issue. > > But I have a problem when I open the tap device in the > > non-privileged container: the tap device currently has the spc_t label > > since the tun_socket inherited the selinux context from the > > super-privileged container who creates it. then libvirt is trying to > change > > the SELinux labels, and since it's not privileged then it fails. > > But I didn't find where and how libvirt is trying to change the tap > > device's label. > > > > Can you point me to that specific code on libvirt? > > If the SELinux policy that libvirtd is running under prevents it from > re-labelling, then TAP devices label failure is just going to be one > out of 100's of labelling failures. > IIUC normally libvirtd would use devices created by itself, so there shouldn't be relabeling failures, right? > > Either the SELinux policy needs to be changed to allow libvirtd to > relabel stuff in the normal manner, or you will have to turn off > SELinux support in libvirtd. in /etc/libvirt/qemu.conf via the > param security_driver = "none". If you turn off SELinux in > libvirt, then you no longer have separation of QEMU processes > which may be a security flaw depending on your deplyoment > scenario. > turning SELinux in libvirtd off or allowing libvirt to relabel are tempting options but it is not an option I'm afraid due to security concerns. Our plan (or at least this particular effort) is to try to relabel the tun-socket after created in the super-privileged container to be the same one as the one used in the unprivileged one, then it won't have an issue consuming it. btw the change I (or rather Migule's my teammate) made is in this PR, where I want to add a tap device in virt-handler (i.e. the super privileged container) to be further uses in virt-launcher (i.e. the non-privileged container): https://github.com/kubevirt/kubevirt/pull/3290 > > Regards, > Daniel > -- > |: https://berrange.com -o- > https://www.flickr.com/photos/dberrange :| > |: https://libvirt.org -o- > https://fstop138.berrange.com :| > |: https://entangle-photo.org-o- > https://www.instagram.com/dberrange :| > >
Re: SELinux labels change in libvirt
On Tue, Jul 14, 2020 at 03:21:17PM +0300, Ram Lavi wrote: > Hello all, > > tl;dr, can you point me to the point in the libvirt repo where it's trying > to change a tap-device's SELinux label? > > I am trying to create a tap device with libvirt on a > super-privileged container, and then use it on another, > unprivileged container with libvirt. > User wise, I know I need the super-privileged container to open the tap > device with the user of the unprivileged one - that I already did and it's > not the issue. > But I have a problem when I open the tap device in the > non-privileged container: the tap device currently has the spc_t label > since the tun_socket inherited the selinux context from the > super-privileged container who creates it. then libvirt is trying to change > the SELinux labels, and since it's not privileged then it fails. > But I didn't find where and how libvirt is trying to change the tap > device's label. > > Can you point me to that specific code on libvirt? If the SELinux policy that libvirtd is running under prevents it from re-labelling, then TAP devices label failure is just going to be one out of 100's of labelling failures. Either the SELinux policy needs to be changed to allow libvirtd to relabel stuff in the normal manner, or you will have to turn off SELinux support in libvirtd. in /etc/libvirt/qemu.conf via the param security_driver = "none". If you turn off SELinux in libvirt, then you no longer have separation of QEMU processes which may be a security flaw depending on your deplyoment scenario. Regards, Daniel -- |: https://berrange.com -o-https://www.flickr.com/photos/dberrange :| |: https://libvirt.org -o-https://fstop138.berrange.com :| |: https://entangle-photo.org-o-https://www.instagram.com/dberrange :|
