Re: Possible issue with linux xattr support?

[Felix Palmen]

* Dmitry Chagin  [20230829 23:16]:
> On Tue, Aug 29, 2023 at 03:02:58PM -0400, Shawn Webb wrote:
> > Back in 2019, I had a similar issue: I needed access to be able to
> > read/write to the system extended attribute namespace from within a
> > jailed context. I wrote a rather simple patch that provides that
> > support on a per-jail basis:
> > 
> > https://git.hardenedbsd.org/hardenedbsd/HardenedBSD/-/commit/96c85982b45e44a6105664c7068a92d0a61da2a3
> > 
> > Hopefully that's useful to someone.
> > 
> 
> Nice, thank you. I'd prefer to disable it by default, like on a host.

When it's disabled by default, it will require additional configuration
to make "Linux jails" work again.

-- 
 Felix Palmen  {private}   fe...@palmen-it.de
 -- ports committer -- {web}  http://palmen-it.de
 {pgp public key}  http://palmen-it.de/pub.txt
 {pgp fingerprint} 6936 13D5 5BBF 4837 B212  3ACC 54AD E006 9879 F231


signature.asc
Description: PGP signature

Re: Possible issue with linux xattr support?

[Alexander Leidinger]

Am 2023-08-29 21:31, schrieb Felix Palmen:

* Shawn Webb  [20230829 15:25]:

On Tue, Aug 29, 2023 at 09:15:03PM +0200, Felix Palmen wrote:
> * Kyle Evans  [20230829 14:07]:
> > On 8/29/23 14:02, Shawn Webb wrote:
> > > Back in 2019, I had a similar issue: I needed access to be able to
> > > read/write to the system extended attribute namespace from within a
> > > jailed context. I wrote a rather simple patch that provides that
> > > support on a per-jail basis:
> > >
> > > 
https://git.hardenedbsd.org/hardenedbsd/HardenedBSD/-/commit/96c85982b45e44a6105664c7068a92d0a61da2a3
> > >
> > > Hopefully that's useful to someone.
> > >
> > > Thanks,
> > >
> >
> > FWIW (which likely isn't much), I like this approach much better; it makes
> > more sense to me that it's a feature controlled by the creator of the jail
> > and not one allowed just by using a compat ABI within a jail.
>
> Well, a typical GNU userland won't work in a jail without this, that's
> what I know now. But I'm certainly with you, it doesn't feel logical
> that a Linux binary can do something in a jail a FreeBSD binary can't.
>
> So, indeed, making it a jail option sounds better.
>
> Unless, bringing back a question raised earlier in this thread: What's
> the reason to restrict this in a jailed context in the first place? IOW,
> could it just be allowed unconditionally?

In HardenedBSD's case, since we use filesystem extended attributes to
toggle exploit mitigations on a per-application basis, there's now a
conceptual security boundary between the host and the jail.

Should the jail and the host share resources, like executables, a
jailed process could toggle an exploit mitigation, and the toggle
would bubble up to the host. So the next time the host executed
/shared/app/executable/here, the security posture of the host would be
affected.


Isn't the sane approach here *not* to share any executables with a jail
other than via a read-only nullfs mount?


In https://reviews.freebsd.org/D40370 I provide infrastructure to 
automatically jail rc.d services. It will use the complete filesystem of 
the system, but uses all the other restrictions of jails. So the answer 
to your questions is "it depends".


Bye,
Alexander.

--
http://www.Leidinger.net alexan...@leidinger.net: PGP 0x8F31830F9F2772BF
http://www.FreeBSD.orgnetch...@freebsd.org  : PGP 0x8F31830F9F2772BF

Re: Possible issue with linux xattr support?

[Alexander Leidinger]

Am 2023-08-29 21:02, schrieb Shawn Webb:


Back in 2019, I had a similar issue: I needed access to be able to
read/write to the system extended attribute namespace from within a
jailed context. I wrote a rather simple patch that provides that
support on a per-jail basis:

https://git.hardenedbsd.org/hardenedbsd/HardenedBSD/-/commit/96c85982b45e44a6105664c7068a92d0a61da2a3


You enabled it by default. I would assume you had a thought about the 
implications... any memories about it?


What I'm after is:
 - What can go wrong if we enable it by default?
 - Why would we like to disable it (or any ideas why it is disabled by 
default in FreeBSD)?


Depending in the answers we may even use a simpler patch and have it 
allowed in jails even without the possibility to configure it.


Bye,
Alexander.

--
http://www.Leidinger.net alexan...@leidinger.net: PGP 0x8F31830F9F2772BF
http://www.FreeBSD.orgnetch...@freebsd.org  : PGP 0x8F31830F9F2772BF

ZFS Page Derefrence

[Cy Schubert]

Hi

Just got the following panic on an and64 machine running poudriere building 
i386 packages.

panic: vm_page_dequeue_deferred: page 0xfe000b222808 has unexpected 
queue state^M
cpuid = 1^M
time = 1693359541^M
KDB: stack backtrace:^M
db_trace_self_wrapper() at db_trace_self_wrapper+0x2b/frame 
0xfe00bf55d600^M
vpanic() at vpanic+0x132/frame 0xfe00bf55d730^M
panic() at panic+0x43/frame 0xfe00bf55d790^M
vm_page_dequeue_deferred() at vm_page_dequeue_deferred+0xb2/frame 
0xfe00bf55d7a0^M
vm_page_free_prep() at vm_page_free_prep+0x11b/frame 0xfe00bf55d7c0^M
vm_page_free_toq() at vm_page_free_toq+0x12/frame 0xfe00bf55d7f0^M
vm_object_page_remove() at vm_object_page_remove+0xb6/frame 
0xfe00bf55d850^M
vn_pages_remove_valid() at vn_pages_remove_valid+0x48/frame 
0xfe00bf55d880^M
zfs_rezget() at zfs_rezget+0x35/frame 0xfe00bf55da60^M
zfs_resume_fs() at zfs_resume_fs+0x1c8/frame 0xfe00bf55dab0^M
zfs_ioc_rollback() at zfs_ioc_rollback+0x157/frame 0xfe00bf55db00^M
zfsdev_ioctl_common() at zfsdev_ioctl_common+0x612/frame 
0xfe00bf55dbc0^M
zfsdev_ioctl() at zfsdev_ioctl+0x12a/frame 0xfe00bf55dbf0^M
devfs_ioctl() at devfs_ioctl+0xd2/frame 0xfe00bf55dc40^M
vn_ioctl() at vn_ioctl+0xc2/frame 0xfe00bf55dcb0^M
devfs_ioctl_f() at devfs_ioctl_f+0x1e/frame 0xfe00bf55dcd0^M
kern_ioctl() at kern_ioctl+0x286/frame 0xfe00bf55dd30^M
sys_ioctl() at sys_ioctl+0x152/frame 0xfe00bf55de00^M
amd64_syscall() at amd64_syscall+0x138/frame 0xfe00bf55df30^M
fast_syscall_common() at fast_syscall_common+0xf8/frame 0xfe00bf55df30^M
--- syscall (54, FreeBSD ELF64, ioctl), rip = 0x191264a4fbca, rsp = 
0x19125ca905c8, rbp = 0x19125c
a90640 ---^M
Uptime: 10h14m15s^M
Dumping 2829 out of 7998 MB:..1%..11%..21%..31%..41%..51%..61%..71%..81%..91
%^M
Dump complete^M
Automatic reboot in 15 seconds - press a key on the console to abort^M
Rebooting...^M
cpu_reset: Restarting BSP^M
cpu_reset_proxy: Stopped CPU 1^M
^M
1  FreeBSD^M
2  FreeBSD^M
3  FreeBSD^M
4  FreeBSD^M
5  Drive 1^M

uname reports,

FreeBSD bob 15.0-CURRENT FreeBSD 15.0-CURRENT amd64 150 #1 
komquats-n265075-2e8edbc285cf: Tue Aug 29 03:51:59 PDT 2023 
root@cwsys:/export/obj/opt/src/git-src/amd64.amd64/sys/BREAK2 amd64

My BREAK2 kernel removes devices I don't use and enables keystrokes to 
interrupt the system from the conosle (conserver). Local patches affect 
ipfilter only.

Head of core.txt:

__curthread () at /opt/src/git-src/sys/amd64/include/pcpu_aux.h:57
57  __asm("movq %%gs:%P1,%0" : "=r" (td) : "n" (offsetof(struct 
pcpu
,
(kgdb) #0  __curthread () at /opt/src/git-src/sys/amd64/include/pcpu_aux.h:5
7
#1  doadump (textdump=textdump@entry=1)
at /opt/src/git-src/sys/kern/kern_shutdown.c:405
#2  0x806c1b30 in kern_reboot (howto=260)
at /opt/src/git-src/sys/kern/kern_shutdown.c:526
#3  0x806c202f in vpanic (
fmt=0x80b5da55 "%s: page %p has unexpected queue state",
ap=ap@entry=0xfe00bf55d770)
at /opt/src/git-src/sys/kern/kern_shutdown.c:970
#4  0x806c1dd3 in panic (fmt=)
at /opt/src/git-src/sys/kern/kern_shutdown.c:894
#5  0x809daab2 in vm_page_dequeue_deferred (m=,
m@entry=0xfe000b222808) at /opt/src/git-src/sys/vm/vm_page.c:3790
#6  0x809ddfeb in vm_page_free_prep (m=m@entry=0xfe000b222808)
at /opt/src/git-src/sys/vm/vm_page.c:3928
#7  0x809d5b52 in vm_page_free_toq (m=,
m@entry=0xfe000b222808) at /opt/src/git-src/sys/vm/vm_page.c:3970
#8  0x809d5b3b in vm_page_free (m=,
m@entry=0xfe000b222808) at /opt/src/git-src/sys/vm/vm_page.c:1328
#9  0x809d0906 in vm_object_page_remove (object=0xf8018bff0e70,
start=0, end=0, options=4) at /opt/src/git-src/sys/vm/vm_object.c:2157
#10 0x807d3218 in vn_pages_remove_valid (vp=,
start=start@entry=0, end=end@entry=0)
at /opt/src/git-src/sys/kern/vfs_vnops.c:2558
#11 0x816b7315 in zfs_rezget (zp=zp@entry=0xf801768b6ae0)
at /opt/src/git-src/sys/contrib/openzfs/module/os/freebsd/zfs/zfs_znode.
c:11
00
#12 0x816a6808 in zfs_resume_fs (zfsvfs=,
ds=ds@entry=0xf8007b609000)
at /opt/src/git-src/sys/contrib/openzfs/module/os/freebsd/zfs/zfs_vfsops
.c:2
006
#13 0x81867e87 in zfs_ioc_rollback (
fsname=0xfe00fbf42000 "bob/poudriere/bob/jails/HEADi386-new-ports-re
f/06
", fsname@entry=,
innvl=,

innvl@entry=,
outnvl=,
outnvl@entry=)
at /opt/src/git-src/sys/contrib/openzfs/module/zfs/zfs_ioctl.c:4407
#14 0x818635c2 in zfsdev_ioctl_common (vecnum=vecnum@entry=25,
zc=zc@entry=0xfe00fbf42000, flag=flag@entry=0)
at /opt/src/git-src/sys/contrib/openzfs/module/zfs/zfs_ioctl.c:7798
#15 0x81696fca in zfsdev_ioctl (dev=,
zcmd=,
zcmd@entry=,
arg=0xfe00bf55dd50 "\017",
arg@entry=,
flag=, td=)
at /opt/src/git-src/sys/contrib/openzfs/module/os/freebsd/zfs/kmod_core.
c:16
8
#16 0x8054b482 in 

Re: Possible issue with linux xattr support?

[Kyle Evans]

On 8/29/23 14:15, Felix Palmen wrote:

* Kyle Evans  [20230829 14:07]:

On 8/29/23 14:02, Shawn Webb wrote:

Back in 2019, I had a similar issue: I needed access to be able to
read/write to the system extended attribute namespace from within a
jailed context. I wrote a rather simple patch that provides that
support on a per-jail basis:

https://git.hardenedbsd.org/hardenedbsd/HardenedBSD/-/commit/96c85982b45e44a6105664c7068a92d0a61da2a3

Hopefully that's useful to someone.

Thanks,



FWIW (which likely isn't much), I like this approach much better; it makes
more sense to me that it's a feature controlled by the creator of the jail
and not one allowed just by using a compat ABI within a jail.


Well, a typical GNU userland won't work in a jail without this, that's
what I know now. But I'm certainly with you, it doesn't feel logical
that a Linux binary can do something in a jail a FreeBSD binary can't.

So, indeed, making it a jail option sounds better.

Unless, bringing back a question raised earlier in this thread: What's
the reason to restrict this in a jailed context in the first place? IOW,
could it just be allowed unconditionally?



I don't think we can answer this definitively, FreeBSD has a pretty wide 
variety of users. I note that we don't /need/ to answer it, either, with 
Shawn's patch; it defaults to system xattrs allowed and an individual 
sysadmin can make that decision for their own context (and supporting 
the knob is relatively low-cost).


The only part I'm not sure I agree with is the addition of the new flag 
to PR_ALLOW_DIFFERENCES. That allows it to be disabled by system root 
for jail "foo", but root in jail "foo" can create another jail "foo.bar" 
in which it *is* enabled (rather than only allowing "foo.bar" to have it 
enabled if its parent does). IMO the name PR_ALLOW_DIFFERENCES is a bit 
off, because to me it would imply that it just allows the flag to be set 
in one jail and unset in its child jail.


Thanks,

Kyle Evans

Re: Possible issue with linux xattr support?

[Dmitry Chagin]

On Tue, Aug 29, 2023 at 03:02:58PM -0400, Shawn Webb wrote:
> On Tue, Aug 29, 2023 at 05:45:51PM +0300, Dmitry Chagin wrote:
> > On Tue, Aug 29, 2023 at 12:59:11PM +0200, Felix Palmen wrote:
> > > * Dmitry Chagin  [20230828 18:57]:
> > > > On Mon, Aug 28, 2023 at 08:03:33AM +0200, Felix Palmen wrote:
> > > > > * Cy Schubert  [20230827 16:59]:
> > > > > > 
> > > > > > If we are to break it to fix a problem, maybe a sysctl to 
> > > > > > enable/disable then?
> > > > > 
> > > > > IMHO depends on the exact nature of the problem. If it's confirmed 
> > > > > that
> > > > > it (always and only) breaks for jailed processes, just disabling it 
> > > > > for
> > > > > them would be the better workaround. "No-op" calls won't break 
> > > > > anything.
> > > > > 
> > > > 
> > > > please, try: https://people.freebsd.org/~dchagin/xattrerror.patch
> > > 
> > > Thanks, I can confirm this avoids the issue in both cases I experienced
> > > (install from GNU coreutils and python).
> > > 
> > thanks, this is the first half of the fix, it works for you due to you
> > are running tools under unprivileged user, afaiu. The second I have
> > tested by myself :)
> > 
> > > If I understand this patch correctly, it completely avoids EPERM,
> > > masking it as not supported, so callers should consider it non-fatal,
> > > allowing to silently ignore writing of "system" attributes while still
> > > keeping other functionality?
> > > 
> > system namespace is accessible only for privileged user, for others Linux
> > returns ENOTSUP. So many tools ignores this error, eg ls.
> > 
> > the second: https://people.freebsd.org/~dchagin/sea_jailed.patch
> > 
> > Try this under privileged user, please.
> 
> Back in 2019, I had a similar issue: I needed access to be able to
> read/write to the system extended attribute namespace from within a
> jailed context. I wrote a rather simple patch that provides that
> support on a per-jail basis:
> 
> https://git.hardenedbsd.org/hardenedbsd/HardenedBSD/-/commit/96c85982b45e44a6105664c7068a92d0a61da2a3
> 
> Hopefully that's useful to someone.
> 

Nice, thank you. I'd prefer to disable it by default, like on a host.

Re: Possible issue with linux xattr support?

[Shawn Webb]

On Tue, Aug 29, 2023 at 09:31:46PM +0200, Felix Palmen wrote:
> * Shawn Webb  [20230829 15:25]:
> > On Tue, Aug 29, 2023 at 09:15:03PM +0200, Felix Palmen wrote:
> > > * Kyle Evans  [20230829 14:07]:
> > > > On 8/29/23 14:02, Shawn Webb wrote:
> > > > > Back in 2019, I had a similar issue: I needed access to be able to
> > > > > read/write to the system extended attribute namespace from within a
> > > > > jailed context. I wrote a rather simple patch that provides that
> > > > > support on a per-jail basis:
> > > > > 
> > > > > https://git.hardenedbsd.org/hardenedbsd/HardenedBSD/-/commit/96c85982b45e44a6105664c7068a92d0a61da2a3
> > > > > 
> > > > > Hopefully that's useful to someone.
> > > > > 
> > > > > Thanks,
> > > > > 
> > > > 
> > > > FWIW (which likely isn't much), I like this approach much better; it 
> > > > makes
> > > > more sense to me that it's a feature controlled by the creator of the 
> > > > jail
> > > > and not one allowed just by using a compat ABI within a jail.
> > > 
> > > Well, a typical GNU userland won't work in a jail without this, that's
> > > what I know now. But I'm certainly with you, it doesn't feel logical
> > > that a Linux binary can do something in a jail a FreeBSD binary can't.
> > > 
> > > So, indeed, making it a jail option sounds better.
> > > 
> > > Unless, bringing back a question raised earlier in this thread: What's
> > > the reason to restrict this in a jailed context in the first place? IOW,
> > > could it just be allowed unconditionally?
> > 
> > In HardenedBSD's case, since we use filesystem extended attributes to
> > toggle exploit mitigations on a per-application basis, there's now a
> > conceptual security boundary between the host and the jail.
> > 
> > Should the jail and the host share resources, like executables, a
> > jailed process could toggle an exploit mitigation, and the toggle
> > would bubble up to the host. So the next time the host executed
> > /shared/app/executable/here, the security posture of the host would be
> > affected.
> 
> Isn't the sane approach here *not* to share any executables with a jail
> other than via a read-only nullfs mount?

I thought about that, too, but nullfs is not guaranteed to be
available or applicable in all environments.

Thanks,

-- 
Shawn Webb
Cofounder / Security Engineer
HardenedBSD

https://git.hardenedbsd.org/hardenedbsd/pubkeys/-/raw/master/Shawn_Webb/03A4CBEBB82EA5A67D9F3853FF2E67A277F8E1FA.pub.asc


signature.asc
Description: PGP signature

Re: Possible issue with linux xattr support?

[Felix Palmen]

* Shawn Webb  [20230829 15:25]:
> On Tue, Aug 29, 2023 at 09:15:03PM +0200, Felix Palmen wrote:
> > * Kyle Evans  [20230829 14:07]:
> > > On 8/29/23 14:02, Shawn Webb wrote:
> > > > Back in 2019, I had a similar issue: I needed access to be able to
> > > > read/write to the system extended attribute namespace from within a
> > > > jailed context. I wrote a rather simple patch that provides that
> > > > support on a per-jail basis:
> > > > 
> > > > https://git.hardenedbsd.org/hardenedbsd/HardenedBSD/-/commit/96c85982b45e44a6105664c7068a92d0a61da2a3
> > > > 
> > > > Hopefully that's useful to someone.
> > > > 
> > > > Thanks,
> > > > 
> > > 
> > > FWIW (which likely isn't much), I like this approach much better; it makes
> > > more sense to me that it's a feature controlled by the creator of the jail
> > > and not one allowed just by using a compat ABI within a jail.
> > 
> > Well, a typical GNU userland won't work in a jail without this, that's
> > what I know now. But I'm certainly with you, it doesn't feel logical
> > that a Linux binary can do something in a jail a FreeBSD binary can't.
> > 
> > So, indeed, making it a jail option sounds better.
> > 
> > Unless, bringing back a question raised earlier in this thread: What's
> > the reason to restrict this in a jailed context in the first place? IOW,
> > could it just be allowed unconditionally?
> 
> In HardenedBSD's case, since we use filesystem extended attributes to
> toggle exploit mitigations on a per-application basis, there's now a
> conceptual security boundary between the host and the jail.
> 
> Should the jail and the host share resources, like executables, a
> jailed process could toggle an exploit mitigation, and the toggle
> would bubble up to the host. So the next time the host executed
> /shared/app/executable/here, the security posture of the host would be
> affected.

Isn't the sane approach here *not* to share any executables with a jail
other than via a read-only nullfs mount?

> FreeBSD uses ELF header tagging, not filesystem extended attributes,
> to toggle exploit mitigations. So my description above is moot for
> FreeBSD users. I'm just hoping to share a unique perspective.

Thanks!

-- 
 Felix Palmen  {private}   fe...@palmen-it.de
 -- ports committer -- {web}  http://palmen-it.de
 {pgp public key}  http://palmen-it.de/pub.txt
 {pgp fingerprint} 6936 13D5 5BBF 4837 B212  3ACC 54AD E006 9879 F231


signature.asc
Description: PGP signature

Re: Possible issue with linux xattr support?

[Shawn Webb]

On Tue, Aug 29, 2023 at 09:15:03PM +0200, Felix Palmen wrote:
> * Kyle Evans  [20230829 14:07]:
> > On 8/29/23 14:02, Shawn Webb wrote:
> > > Back in 2019, I had a similar issue: I needed access to be able to
> > > read/write to the system extended attribute namespace from within a
> > > jailed context. I wrote a rather simple patch that provides that
> > > support on a per-jail basis:
> > > 
> > > https://git.hardenedbsd.org/hardenedbsd/HardenedBSD/-/commit/96c85982b45e44a6105664c7068a92d0a61da2a3
> > > 
> > > Hopefully that's useful to someone.
> > > 
> > > Thanks,
> > > 
> > 
> > FWIW (which likely isn't much), I like this approach much better; it makes
> > more sense to me that it's a feature controlled by the creator of the jail
> > and not one allowed just by using a compat ABI within a jail.
> 
> Well, a typical GNU userland won't work in a jail without this, that's
> what I know now. But I'm certainly with you, it doesn't feel logical
> that a Linux binary can do something in a jail a FreeBSD binary can't.
> 
> So, indeed, making it a jail option sounds better.
> 
> Unless, bringing back a question raised earlier in this thread: What's
> the reason to restrict this in a jailed context in the first place? IOW,
> could it just be allowed unconditionally?

In HardenedBSD's case, since we use filesystem extended attributes to
toggle exploit mitigations on a per-application basis, there's now a
conceptual security boundary between the host and the jail.

Should the jail and the host share resources, like executables, a
jailed process could toggle an exploit mitigation, and the toggle
would bubble up to the host. So the next time the host executed
/shared/app/executable/here, the security posture of the host would be
affected.

FreeBSD uses ELF header tagging, not filesystem extended attributes,
to toggle exploit mitigations. So my description above is moot for
FreeBSD users. I'm just hoping to share a unique perspective.

Thanks,

-- 
Shawn Webb
Cofounder / Security Engineer
HardenedBSD

https://git.hardenedbsd.org/hardenedbsd/pubkeys/-/raw/master/Shawn_Webb/03A4CBEBB82EA5A67D9F3853FF2E67A277F8E1FA.pub.asc


signature.asc
Description: PGP signature

Re: Possible issue with linux xattr support?

[Felix Palmen]

* Kyle Evans  [20230829 14:07]:
> On 8/29/23 14:02, Shawn Webb wrote:
> > Back in 2019, I had a similar issue: I needed access to be able to
> > read/write to the system extended attribute namespace from within a
> > jailed context. I wrote a rather simple patch that provides that
> > support on a per-jail basis:
> > 
> > https://git.hardenedbsd.org/hardenedbsd/HardenedBSD/-/commit/96c85982b45e44a6105664c7068a92d0a61da2a3
> > 
> > Hopefully that's useful to someone.
> > 
> > Thanks,
> > 
> 
> FWIW (which likely isn't much), I like this approach much better; it makes
> more sense to me that it's a feature controlled by the creator of the jail
> and not one allowed just by using a compat ABI within a jail.

Well, a typical GNU userland won't work in a jail without this, that's
what I know now. But I'm certainly with you, it doesn't feel logical
that a Linux binary can do something in a jail a FreeBSD binary can't.

So, indeed, making it a jail option sounds better.

Unless, bringing back a question raised earlier in this thread: What's
the reason to restrict this in a jailed context in the first place? IOW,
could it just be allowed unconditionally?

Cheers, Felix

-- 
 Felix Palmen  {private}   fe...@palmen-it.de
 -- ports committer -- {web}  http://palmen-it.de
 {pgp public key}  http://palmen-it.de/pub.txt
 {pgp fingerprint} 6936 13D5 5BBF 4837 B212  3ACC 54AD E006 9879 F231


signature.asc
Description: PGP signature

Re: Possible issue with linux xattr support?

[Kyle Evans]

On 8/29/23 14:02, Shawn Webb wrote:

On Tue, Aug 29, 2023 at 05:45:51PM +0300, Dmitry Chagin wrote:

On Tue, Aug 29, 2023 at 12:59:11PM +0200, Felix Palmen wrote:

* Dmitry Chagin  [20230828 18:57]:

On Mon, Aug 28, 2023 at 08:03:33AM +0200, Felix Palmen wrote:

* Cy Schubert  [20230827 16:59]:


If we are to break it to fix a problem, maybe a sysctl to enable/disable then?


IMHO depends on the exact nature of the problem. If it's confirmed that
it (always and only) breaks for jailed processes, just disabling it for
them would be the better workaround. "No-op" calls won't break anything.



please, try: https://people.freebsd.org/~dchagin/xattrerror.patch


Thanks, I can confirm this avoids the issue in both cases I experienced
(install from GNU coreutils and python).


thanks, this is the first half of the fix, it works for you due to you
are running tools under unprivileged user, afaiu. The second I have
tested by myself :)


If I understand this patch correctly, it completely avoids EPERM,
masking it as not supported, so callers should consider it non-fatal,
allowing to silently ignore writing of "system" attributes while still
keeping other functionality?


system namespace is accessible only for privileged user, for others Linux
returns ENOTSUP. So many tools ignores this error, eg ls.

the second: https://people.freebsd.org/~dchagin/sea_jailed.patch

Try this under privileged user, please.


Back in 2019, I had a similar issue: I needed access to be able to
read/write to the system extended attribute namespace from within a
jailed context. I wrote a rather simple patch that provides that
support on a per-jail basis:

https://git.hardenedbsd.org/hardenedbsd/HardenedBSD/-/commit/96c85982b45e44a6105664c7068a92d0a61da2a3

Hopefully that's useful to someone.

Thanks,



FWIW (which likely isn't much), I like this approach much better; it 
makes more sense to me that it's a feature controlled by the creator of 
the jail and not one allowed just by using a compat ABI within a jail.


Thanks,

Kyle Evans

Re: Possible issue with linux xattr support?

[Shawn Webb]

On Tue, Aug 29, 2023 at 05:45:51PM +0300, Dmitry Chagin wrote:
> On Tue, Aug 29, 2023 at 12:59:11PM +0200, Felix Palmen wrote:
> > * Dmitry Chagin  [20230828 18:57]:
> > > On Mon, Aug 28, 2023 at 08:03:33AM +0200, Felix Palmen wrote:
> > > > * Cy Schubert  [20230827 16:59]:
> > > > > 
> > > > > If we are to break it to fix a problem, maybe a sysctl to 
> > > > > enable/disable then?
> > > > 
> > > > IMHO depends on the exact nature of the problem. If it's confirmed that
> > > > it (always and only) breaks for jailed processes, just disabling it for
> > > > them would be the better workaround. "No-op" calls won't break anything.
> > > > 
> > > 
> > > please, try: https://people.freebsd.org/~dchagin/xattrerror.patch
> > 
> > Thanks, I can confirm this avoids the issue in both cases I experienced
> > (install from GNU coreutils and python).
> > 
> thanks, this is the first half of the fix, it works for you due to you
> are running tools under unprivileged user, afaiu. The second I have
> tested by myself :)
> 
> > If I understand this patch correctly, it completely avoids EPERM,
> > masking it as not supported, so callers should consider it non-fatal,
> > allowing to silently ignore writing of "system" attributes while still
> > keeping other functionality?
> > 
> system namespace is accessible only for privileged user, for others Linux
> returns ENOTSUP. So many tools ignores this error, eg ls.
> 
> the second: https://people.freebsd.org/~dchagin/sea_jailed.patch
> 
> Try this under privileged user, please.

Back in 2019, I had a similar issue: I needed access to be able to
read/write to the system extended attribute namespace from within a
jailed context. I wrote a rather simple patch that provides that
support on a per-jail basis:

https://git.hardenedbsd.org/hardenedbsd/HardenedBSD/-/commit/96c85982b45e44a6105664c7068a92d0a61da2a3

Hopefully that's useful to someone.

Thanks,

-- 
Shawn Webb
Cofounder / Security Engineer
HardenedBSD

https://git.hardenedbsd.org/hardenedbsd/pubkeys/-/raw/master/Shawn_Webb/03A4CBEBB82EA5A67D9F3853FF2E67A277F8E1FA.pub.asc


signature.asc
Description: PGP signature

Re: Possible issue with linux xattr support?

[Felix Palmen]

* Dmitry Chagin  [20230829 21:12]:
> On Tue, Aug 29, 2023 at 07:02:22PM +0200, Felix Palmen wrote:
> > So, using user.* works, using system.* doesn't, and maybe a bit
> > surprising(?), dumping all attributes which by default excludes the
> > system namespace doesn't work either.
> > 
> 
> As expected, the second patch intended to allow access to system
> namespace in jailed env.

Not exactly, according to the manual, 'getfattr -d' *excludes* the
system namespace by default, so I would have expected it to work. But
maybe that's an issue with the getfattr tool itself.

> > I still wonder, is the first patch needed anyways? Maybe I fail to
> > understand something here. Won't it map *every* EPERM to ENOSUP and
> > can't this be an issue?
> > 
> 
> fine, thanks. Gnu tools running under unprivileged user will fail, eg
> ls, which uses getfattr() to get the posix acl

Ah, I see, that would still break "Linux jails" then...

Thanks again for quickly fixing this! Can this still make 14.0-RELEASE?

Cheers, Felix

-- 
 Felix Palmen  {private}   fe...@palmen-it.de
 -- ports committer -- {web}  http://palmen-it.de
 {pgp public key}  http://palmen-it.de/pub.txt
 {pgp fingerprint} 6936 13D5 5BBF 4837 B212  3ACC 54AD E006 9879 F231


signature.asc
Description: PGP signature

Re: Possible issue with linux xattr support?

[Dmitry Chagin]

On Tue, Aug 29, 2023 at 07:02:22PM +0200, Felix Palmen wrote:
> * Dmitry Chagin  [20230829 17:45]:
> > On Tue, Aug 29, 2023 at 12:59:11PM +0200, Felix Palmen wrote:
> > > Thanks, I can confirm this avoids the issue in both cases I experienced
> > > (install from GNU coreutils and python).
> > > 
> > thanks, this is the first half of the fix, it works for you due to you
> > are running tools under unprivileged user, afaiu. The second I have
> > tested by myself :)
> 
> Sure, poudriere is running all builds as "nobody" by default.
> 
> > > If I understand this patch correctly, it completely avoids EPERM,
> > > masking it as not supported, so callers should consider it non-fatal,
> > > allowing to silently ignore writing of "system" attributes while still
> > > keeping other functionality?
> > > 
> > system namespace is accessible only for privileged user, for others Linux
> > returns ENOTSUP. So many tools ignores this error, eg ls.
> > 
> > the second: https://people.freebsd.org/~dchagin/sea_jailed.patch
> 
> Ok, I did some tests in a poudriere jail using Linux bash, as root.
> First, with only the first patch:
> 
> | bash-5.2# getfattr -d /bin/sh
> | getfattr: /bin/sh: Operation not supported
> | bash-5.2# setfattr -n user.foo -v bar /bin/sh
> | bash-5.2# getfattr -n user.foo /bin/sh
> | getfattr: Removing leading '/' from absolute path names
> | # file: bin/sh
> | user.foo="bar"
> | bash-5.2# setfattr -x user.foo /bin/sh
> | bash-5.2# setfattr -x system.foo /bin/sh
> | setfattr: /bin/sh: Operation not supported
> 
> So, using user.* works, using system.* doesn't, and maybe a bit
> surprising(?), dumping all attributes which by default excludes the
> system namespace doesn't work either.
> 

As expected, the second patch intended to allow access to system
namespace in jailed env.

> Then with the second patch applied as well:
> 
> | bash-5.2# getfattr -d /bin/sh
> | bash-5.2# setfattr -n system.foo -v bar /bin/sh
> | bash-5.2# getfattr -d /bin/sh -m-
> | getfattr: Removing leading '/' from absolute path names
> | # file: bin/sh
> | system.foo="bar"
> | 
> | bash-5.2# setfattr -x system.foo /bin/sh
> | bash-5.2# getfattr -d /bin/sh -m-
> | bash-5.2#
> 
> This looks perfectly fine, thanks a lot!
> 
> I still wonder, is the first patch needed anyways? Maybe I fail to
> understand something here. Won't it map *every* EPERM to ENOSUP and
> can't this be an issue?
> 

fine, thanks. Gnu tools running under unprivileged user will fail, eg
ls, which uses getfattr() to get the posix acl

> Cheers, Felix
> 
> -- 
>  Felix Palmen  {private}   fe...@palmen-it.de
>  -- ports committer -- {web}  http://palmen-it.de
>  {pgp public key}  http://palmen-it.de/pub.txt
>  {pgp fingerprint} 6936 13D5 5BBF 4837 B212  3ACC 54AD E006 9879 F231

Re: Possible issue with linux xattr support?

[Felix Palmen]

* Dmitry Chagin  [20230829 17:45]:
> On Tue, Aug 29, 2023 at 12:59:11PM +0200, Felix Palmen wrote:
> > Thanks, I can confirm this avoids the issue in both cases I experienced
> > (install from GNU coreutils and python).
> > 
> thanks, this is the first half of the fix, it works for you due to you
> are running tools under unprivileged user, afaiu. The second I have
> tested by myself :)

Sure, poudriere is running all builds as "nobody" by default.

> > If I understand this patch correctly, it completely avoids EPERM,
> > masking it as not supported, so callers should consider it non-fatal,
> > allowing to silently ignore writing of "system" attributes while still
> > keeping other functionality?
> > 
> system namespace is accessible only for privileged user, for others Linux
> returns ENOTSUP. So many tools ignores this error, eg ls.
> 
> the second: https://people.freebsd.org/~dchagin/sea_jailed.patch

Ok, I did some tests in a poudriere jail using Linux bash, as root.
First, with only the first patch:

| bash-5.2# getfattr -d /bin/sh
| getfattr: /bin/sh: Operation not supported
| bash-5.2# setfattr -n user.foo -v bar /bin/sh
| bash-5.2# getfattr -n user.foo /bin/sh
| getfattr: Removing leading '/' from absolute path names
| # file: bin/sh
| user.foo="bar"
| bash-5.2# setfattr -x user.foo /bin/sh
| bash-5.2# setfattr -x system.foo /bin/sh
| setfattr: /bin/sh: Operation not supported

So, using user.* works, using system.* doesn't, and maybe a bit
surprising(?), dumping all attributes which by default excludes the
system namespace doesn't work either.

Then with the second patch applied as well:

| bash-5.2# getfattr -d /bin/sh
| bash-5.2# setfattr -n system.foo -v bar /bin/sh
| bash-5.2# getfattr -d /bin/sh -m-
| getfattr: Removing leading '/' from absolute path names
| # file: bin/sh
| system.foo="bar"
| 
| bash-5.2# setfattr -x system.foo /bin/sh
| bash-5.2# getfattr -d /bin/sh -m-
| bash-5.2#

This looks perfectly fine, thanks a lot!

I still wonder, is the first patch needed anyways? Maybe I fail to
understand something here. Won't it map *every* EPERM to ENOSUP and
can't this be an issue?

Cheers, Felix

-- 
 Felix Palmen  {private}   fe...@palmen-it.de
 -- ports committer -- {web}  http://palmen-it.de
 {pgp public key}  http://palmen-it.de/pub.txt
 {pgp fingerprint} 6936 13D5 5BBF 4837 B212  3ACC 54AD E006 9879 F231


signature.asc
Description: PGP signature

Re: Possible issue with linux xattr support?

[Dmitry Chagin]

On Tue, Aug 29, 2023 at 12:59:11PM +0200, Felix Palmen wrote:
> * Dmitry Chagin  [20230828 18:57]:
> > On Mon, Aug 28, 2023 at 08:03:33AM +0200, Felix Palmen wrote:
> > > * Cy Schubert  [20230827 16:59]:
> > > > 
> > > > If we are to break it to fix a problem, maybe a sysctl to 
> > > > enable/disable then?
> > > 
> > > IMHO depends on the exact nature of the problem. If it's confirmed that
> > > it (always and only) breaks for jailed processes, just disabling it for
> > > them would be the better workaround. "No-op" calls won't break anything.
> > > 
> > 
> > please, try: https://people.freebsd.org/~dchagin/xattrerror.patch
> 
> Thanks, I can confirm this avoids the issue in both cases I experienced
> (install from GNU coreutils and python).
> 
thanks, this is the first half of the fix, it works for you due to you
are running tools under unprivileged user, afaiu. The second I have
tested by myself :)

> If I understand this patch correctly, it completely avoids EPERM,
> masking it as not supported, so callers should consider it non-fatal,
> allowing to silently ignore writing of "system" attributes while still
> keeping other functionality?
> 
system namespace is accessible only for privileged user, for others Linux
returns ENOTSUP. So many tools ignores this error, eg ls.

the second: https://people.freebsd.org/~dchagin/sea_jailed.patch

Try this under privileged user, please.

> I wonder whether this could cause trouble in other scenarios (like a
> read-only fs or actually missing file permissions)?
> 
> Cheers, Felix
> 
> -- 
>  Felix Palmen  {private}   fe...@palmen-it.de
>  -- ports committer -- {web}  http://palmen-it.de
>  {pgp public key}  http://palmen-it.de/pub.txt
>  {pgp fingerprint} 6936 13D5 5BBF 4837 B212  3ACC 54AD E006 9879 F231

Re: Possible issue with linux xattr support?

[Felix Palmen]

* Dmitry Chagin  [20230828 18:57]:
> On Mon, Aug 28, 2023 at 08:03:33AM +0200, Felix Palmen wrote:
> > * Cy Schubert  [20230827 16:59]:
> > > 
> > > If we are to break it to fix a problem, maybe a sysctl to enable/disable 
> > > then?
> > 
> > IMHO depends on the exact nature of the problem. If it's confirmed that
> > it (always and only) breaks for jailed processes, just disabling it for
> > them would be the better workaround. "No-op" calls won't break anything.
> > 
> 
> please, try: https://people.freebsd.org/~dchagin/xattrerror.patch

Thanks, I can confirm this avoids the issue in both cases I experienced
(install from GNU coreutils and python).

If I understand this patch correctly, it completely avoids EPERM,
masking it as not supported, so callers should consider it non-fatal,
allowing to silently ignore writing of "system" attributes while still
keeping other functionality?

I wonder whether this could cause trouble in other scenarios (like a
read-only fs or actually missing file permissions)?

Cheers, Felix

-- 
 Felix Palmen  {private}   fe...@palmen-it.de
 -- ports committer -- {web}  http://palmen-it.de
 {pgp public key}  http://palmen-it.de/pub.txt
 {pgp fingerprint} 6936 13D5 5BBF 4837 B212  3ACC 54AD E006 9879 F231


signature.asc
Description: PGP signature

(264305, 268393, 272878) 14-ALPHA2 panic on cold boot

[Graham Perrin]

On 29/08/2023 09:09, cglogic wrote:

… I found two related bug reports:
https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=264305
https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=268393

My backtrace is the same as in reported issues. So I did not write it 
from a screen to a paper.


As I understand, this bug affects all AMD Zen2, Zen3 and Zen4 
computers and laptops.
We are near FreeBSD 14.0 release now, maybe it will be possible to fix 
it before release.
; I'll 
discuss with the Bugzilla triage team.