Re: Re: LXC unpreviliged problem with aufs mounted on nfs
OK, thanks a lot. BR. __ Michael Mao From: [1]J. R. Okajima Date: 2020-03-23 23:36 To: [2]hom...@163.com CC: [3]aufs-users Subject: Re: LXC unpreviliged problem with aufs mounted on nfs "hom...@163.com": > Does aufs has the interface that I can use to write a hook to replace some functions, like "chown" in aufs' VFS interface? No, but linux kernel has a feature called KPROBE. See Documentation/kprobes.txt. J. R. Okajima References 1. mailto:hooanon...@gmail.com 2. mailto:hom...@163.com 3. mailto:aufs-users@lists.sourceforge.net
Re: Re: LXC unpreviliged problem with aufs mounted on nfs
Hi Okajima San, Does aufs has the interface that I can use to write a hook to replace some functions, like "chown" in aufs' VFS interface? Cause my nfs server can't give me the same permission to access the files, as the client does. (Because client is using the linux namespace(unprivileged container) to access the local files, but the namespace doesn't work on the nfs server's files) So, I want to replace some functions in the aufs filesystem to simulate the client permission validation. __ Michael Mao From: [1]J. R. Okajima Date: 2020-03-22 13:58 To: [2]hom...@163.com CC: [3]aufs-users Subject: Re: LXC unpreviliged problem with aufs mounted on nfs "hom...@163.com": > I found the error in packetdump2: setattr ERROR: Operation not permitted I was going to write exact same thing. :-) > I think aufs works well when mount the local path as the first writable branch. Maybe local container using fuse, so container root user can do setattr success. But in nfs server side, no fuse filesystem using, so the remote container( relate to the nfs server side) can't do setattr to the nfs server file? You should see the behaviour on your nfs server, particulary why the server returned the error for NFS SETATTR request. I'd suggest you to check the parameters in the request packet first. J. R. Okajima References 1. mailto:hooanon...@gmail.com 2. mailto:hom...@163.com 3. mailto:aufs-users@lists.sourceforge.net
Re: Re: LXC unpreviliged problem with aufs mounted on nfs
Hi, Okajima San, Really appreciate for you help. Let me try to find out what's going on in the nfs server side. I will let you know when I find any clue. __ Michael Mao From: [1]J. R. Okajima Date: 2020-03-22 13:58 To: [2]hom...@163.com CC: [3]aufs-users Subject: Re: LXC unpreviliged problem with aufs mounted on nfs "hom...@163.com": > I found the error in packetdump2: setattr ERROR: Operation not permitted I was going to write exact same thing. :-) > I think aufs works well when mount the local path as the first writable branch. Maybe local container using fuse, so container root user can do setattr success. But in nfs server side, no fuse filesystem using, so the remote container( relate to the nfs server side) can't do setattr to the nfs server file? You should see the behaviour on your nfs server, particulary why the server returned the error for NFS SETATTR request. I'd suggest you to check the parameters in the request packet first. J. R. Okajima References 1. mailto:hooanon...@gmail.com 2. mailto:hom...@163.com 3. mailto:aufs-users@lists.sourceforge.net
Re: Re: LXC unpreviliged problem with aufs mounted on nfs
Hi, Okajima San,
I found the error in packetdump2: setattr ERROR: Operation not
permitted
I think aufs works well when mount the local path as the first
writable branch. Maybe local container using fuse, so container root
user can do setattr success. But in nfs server side, no fuse filesystem
using, so the remote container( relate to the nfs server side) can't do
setattr to the nfs server file?
__
Michael Mao
From: [1]hom...@163.com
Date: 2020-03-22 13:15
To: [2]hooanon05g
CC: [3]aufs-users
Subject: Re: Re: LXC unpreviliged problem with aufs mounted on nfs
ps: last packetdump1 is the tcp data of command running:
useradd newuser,
and got the warnning: useradd: failure while writing changes to
/etc/shadow
this attachment packetdump2 is the data of command:
chown _apt:root aaae
and got the warnning: chown: changing ownership of './aaae':
Operation not permitted
Michael Mao
From: hom...@163.com
Date: 2020-03-22 13:10
To: hooanon05g
CC: aufs-users
Subject: Re: Re: LXC unpreviliged problem with aufs mounted on nfs
Hi, Okajima San,
Please refer to the attachment of the tcp packdump file.
Michael Mao
From: hom...@163.com
Date: 2020-03-22 12:35
To: hooanon05g
CC: aufs-users
Subject: Re: Re: LXC unpreviliged problem with aufs mounted on nfs
Hi, Okajima San,
Thanks. That will be easier for me to manage the aufs mount with
the xino option.
Yes, Problem is still there after I reboot the system.
About the LSM, I just stop the AppArmor service, and setenforce 0
to close the Selinux. It seems not work.
About the XATTR, I found some info about fuse_xattrs , which can
simulate the NFS XATTR. But infomation is too few to check it out.
I am trying to building the kernel, but the network is slow, it
will take a long time.
I will try the packet dumping with tcpdump. Only NFS packet data
needed?
Michael Mao
From: J. R. Okajima
Date: 2020-03-22 11:59
To: hom...@163.com
CC: aufs-users
Subject: Re: LXC unpreviliged problem with aufs mounted on nfs
"hom...@163.com":
> I thought the xino option is necessary for aufs mount when has
branch of nfs filesystem.
Generally you don't need to specify xino option. Refer to aufs manual
for its default path.
> When I reboot the system, the xino option works. Now I add the
xino option to mount again. Now I didn't find that warnning in kernel
log.
But the problem of chown still remains, right?
Even if we make the kernel log very verbose (by chaging printk under
procfs), it didn't give us good info.
Hmm, have you ever tried collecting network packet, by wireshark or
something? If we can see the packets between nfs client and server, we
may be able find which side the problem exists on.
I thought the problem is related to XATTR or LSM because there ever
have
benn the similar problems. But your repoort indicated it was not. Now
I'd like to narrow down the range (layers) where the problem happens.
Current candidates are vfs, aufs, nfs(client) and the server side.
Since you have no experience building your kernel, we have no good
method to see vfs and aufs. If we can see the packets between nfs
server and client, and the client sends the request correctly, then it
means the problem exists on the server side.
Can you try packet dumping?
J. R. Okajima
Hi, Okajima San,
Thanks. That will be easier for me to manage the aufs mount with
the xino option.
Yes, Problem is still there after I reboot the system.
About the LSM, I just stop the AppArmor service, and setenforce
0
to close the Selinux. It seems not work.
About the XATTR, I found some info about fuse_xattrs , which can
simulate the NFS XATTR. But infomation is too few to check it out.
I am trying to building the kernel, but the network is slow, it
will take a long time.
I will try the packet dumping with tcpdump. Only NFS packet
data
needed?
__
Michael Mao
From: [1]J. R. Okajima
Date: 2020-03-22 11:59
To: [2]hom...@163.com
CC: [3]aufs-users
Subject: Re: LXC unpreviliged problem with aufs mounted on nfs
"hom...@163.com":
> I thought the xino option is necessary for aufs mount when has
branch of nfs filesystem.
Generally you don't need to specify xino option. Refer to aufs
manual
for its default path.
> When I reboot the system, the xino option works. Now I add the
xino option to mount again. Now I didn't find that warnning in
kernel
log.
But the probl
Re: Re: LXC unpreviliged problem with aufs mounted on nfs
ps: last packetdump1 is the tcp data of command running:
useradd newuser,
and got the warnning: useradd: failure while writing changes to
/etc/shadow
this attachment packetdump2 is the data of command:
chown _apt:root aaae
and got the warnning: chown: changing ownership of './aaae':
Operation not permitted
__
Michael Mao
From: [1]hom...@163.com
Date: 2020-03-22 13:10
To: [2]hooanon05g
CC: [3]aufs-users
Subject: Re: Re: LXC unpreviliged problem with aufs mounted on nfs
Hi, Okajima San,
Please refer to the attachment of the tcp packdump file.
__
Michael Mao
From: [4]hom...@163.com
Date: 2020-03-22 12:35
To: [5]hooanon05g
CC: [6]aufs-users
Subject: Re: Re: LXC unpreviliged problem with aufs mounted on nfs
Hi, Okajima San,
Thanks. That will be easier for me to manage the aufs mount with
the xino option.
Yes, Problem is still there after I reboot the system.
About the LSM, I just stop the AppArmor service, and setenforce 0
to close the Selinux. It seems not work.
About the XATTR, I found some info about fuse_xattrs , which can
simulate the NFS XATTR. But infomation is too few to check it out.
I am trying to building the kernel, but the network is slow, it
will take a long time.
I will try the packet dumping with tcpdump. Only NFS packet data
needed?
Michael Mao
From: J. R. Okajima
Date: 2020-03-22 11:59
To: hom...@163.com
CC: aufs-users
Subject: Re: LXC unpreviliged problem with aufs mounted on nfs
"hom...@163.com":
> I thought the xino option is necessary for aufs mount when has
branch of nfs filesystem.
Generally you don't need to specify xino option. Refer to aufs manual
for its default path.
> When I reboot the system, the xino option works. Now I add the
xino option to mount again. Now I didn't find that warnning in kernel
log.
But the problem of chown still remains, right?
Even if we make the kernel log very verbose (by chaging printk under
procfs), it didn't give us good info.
Hmm, have you ever tried collecting network packet, by wireshark or
something? If we can see the packets between nfs client and server, we
may be able find which side the problem exists on.
I thought the problem is related to XATTR or LSM because there ever
have
benn the similar problems. But your repoort indicated it was not. Now
I'd like to narrow down the range (layers) where the problem happens.
Current candidates are vfs, aufs, nfs(client) and the server side.
Since you have no experience building your kernel, we have no good
method to see vfs and aufs. If we can see the packets between nfs
server and client, and the client sends the request correctly, then it
means the problem exists on the server side.
Can you try packet dumping?
J. R. Okajima
Hi, Okajima San,
Thanks. That will be easier for me to manage the aufs mount with
the xino option.
Yes, Problem is still there after I reboot the system.
About the LSM, I just stop the AppArmor service, and setenforce
0
to close the Selinux. It seems not work.
About the XATTR, I found some info about fuse_xattrs , which can
simulate the NFS XATTR. But infomation is too few to check it out.
I am trying to building the kernel, but the network is slow, it
will take a long time.
I will try the packet dumping with tcpdump. Only NFS packet
data
needed?
__
Michael Mao
From: [1]J. R. Okajima
Date: 2020-03-22 11:59
To: [2]hom...@163.com
CC: [3]aufs-users
Subject: Re: LXC unpreviliged problem with aufs mounted on nfs
"hom...@163.com":
> I thought the xino option is necessary for aufs mount when has
branch of nfs filesystem.
Generally you don't need to specify xino option. Refer to aufs
manual
for its default path.
> When I reboot the system, the xino option works. Now I add the
xino option to mount again. Now I didn't find that warnning in
kernel
log.
But the problem of chown still remains, right?
Even if we make the kernel log very verbose (by chaging printk under
procfs), it didn't give us good info.
Hmm, have you ever tried collecting network packet, by wireshark or
something? If we can see the packets between nfs client and server,
we
may be able find which side the problem exists on.
I thought the problem is related to XATTR or LSM because there ever
have
benn the similar problems. But your repoort indica
Re: Re: LXC unpreviliged problem with aufs mounted on nfs
Sorry, fix this: That will be easier for me to manage the aufs mount without the xino option. __ Michael Mao From: [1]hom...@163.com Date: 2020-03-22 12:35 To: [2]hooanon05g CC: [3]aufs-users Subject: Re: Re: LXC unpreviliged problem with aufs mounted on nfs Hi, Okajima San, Thanks. That will be easier for me to manage the aufs mount with the xino option. Yes, Problem is still there after I reboot the system. About the LSM, I just stop the AppArmor service, and setenforce 0 to close the Selinux. It seems not work. About the XATTR, I found some info about fuse_xattrs , which can simulate the NFS XATTR. But infomation is too few to check it out. I am trying to building the kernel, but the network is slow, it will take a long time. I will try the packet dumping with tcpdump. Only NFS packet data needed? __ Michael Mao From: [4]J. R. Okajima Date: 2020-03-22 11:59 To: [5]hom...@163.com CC: [6]aufs-users Subject: Re: LXC unpreviliged problem with aufs mounted on nfs "hom...@163.com": > I thought the xino option is necessary for aufs mount when has branch of nfs filesystem. Generally you don't need to specify xino option. Refer to aufs manual for its default path. > When I reboot the system, the xino option works. Now I add the xino option to mount again. Now I didn't find that warnning in kernel log. But the problem of chown still remains, right? Even if we make the kernel log very verbose (by chaging printk under procfs), it didn't give us good info. Hmm, have you ever tried collecting network packet, by wireshark or something? If we can see the packets between nfs client and server, we may be able find which side the problem exists on. I thought the problem is related to XATTR or LSM because there ever have benn the similar problems. But your repoort indicated it was not. Now I'd like to narrow down the range (layers) where the problem happens. Current candidates are vfs, aufs, nfs(client) and the server side. Since you have no experience building your kernel, we have no good method to see vfs and aufs. If we can see the packets between nfs server and client, and the client sends the request correctly, then it means the problem exists on the server side. Can you try packet dumping? J. R. Okajima References 1. mailto:hom...@163.com 2. mailto:hooanon...@gmail.com 3. mailto:aufs-users@lists.sourceforge.net 4. mailto:hooanon...@gmail.com 5. mailto:hom...@163.com 6. mailto:aufs-users@lists.sourceforge.net
Re: Re: LXC unpreviliged problem with aufs mounted on nfs
Hi, Okajima San, Thanks. That will be easier for me to manage the aufs mount with the xino option. Yes, Problem is still there after I reboot the system. About the LSM, I just stop the AppArmor service, and setenforce 0 to close the Selinux. It seems not work. About the XATTR, I found some info about fuse_xattrs , which can simulate the NFS XATTR. But infomation is too few to check it out. I am trying to building the kernel, but the network is slow, it will take a long time. I will try the packet dumping with tcpdump. Only NFS packet data needed? __ Michael Mao From: [1]J. R. Okajima Date: 2020-03-22 11:59 To: [2]hom...@163.com CC: [3]aufs-users Subject: Re: LXC unpreviliged problem with aufs mounted on nfs "hom...@163.com": > I thought the xino option is necessary for aufs mount when has branch of nfs filesystem. Generally you don't need to specify xino option. Refer to aufs manual for its default path. > When I reboot the system, the xino option works. Now I add the xino option to mount again. Now I didn't find that warnning in kernel log. But the problem of chown still remains, right? Even if we make the kernel log very verbose (by chaging printk under procfs), it didn't give us good info. Hmm, have you ever tried collecting network packet, by wireshark or something? If we can see the packets between nfs client and server, we may be able find which side the problem exists on. I thought the problem is related to XATTR or LSM because there ever have benn the similar problems. But your repoort indicated it was not. Now I'd like to narrow down the range (layers) where the problem happens. Current candidates are vfs, aufs, nfs(client) and the server side. Since you have no experience building your kernel, we have no good method to see vfs and aufs. If we can see the packets between nfs server and client, and the client sends the request correctly, then it means the problem exists on the server side. Can you try packet dumping? J. R. Okajima References 1. mailto:hooanon...@gmail.com 2. mailto:hom...@163.com 3. mailto:aufs-users@lists.sourceforge.net
Re: Re: LXC unpreviliged problem with aufs mounted on nfs
Hi, Okajima San, I saw some info about aufs from web, like : [1]http://gdr.geekhood.net/gdrwpl/aufs-nfs.php, It says: Also, when exporting via NFS, the AUFS mount should have xino=/tmp/somefile option attached, with the file different for different share. And in [2]http://aufs.sourceforge.net/aufs5/man.html#Exporting%20Aufs%20via %20NFS, Exporting Aufs via NFS The External Inode Number Bitmap, Translation Table and Generation Table (xino) is required since NFS file handle is based upon inode number. The mount option 'xino' is enabled by default. I thought the xino option is necessary for aufs mount when has branch of nfs filesystem. When I reboot the system, the xino option works. Now I add the xino option to mount again. Now I didn't find that warnning in kernel log. __ Michael Mao From: [3]J. R. Okajima Date: 2020-03-22 11:25 To: [4]hom...@163.com CC: [5]aufs-users Subject: Re: LXC unpreviliged problem with aufs mounted on nfs "hom...@163.com": > About the xino, I gave the xino option first time , but after some kind of operations, I found aufs can't mount success if I still give the same xino path, even if I umount the path. So I remove the xino option to mount the path now. > I will reboot the system and try again mount with the xino. Hmm, you might not understand why it failed. The log > Mar 22 10:22:44 zss-node1 kernel: [62451.849348] aufs au_xino_create:819:mount[30959]: open /tmp/aaa(-17) means "17:EEXIST:File exists". The filepath given to xino must be a new one since aufs will create it. But why do you want to add xino=? Any particular reason? J. R. Okajima References 1. http://gdr.geekhood.net/gdrwpl/aufs-nfs.php 2. http://aufs.sourceforge.net/aufs5/man.html#Exporting%20Aufs%20via%20NFS 3. mailto:hooanon...@gmail.com 4. mailto:hom...@163.com 5. mailto:aufs-users@lists.sourceforge.net
Re: Re: LXC unpreviliged problem with aufs mounted on nfs
Hi, Okajima San, OK, I try to gather them to you. About the xino, I gave the xino option first time , but after some kind of operations, I found aufs can't mount success if I still give the same xino path, even if I umount the path. So I remove the xino option to mount the path now. I will reboot the system and try again mount with the xino. BTW, I don't know if you will login to my server and debug by yourself. I think that may be more effective. And I don't know what online IM tool you are using, such as wechat in China. Maybe we can add each other to communicate online. BR. __ Michael Mao From: [1]J. R. Okajima Date: 2020-03-22 10:54 To: [2]hom...@163.com CC: [3]aufs-users Subject: Re: LXC unpreviliged problem with aufs mounted on nfs "hom...@163.com": > Sorry, please refer to below: The log is not so informative unfortunately, but I can see this line. > Mar 22 10:22:44 zss-node1 kernel: [62451.849348] aufs au_xino_create:819:mount[30959]: open /tmp/aaa(-17) Did you give xino= option? Are you running aufs with noxino? Next time when you post, please attach these info. (from aufs README file) When you have any problems or strange behaviour in aufs, please let me know with: - /proc/mounts (instead of the output of mount(8)) - /sys/module/aufs/* - /sys/fs/aufs/* (if you have them) - /debug/aufs/* (if you have them) - linux kernel version if your kernel is not plain, for example modified by distributor, the url where i can download its source is necessary too. - aufs version which was printed at loading the module or booting the system, instead of the date you downloaded. - configuration (define/undefine CONFIG_AUFS_xxx) - kernel configuration or /proc/config.gz (if you have it) - LSM (linux security module, if you are using) - behaviour which you think to be incorrect J. R. Okajima References 1. mailto:hooanon...@gmail.com 2. mailto:hom...@163.com 3. mailto:aufs-users@lists.sourceforge.net
Re: Re: LXC unpreviliged problem with aufs mounted on nfs
l="lxd-aaa_//&:lxd-aaa_:unconfined" name="/usr/lib/NetworkManager/nm-dhcp-client.action" pid=31683 comm="apparmor_parser" Mar 22 10:23:19 zss-node1 kernel: [62487.116326] audit: type=1400 audit(1584843799.408:379): apparmor="STATUS" operation="profile_load" label="lxd-aaa_//&:lxd-aaa_:unconfined" name="/usr/lib/NetworkManager/nm-dhcp-helper" pid=31683 comm="apparmor_parser" Mar 22 10:23:19 zss-node1 kernel: [62487.118846] audit: type=1400 audit(1584843799.408:380): apparmor="STATUS" operation="profile_load" label="lxd-aaa_//&:lxd-aaa_:unconfined" name="/usr/lib/connman/scripts/dhclient-script" pid=31683 comm="apparmor_parser" __ Michael Mao From: [1]J. R. Okajima Date: 2020-03-22 10:36 To: [2]hom...@163.com CC: [3]aufs-users Subject: Re: LXC unpreviliged problem with aufs mounted on nfs "hom...@163.com": > Please refer to the attachment. I am afraid you forgot the kernel log. J. R. Okajima References 1. mailto:hooanon...@gmail.com 2. mailto:hom...@163.com 3. mailto:aufs-users@lists.sourceforge.net
Re: Re: LXC unpreviliged problem with aufs mounted on nfs
Hi,Okajima San, Please refer to the attachment. __ Michael Mao From: [1]J. R. Okajima Date: 2020-03-22 09:17 To: [2]hom...@163.com CC: [3]aufs-users Subject: Re: LXC unpreviliged problem with aufs mounted on nfs "hom...@163.com": > I never open the kernel debug. Is there any quick way to open CONFIG_AUFS_DEBUG? If you have never experienced configuring and building your kernel, then forget about CONFIG_AUFS_DEBUG. There still be another (but less effective) method, I wonder. Assuming the kernel splats some msgs but you don't recieve them, let's try this. # id # cat /proc/sys/kernel/printk (assume "4 4 1 7") # echo 8 >> /proc/sys/kernel/printk # strace chown apt:root ./aaae # echo 4 >> /proc/sys/kernel/printk (here '4' is the first value of the original) J. R. Okajima References 1. mailto:hooanon...@gmail.com 2. mailto:hom...@163.com 3. mailto:aufs-users@lists.sourceforge.net strace chown _apt in unpreviliged container by printk .txt Description: Binary data
Re: Re: LXC unpreviliged problem with aufs mounted on nfs
Hi, Okajima San, I never open the kernel debug. Is there any quick way to open CONFIG_AUFS_DEBUG? I am downloading the [1]https://github.com/sfjro/aufs4-linux.git, but the network is so slow that may take me a long time to finish it. __ Michael Mao From: [2]J. R. Okajima Date: 2020-03-21 23:20 To: [3]hom...@163.com CC: [4]aufs-users Subject: Re: LXC unpreviliged problem with aufs mounted on nfs "hom...@163.com": > I have the AppArmor and Seccomp closed, but still got the warning. Ok. Now define our test command as "chown apt:root ./aaae" and let's step forward. We already know that chown failed because fchownat(2) returned EPERM. (from your previous log) fchownat(AT_FDCWD, "./aaae", 104, 0, 0) = -1 EPERM (Operation not permitted) Assuming there is no error in VFS and LSM, let's track down aufs. - enable CONFIG_AUFS_DEBUG - set a module parameter debug=1 just before chown, and reset to debug=0 just after chown # id # echo 1 >> /sys/module/aufs/parameter/debug # strace chown apt:root ./aaae # echo 0 >> /sys/module/aufs/parameter/debug and show me the strace output and the kernel log. Just to make sure, you coundn't find any related msg in your LSM logs, right? And this is unrelated to capability, right? J. . Okajima References 1. https://github.com/sfjro/aufs4-linux.git, 2. mailto:hooanon...@gmail.com 3. mailto:hom...@163.com 4. mailto:aufs-users@lists.sourceforge.net
Re: Re: LXC unpreviliged problem with aufs mounted on nfs
Hi, Okajima San, I have the AppArmor and Seccomp closed, but still got the warning. __ Michael Mao From: [1]hom...@163.com Date: 2020-03-21 22:45 To: [2]hooanon05g CC: [3]aufs-users Subject: Re: Re: LXC unpreviliged problem with aufs mounted on nfs Hi, Okajima San, OK. Since I am using LXD to drive the LXC, and AppArmor is default profile for LXD driver. I need to study how to close this function in LXD. __ Michael Mao From: [4]J. R. Okajima Date: 2020-03-21 22:26 To: [5]hom...@163.com CC: [6]aufs-users Subject: Re: LXC unpreviliged problem with aufs mounted on nfs "hom...@163.com": > I run the command : useradd 1234, and got the warning: useradd: failure while writing changes to /etc/shadow > and run: chown _apt:root ./aaae (aaae is a txt file), and got warning: chown: changing ownership of './aaae': Operation not permitted Before diving into these strace logs, I'd want you to confirm one more thing. Are there any msg left from your AppArmor or any other LSM? If you stop all LSM temporarly and test again, what will happen? J. R. Okajima References 1. mailto:hom...@163.com 2. mailto:hooanon...@gmail.com 3. mailto:aufs-users@lists.sourceforge.net 4. mailto:hooanon...@gmail.com 5. mailto:hom...@163.com 6. mailto:aufs-users@lists.sourceforge.net
Re: Re: LXC unpreviliged problem with aufs mounted on nfs
Hi, Okajima San, OK. Since I am using LXD to drive the LXC, and AppArmor is default profile for LXD driver. I need to study how to close this function in LXD. __ Michael Mao From: [1]J. R. Okajima Date: 2020-03-21 22:26 To: [2]hom...@163.com CC: [3]aufs-users Subject: Re: LXC unpreviliged problem with aufs mounted on nfs "hom...@163.com": > I run the command : useradd 1234, and got the warning: useradd: failure while writing changes to /etc/shadow > and run: chown _apt:root ./aaae (aaae is a txt file), and got warning: chown: changing ownership of './aaae': Operation not permitted Before diving into these strace logs, I'd want you to confirm one more thing. Are there any msg left from your AppArmor or any other LSM? If you stop all LSM temporarly and test again, what will happen? J. R. Okajima References 1. mailto:hooanon...@gmail.com 2. mailto:hom...@163.com 3. mailto:aufs-users@lists.sourceforge.net
Re: Re: LXC unpreviliged problem with aufs mounted on nfs
Hi, Okajima San, Please find the attachments which are the outputs of comannds execution in the unpreviliged container. I run the command : useradd 1234, and got the warning: useradd: failure while writing changes to /etc/shadow and run: chown _apt:root ./aaae (aaae is a txt file), and got warning: chown: changing ownership of './aaae': Operation not permitted But the kernel log still seems the same. __ Michael Mao From: [1]J. R. Okajima Date: 2020-03-21 21:56 To: [2]hom...@163.com CC: [3]aufs-users Subject: Re: LXC unpreviliged problem with aufs mounted on nfs "hom...@163.com": > Attachments is the output in the unpreviliged container and parent server. I cannot see any error in your "strace touch newfile in unpreviliged containers.txt" and "strace touch newfile in parent server.txt". Please define a command to reproduce the error. J. R. Okajima References 1. mailto:hooanon...@gmail.com 2. mailto:hom...@163.com 3. mailto:aufs-users@lists.sourceforge.net strace useradd 1234 in unpreviliged container.txt Description: Binary data strace chown _apt in unpreviliged container.txt Description: Binary data kern.log.txt Description: Binary data
Re: Re: LXC unpreviliged problem with aufs mounted on nfs
Hi, Okajima San, Attachments is the output in the unpreviliged container and parent server. It seems the kernel log doesn't change when touch newfile. Please let me know if any question. BR. __ Michael Mao From: [1]J. R. Okajima Date: 2020-03-21 20:35 To: [2]hom...@163.com CC: [3]aufs-users Subject: Re: LXC unpreviliged problem with aufs mounted on nfs "hom...@163.com": > Below attached is the /var/log/kern.log and /var/log/syslog, which logs the info from nfs client side ubuntu-v18.04 reboot to the LXC rootfs mount by aufs and LXC startup . I didn't find any info about XATTR. > Did I miss any thing? or Should I show you the log file of the nfs server side ? As a next step, - define your test command such as "touch newfile" or "rm oldfile" - run "strace touch newfile" and send me the output - and check the kernel log when you ran "touch newfile" J. R. Okajima References 1. mailto:hooanon...@gmail.com 2. mailto:hom...@163.com 3. mailto:aufs-users@lists.sourceforge.net strace touch newfile in unpreviliged containers.txt Description: Binary data strace touch newfile in parent server.txt Description: Binary data kern.log.txt Description: Binary data
Re: Re: LXC unpreviliged problem with aufs mounted on nfs
Hi, Okajima San, BTW, I got some infomation from [1]http://aufs.sourceforge.net/, that describe the aufs module parameter: allow_userns= Y | N Allows an unprivileged mount under user namespace. Userns mount to put AUFS into a chroot environment can be useful while it as a security worry. This parameter sets an internal flag FS_USERNS_MOUNT and allows userns unconditionally. See the discussion in http://www.mail-archive.com/aufs-users@lists.sourceforge.net/msg04266.h tml and its thread. The default is 'N'. If CONFIG_USER_NS is disabled, this parameter is meaningless. Should I mount the container rootfs path with aufs under the unpreviliged user? My ubuntu parent server has the aufs kernel support : grep aufs /proc/filesystems nodev aufs And I also setup the aufs module. and run " modprobe aufs allow_userns=Y" ,(and checked the CONFIG_USER_NS is enabled). and tried to mount the container rootfs by root user. But the unpreviliged container still can't work well. When I mount the container rootfs by unpreviliged user, I got the warn: mount: only root can use "--options" option Looking forward to you reply. Thanks. BR. __ Michael Mao From: [2]hom...@163.com Date: 2020-03-21 17:29 To: [3]hooanon05g CC: [4]aufs-users Subject: Re: Re: LXC unpreviliged problem with aufs mounted on nfs Hi,Mr. Okajima, Below attached is the /var/log/kern.log and /var/log/syslog, which logs the info from nfs client side ubuntu-v18.04 reboot to the LXC rootfs mount by aufs and LXC startup . I didn't find any info about XATTR. Did I miss any thing? or Should I show you the log file of the nfs server side ? thanks a lot. /var/log/kern.log Mar 21 17:02:06 zss-node1 kernel: [9.157693] random: (sd-executor): uninitialized urandom read (16 bytes read) Mar 21 17:02:06 zss-node1 kernel: [9.529600] Loading iSCSI transport class v2.0-870. Mar 21 17:02:06 zss-node1 kernel: [9.537351] iscsi: registered transport (tcp) Mar 21 17:02:06 zss-node1 kernel: [9.539466] EXT4-fs (vda1): re-mounted. Opts: errors=remount-ro Mar 21 17:02:06 zss-node1 kernel: [9.745084] RPC: Registered named UNIX socket transport module. Mar 21 17:02:06 zss-node1 kernel: [9.746528] RPC: Registered udp transport module. Mar 21 17:02:06 zss-node1 kernel: [9.747939] RPC: Registered tcp transport module. Mar 21 17:02:06 zss-node1 kernel: [9.749352] RPC: Registered tcp NFSv4.1 backchannel transport module. Mar 21 17:02:06 zss-node1 kernel: [9.808824] iscsi: registered transport (iser) Mar 21 17:02:06 zss-node1 kernel: [ 10.193888] RAPL PMU: API unit is 2^-32 Joules, 3 fixed counters, 10737418240 ms ovfl timer Mar 21 17:02:06 zss-node1 kernel: [ 10.193889] RAPL PMU: hw unit of domain pp0-core 2^-0 Joules Mar 21 17:02:06 zss-node1 kernel: [ 10.193890] RAPL PMU: hw unit of domain package 2^-0 Joules Mar 21 17:02:06 zss-node1 kernel: [ 10.193890] RAPL PMU: hw unit of domain dram 2^-16 Joules Mar 21 17:02:06 zss-node1 kernel: [ 10.206705] shpchp: Standard Hot Plug PCI Controller Driver version: 0.4 Mar 21 17:02:06 zss-node1 kernel: [ 10.804762] spl: loading out-of-tree module taints kernel. Mar 21 17:02:06 zss-node1 kernel: [ 10.805915] SPL: Loaded module v0.7.5-1ubuntu1 Mar 21 17:02:06 zss-node1 kernel: [ 10.836922] znvpair: module license 'CDDL' taints kernel. Mar 21 17:02:06 zss-node1 kernel: [ 10.836923] Disabling lock debugging due to kernel taint Mar 21 17:02:06 zss-node1 kernel: [ 13.474100] ZFS: Loaded module v0.7.5-1ubuntu16.4, ZFS pool version 5000, ZFS filesystem version 5 Mar 21 17:02:06 zss-node1 kernel: [ 14.104544] audit: type=1400 audit(1584781326.408:2): apparmor="STATUS" operation="profile_load" profile="unconfined" name="/usr/bin/lxc-start" pid=1238 comm="apparmor_parser" Mar 21 17:02:06 zss-node1 kernel: [ 14.107425] audit: type=1400 audit(1584781326.408:3): apparmor="STATUS" operation="profile_load" profile="unconfined" name="/usr/bin/man" pid=1239 comm="apparmor_parser" Mar 21 17:02:06 zss-node1 kernel: [ 14.107426] audit: type=1400 audit(1584781326.408:4): apparmor="STATUS" operation="profile_load" profile="unconfined" name="man_filter" pid=1239 comm="apparmor_parser" Mar 21 17:02:06 zss-node1 kernel: [ 14.107428] audit: type=1400 audit(1584781326.408:5): apparmor="STATUS" operation="profile_load" profile="unconfined" name="man_groff" pid=1239 comm="apparmor_parser" Mar 21 17:02:
Re: Re: LXC unpreviliged problem with aufs mounted on nfs
=3643 comm="apparmor_parser" Mar 21 17:08:46 zss-node1 kernel: [ 414.195613] audit: type=1400 audit(1584781726.496:46): apparmor="STATUS" operation="profile_load" label="lxd-bbb_//&:lxd-bbb_:unconfined" name="/usr/lib/NetworkManager/nm-dhcp-client.action" pid=3643 comm="apparmor_parser" Mar 21 17:08:46 zss-node1 kernel: [ 414.195614] audit: type=1400 audit(1584781726.496:47): apparmor="STATUS" operation="profile_load" label="lxd-bbb_//&:lxd-bbb_:unconfined" name="/usr/lib/NetworkManager/nm-dhcp-helper" pid=3643 comm="apparmor_parser" Mar 21 17:08:46 zss-node1 kernel: [ 414.195616] audit: type=1400 audit(1584781726.496:48): apparmor="STATUS" operation="profile_load" label="lxd-bbb_//&:lxd-bbb_:unconfined" name="/usr/lib/connman/scripts/dhclient-script" pid=3643 comm="apparmor_parser" Mar 21 17:15:51 zss-node1 systemd[1]: Started Session 3 of user root. Mar 21 17:16:48 zss-node1 systemd-resolved[1250]: Grace period over, resuming full feature set (UDP+EDNS0) for DNS server 114.114.115.115. Mar 21 17:17:01 zss-node1 CRON[5298]: (root) CMD ( cd / && run-parts --report /etc/cron.hourly) Mar 21 17:17:02 zss-node1 systemd[1]: Starting Cleanup of Temporary Directories... Mar 21 17:17:02 zss-node1 systemd[1]: Started Cleanup of Temporary Directories. Mar 21 17:20:43 zss-node1 systemd[1]: Starting Message of the Day... Mar 21 17:20:43 zss-node1 systemd-resolved[1250]: Using degraded feature set (UDP) for DNS server 114.114.115.115. Mar 21 17:20:51 zss-node1 50-motd-news[5318]: * Latest Kubernetes 1.18 beta is now available for your laptop, NUC, cloud Mar 21 17:20:51 zss-node1 50-motd-news[5318]:instance or Raspberry Pi, with automatic updates to the final GA release. Mar 21 17:20:51 zss-node1 50-motd-news[5318]: sudo snap install microk8s --channel=1.18/beta --classic Mar 21 17:20:51 zss-node1 50-motd-news[5318]: * Multipass 1.1 adds proxy support for developers behind enterprise Mar 21 17:20:51 zss-node1 50-motd-news[5318]:firewalls. Rapid prototyping for cloud operations just got easier. Mar 21 17:20:51 zss-node1 50-motd-news[5318]: https://multipass.run/ Mar 21 17:20:51 zss-node1 systemd[1]: Started Message of the Day. __ Michael Mao From: [1]J. R. Okajima Date: 2020-03-21 14:14 To: [2]homalu CC: [3]aufs-users Subject: Re: LXC unpreviliged problem with aufs mounted on nfs homalu: > My NFS export file is like this : > /mnt/NFSmountpoint 172.16.0.0/24(rw,fsid=0,insecure,no_root_squash,no_subtree_check,sync,c rossmnt). > Is this OK? I hope so, but I am not sure. As a next step, I'd suggest you to check the kernel log on nfs client, whether aufs complained about XATTR on nfs. J. R. Okajima References 1. mailto:hooanon...@gmail.com 2. mailto:hom...@163.com 3. mailto:aufs-users@lists.sourceforge.net
