[Kernel-packages] [Bug 1617388] Re: When using overlayfs with kernel 4.4, some files cannot be deleted.

[oleg]

> This also means that a kuid outside of the range 10-165535
> is literally meaningless with respect to that mount - the kernel
> has no way to map it to an id valid in the filesystem. 
> That is the real reason for the patch.

That is a compelling point.  mounts inside a user-namespace are only
well-defined if every file's uid lies within the user-namespace.

I wish I had realised that prior to filing the bug report, and am
now happy to close it as not-a-bug (-:  Thank you for your
highly informative replies.

** Changed in: linux (Ubuntu)
   Status: Triaged => Invalid

-- 
You received this bug notification because you are a member of Kernel
Packages, which is subscribed to linux in Ubuntu.
https://bugs.launchpad.net/bugs/1617388

Title:
  When using overlayfs with kernel 4.4, some files cannot be deleted.

Status in linux package in Ubuntu:
  Invalid

Bug description:
  #!/bin/bash
  # -
  # This script exhibits a bug in overlayfs in kernel 4.4.
  # The bug is not present in kernel 4.2.
  # The bug can be reproduced in an x86_64 virtual-machine;
  # 32-bit has not been tested.
  #
  # With kernel 4.2, the script output ends with:
  #   "script completed without encountering a kernel bug"
  #
  # With kernel 4.4, the script output ends with:
  #   "rm: cannot remove ‘mnt_ovl/sub/sub.txt’:
  #Value too large for defined data type"
  #
  # The script depends upon lxc-usernsexec (part of the lxc1 package) to
  # create a user-namespace.
  #
  # The script should be run as a normal user (not root), in a directory where
  # the user has write-permission:
  # ./script
  # 

  cleanup()
  {
     [[ -d "$storedir" ]] || exit 1
     cd "$storedir"   || exit 1
     [[ -d "$tmpdir"   ]] || exit 1
     lxc-usernsexec -m b:0:1000:1 -m b:10:10:1 -- rm -rf "$tmpdir"
  }

  trap cleanup EXIT

  set -e
  storedir="$(pwd)"

  # create tmpdir
  tmpdir="$(mktemp -d --tmpdir=.)"

  cd "$tmpdir"

  # create lowerdir for overlay
  mkdir -p lower/sub
  touch lower/lower.txt lower/sub/sub.txt

  cd ..

  chmod -R a+rwX "$tmpdir"

  # run a script in a user namepace
  lxc-usernsexec -m b:0:10:65534 --  bash << EOF
     set -e
     cd "$tmpdir"

     # create tmpfs
     mkdir mnt_tmpfs
     mount -t tmpfs tmpfs mnt_tmpfs

     # create upperdir and workdir for overlay
     mkdir mnt_tmpfs/{upper,work}

     # mount overlay
     mkdir mnt_ovl
     mount -t overlay \
     -o lowerdir=lower,upperdir=mnt_tmpfs/upper,workdir=mnt_tmpfs/work \
     overlay mnt_ovl

     echo 'overlay directory listing'
     ls -RF mnt_ovl
     echo ''

     set -x
     rm mnt_ovl/lower.txt# always succeeds
     rm mnt_ovl/sub/sub.txt  # fails with kernel 4.4+

     set +x
     echo 'script completed without encountering a kernel bug'

  EOF

To manage notifications about this bug go to:
https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1617388/+subscriptions

-- 
Mailing list: https://launchpad.net/~kernel-packages
Post to : kernel-packages@lists.launchpad.net
Unsubscribe : https://launchpad.net/~kernel-packages
More help   : https://help.launchpad.net/ListHelp

Re: [Kernel-packages] [Bug 1617388] Re: When using overlayfs with kernel 4.4, some files cannot be deleted.

[Seth Forshee]

On Thu, Sep 01, 2016 at 03:11:30PM -, oleg wrote:
> Question (related to UBUNTU: SAUCE: fs: Refuse uid/gid changes which don't 
> map into s_user_ns)
> ---
> 
> When an overlayfs is mounted inside a user-namespace, should it permit
> the creation of inodes in the upper layer with uids that are outside the
> user-namespace?
> 
> 
> My Tentative Answer  
> ---
> 
> (TLDR; yes)
> If a directory D is granted world-write permissions (in the initial 
> namespace) via 'chmod -R a+rwX', then any user is permitted to edit or delete 
> files in D, even if the files and D are owned by root.

Some operations are still restricted for normal users though, like
chowning a file that user doesn't own or setting certain xattrs.

> If we subsequently enter a user-namespace, then we are still permitted
> to edit files in D.
> 
> But if we enter a user-namespace and subsequently utilize D as the lower
> layer of an overlayfs, then editing files in D necessitates the creation
> of files in the upper layer with uids which are outside the user
> namespace.

And copying up files copies up all xattrs, setuid, setgid, etc. So what
if the lowerdir was in a nosuid mount and we copy up a suid-root file to
an upperdir that is in a mount without nosuid?

Actually we have protections against that specific scenario. I give it
as an example of how copy-up can be dangerous if the user doesn't
control the ids of the file being copied.

> While restriciting the permissible range of uids in the upper layer may
> well enhance security, it also limits the utility of overlayfs.
> overlayfs will sometimes deny permissions which were granted in the
> initial namespace.  overlayfs will remain useful for mounting a rootfs
> (since all uids are within the user-namespace), but not for mounting
> directories onto the rootfs.

Security is part of it, but not the full story. tmpfs is a little
unusual though, it's easier to understand if you think of using a
filesystem with a backing store (understanding that it's generally not
possible to mount these filesystems in user namespaces, but some support
for that is coming).

When we do a mount of a filesystem in a user namespace, the kernel will
interpret the uids in that filesystem as being in the user ns. So if
your user ns mapping is 0:10:65536 id 0 in the filesystem will be
mapped to id 10 in the kernel. That's done for a couple of reasons -
it means that id 0 in the filesystem shows up as id 0 in the user ns
where you mounted the filesystem, and it means a user can't inject
inodes into the kernel with an id that the user doesn't have control of
(i.e. any id not mapped into the user ns).

This also means that a kuid outside of the rante 10-165535 is
literally meaningless with respect to that mount - the kernel has no way
to map it to an id valid in the filesystem. That is the real reason for
the patch.

Back to tmpfs - since it has no backing store that restriction isn't
needed for that reason, and we could treat all tmpfs mounts as being in
init_user_ns. However the way it is now does have the benefit of
reducing the kernel's attack surface. And as of 4.8-rc this patch is
upstream, so upstream tmpfs is going to behave the same way xenial does
now.

I'm still mulling all of this over. The truth is that this is a
regression in Ubuntu because we allow overlayfs mounts in user
namespaces, but upstream does not so there's no regression there and
thus it may be difficult to convince upstream to change the behavior.

-- 
You received this bug notification because you are a member of Kernel
Packages, which is subscribed to linux in Ubuntu.
https://bugs.launchpad.net/bugs/1617388

Title:
  When using overlayfs with kernel 4.4, some files cannot be deleted.

Status in linux package in Ubuntu:
  Triaged

Bug description:
  #!/bin/bash
  # -
  # This script exhibits a bug in overlayfs in kernel 4.4.
  # The bug is not present in kernel 4.2.
  # The bug can be reproduced in an x86_64 virtual-machine;
  # 32-bit has not been tested.
  #
  # With kernel 4.2, the script output ends with:
  #   "script completed without encountering a kernel bug"
  #
  # With kernel 4.4, the script output ends with:
  #   "rm: cannot remove ‘mnt_ovl/sub/sub.txt’:
  #Value too large for defined data type"
  #
  # The script depends upon lxc-usernsexec (part of the lxc1 package) to
  # create a user-namespace.
  #
  # The script should be run as a normal user (not root), in a directory where
  # the user has write-permission:
  # ./script
  # 

  cleanup()
  {
     [[ -d "$storedir" ]] || exit 1
     cd "$storedir"   || exit 1
     [[ -d "$tmpdir"   ]] || exit 1
     lxc-usernsexec -m b:0:1000:1 -m b:10:10:1 -- rm -rf "$tmpdir"
  }

  trap cleanup EXIT

  set -e
  storedir="$(pwd)"

  # create tmpdir
  tmpdir="$(mktemp -d --tmpdir=.)"

 

[Kernel-packages] [Bug 1617388] Re: When using overlayfs with kernel 4.4, some files cannot be deleted.

[oleg]

Question (related to UBUNTU: SAUCE: fs: Refuse uid/gid changes which don't map 
into s_user_ns)
---

When an overlayfs is mounted inside a user-namespace, should it permit
the creation of inodes in the upper layer with uids that are outside the
user-namespace?


My Tentative Answer  
---

(TLDR; yes)
If a directory D is granted world-write permissions (in the initial namespace) 
via 'chmod -R a+rwX', then any user is permitted to edit or delete files in D, 
even if the files and D are owned by root.

If we subsequently enter a user-namespace, then we are still permitted
to edit files in D.

But if we enter a user-namespace and subsequently utilize D as the lower
layer of an overlayfs, then editing files in D necessitates the creation
of files in the upper layer with uids which are outside the user
namespace.

While restriciting the permissible range of uids in the upper layer may
well enhance security, it also limits the utility of overlayfs.
overlayfs will sometimes deny permissions which were granted in the
initial namespace.  overlayfs will remain useful for mounting a rootfs
(since all uids are within the user-namespace), but not for mounting
directories onto the rootfs.

-- 
You received this bug notification because you are a member of Kernel
Packages, which is subscribed to linux in Ubuntu.
https://bugs.launchpad.net/bugs/1617388

Title:
  When using overlayfs with kernel 4.4, some files cannot be deleted.

Status in linux package in Ubuntu:
  Triaged

Bug description:
  #!/bin/bash
  # -
  # This script exhibits a bug in overlayfs in kernel 4.4.
  # The bug is not present in kernel 4.2.
  # The bug can be reproduced in an x86_64 virtual-machine;
  # 32-bit has not been tested.
  #
  # With kernel 4.2, the script output ends with:
  #   "script completed without encountering a kernel bug"
  #
  # With kernel 4.4, the script output ends with:
  #   "rm: cannot remove ‘mnt_ovl/sub/sub.txt’:
  #Value too large for defined data type"
  #
  # The script depends upon lxc-usernsexec (part of the lxc1 package) to
  # create a user-namespace.
  #
  # The script should be run as a normal user (not root), in a directory where
  # the user has write-permission:
  # ./script
  # 

  cleanup()
  {
     [[ -d "$storedir" ]] || exit 1
     cd "$storedir"   || exit 1
     [[ -d "$tmpdir"   ]] || exit 1
     lxc-usernsexec -m b:0:1000:1 -m b:10:10:1 -- rm -rf "$tmpdir"
  }

  trap cleanup EXIT

  set -e
  storedir="$(pwd)"

  # create tmpdir
  tmpdir="$(mktemp -d --tmpdir=.)"

  cd "$tmpdir"

  # create lowerdir for overlay
  mkdir -p lower/sub
  touch lower/lower.txt lower/sub/sub.txt

  cd ..

  chmod -R a+rwX "$tmpdir"

  # run a script in a user namepace
  lxc-usernsexec -m b:0:10:65534 --  bash << EOF
     set -e
     cd "$tmpdir"

     # create tmpfs
     mkdir mnt_tmpfs
     mount -t tmpfs tmpfs mnt_tmpfs

     # create upperdir and workdir for overlay
     mkdir mnt_tmpfs/{upper,work}

     # mount overlay
     mkdir mnt_ovl
     mount -t overlay \
     -o lowerdir=lower,upperdir=mnt_tmpfs/upper,workdir=mnt_tmpfs/work \
     overlay mnt_ovl

     echo 'overlay directory listing'
     ls -RF mnt_ovl
     echo ''

     set -x
     rm mnt_ovl/lower.txt# always succeeds
     rm mnt_ovl/sub/sub.txt  # fails with kernel 4.4+

     set +x
     echo 'script completed without encountering a kernel bug'

  EOF

To manage notifications about this bug go to:
https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1617388/+subscriptions

-- 
Mailing list: https://launchpad.net/~kernel-packages
Post to : kernel-packages@lists.launchpad.net
Unsubscribe : https://launchpad.net/~kernel-packages
More help   : https://help.launchpad.net/ListHelp

[Kernel-packages] [Bug 1617388] Re: When using overlayfs with kernel 4.4, some files cannot be deleted.

[oleg]

> First, starting in xenial the tmpfs mount isn't going to allow any
inodes to be created in it that > are not mapped into the user
namespace.

Is this specific to tmpfs?  I was unable to locate the relevant
code/commit.

-- 
You received this bug notification because you are a member of Kernel
Packages, which is subscribed to linux in Ubuntu.
https://bugs.launchpad.net/bugs/1617388

Title:
  When using overlayfs with kernel 4.4, some files cannot be deleted.

Status in linux package in Ubuntu:
  Triaged

Bug description:
  #!/bin/bash
  # -
  # This script exhibits a bug in overlayfs in kernel 4.4.
  # The bug is not present in kernel 4.2.
  # The bug can be reproduced in an x86_64 virtual-machine;
  # 32-bit has not been tested.
  #
  # With kernel 4.2, the script output ends with:
  #   "script completed without encountering a kernel bug"
  #
  # With kernel 4.4, the script output ends with:
  #   "rm: cannot remove ‘mnt_ovl/sub/sub.txt’:
  #Value too large for defined data type"
  #
  # The script depends upon lxc-usernsexec (part of the lxc1 package) to
  # create a user-namespace.
  #
  # The script should be run as a normal user (not root), in a directory where
  # the user has write-permission:
  # ./script
  # 

  cleanup()
  {
     [[ -d "$storedir" ]] || exit 1
     cd "$storedir"   || exit 1
     [[ -d "$tmpdir"   ]] || exit 1
     lxc-usernsexec -m b:0:1000:1 -m b:10:10:1 -- rm -rf "$tmpdir"
  }

  trap cleanup EXIT

  set -e
  storedir="$(pwd)"

  # create tmpdir
  tmpdir="$(mktemp -d --tmpdir=.)"

  cd "$tmpdir"

  # create lowerdir for overlay
  mkdir -p lower/sub
  touch lower/lower.txt lower/sub/sub.txt

  cd ..

  chmod -R a+rwX "$tmpdir"

  # run a script in a user namepace
  lxc-usernsexec -m b:0:10:65534 --  bash << EOF
     set -e
     cd "$tmpdir"

     # create tmpfs
     mkdir mnt_tmpfs
     mount -t tmpfs tmpfs mnt_tmpfs

     # create upperdir and workdir for overlay
     mkdir mnt_tmpfs/{upper,work}

     # mount overlay
     mkdir mnt_ovl
     mount -t overlay \
     -o lowerdir=lower,upperdir=mnt_tmpfs/upper,workdir=mnt_tmpfs/work \
     overlay mnt_ovl

     echo 'overlay directory listing'
     ls -RF mnt_ovl
     echo ''

     set -x
     rm mnt_ovl/lower.txt# always succeeds
     rm mnt_ovl/sub/sub.txt  # fails with kernel 4.4+

     set +x
     echo 'script completed without encountering a kernel bug'

  EOF

To manage notifications about this bug go to:
https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1617388/+subscriptions

-- 
Mailing list: https://launchpad.net/~kernel-packages
Post to : kernel-packages@lists.launchpad.net
Unsubscribe : https://launchpad.net/~kernel-packages
More help   : https://help.launchpad.net/ListHelp

[Kernel-packages] [Bug 1617388] Re: When using overlayfs with kernel 4.4, some files cannot be deleted.

[oleg]

Real-world-use case: running ephemeral, containerized instances of an
app:

There may be multiple instances of the app, each running
in a separate ephemeral container with a unique uid_map.

The app's configuration directory is permanently stored on the host,
and is mounted in each container using overlayfs.

Permission for each container to access the configuration directory is granted 
via
either 'chmod a+rwX' or 'setfacl -m -u:container_uid:rwX'.

In certain situations it is desirable for the upper layer of overlayfs to be 
tmpfs:
   -read-only host filesystem
   -the host's disk is writeable but very slow (eg thumb-drives, nfs)
   -the host filesystem is btrfs 
(cannot be used as the upper layer with the 4.4 kernel. fixed in 4.7)

-- 
You received this bug notification because you are a member of Kernel
Packages, which is subscribed to linux in Ubuntu.
https://bugs.launchpad.net/bugs/1617388

Title:
  When using overlayfs with kernel 4.4, some files cannot be deleted.

Status in linux package in Ubuntu:
  Triaged

Bug description:
  #!/bin/bash
  # -
  # This script exhibits a bug in overlayfs in kernel 4.4.
  # The bug is not present in kernel 4.2.
  # The bug can be reproduced in an x86_64 virtual-machine;
  # 32-bit has not been tested.
  #
  # With kernel 4.2, the script output ends with:
  #   "script completed without encountering a kernel bug"
  #
  # With kernel 4.4, the script output ends with:
  #   "rm: cannot remove ‘mnt_ovl/sub/sub.txt’:
  #Value too large for defined data type"
  #
  # The script depends upon lxc-usernsexec (part of the lxc1 package) to
  # create a user-namespace.
  #
  # The script should be run as a normal user (not root), in a directory where
  # the user has write-permission:
  # ./script
  # 

  cleanup()
  {
     [[ -d "$storedir" ]] || exit 1
     cd "$storedir"   || exit 1
     [[ -d "$tmpdir"   ]] || exit 1
     lxc-usernsexec -m b:0:1000:1 -m b:10:10:1 -- rm -rf "$tmpdir"
  }

  trap cleanup EXIT

  set -e
  storedir="$(pwd)"

  # create tmpdir
  tmpdir="$(mktemp -d --tmpdir=.)"

  cd "$tmpdir"

  # create lowerdir for overlay
  mkdir -p lower/sub
  touch lower/lower.txt lower/sub/sub.txt

  cd ..

  chmod -R a+rwX "$tmpdir"

  # run a script in a user namepace
  lxc-usernsexec -m b:0:10:65534 --  bash << EOF
     set -e
     cd "$tmpdir"

     # create tmpfs
     mkdir mnt_tmpfs
     mount -t tmpfs tmpfs mnt_tmpfs

     # create upperdir and workdir for overlay
     mkdir mnt_tmpfs/{upper,work}

     # mount overlay
     mkdir mnt_ovl
     mount -t overlay \
     -o lowerdir=lower,upperdir=mnt_tmpfs/upper,workdir=mnt_tmpfs/work \
     overlay mnt_ovl

     echo 'overlay directory listing'
     ls -RF mnt_ovl
     echo ''

     set -x
     rm mnt_ovl/lower.txt# always succeeds
     rm mnt_ovl/sub/sub.txt  # fails with kernel 4.4+

     set +x
     echo 'script completed without encountering a kernel bug'

  EOF

To manage notifications about this bug go to:
https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1617388/+subscriptions

-- 
Mailing list: https://launchpad.net/~kernel-packages
Post to : kernel-packages@lists.launchpad.net
Unsubscribe : https://launchpad.net/~kernel-packages
More help   : https://help.launchpad.net/ListHelp

[Kernel-packages] [Bug 1617388] Re: When using overlayfs with kernel 4.4, some files cannot be deleted.

[Seth Forshee]

I had a slight misunderstanding when I looked at the code previously.
The copy up is of the parent directory, which makes sense because it
needs to modify one of the dirents in that directory. Which by extension
means that every ancestor of the dirent being unlinked needs to be
copied up.

So the problem is not the owner of the inode which is the target of the
unlink, but those of the ancestor directories. The tmpfs mount in the
user ns cannot contain uids not mapped into the user ns, which is why
the copy up fails. Since the directory will hang around after the unlink
finishes, we don't want to change its ownership, but a directory with
that ownership cannot exist in the tmpfs mount.

The problem may indeed be intractable then. Is this breaking some real-
world use case?

Note that even if we could do the copy up, I'm not sure that we should.
Generally speaking we don't want overlayfs to allow the user to create
objects in the upperdir that the mounter of the overlayfs filesystem
could not have created by other means.

-- 
You received this bug notification because you are a member of Kernel
Packages, which is subscribed to linux in Ubuntu.
https://bugs.launchpad.net/bugs/1617388

Title:
  When using overlayfs with kernel 4.4, some files cannot be deleted.

Status in linux package in Ubuntu:
  Triaged

Bug description:
  #!/bin/bash
  # -
  # This script exhibits a bug in overlayfs in kernel 4.4.
  # The bug is not present in kernel 4.2.
  # The bug can be reproduced in an x86_64 virtual-machine;
  # 32-bit has not been tested.
  #
  # With kernel 4.2, the script output ends with:
  #   "script completed without encountering a kernel bug"
  #
  # With kernel 4.4, the script output ends with:
  #   "rm: cannot remove ‘mnt_ovl/sub/sub.txt’:
  #Value too large for defined data type"
  #
  # The script depends upon lxc-usernsexec (part of the lxc1 package) to
  # create a user-namespace.
  #
  # The script should be run as a normal user (not root), in a directory where
  # the user has write-permission:
  # ./script
  # 

  cleanup()
  {
     [[ -d "$storedir" ]] || exit 1
     cd "$storedir"   || exit 1
     [[ -d "$tmpdir"   ]] || exit 1
     lxc-usernsexec -m b:0:1000:1 -m b:10:10:1 -- rm -rf "$tmpdir"
  }

  trap cleanup EXIT

  set -e
  storedir="$(pwd)"

  # create tmpdir
  tmpdir="$(mktemp -d --tmpdir=.)"

  cd "$tmpdir"

  # create lowerdir for overlay
  mkdir -p lower/sub
  touch lower/lower.txt lower/sub/sub.txt

  cd ..

  chmod -R a+rwX "$tmpdir"

  # run a script in a user namepace
  lxc-usernsexec -m b:0:10:65534 --  bash << EOF
     set -e
     cd "$tmpdir"

     # create tmpfs
     mkdir mnt_tmpfs
     mount -t tmpfs tmpfs mnt_tmpfs

     # create upperdir and workdir for overlay
     mkdir mnt_tmpfs/{upper,work}

     # mount overlay
     mkdir mnt_ovl
     mount -t overlay \
     -o lowerdir=lower,upperdir=mnt_tmpfs/upper,workdir=mnt_tmpfs/work \
     overlay mnt_ovl

     echo 'overlay directory listing'
     ls -RF mnt_ovl
     echo ''

     set -x
     rm mnt_ovl/lower.txt# always succeeds
     rm mnt_ovl/sub/sub.txt  # fails with kernel 4.4+

     set +x
     echo 'script completed without encountering a kernel bug'

  EOF

To manage notifications about this bug go to:
https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1617388/+subscriptions

-- 
Mailing list: https://launchpad.net/~kernel-packages
Post to : kernel-packages@lists.launchpad.net
Unsubscribe : https://launchpad.net/~kernel-packages
More help   : https://help.launchpad.net/ListHelp

Re: [Kernel-packages] [Bug 1617388] Re: When using overlayfs with kernel 4.4, some files cannot be deleted.

[Seth Forshee]

On Mon, Aug 29, 2016 at 06:26:54PM -, oleg wrote:
> Naive question regarding copy-up:  Do the files in the upperdir and
> lowerdir need to have the same owner?  If not, could copy-up be amended
> so that it behaves similarly to the copying of a root-owned file by a
> non-root user?  Ie the ownership of the file in the upperdir could
> change to that of the user doing the copying/deleting.

Typically yes, because copy up is what happens when you modify a file
from the lowerdir. The file shouldn't change ownership as a result of
copy-up. In the case of unlink it shouldn't matter though.

-- 
You received this bug notification because you are a member of Kernel
Packages, which is subscribed to linux in Ubuntu.
https://bugs.launchpad.net/bugs/1617388

Title:
  When using overlayfs with kernel 4.4, some files cannot be deleted.

Status in linux package in Ubuntu:
  Triaged

Bug description:
  #!/bin/bash
  # -
  # This script exhibits a bug in overlayfs in kernel 4.4.
  # The bug is not present in kernel 4.2.
  # The bug can be reproduced in an x86_64 virtual-machine;
  # 32-bit has not been tested.
  #
  # With kernel 4.2, the script output ends with:
  #   "script completed without encountering a kernel bug"
  #
  # With kernel 4.4, the script output ends with:
  #   "rm: cannot remove ‘mnt_ovl/sub/sub.txt’:
  #Value too large for defined data type"
  #
  # The script depends upon lxc-usernsexec (part of the lxc1 package) to
  # create a user-namespace.
  #
  # The script should be run as a normal user (not root), in a directory where
  # the user has write-permission:
  # ./script
  # 

  cleanup()
  {
     [[ -d "$storedir" ]] || exit 1
     cd "$storedir"   || exit 1
     [[ -d "$tmpdir"   ]] || exit 1
     lxc-usernsexec -m b:0:1000:1 -m b:10:10:1 -- rm -rf "$tmpdir"
  }

  trap cleanup EXIT

  set -e
  storedir="$(pwd)"

  # create tmpdir
  tmpdir="$(mktemp -d --tmpdir=.)"

  cd "$tmpdir"

  # create lowerdir for overlay
  mkdir -p lower/sub
  touch lower/lower.txt lower/sub/sub.txt

  cd ..

  chmod -R a+rwX "$tmpdir"

  # run a script in a user namepace
  lxc-usernsexec -m b:0:10:65534 --  bash << EOF
     set -e
     cd "$tmpdir"

     # create tmpfs
     mkdir mnt_tmpfs
     mount -t tmpfs tmpfs mnt_tmpfs

     # create upperdir and workdir for overlay
     mkdir mnt_tmpfs/{upper,work}

     # mount overlay
     mkdir mnt_ovl
     mount -t overlay \
     -o lowerdir=lower,upperdir=mnt_tmpfs/upper,workdir=mnt_tmpfs/work \
     overlay mnt_ovl

     echo 'overlay directory listing'
     ls -RF mnt_ovl
     echo ''

     set -x
     rm mnt_ovl/lower.txt# always succeeds
     rm mnt_ovl/sub/sub.txt  # fails with kernel 4.4+

     set +x
     echo 'script completed without encountering a kernel bug'

  EOF

To manage notifications about this bug go to:
https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1617388/+subscriptions

-- 
Mailing list: https://launchpad.net/~kernel-packages
Post to : kernel-packages@lists.launchpad.net
Unsubscribe : https://launchpad.net/~kernel-packages
More help   : https://help.launchpad.net/ListHelp

[Kernel-packages] [Bug 1617388] Re: When using overlayfs with kernel 4.4, some files cannot be deleted.

[oleg]

Thank you for the detailed explanation.

Since the underlying problem is that overlayfs is not (yet very)
namespace aware, this issue may be intractable.

Naive question regarding copy-up:  Do the files in the upperdir and
lowerdir need to have the same owner?  If not, could copy-up be amended
so that it behaves similarly to the copying of a root-owned file by a
non-root user?  Ie the ownership of the file in the upperdir could
change to that of the user doing the copying/deleting.

-- 
You received this bug notification because you are a member of Kernel
Packages, which is subscribed to linux in Ubuntu.
https://bugs.launchpad.net/bugs/1617388

Title:
  When using overlayfs with kernel 4.4, some files cannot be deleted.

Status in linux package in Ubuntu:
  Triaged

Bug description:
  #!/bin/bash
  # -
  # This script exhibits a bug in overlayfs in kernel 4.4.
  # The bug is not present in kernel 4.2.
  # The bug can be reproduced in an x86_64 virtual-machine;
  # 32-bit has not been tested.
  #
  # With kernel 4.2, the script output ends with:
  #   "script completed without encountering a kernel bug"
  #
  # With kernel 4.4, the script output ends with:
  #   "rm: cannot remove ‘mnt_ovl/sub/sub.txt’:
  #Value too large for defined data type"
  #
  # The script depends upon lxc-usernsexec (part of the lxc1 package) to
  # create a user-namespace.
  #
  # The script should be run as a normal user (not root), in a directory where
  # the user has write-permission:
  # ./script
  # 

  cleanup()
  {
     [[ -d "$storedir" ]] || exit 1
     cd "$storedir"   || exit 1
     [[ -d "$tmpdir"   ]] || exit 1
     lxc-usernsexec -m b:0:1000:1 -m b:10:10:1 -- rm -rf "$tmpdir"
  }

  trap cleanup EXIT

  set -e
  storedir="$(pwd)"

  # create tmpdir
  tmpdir="$(mktemp -d --tmpdir=.)"

  cd "$tmpdir"

  # create lowerdir for overlay
  mkdir -p lower/sub
  touch lower/lower.txt lower/sub/sub.txt

  cd ..

  chmod -R a+rwX "$tmpdir"

  # run a script in a user namepace
  lxc-usernsexec -m b:0:10:65534 --  bash << EOF
     set -e
     cd "$tmpdir"

     # create tmpfs
     mkdir mnt_tmpfs
     mount -t tmpfs tmpfs mnt_tmpfs

     # create upperdir and workdir for overlay
     mkdir mnt_tmpfs/{upper,work}

     # mount overlay
     mkdir mnt_ovl
     mount -t overlay \
     -o lowerdir=lower,upperdir=mnt_tmpfs/upper,workdir=mnt_tmpfs/work \
     overlay mnt_ovl

     echo 'overlay directory listing'
     ls -RF mnt_ovl
     echo ''

     set -x
     rm mnt_ovl/lower.txt# always succeeds
     rm mnt_ovl/sub/sub.txt  # fails with kernel 4.4+

     set +x
     echo 'script completed without encountering a kernel bug'

  EOF

To manage notifications about this bug go to:
https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1617388/+subscriptions

-- 
Mailing list: https://launchpad.net/~kernel-packages
Post to : kernel-packages@lists.launchpad.net
Unsubscribe : https://launchpad.net/~kernel-packages
More help   : https://help.launchpad.net/ListHelp

[Kernel-packages] [Bug 1617388] Re: When using overlayfs with kernel 4.4, some files cannot be deleted.

[Seth Forshee]

I'm pretty sure this is related to copy up. For some reason, when
unlinking a file in the lowerdir that isn't in the upperdir overlayfs
first copies up the file from the lowerdir, then replaces it with a
whiteout. There are a couple of problems with letting the user create
the inode in the upperdir though.

First, starting in xenial the tmpfs mount isn't going to allow any
inodes to be created in it that are not mapped into the user namespace.
This is generally a sensible policy as it presents users from inserting
inodes into the system owned by users over which they have no
privileges.

Second, even if the upperdir wasn't limited in this matter, it's not
really a good idea to let a user create inodes owned by another user
without having privileges towards that user. In this case it's under
kernel control and immediately replaced by a whiteout, so it probably
doesn't pose a problem in reality. But generally allowing copy up of
such an inode to a mount over which the user is privileged could be
problematic.

So I'm going to have to think about this as we need to proceed very
carefully. Ideally we can just avoid the copy up and write the whiteout
directly, but I assume there must be some reason the code doesn't
already work that way.

-- 
You received this bug notification because you are a member of Kernel
Packages, which is subscribed to linux in Ubuntu.
https://bugs.launchpad.net/bugs/1617388

Title:
  When using overlayfs with kernel 4.4, some files cannot be deleted.

Status in linux package in Ubuntu:
  Triaged

Bug description:
  #!/bin/bash
  # -
  # This script exhibits a bug in overlayfs in kernel 4.4.
  # The bug is not present in kernel 4.2.
  # The bug can be reproduced in an x86_64 virtual-machine;
  # 32-bit has not been tested.
  #
  # With kernel 4.2, the script output ends with:
  #   "script completed without encountering a kernel bug"
  #
  # With kernel 4.4, the script output ends with:
  #   "rm: cannot remove ‘mnt_ovl/sub/sub.txt’:
  #Value too large for defined data type"
  #
  # The script depends upon lxc-usernsexec (part of the lxc1 package) to
  # create a user-namespace.
  #
  # The script should be run as a normal user (not root), in a directory where
  # the user has write-permission:
  # ./script
  # 

  cleanup()
  {
     [[ -d "$storedir" ]] || exit 1
     cd "$storedir"   || exit 1
     [[ -d "$tmpdir"   ]] || exit 1
     lxc-usernsexec -m b:0:1000:1 -m b:10:10:1 -- rm -rf "$tmpdir"
  }

  trap cleanup EXIT

  set -e
  storedir="$(pwd)"

  # create tmpdir
  tmpdir="$(mktemp -d --tmpdir=.)"

  cd "$tmpdir"

  # create lowerdir for overlay
  mkdir -p lower/sub
  touch lower/lower.txt lower/sub/sub.txt

  cd ..

  chmod -R a+rwX "$tmpdir"

  # run a script in a user namepace
  lxc-usernsexec -m b:0:10:65534 --  bash << EOF
     set -e
     cd "$tmpdir"

     # create tmpfs
     mkdir mnt_tmpfs
     mount -t tmpfs tmpfs mnt_tmpfs

     # create upperdir and workdir for overlay
     mkdir mnt_tmpfs/{upper,work}

     # mount overlay
     mkdir mnt_ovl
     mount -t overlay \
     -o lowerdir=lower,upperdir=mnt_tmpfs/upper,workdir=mnt_tmpfs/work \
     overlay mnt_ovl

     echo 'overlay directory listing'
     ls -RF mnt_ovl
     echo ''

     set -x
     rm mnt_ovl/lower.txt# always succeeds
     rm mnt_ovl/sub/sub.txt  # fails with kernel 4.4+

     set +x
     echo 'script completed without encountering a kernel bug'

  EOF

To manage notifications about this bug go to:
https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1617388/+subscriptions

-- 
Mailing list: https://launchpad.net/~kernel-packages
Post to : kernel-packages@lists.launchpad.net
Unsubscribe : https://launchpad.net/~kernel-packages
More help   : https://help.launchpad.net/ListHelp

[Kernel-packages] [Bug 1617388] Re: When using overlayfs with kernel 4.4, some files cannot be deleted.

[Joseph Salisbury]

** Changed in: linux (Ubuntu)
   Importance: Undecided => Medium

** Tags added: kernel-dakye

** Tags removed: kernel-dakye
** Tags added: kernel-da-key

** Changed in: linux (Ubuntu)
   Status: Confirmed => Triaged

-- 
You received this bug notification because you are a member of Kernel
Packages, which is subscribed to linux in Ubuntu.
https://bugs.launchpad.net/bugs/1617388

Title:
  When using overlayfs with kernel 4.4, some files cannot be deleted.

Status in linux package in Ubuntu:
  Triaged

Bug description:
  #!/bin/bash
  # -
  # This script exhibits a bug in overlayfs in kernel 4.4.
  # The bug is not present in kernel 4.2.
  # The bug can be reproduced in an x86_64 virtual-machine;
  # 32-bit has not been tested.
  #
  # With kernel 4.2, the script output ends with:
  #   "script completed without encountering a kernel bug"
  #
  # With kernel 4.4, the script output ends with:
  #   "rm: cannot remove ‘mnt_ovl/sub/sub.txt’:
  #Value too large for defined data type"
  #
  # The script depends upon lxc-usernsexec (part of the lxc1 package) to
  # create a user-namespace.
  #
  # The script should be run as a normal user (not root), in a directory where
  # the user has write-permission:
  # ./script
  # 

  cleanup()
  {
     [[ -d "$storedir" ]] || exit 1
     cd "$storedir"   || exit 1
     [[ -d "$tmpdir"   ]] || exit 1
     lxc-usernsexec -m b:0:1000:1 -m b:10:10:1 -- rm -rf "$tmpdir"
  }

  trap cleanup EXIT

  set -e
  storedir="$(pwd)"

  # create tmpdir
  tmpdir="$(mktemp -d --tmpdir=.)"

  cd "$tmpdir"

  # create lowerdir for overlay
  mkdir -p lower/sub
  touch lower/lower.txt lower/sub/sub.txt

  cd ..

  chmod -R a+rwX "$tmpdir"

  # run a script in a user namepace
  lxc-usernsexec -m b:0:10:65534 --  bash << EOF
     set -e
     cd "$tmpdir"

     # create tmpfs
     mkdir mnt_tmpfs
     mount -t tmpfs tmpfs mnt_tmpfs

     # create upperdir and workdir for overlay
     mkdir mnt_tmpfs/{upper,work}

     # mount overlay
     mkdir mnt_ovl
     mount -t overlay \
     -o lowerdir=lower,upperdir=mnt_tmpfs/upper,workdir=mnt_tmpfs/work \
     overlay mnt_ovl

     echo 'overlay directory listing'
     ls -RF mnt_ovl
     echo ''

     set -x
     rm mnt_ovl/lower.txt# always succeeds
     rm mnt_ovl/sub/sub.txt  # fails with kernel 4.4+

     set +x
     echo 'script completed without encountering a kernel bug'

  EOF

To manage notifications about this bug go to:
https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1617388/+subscriptions

-- 
Mailing list: https://launchpad.net/~kernel-packages
Post to : kernel-packages@lists.launchpad.net
Unsubscribe : https://launchpad.net/~kernel-packages
More help   : https://help.launchpad.net/ListHelp

[Kernel-packages] [Bug 1617388] Re: When using overlayfs with kernel 4.4, some files cannot be deleted.

[oleg]

** Tags added: bisect-done kernel-bug

-- 
You received this bug notification because you are a member of Kernel
Packages, which is subscribed to linux in Ubuntu.
https://bugs.launchpad.net/bugs/1617388

Title:
  When using overlayfs with kernel 4.4, some files cannot be deleted.

Status in linux package in Ubuntu:
  Confirmed

Bug description:
  #!/bin/bash
  # -
  # This script exhibits a bug in overlayfs in kernel 4.4.
  # The bug is not present in kernel 4.2.
  # The bug can be reproduced in an x86_64 virtual-machine;
  # 32-bit has not been tested.
  #
  # With kernel 4.2, the script output ends with:
  #   "script completed without encountering a kernel bug"
  #
  # With kernel 4.4, the script output ends with:
  #   "rm: cannot remove ‘mnt_ovl/sub/sub.txt’:
  #Value too large for defined data type"
  #
  # The script depends upon lxc-usernsexec (part of the lxc1 package) to
  # create a user-namespace.
  #
  # The script should be run as a normal user (not root), in a directory where
  # the user has write-permission:
  # ./script
  # 

  cleanup()
  {
     [[ -d "$storedir" ]] || exit 1
     cd "$storedir"   || exit 1
     [[ -d "$tmpdir"   ]] || exit 1
     lxc-usernsexec -m b:0:1000:1 -m b:10:10:1 -- rm -rf "$tmpdir"
  }

  trap cleanup EXIT

  set -e
  storedir="$(pwd)"

  # create tmpdir
  tmpdir="$(mktemp -d --tmpdir=.)"

  cd "$tmpdir"

  # create lowerdir for overlay
  mkdir -p lower/sub
  touch lower/lower.txt lower/sub/sub.txt

  cd ..

  chmod -R a+rwX "$tmpdir"

  # run a script in a user namepace
  lxc-usernsexec -m b:0:10:65534 --  bash << EOF
     set -e
     cd "$tmpdir"

     # create tmpfs
     mkdir mnt_tmpfs
     mount -t tmpfs tmpfs mnt_tmpfs

     # create upperdir and workdir for overlay
     mkdir mnt_tmpfs/{upper,work}

     # mount overlay
     mkdir mnt_ovl
     mount -t overlay \
     -o lowerdir=lower,upperdir=mnt_tmpfs/upper,workdir=mnt_tmpfs/work \
     overlay mnt_ovl

     echo 'overlay directory listing'
     ls -RF mnt_ovl
     echo ''

     set -x
     rm mnt_ovl/lower.txt# always succeeds
     rm mnt_ovl/sub/sub.txt  # fails with kernel 4.4+

     set +x
     echo 'script completed without encountering a kernel bug'

  EOF

To manage notifications about this bug go to:
https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1617388/+subscriptions

-- 
Mailing list: https://launchpad.net/~kernel-packages
Post to : kernel-packages@lists.launchpad.net
Unsubscribe : https://launchpad.net/~kernel-packages
More help   : https://help.launchpad.net/ListHelp

[Kernel-packages] [Bug 1617388] Re: When using overlayfs with kernel 4.4, some files cannot be deleted.

[oleg]

For reference, here is the bad commit:

commit eac15dc
Author: Seth Forshee 
Date:   Wed Nov 19 11:00:56 2014 -0600

UBUNTU: SAUCE: fs: Refuse uid/gid changes which don't map into s_user_ns

Add checks to inode_change_ok to verify that uid and gid changes
will map into the superblock's user namespace. If they do not
fail with -EOVERFLOW. This cannot be overriden with ATTR_FORCE.

Signed-off-by: Seth Forshee 
Acked-by: Serge Hallyn 
Signed-off-by: Tim Gardner 

diff --git a/fs/attr.c b/fs/attr.c
index 6530ced..55b46e3 100644
--- a/fs/attr.c
+++ b/fs/attr.c
@@ -42,6 +42,17 @@ int inode_change_ok(const struct inode *inode, struct iattr 
*attr)
return error;
}
 
+   /*
+* Verify that uid/gid changes are valid in the target namespace
+* of the superblock. This cannot be overriden using ATTR_FORCE.
+*/
+   if (ia_valid & ATTR_UID &&
+   from_kuid(inode->i_sb->s_user_ns, attr->ia_uid) == (uid_t)-1)
+   return -EOVERFLOW;
+   if (ia_valid & ATTR_GID &&
+   from_kgid(inode->i_sb->s_user_ns, attr->ia_gid) == (gid_t)-1)
+   return -EOVERFLOW;
+
/* If force is set do it anyway. */
if (ia_valid & ATTR_FORCE)
return 0;

-- 
You received this bug notification because you are a member of Kernel
Packages, which is subscribed to linux in Ubuntu.
https://bugs.launchpad.net/bugs/1617388

Title:
  When using overlayfs with kernel 4.4, some files cannot be deleted.

Status in linux package in Ubuntu:
  Confirmed

Bug description:
  #!/bin/bash
  # -
  # This script exhibits a bug in overlayfs in kernel 4.4.
  # The bug is not present in kernel 4.2.
  # The bug can be reproduced in an x86_64 virtual-machine;
  # 32-bit has not been tested.
  #
  # With kernel 4.2, the script output ends with:
  #   "script completed without encountering a kernel bug"
  #
  # With kernel 4.4, the script output ends with:
  #   "rm: cannot remove ‘mnt_ovl/sub/sub.txt’:
  #Value too large for defined data type"
  #
  # The script depends upon lxc-usernsexec (part of the lxc1 package) to
  # create a user-namespace.
  #
  # The script should be run as a normal user (not root), in a directory where
  # the user has write-permission:
  # ./script
  # 

  cleanup()
  {
     [[ -d "$storedir" ]] || exit 1
     cd "$storedir"   || exit 1
     [[ -d "$tmpdir"   ]] || exit 1
     lxc-usernsexec -m b:0:1000:1 -m b:10:10:1 -- rm -rf "$tmpdir"
  }

  trap cleanup EXIT

  set -e
  storedir="$(pwd)"

  # create tmpdir
  tmpdir="$(mktemp -d --tmpdir=.)"

  cd "$tmpdir"

  # create lowerdir for overlay
  mkdir -p lower/sub
  touch lower/lower.txt lower/sub/sub.txt

  cd ..

  chmod -R a+rwX "$tmpdir"

  # run a script in a user namepace
  lxc-usernsexec -m b:0:10:65534 --  bash << EOF
     set -e
     cd "$tmpdir"

     # create tmpfs
     mkdir mnt_tmpfs
     mount -t tmpfs tmpfs mnt_tmpfs

     # create upperdir and workdir for overlay
     mkdir mnt_tmpfs/{upper,work}

     # mount overlay
     mkdir mnt_ovl
     mount -t overlay \
     -o lowerdir=lower,upperdir=mnt_tmpfs/upper,workdir=mnt_tmpfs/work \
     overlay mnt_ovl

     echo 'overlay directory listing'
     ls -RF mnt_ovl
     echo ''

     set -x
     rm mnt_ovl/lower.txt# always succeeds
     rm mnt_ovl/sub/sub.txt  # fails with kernel 4.4+

     set +x
     echo 'script completed without encountering a kernel bug'

  EOF

To manage notifications about this bug go to:
https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1617388/+subscriptions

-- 
Mailing list: https://launchpad.net/~kernel-packages
Post to : kernel-packages@lists.launchpad.net
Unsubscribe : https://launchpad.net/~kernel-packages
More help   : https://help.launchpad.net/ListHelp

[Kernel-packages] [Bug 1617388] Re: When using overlayfs with kernel 4.4, some files cannot be deleted.

[oleg]

git bisect of the ubuntu-xenial tree:

last good commit: 0e9c9b5 UBUNTU: SAUCE: cred: Reject inodes with invalid ids 
in set_create_file_as()
first bad commit: eac15dc UBUNTU: SAUCE: fs: Refuse uid/gid changes which don't 
map into s_user_ns

-- 
You received this bug notification because you are a member of Kernel
Packages, which is subscribed to linux in Ubuntu.
https://bugs.launchpad.net/bugs/1617388

Title:
  When using overlayfs with kernel 4.4, some files cannot be deleted.

Status in linux package in Ubuntu:
  Confirmed

Bug description:
  #!/bin/bash
  # -
  # This script exhibits a bug in overlayfs in kernel 4.4.
  # The bug is not present in kernel 4.2.
  # The bug can be reproduced in an x86_64 virtual-machine;
  # 32-bit has not been tested.
  #
  # With kernel 4.2, the script output ends with:
  #   "script completed without encountering a kernel bug"
  #
  # With kernel 4.4, the script output ends with:
  #   "rm: cannot remove ‘mnt_ovl/sub/sub.txt’:
  #Value too large for defined data type"
  #
  # The script depends upon lxc-usernsexec (part of the lxc1 package) to
  # create a user-namespace.
  #
  # The script should be run as a normal user (not root), in a directory where
  # the user has write-permission:
  # ./script
  # 

  cleanup()
  {
     [[ -d "$storedir" ]] || exit 1
     cd "$storedir"   || exit 1
     [[ -d "$tmpdir"   ]] || exit 1
     lxc-usernsexec -m b:0:1000:1 -m b:10:10:1 -- rm -rf "$tmpdir"
  }

  trap cleanup EXIT

  set -e
  storedir="$(pwd)"

  # create tmpdir
  tmpdir="$(mktemp -d --tmpdir=.)"

  cd "$tmpdir"

  # create lowerdir for overlay
  mkdir -p lower/sub
  touch lower/lower.txt lower/sub/sub.txt

  cd ..

  chmod -R a+rwX "$tmpdir"

  # run a script in a user namepace
  lxc-usernsexec -m b:0:10:65534 --  bash << EOF
     set -e
     cd "$tmpdir"

     # create tmpfs
     mkdir mnt_tmpfs
     mount -t tmpfs tmpfs mnt_tmpfs

     # create upperdir and workdir for overlay
     mkdir mnt_tmpfs/{upper,work}

     # mount overlay
     mkdir mnt_ovl
     mount -t overlay \
     -o lowerdir=lower,upperdir=mnt_tmpfs/upper,workdir=mnt_tmpfs/work \
     overlay mnt_ovl

     echo 'overlay directory listing'
     ls -RF mnt_ovl
     echo ''

     set -x
     rm mnt_ovl/lower.txt# always succeeds
     rm mnt_ovl/sub/sub.txt  # fails with kernel 4.4+

     set +x
     echo 'script completed without encountering a kernel bug'

  EOF

To manage notifications about this bug go to:
https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1617388/+subscriptions

-- 
Mailing list: https://launchpad.net/~kernel-packages
Post to : kernel-packages@lists.launchpad.net
Unsubscribe : https://launchpad.net/~kernel-packages
More help   : https://help.launchpad.net/ListHelp

[Kernel-packages] [Bug 1617388] Re: When using overlayfs with kernel 4.4, some files cannot be deleted.

[oleg]

** Description changed:

  #!/bin/bash
  # -
  # This script exhibits a bug in overlayfs in kernel 4.4.
  # The bug is not present in kernel 4.2.
  # The bug can be reproduced in an x86_64 virtual-machine;
  # 32-bit has not been tested.
  #
  # With kernel 4.2, the script output ends with:
  #   "script completed without encountering a kernel bug"
  #
  # With kernel 4.4, the script output ends with:
  #   "rm: cannot remove ‘mnt_ovl/sub/sub.txt’:
  #Value too large for defined data type"
  #
  # The script depends upon lxc-usernsexec (part of the lxc1 package) to
  # create a user-namespace.
  #
  # The script should be run as a normal user (not root), in a directory where
  # the user has write-permission:
  # ./script
  # 
  
  cleanup()
  {
     [[ -d "$storedir" ]] || exit 1
     cd "$storedir"   || exit 1
     [[ -d "$tmpdir"   ]] || exit 1
     lxc-usernsexec -m b:0:1000:1 -m b:10:10:1 -- rm -rf "$tmpdir"
  }
  
  trap cleanup EXIT
  
  set -e
  storedir="$(pwd)"
  
  # create tmpdir
  tmpdir="$(mktemp -d --tmpdir=.)"
  
  cd "$tmpdir"
  
  # create lowerdir for overlay
  mkdir -p lower/sub
  touch lower/lower.txt lower/sub/sub.txt
+ 
  cd ..
  
  chmod -R a+rwX "$tmpdir"
  
  # run a script in a user namepace
  lxc-usernsexec -m b:0:10:65534 --  bash << EOF
     set -e
     cd "$tmpdir"
  
     # create tmpfs
     mkdir mnt_tmpfs
     mount -t tmpfs tmpfs mnt_tmpfs
  
     # create upperdir and workdir for overlay
     mkdir mnt_tmpfs/{upper,work}
  
     # mount overlay
     mkdir mnt_ovl
     mount -t overlay \
     -o lowerdir=lower,upperdir=mnt_tmpfs/upper,workdir=mnt_tmpfs/work \
     overlay mnt_ovl
  
     echo 'overlay directory listing'
     ls -RF mnt_ovl
     echo ''
  
     set -x
     rm mnt_ovl/lower.txt# always succeeds
     rm mnt_ovl/sub/sub.txt  # fails with kernel 4.4+
  
     set +x
     echo 'script completed without encountering a kernel bug'
  
  EOF

-- 
You received this bug notification because you are a member of Kernel
Packages, which is subscribed to linux in Ubuntu.
https://bugs.launchpad.net/bugs/1617388

Title:
  When using overlayfs with kernel 4.4, some files cannot be deleted.

Status in linux package in Ubuntu:
  Confirmed

Bug description:
  #!/bin/bash
  # -
  # This script exhibits a bug in overlayfs in kernel 4.4.
  # The bug is not present in kernel 4.2.
  # The bug can be reproduced in an x86_64 virtual-machine;
  # 32-bit has not been tested.
  #
  # With kernel 4.2, the script output ends with:
  #   "script completed without encountering a kernel bug"
  #
  # With kernel 4.4, the script output ends with:
  #   "rm: cannot remove ‘mnt_ovl/sub/sub.txt’:
  #Value too large for defined data type"
  #
  # The script depends upon lxc-usernsexec (part of the lxc1 package) to
  # create a user-namespace.
  #
  # The script should be run as a normal user (not root), in a directory where
  # the user has write-permission:
  # ./script
  # 

  cleanup()
  {
     [[ -d "$storedir" ]] || exit 1
     cd "$storedir"   || exit 1
     [[ -d "$tmpdir"   ]] || exit 1
     lxc-usernsexec -m b:0:1000:1 -m b:10:10:1 -- rm -rf "$tmpdir"
  }

  trap cleanup EXIT

  set -e
  storedir="$(pwd)"

  # create tmpdir
  tmpdir="$(mktemp -d --tmpdir=.)"

  cd "$tmpdir"

  # create lowerdir for overlay
  mkdir -p lower/sub
  touch lower/lower.txt lower/sub/sub.txt

  cd ..

  chmod -R a+rwX "$tmpdir"

  # run a script in a user namepace
  lxc-usernsexec -m b:0:10:65534 --  bash << EOF
     set -e
     cd "$tmpdir"

     # create tmpfs
     mkdir mnt_tmpfs
     mount -t tmpfs tmpfs mnt_tmpfs

     # create upperdir and workdir for overlay
     mkdir mnt_tmpfs/{upper,work}

     # mount overlay
     mkdir mnt_ovl
     mount -t overlay \
     -o lowerdir=lower,upperdir=mnt_tmpfs/upper,workdir=mnt_tmpfs/work \
     overlay mnt_ovl

     echo 'overlay directory listing'
     ls -RF mnt_ovl
     echo ''

     set -x
     rm mnt_ovl/lower.txt# always succeeds
     rm mnt_ovl/sub/sub.txt  # fails with kernel 4.4+

     set +x
     echo 'script completed without encountering a kernel bug'

  EOF

To manage notifications about this bug go to:
https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1617388/+subscriptions

-- 
Mailing list: https://launchpad.net/~kernel-packages
Post to : kernel-packages@lists.launchpad.net
Unsubscribe : https://launchpad.net/~kernel-packages
More help   : https://help.launchpad.net/ListHelp

[Kernel-packages] [Bug 1617388] Re: When using overlayfs with kernel 4.4, some files cannot be deleted.

[oleg]

More Apport-Collect data:

AlsaDevices:
 total 0
 crw-rw 1 root audio 116, 1 Aug 26 15:09 seq
 crw-rw 1 root audio 116, 33 Aug 26 15:09 timer
AplayDevices: Error: [Errno 2] No such file or directory
ApportVersion: 2.14.1-0ubuntu3.21
Architecture: amd64
ArecordDevices: Error: [Errno 2] No such file or directory
AudioDevicesInUse: Error: command ['fuser', '-v', '/dev/snd/seq', 
'/dev/snd/timer'] failed with exit code 1:
CRDA: Error: [Errno 2] No such file or directory
DistroRelease: Ubuntu 14.04
IwConfig: Error: [Errno 2] No such file or directory
Lsusb: Error: command ['lsusb'] failed with exit code 1: unable to initialize 
libusb: -99
MachineType: QEMU Standard PC (i440FX + PIIX, 1996)
Package: linux (not installed)
PciMultimedia:

ProcEnviron:
 TERM=vt102
 PATH=(custom, no user)
 XDG_RUNTIME_DIR=
 LANG=en_US.UTF-8
 SHELL=/bin/bash
ProcFB:

ProcKernelCmdLine: BOOT_IMAGE=/boot/vmlinuz-4.4.0-34-generic 
root=UUID=78cee363-0cf0-46ce-bd06-d5601344c627 ro console=tty1 console=ttyS0
ProcVersionSignature: User Name 4.4.0-34.53~14.04.1-generic 4.4.15
RelatedPackageVersions:
 linux-restricted-modules-4.4.0-34-generic N/A
 linux-backports-modules-4.4.0-34-generic N/A
 linux-firmware N/A
RfKill: Error: [Errno 2] No such file or directory
Tags: trusty uec-images
Uname: Linux 4.4.0-34-generic x86_64
UpgradeStatus: No upgrade log present (probably fresh install)
UserGroups: adm audio cdrom dialout dip floppy netdev plugdev sudo video
_MarkForUpload: True
dmi.bios.date: 01/01/2011
dmi.bios.vendor: Bochs
dmi.bios.version: Bochs
dmi.chassis.type: 1
dmi.chassis.vendor: Bochs
dmi.modalias: 
dmi:bvnBochs:bvrBochs:bd01/01/2011:svnQEMU:pnStandardPC(i440FX+PIIX,1996):pvrpc-i440fx-trusty:cvnBochs:ct1:cvr:
dmi.product.name: Standard PC (i440FX + PIIX, 1996)
dmi.product.version: pc-i440fx-trusty
dmi.sys.vendor: QEMU

** Description changed:

  #!/bin/bash
  # -
  # This script exhibits a bug in overlayfs in kernel 4.4.
  # The bug is not present in kernel 4.2.
  # The bug can be reproduced in an x86_64 virtual-machine;
  # 32-bit has not been tested.
  #
  # With kernel 4.2, the script output ends with:
  #   "script completed without encountering a kernel bug"
  #
  # With kernel 4.4, the script output ends with:
  #   "rm: cannot remove ‘mnt_ovl/sub/sub.txt’:
  #Value too large for defined data type"
  #
  # The script depends upon lxc-usernsexec (part of the lxc1 package) to
  # create a user-namespace.
  #
  # The script should be run as a normal user (not root), in a directory where
  # the user has write-permission:
  # ./script
  # 
  
  cleanup()
  {
     [[ -d "$storedir" ]] || exit 1
     cd "$storedir"   || exit 1
     [[ -d "$tmpdir"   ]] || exit 1
     lxc-usernsexec -m b:0:1000:1 -m b:10:10:1 -- rm -rf "$tmpdir"
  }
  
  trap cleanup EXIT
  
  set -e
  storedir="$(pwd)"
  
  # create tmpdir
  tmpdir="$(mktemp -d --tmpdir=.)"
  
  cd "$tmpdir"
  
  # create lowerdir for overlay
  mkdir -p lower/sub
  touch lower/lower.txt lower/sub/sub.txt
  cd ..
  
  chmod -R a+rwX "$tmpdir"
  
  # run a script in a user namepace
  lxc-usernsexec -m b:0:10:65534 --  bash << EOF
     set -e
     cd "$tmpdir"
  
     # create tmpfs
     mkdir mnt_tmpfs
     mount -t tmpfs tmpfs mnt_tmpfs
  
     # create upperdir and workdir for overlay
     mkdir mnt_tmpfs/{upper,work}
  
     # mount overlay
     mkdir mnt_ovl
     mount -t overlay \
     -o lowerdir=lower,upperdir=mnt_tmpfs/upper,workdir=mnt_tmpfs/work \
-overlay mnt_ovl
+    overlay mnt_ovl
  
     echo 'overlay directory listing'
     ls -RF mnt_ovl
     echo ''
  
     set -x
     rm mnt_ovl/lower.txt# always succeeds
     rm mnt_ovl/sub/sub.txt  # fails with kernel 4.4+
  
     set +x
     echo 'script completed without encountering a kernel bug'
  
  EOF
- --- 
- AlsaDevices:
-  total 0
-  crw-rw 1 root audio 116,  1 Aug 26 15:09 seq
-  crw-rw 1 root audio 116, 33 Aug 26 15:09 timer
- AplayDevices: Error: [Errno 2] No such file or directory
- ApportVersion: 2.14.1-0ubuntu3.21
- Architecture: amd64
- ArecordDevices: Error: [Errno 2] No such file or directory
- AudioDevicesInUse: Error: command ['fuser', '-v', '/dev/snd/seq', 
'/dev/snd/timer'] failed with exit code 1:
- CRDA: Error: [Errno 2] No such file or directory
- DistroRelease: Ubuntu 14.04
- IwConfig: Error: [Errno 2] No such file or directory
- Lsusb: Error: command ['lsusb'] failed with exit code 1: unable to initialize 
libusb: -99
- MachineType: QEMU Standard PC (i440FX + PIIX, 1996)
- Package: linux (not installed)
- PciMultimedia:
-  
- ProcEnviron:
-  TERM=vt102
-  PATH=(custom, no user)
-  XDG_RUNTIME_DIR=
-  LANG=en_US.UTF-8
-  SHELL=/bin/bash
- ProcFB:
-  
- ProcKernelCmdLine: BOOT_IMAGE=/boot/vmlinuz-4.4.0-34-generic 
root=UUID=78cee363-0cf0-46ce-bd06-d5601344c627 ro console=tty1 console=ttyS0
- 

[Kernel-packages] [Bug 1617388] Re: When using overlayfs with kernel 4.4, some files cannot be deleted.

[oleg]

apport information

** Tags added: apport-collected trusty uec-images

** Description changed:

  #!/bin/bash
  # -
  # This script exhibits a bug in overlayfs in kernel 4.4.
  # The bug is not present in kernel 4.2.
  # The bug can be reproduced in an x86_64 virtual-machine;
  # 32-bit has not been tested.
  #
  # With kernel 4.2, the script output ends with:
  #   "script completed without encountering a kernel bug"
  #
  # With kernel 4.4, the script output ends with:
  #   "rm: cannot remove ‘mnt_ovl/sub/sub.txt’:
  #Value too large for defined data type"
  #
  # The script depends upon lxc-usernsexec (part of the lxc1 package) to
  # create a user-namespace.
  #
  # The script should be run as a normal user (not root), in a directory where
  # the user has write-permission:
  # ./script
  # 
  
  cleanup()
  {
     [[ -d "$storedir" ]] || exit 1
     cd "$storedir"   || exit 1
     [[ -d "$tmpdir"   ]] || exit 1
     lxc-usernsexec -m b:0:1000:1 -m b:10:10:1 -- rm -rf "$tmpdir"
  }
  
  trap cleanup EXIT
  
  set -e
  storedir="$(pwd)"
  
  # create tmpdir
  tmpdir="$(mktemp -d --tmpdir=.)"
  
  cd "$tmpdir"
  
  # create lowerdir for overlay
  mkdir -p lower/sub
  touch lower/lower.txt lower/sub/sub.txt
  cd ..
  
  chmod -R a+rwX "$tmpdir"
  
  # run a script in a user namepace
  lxc-usernsexec -m b:0:10:65534 --  bash << EOF
     set -e
     cd "$tmpdir"
  
     # create tmpfs
     mkdir mnt_tmpfs
     mount -t tmpfs tmpfs mnt_tmpfs
  
     # create upperdir and workdir for overlay
     mkdir mnt_tmpfs/{upper,work}
  
     # mount overlay
     mkdir mnt_ovl
     mount -t overlay \
     -o lowerdir=lower,upperdir=mnt_tmpfs/upper,workdir=mnt_tmpfs/work \
 overlay mnt_ovl
  
     echo 'overlay directory listing'
     ls -RF mnt_ovl
     echo ''
  
     set -x
     rm mnt_ovl/lower.txt# always succeeds
     rm mnt_ovl/sub/sub.txt  # fails with kernel 4.4+
  
     set +x
     echo 'script completed without encountering a kernel bug'
  
  EOF
+ --- 
+ AlsaDevices:
+  total 0
+  crw-rw 1 root audio 116,  1 Aug 26 15:09 seq
+  crw-rw 1 root audio 116, 33 Aug 26 15:09 timer
+ AplayDevices: Error: [Errno 2] No such file or directory
+ ApportVersion: 2.14.1-0ubuntu3.21
+ Architecture: amd64
+ ArecordDevices: Error: [Errno 2] No such file or directory
+ AudioDevicesInUse: Error: command ['fuser', '-v', '/dev/snd/seq', 
'/dev/snd/timer'] failed with exit code 1:
+ CRDA: Error: [Errno 2] No such file or directory
+ DistroRelease: Ubuntu 14.04
+ IwConfig: Error: [Errno 2] No such file or directory
+ Lsusb: Error: command ['lsusb'] failed with exit code 1: unable to initialize 
libusb: -99
+ MachineType: QEMU Standard PC (i440FX + PIIX, 1996)
+ Package: linux (not installed)
+ PciMultimedia:
+  
+ ProcEnviron:
+  TERM=vt102
+  PATH=(custom, no user)
+  XDG_RUNTIME_DIR=
+  LANG=en_US.UTF-8
+  SHELL=/bin/bash
+ ProcFB:
+  
+ ProcKernelCmdLine: BOOT_IMAGE=/boot/vmlinuz-4.4.0-34-generic 
root=UUID=78cee363-0cf0-46ce-bd06-d5601344c627 ro console=tty1 console=ttyS0
+ ProcVersionSignature: User Name 4.4.0-34.53~14.04.1-generic 4.4.15
+ RelatedPackageVersions:
+  linux-restricted-modules-4.4.0-34-generic N/A
+  linux-backports-modules-4.4.0-34-generic  N/A
+  linux-firmwareN/A
+ RfKill: Error: [Errno 2] No such file or directory
+ Tags:  trusty uec-images
+ Uname: Linux 4.4.0-34-generic x86_64
+ UpgradeStatus: No upgrade log present (probably fresh install)
+ UserGroups: adm audio cdrom dialout dip floppy netdev plugdev sudo video
+ _MarkForUpload: True
+ dmi.bios.date: 01/01/2011
+ dmi.bios.vendor: Bochs
+ dmi.bios.version: Bochs
+ dmi.chassis.type: 1
+ dmi.chassis.vendor: Bochs
+ dmi.modalias: 
dmi:bvnBochs:bvrBochs:bd01/01/2011:svnQEMU:pnStandardPC(i440FX+PIIX,1996):pvrpc-i440fx-trusty:cvnBochs:ct1:cvr:
+ dmi.product.name: Standard PC (i440FX + PIIX, 1996)
+ dmi.product.version: pc-i440fx-trusty
+ dmi.sys.vendor: QEMU

** Attachment added: "BootDmesg.txt"
   
https://bugs.launchpad.net/bugs/1617388/+attachment/4728459/+files/BootDmesg.txt

-- 
You received this bug notification because you are a member of Kernel
Packages, which is subscribed to linux in Ubuntu.
https://bugs.launchpad.net/bugs/1617388

Title:
  When using overlayfs with kernel 4.4, some files cannot be deleted.

Status in linux package in Ubuntu:
  Confirmed

Bug description:
  #!/bin/bash
  # -
  # This script exhibits a bug in overlayfs in kernel 4.4.
  # The bug is not present in kernel 4.2.
  # The bug can be reproduced in an x86_64 virtual-machine;
  # 32-bit has not been tested.
  #
  # With kernel 4.2, the script output ends with:
  #   "script completed without encountering a kernel bug"
  #
  # With kernel 4.4, the script output ends with:
  #   "rm: cannot