Hi, First, +1, great idea, I don't know why we never did this before, especially given that the alpine tarball we use for integration tests is also manually doctored for this reason.
On Fri, 2020-07-03 at 19:58 +0200, Sander Striker wrote: > On Fri, Jul 3, 2020 at 7:16 PM Benjamin Schubert < > [email protected]> > wrote: > > > Hey everyone > > > > TLDR: I think we should by default filter `dev` files from `tar` > > archives > > when staging. > > > > I have a problem that I sometimes encounter: I want to use a stage3 > > archive as the basic for a > > BuildStream project. Some of those tar files contain `dev` devices > > (/dev/* > > mainly), and I can > > therefore not open them with BuildStream, since I am not root and > > therefore don't have enough > > permissions for this. > > > > This seems to be a common theme in source plugins. > > This is indeed a common theme for some plugins (only `tar` and `ostree` spring to mind, while other source plugins normally import source code, which the core takes some precautions to ensure that permissions are consistent across environments). For `ostree`, device files are already not permitted, and extended attributes I believe are lost with ostree's "user-mode" checkouts. For `tar` we just fail if the regular user doesn't have permissions (not very ideal). > > Currently the only way of doing that with BuildStream is re-taring > > the > > file by myself. > > > > I don't think that is a good user experience, and I also think that > > this > > is not the default expectation > > when using BuildStream. > > > > I'd therefore suggest two things: > > > > - Add a `exclude-dev-nodes` option to `tar` to allow filtering out > > those > > special files > > - Make this option true by default, since I think that is the > > default > > expectation in BuildStream? > > > > What do you think? > > > > Can we not address this in the core rather than in the plugins? > That would be interesting, but probably require some major changes, like enforcing that sources stage into a virtual directory instead of a directory on disk, might not be an easy task for most normal plugins like git or other VCS plugins. Maybe we could indeed offer an alternative virtual source staging API down the line. It makes sense to keep the regular staging API in place for most simple source plugins which stage source code (stage to a directory with post staging permission doctoring from the core), later, when allowing staging to a virtual directory: we could control what is allowed to be staged more selectively, depending on the project's explicit requirements and local/remote sandbox and storage capabilities (e.g., whether remote artifact servers allow storing dev nodes etc), in that case we could have a core-centric way of defining what the user allows these sources to stage. I would also imagine that not only storage capability and project requirements need to be considered, but also sandbox policies; e.g. we might support static dev nodes (perhaps not "real" ones), but probably disallow staging them into /dev, which the sandbox wants to control. In the short term, I would support Ben's proposal, except I would propose to *not* offer any option, and only offer an option at a later date when device files and extended attributes/setcap attributes etc, could be supported by regular (non-root) users (right now I doubt this is a breaking change, as tarballs which require that you are root, are just not usable anyway). Moving forward without an option at this stage would grant us more freedom in supporting something more elaborate and granular in the future. Cheers, -Tristan
