staehle opened a new issue, #1773: URL: https://github.com/apache/buildstream/issues/1773
Hi, me again. (If there's a better way for me to submit things like this, let me know. There isn't really a clear "how to contact BuildStream Dev team") When doing a "bst artifact checkout", the default argument to "--deps" is "run". Can this be made a user-configurable option so we can set it to "none" by default via https://docs.buildstream.build/master/using_config.html or as a project.conf value? Reasoning: The components we're cross-compiling have these base dependencies as "build and runtime" dependencies: ``` depends: - base/base.bst - toolchain/cross-compiler.bst ``` When running "bst artifact checkout", the artifact list is as I would expect (using cross-compiled zlib as an example): ``` $ bst -o target_arch aarch64 -o platform rpi artifact list-contents components/target/zlib.bst [--:--:--][ ][ main:core activity ] START Loading elements [00:00:00][ ][ main:core activity ] SUCCESS Loading elements [--:--:--][ ][ main:core activity ] START Resolving elements [00:00:00][ ][ main:core activity ] SUCCESS Resolving elements [--:--:--][ ][ main:core activity ] START Initializing remote caches [00:00:00][ ][ main:core activity ] SUCCESS Initializing remote caches [--:--:--][ ][ main:core activity ] START Query cache [00:00:00][ ][ main:core activity ] SUCCESS Query cache components/target/zlib.bst: robs robs/staging robs/staging/image robs/staging/image/usr robs/staging/image/usr/lib robs/staging/image/usr/lib/libz.so robs/staging/image/usr/lib/libz.so.1 robs/staging/image/usr/lib/libz.so.1.2.12 robs/staging/sysroot robs/staging/sysroot/usr robs/staging/sysroot/usr/include robs/staging/sysroot/usr/include/zconf.h robs/staging/sysroot/usr/include/zlib.h robs/staging/sysroot/usr/lib robs/staging/sysroot/usr/lib/libz.a robs/staging/sysroot/usr/lib/libz.so robs/staging/sysroot/usr/lib/libz.so.1 robs/staging/sysroot/usr/lib/libz.so.1.2.12 robs/staging/sysroot/usr/share robs/staging/sysroot/usr/share/man robs/staging/sysroot/usr/share/man/man3 robs/staging/sysroot/usr/share/man/man3/zlib.3 robs/staging/sysroot/usr/share/pkgconfig robs/staging/sysroot/usr/share/pkgconfig/zlib.pc ``` However, this is inconsistent with what "bst artifact checkout" does, which outputs the artifacts as well as the entire sandbox (...why?). I would only be expecting the "robs" directory inside "components/target/zlib/": ``` $ bst -o target_arch aarch64 -o platform rpi artifact checkout components/target/zlib.bst [--:--:--][ ][ main:core activity ] START Loading elements [00:00:00][ ][ main:core activity ] SUCCESS Loading elements [--:--:--][ ][ main:core activity ] START Resolving elements [00:00:00][ ][ main:core activity ] SUCCESS Resolving elements [--:--:--][ ][ main:core activity ] START Initializing remote caches [00:00:00][ ][ main:core activity ] SUCCESS Initializing remote caches [--:--:--][ ][ main:core activity ] START Query cache [00:00:00][ ][ main:core activity ] SUCCESS Query cache [--:--:--][bd873de7][ main:components/target/zlib.bst ] START Staging dependencies [00:00:00][bd873de7][ main:components/target/zlib.bst ] SUCCESS Staging dependencies [--:--:--][bd873de7][ main:components/target/zlib.bst ] START Integrating sandbox [00:00:00][bd873de7][ main:components/target/zlib.bst ] SUCCESS Integrating sandbox [--:--:--][bd873de7][ main:components/target/zlib.bst ] START Checking out files in '<redacted>/components/target/zlib' [00:00:01][bd873de7][ main:components/target/zlib.bst ] SUCCESS Checking out files in '<redacted>/components/target/zlib' $ ls components/target/zlib/ bin dev etc lib lib64 proc robs sbin tmp usr ``` So we initially thought this was an issue with how we were handling dependencies, so had changed the base 'depends' to 'build-depends': ``` build-depends: - base/base.bst - toolchain/cross-compiler.bst ``` And this fixed "artifact checkout" to be what we'd expect. However, it broke "bst shell" since then the base sandbox wasn't there: ``` $ bst -o target_arch aarch64 -o platform rpi shell components/target/zlib.bst [--:--:--][ ][ main:core activity ] START Loading elements [00:00:00][ ][ main:core activity ] SUCCESS Loading elements [--:--:--][ ][ main:core activity ] START Resolving elements [00:00:00][ ][ main:core activity ] SUCCESS Resolving elements [--:--:--][ ][ main:core activity ] START Initializing remote caches [00:00:00][ ][ main:core activity ] SUCCESS Initializing remote caches [--:--:--][ ][ main:core activity ] START Query cache [00:00:00][ ][ main:core activity ] SUCCESS Query cache [--:--:--][bd873de7][ main:components/target/zlib.bst ] START Staging dependencies [00:00:00][bd873de7][ main:components/target/zlib.bst ] SUCCESS Staging dependencies [--:--:--][bd873de7][ main:components/target/zlib.bst ] START Integrating sandbox [00:00:00][bd873de7][ main:components/target/zlib.bst ] SUCCESS Integrating sandbox [--:--:--][bd873de7][ main:components/target/zlib.bst ] STATUS Running command sh -i Error launching shell: Staged artifacts do not provide command 'sh' ``` It was only **_after_** all this that I finally noticed that "artifact checkout" has a "--deps" option, which is "run" by default: https://docs.buildstream.build/master/using_commands.html#cmdoption-bst-artifact-checkout-d Which I'm sure there's a reason for, but the inconsistency between "checkout" and "list-contents" was confusing for multiple people. So, being able to change this default value in some config file would be nice, since our workflow would have us only ever really using that style. Also, the command line is getting quite long with the options we need (related #1770). For example: `$ bst -o target_arch aarch64 -o platform rpi -o debug true artifact checkout --deps none components/target/zlib.bst`, which is kinda... yeah. Thanks :) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
