ssssam opened a new issue, #1740: URL: https://github.com/apache/buildstream/issues/1740
The test test_shell_pull_cached_buildtree sometimes fails. Example log: https://gitlab.com/samthursfield/buildstream-docker-images/-/jobs/2923053985 Failure info displayed by pytest: ``` ___________ test_shell_pull_cached_buildtree[pull-without-buildtree] ___________ share_with_buildtrees = <tests.testutils.artifactshare.ArtifactShare object at 0x7fcb3d6fb3d0> datafiles = local('/home/testuser/buildstream/.tox/py310-nocover/tmp/test_shell_pull_cached_buildtr2') cli = <buildstream._testing.runcli.Cli object at 0x7fcb3c85ead0> pull_deps = 'all', pull_buildtree = False @pytest.mark.datafiles(DATA_DIR) @pytest.mark.skipif(not HAVE_SANDBOX, reason="Only available with a functioning sandbox") @pytest.mark.parametrize( "pull_deps,pull_buildtree", [ # Don't pull at all (None, False), # Pull only dependencies ("build", False), # Pull all elements including the shell element, but without the buildtree ("all", False), # Pull all elements including the shell element, and pull buildtrees ("all", True), ], ids=["no-pull", "pull-only-deps", "pull-without-buildtree", "pull-with-buildtree"], ) def test_shell_pull_cached_buildtree(share_with_buildtrees, datafiles, cli, pull_deps, pull_buildtree): project = str(datafiles) element_name = "build-shell/buildtree.bst" cli.configure({"artifacts": {"servers": [{"url": share_with_buildtrees.repo}]}}) # Optionally pull the buildtree along with `bst artifact pull` maybe_pull_deps(cli, project, element_name, pull_deps, pull_buildtree) # Run the shell and request that required artifacts and buildtrees should be pulled result = cli.run( project=project, args=[ "--pull-buildtrees", "shell", "--build", element_name, "--use-buildtree", "--", "cat", "test", ], ) # In this case, we should succeed every time, regardless of what was # originally available in the local cache. # > result.assert_success() tests/integration/shellbuildtrees.py:325: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ self = <buildstream._testing.runcli.Result object at 0x7fcb3c74e3b0> fail_message = '' def assert_success(self, fail_message=""): > assert self.exit_code == 0, fail_message E AssertionError .tox/py310-nocover/lib64/python3.10/site-packages/buildstream/_testing/runcli.py:107: AssertionError ----------------------------- Captured stdout call ----------------------------- BuildStream exited with code 0 for invocation: bst --no-colors --config /home/testuser/buildstream/.tox/py310-nocover/tmp/test_shell_pull_cached_buildtr2/cache/buildstream.conf --directory /home/testuser/buildstream/.tox/py310-nocover/tmp/test_shell_pull_cached_buildtr2 artifact pull --deps all build-shell/buildtree.bst Program stderr was: [--:--:--][ ][ main:core activity ] START Pull [--:--:--][ ][ main:core activity ] START Loading elements [00:00:00][ ][ main:core activity ] SUCCESS Loading elements 3 subtasks processed [--:--:--][ ][ main:core activity ] START Resolving elements [00:00:00][ ][ main:core activity ] SUCCESS Resolving elements 3 of 3 subtasks processed [--:--:--][ ][ main:core activity ] START Initializing remote caches [00:00:00][ ][ main:core activity ] SUCCESS Initializing remote caches [--:--:--][ ][ main:core activity ] START Query cache [00:00:00][ ][ main:core activity ] SUCCESS Query cache 3 of 3 subtasks processed BuildStream Version 1.95.2+4.gb6b96a6b1 Session Start: Tuesday, 23-08-2022 at 17:12:11 Project: test (/home/testuser/buildstream/.tox/py310-nocover/tmp/test_shell_pull_cached_buildtr2) Targets: build-shell/buildtree.bst User Configuration Configuration File: /home/testuser/buildstream/.tox/py310-nocover/tmp/test_shell_pull_cached_buildtr2/cache/buildstream.conf Cache Directory: /home/testuser/buildstream/.tox/py310-nocover/tmp/test_shell_pull_cached_buildtr2/cache Log Files: /home/testuser/buildstream/.tox/py310-nocover/tmp/test_shell_pull_cached_buildtr2/cache/logs Source Mirrors: /home/testuser/buildstream/.tox/py310-nocover/tmp/test_shell_pull_cached_buildtr2/cache/sources Build Area: /home/testuser/buildstream/.tox/py310-nocover/tmp/test_shell_pull_cached_buildtr2/cache/build Strict Build Plan: Yes Maximum Fetch Tasks: 10 Maximum Build Tasks: 4 Maximum Push Tasks: 4 Maximum Network Retries: 2 Project: test Project Options arch: x86-64 linux: 1 Element Plugins manual: core plugin stack: core plugin import: core plugin Source Plugins tar: core plugin Artifact cache servers * http://localhost:44409/ push: False type: all instance: None loaded from: /home/testuser/buildstream/.tox/py310-nocover/tmp/test_shell_pull_cached_buildtr2/cache/buildstream.conf [line 3 column 4] Pipeline fetch needed 4ed9f63d0077a00322df997adda1c3eb96d9142cd2848074b3325f6f6f9e530e base/base-alpine.bst waiting 024ff07b5cef16947a874b815f01ed5afbe338d2beb69634e9d9f3813efb5782 base.bst waiting 4a47c98a10df39e65e99d471f96edc5b58d4ea5b9b1f221d0be832a8124b8099 build-shell/buildtree.bst =============================================================================== [--:--:--][4ed9f63d][ pull:base/base-alpine.bst ] START test/base-base-alpine/4ed9f63d-pull.261.log [--:--:--][4ed9f63d][ pull:base/base-alpine.bst ] STATUS Pulling artifact 4ed9f63d <- http://localhost:44409/ [--:--:--][4ed9f63d][ pull:base/base-alpine.bst ] STATUS Pulling data for artifact 4ed9f63d <- http://localhost:44409/ [--:--:--][024ff07b][ pull:base.bst ] START test/base/024ff07b-pull.261.log [--:--:--][024ff07b][ pull:base.bst ] STATUS Pulling artifact 024ff07b <- http://localhost:44409/ [--:--:--][024ff07b][ pull:base.bst ] STATUS Pulling data for artifact 024ff07b <- http://localhost:44409/ [--:--:--][4a47c98a][ pull:build-shell/buildtree.bst ] START test/build-shell-buildtree/4a47c98a-pull.261.log [--:--:--][4a47c98a][ pull:build-shell/buildtree.bst ] STATUS Pulling artifact 4a47c98a <- http://localhost:44409/ [--:--:--][024ff07b][ pull:base.bst ] INFO Pulled artifact 024ff07b <- http://localhost:44409/ [00:00:00][024ff07b][ pull:base.bst ] SUCCESS test/base/024ff07b-pull.261.log [--:--:--][4a47c98a][ pull:build-shell/buildtree.bst ] STATUS Pulling data for artifact 4a47c98a <- http://localhost:44409/ [--:--:--][4a47c98a][ pull:build-shell/buildtree.bst ] INFO Pulled artifact 4a47c98a <- http://localhost:44409/ [00:00:00][4a47c98a][ pull:build-shell/buildtree.bst ] SUCCESS test/build-shell-buildtree/4a47c98a-pull.261.log [--:--:--][4ed9f63d][ pull:base/base-alpine.bst ] INFO Pulled artifact 4ed9f63d <- http://localhost:44409/ [00:00:03][4ed9f63d][ pull:base/base-alpine.bst ] SUCCESS test/base-base-alpine/4ed9f63d-pull.261.log [00:00:03][ ][ main:core activity ] SUCCESS Pull Pipeline Summary Total: 3 Session: 3 Pull Queue: processed 3, skipped 0, failed 0 BuildStream exited with code -1 for invocation: bst --no-colors --config /home/testuser/buildstream/.tox/py310-nocover/tmp/test_shell_pull_cached_buildtr2/cache/buildstream.conf --directory /home/testuser/buildstream/.tox/py310-nocover/tmp/test_shell_pull_cached_buildtr2 --pull-buildtrees shell --build build-shell/buildtree.bst --use-buildtree -- cat test Program stderr was: [--:--:--][ ][ main:core activity ] START Loading elements [00:00:00][ ][ main:core activity ] SUCCESS Loading elements 3 subtasks processed [--:--:--][ ][ main:core activity ] START Resolving elements [00:00:00][ ][ main:core activity ] SUCCESS Resolving elements 3 of 3 subtasks processed [--:--:--][ ][ main:core activity ] START Initializing remote caches [00:00:00][ ][ main:core activity ] SUCCESS Initializing remote caches [--:--:--][ ][ main:core activity ] START Query cache [00:00:00][ ][ main:core activity ] SUCCESS Query cache 3 of 3 subtasks processed [--:--:--][ ][ main:core activity ] INFO Attempting to fetch missing or incomplete artifact(s) [--:--:--][4a47c98a][ pull:build-shell/buildtree.bst ] START test/build-shell-buildtree/4a47c98a-pull.261.log [--:--:--][4a47c98a][ pull:build-shell/buildtree.bst ] STATUS Pulling artifact 4a47c98a <- http://localhost:44409/ [--:--:--][4a47c98a][ pull:build-shell/buildtree.bst ] STATUS Pulling data for artifact 4a47c98a <- http://localhost:44409/ [--:--:--][4a47c98a][ pull:build-shell/buildtree.bst ] START test/build-shell-buildtree/4a47c98a-pull.261.log [--:--:--][4a47c98a][ pull:build-shell/buildtree.bst ] STATUS Pulling artifact 4a47c98a <- http://localhost:44409/ [--:--:--][4a47c98a][ pull:build-shell/buildtree.bst ] STATUS Pulling data for artifact 4a47c98a <- http://localhost:44409/ [--:--:--][4a47c98a][ pull:build-shell/buildtree.bst ] INFO Pulled artifact 4a47c98a <- http://localhost:44409/ [--:--:--][4a47c98a][ pull:build-shell/buildtree.bst ] INFO Pulled artifact 4a47c98a <- http://localhost:44409/ [00:00:00][4a47c98a][ pull:build-shell/buildtree.bst ] SUCCESS test/build-shell-buildtree/4a47c98a-pull.261.log [00:00:00][4a47c98a][ pull:build-shell/buildtree.bst ] FAILURE Failed to link '/home/testuser/buildstream/.tox/py310-nocover/tmp/test_shell_pull_cached_buildtr2/cache/artifacts/refs/test/build-shell-buildtree/4a47c98a10df39e65e99d471f96edc5b58d4ea5b9b1f221d0be832a8124b8099 -> /home/testuser/buildstream/.tox/py310-nocover/tmp/test_shell_pull_cached_buildtr2/cache/artifacts/refs/test/build-shell-buildtree/38b730aec720dbddba843c9e3ce1ad8073e39d16cc4d25f752bc89d867fbaec1': [Errno 17] File exists: '/home/testuser/buildstream/.tox/py310-nocover/tmp/test_shell_pull_cached_buildtr2/cache/artifacts/refs/test/build-shell-buildtree/4a47c98a10df39e65e99d471f96edc5b58d4ea5b9b1f221d0be832a8124b8099' -> '/home/testuser/buildstream/.tox/py310-nocover/tmp/test_shell_pull_cached_buildtr2/cache/artifacts/refs/test/build-shell-buildtree/38b730aec720dbddba843c9e3ce1ad8073e39d16cc4d25f752bc89d867fbaec1' Printing the last 20 lines from log file: /home/testuser/buildstream/.tox/py310-nocover/tmp/test_shell_pull_cached_buildtr2/cache/logs/test/build-shell-buildtree/4a47c98a-pull.261.log ====================================================================== BuildStream 1.95.2+4.gb6b96a6b1 - Tuesday, 23-08-2022 at 17:12:11 [--:--:--] START [4a47c98a] build-shell/buildtree.bst: Pull [--:--:--] STATUS [4a47c98a] build-shell/buildtree.bst: Pulling artifact 4a47c98a <- http://localhost:44409/ [--:--:--] STATUS [4a47c98a] build-shell/buildtree.bst: Pulling data for artifact 4a47c98a <- http://localhost:44409/ [--:--:--] INFO [4a47c98a] build-shell/buildtree.bst: Pulled artifact 4a47c98a <- http://localhost:44409/ [00:00:00] SUCCESS [4a47c98a] build-shell/buildtree.bst: Pull BuildStream 1.95.2+4.gb6b96a6b1 - Tuesday, 23-08-2022 at 17:12:14 [--:--:--] START [4a47c98a] build-shell/buildtree.bst: Pull [--:--:--] STATUS [4a47c98a] build-shell/buildtree.bst: Pulling artifact 4a47c98a <- http://localhost:44409/ [--:--:--] STATUS [4a47c98a] build-shell/buildtree.bst: Pulling data for artifact 4a47c98a <- http://localhost:44409/ BuildStream 1.95.2+4.gb6b96a6b1 - Tuesday, 23-08-2022 at 17:12:14 [--:--:--] START [4a47c98a] build-shell/buildtree.bst: Pull [--:--:--] STATUS [4a47c98a] build-shell/buildtree.bst: Pulling artifact 4a47c98a <- http://localhost:44409/ [--:--:--] STATUS [4a47c98a] build-shell/buildtree.bst: Pulling data for artifact 4a47c98a <- http://localhost:44409/ [--:--:--] INFO [4a47c98a] build-shell/buildtree.bst: Pulled artifact 4a47c98a <- http://localhost:44409/ [--:--:--] INFO [4a47c98a] build-shell/buildtree.bst: Pulled artifact 4a47c98a <- http://localhost:44409/ [00:00:00] SUCCESS [4a47c98a] build-shell/buildtree.bst: Pull [00:00:00] FAILURE [4a47c98a] build-shell/buildtree.bst: Failed to link '/home/testuser/buildstream/.tox/py310-nocover/tmp/test_shell_pull_cached_buildtr2/cache/artifacts/refs/test/build-shell-buildtree/4a47c98a10df39e65e99d471f96edc5b58d4ea5b9b1f221d0be832a8124b8099 -> /home/testuser/buildstream/.tox/py310-nocover/tmp/test_shell_pull_cached_buildtr2/cache/artifacts/refs/test/build-shell-buildtree/38b730aec720dbddba843c9e3ce1ad8073e39d16cc4d25f752bc89d867fbaec1': [Errno 17] File exists: '/home/testuser/buildstream/.tox/py310-nocover/tmp/test_shell_pull_cached_buildtr2/cache/artifacts/refs/test/build-shell-buildtree/4a47c98a10df39e65e99d471f96edc5b58d4ea5b9b1f221d0be832a8124b8099' -> '/home/testuser/buildstream/.tox/py310-nocover/tmp/test_shell_pull_cached_buildtr2/cache/artifacts/refs/test/build-shell-buildtree/38b730aec720dbddba843c9e3ce1ad8073e39d16cc4d25f752bc89d867fbaec1' ====================================================================== Error launching shell: ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
