zendesk-kjaanson commented on issue #32743: URL: https://github.com/apache/beam/issues/32743#issuecomment-2706068633
How should the artefact staging to workers work? Not through shared staging directory? I am wondering if me getting the staging volume sharing working might simply have sidestepped the actual issue? When I am looking staging folder in jobmanager and taskmanagers then jobmanager staging looks like this `/tmp/beam-artifact-staging/<flink job id>` ``` -rw-r--r-- 1 root root 126850563 Mar 7 09:39 1-0:ref_Environment_default-workflow.tar.gz -rw-r--r-- 1 root root 7406 Mar 7 09:39 2-0:ref_Environment_default-pickled_main_session -rw-r--r-- 1 root root 3257 Mar 7 09:39 3-0:ref_Environment_default-submission_environment_dependencies.txt ``` However taskmanager logs show that ArtefactRetrievalService is not able to fetch anything unless the staging volume is actually shared between jobmanager and taskmanagers. This error is in task managers logs ``` java.io.FileNotFoundException: /tmp/beam-artifact-staging/<flink job id>/3-0:ref_Environment_default-submission_environment_dependencies.txt (No such file or directory) ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: github-unsubscr...@beam.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
