tvalentyn commented on code in PR #28564:
URL: https://github.com/apache/beam/pull/28564#discussion_r1508325095
##########
sdks/python/apache_beam/runners/portability/stager.py:
##########
@@ -365,6 +371,11 @@ def create_job_resources(options, # type: PipelineOptions
Stager._create_file_stage_to_artifact(
pickled_session_file, names.PICKLED_MAIN_SESSION_FILE))
+ # stage the submission environment dependencies, if enabled.
Review Comment:
let's provide a way to disable this behavior via experiment. For example:
```
if (log_submission_env_dependencies and not
options.view_as(DebugOptions).lookup_experiment('disable_logging_submission_environment'))
```
you can use that experiment in one of the tests that you had issues.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
