sagget opened a new issue, #30214: URL: https://github.com/apache/beam/issues/30214
### What happened? Hi all, When attempting to bundle a beam job like the wordcount example into a jar for execution on Spark it appears that this step requires local execution. I am following: https://beam.apache.org/documentation/runners/spark/#kubernetes. When running: ``` python -m beam_example_wc \ --runner=SparkRunner \ --output_executable_path=./wc_job.jar \ --environment_type=PROCESS \ --environment_config='{\"command\": \"/opt/apache/beam/boot\"}' \ --spark_version=3 ``` I get an error: `OSError No files found based on the file pattern s3://bucket/path`. My local machine running this command does not have access to the input files on s3. But Spark does. The comment on [output_executable_path](https://github.com/apache/beam/blob/master/sdks/python/apache_beam/options/pipeline_options.py#L1518) states it builds the jar rather than running. If that's the case I'm confused why I get an access error? Not running the beam code should mean my local machine does not need to talk to s3? Any help would be appreciated! Thanks! ### Issue Priority Priority: 3 (minor) ### Issue Components - [X] Component: Python SDK - [ ] Component: Java SDK - [ ] Component: Go SDK - [ ] Component: Typescript SDK - [ ] Component: IO connector - [ ] Component: Beam YAML - [ ] Component: Beam examples - [ ] Component: Beam playground - [ ] Component: Beam katas - [ ] Component: Website - [ ] Component: Spark Runner - [ ] Component: Flink Runner - [ ] Component: Samza Runner - [ ] Component: Twister2 Runner - [ ] Component: Hazelcast Jet Runner - [ ] Component: Google Cloud Dataflow Runner -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
