I was trying to use the --sdk_location parameter in a python pipeline, to allow users to run a snapshot SDK. Though it looks like it hit a type error after downloading the .wdl file.
Perhaps this code is assuming that remote files downloaded are text type, not bytes type? Have I done something wrong? Or is this a bug? Any ideas? Thanks for taking a look, Alex Using the --sdk_location parameter (Full command line <https://paste.googleplex.com/5792777008840704>) --sdk_location= https://storage.googleapis.com/beam-wheels-staging/master/94f9e7fd4cae0f8aa6587d2cf14887f1c4827485-198203585/apache_beam-2.24.0.dev0-cp27-cp27m-macosx_10_9_x86_64.whl INFO:apache_beam.runners.portability.stager:Failed to download Artifact from https://storage.googleapis.com/beam-wheels-staging/master/94f9e7fd4cae0f8aa6587d2cf14887f1c4827485-198203585/apache_beam-2.24.0.dev0-cp27-cp27m-macosx_10_9_x86_64.whl Traceback (most recent call last): File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/runpy.py", line 193, in _run_module_as_main "__main__", mod_spec) File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/runpy.py", line 85, in _run_code exec(code, run_globals) File "/Users/ajamato/beam/beam-sdk-download-test/venv/lib/python3.6/site-packages/apache_beam/examples/wordcount.py", line 142, in <module> run() File "/Users/ajamato/beam/beam-sdk-download-test/venv/lib/python3.6/site-packages/apache_beam/examples/wordcount.py", line 121, in run result = p.run() File "/Users/ajamato/beam/beam-sdk-download-test/venv/lib/python3.6/site-packages/apache_beam/pipeline.py", line 521, in run allow_proto_holders=True).run(False) File "/Users/ajamato/beam/beam-sdk-download-test/venv/lib/python3.6/site-packages/apache_beam/pipeline.py", line 534, in run return self.runner.run_pipeline(self, self._options) File "/Users/ajamato/beam/beam-sdk-download-test/venv/lib/python3.6/site-packages/apache_beam/runners/dataflow/dataflow_runner.py", line 479, in run_pipeline artifacts=environments.python_sdk_dependencies(options))) File "/Users/ajamato/beam/beam-sdk-download-test/venv/lib/python3.6/site-packages/apache_beam/transforms/environments.py", line 611, in python_sdk_dependencies staged_name in stager.Stager.create_job_resources(options, tmp_dir)) File "/Users/ajamato/beam/beam-sdk-download-test/venv/lib/python3.6/site-packages/apache_beam/runners/portability/stager.py", line 235, in create_job_resources resources.extend(Stager._create_beam_sdk(sdk_remote_location, temp_dir)) File "/Users/ajamato/beam/beam-sdk-download-test/venv/lib/python3.6/site-packages/apache_beam/runners/portability/stager.py", line 657, in _create_beam_sdk Stager._download_file(sdk_remote_location, local_download_file) File "/Users/ajamato/beam/beam-sdk-download-test/venv/lib/python3.6/site-packages/apache_beam/runners/portability/stager.py", line 375, in _download_file f.write(content) TypeError: write() argument must be str, not bytes
