ConverJens commented on pull request #13723:
URL: https://github.com/apache/beam/pull/13723#issuecomment-766708288
@dandy10
I added some logging in s3io.py just before the s3 client is initialised as
you suggested:
```
logging.info("boto3 options")
logging.info(options)
self.client = boto3_client.Client(options=options)
```
From the output it seems as the pipeline options are indeed empty:
```
INFO:apache_beam.runners.portability.local_job_service:Worker: severity:
INFO timestamp { seconds: 1611569401 nanos: 499072313 } message: "boto3
options" instruction_id: "bundle_33" transform_id:
"WriteSplit[train]/Write/Write/WriteImpl/WriteBundles" log_location:
"/usr/local/lib/python3.7/dist-packages/apache_beam/io/aws/s3io.py:67" thread:
"Thread-14"
INFO:apache_beam.runners.portability.local_job_service:Worker: severity:
INFO timestamp { seconds: 1611569401 nanos: 499203681 } message:
"PipelineOptions()" instruction_id: "bundle_33" transform_id:
"WriteSplit[train]/Write/Write/WriteImpl/WriteBundles" log_location:
"/usr/local/lib/python3.7/dist-packages/apache_beam/io/aws/s3io.py:68" thread:
"Thread-14"
```
Any idea where to continue troubleshooting?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
