goanpeca opened a new issue, #39573: URL: https://github.com/apache/beam/issues/39573
### What needs to happen? `sdks/python/apache_beam/io/aws/clients/s3/boto3_client.py` constructs its boto3 client without a botocore `Config`, so S3 requests from Beam's Python AWS I/O carry only the default `Boto3/... botocore/...` user agent. Object store operators cannot attribute that traffic to Beam, which makes it harder to diagnose throughput, throttling, or request-volume questions raised by Beam users. Beam already does this for its other cloud I/O connectors. `apache_beam/io/gcp/gcsio.py:181`, `apache_beam/io/gcp/bigquery_tools.py:365`, and `apache_beam/io/gcp/datastore/v1new/helper.py:55` all set a Beam-identifying user agent on their clients. The S3 client is the inconsistent one. The change is to pass `config=Config(user_agent_extra='apache-beam/<version>')` when building the client, reading the version from `apache_beam.version.__version__`. `user_agent_extra` appends to the standard botocore user agent rather than replacing it, so nothing that parses the existing string changes. This benefits operators of Amazon S3 and of any other implementation of the S3 API equally. I have a working branch with a unit test that asserts the version string appears in `client.meta.config.user_agent`, and can open a PR against this issue. ### Issue Priority Priority: 3 (nice-to-have improvement) ### Issue Components - Component: Python SDK - Component: IO connector -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
