Krzysztof Perchlicki created BEAM-10133:
-------------------------------------------
Summary: Add possibility for multiple region client
Key: BEAM-10133
URL: https://issues.apache.org/jira/browse/BEAM-10133
Project: Beam
Issue Type: Improvement
Components: io-java-aws
Reporter: Krzysztof Perchlicki
h3. What?
I recently had an use case that I can not fulfill due to actual AWS Writer
limitation.
What I want to do is to be able to save dynamically element into S3 buckets
which are in different regions.
For now the region for AWS is passed only on application start and it is
required.
This region is used by {{AWSClient}} with each of write to S3. Unfortunatelly
in case of writing to bucket in different region there is 400 response code.
What would be nice in my opinion it would be to have possibility for creating
dynamically {{AWSClients}} based on some kind of {{RegionResolver}}(?)
It could be done as it is done for {{AWSCredentialsProvider}} option.
[https://beam.apache.org/releases/javadoc/2.5.0/org/apache/beam/sdk/io/aws/options/AwsOptions.html#setAwsCredentialsProvider-com.amazonaws.auth.AWSCredentialsProvider-]
So user declares custom class for resolving region (by key value from e.g.
{{dynamicWriting}}) and this function is used to create {{AWSClient}} with each
of write request. By default it could just return provided in options
{{regionName}} (which could be optional in that case)
I am just thinking loudly about my idea but you as creators knows the best :)
h3. Why?
To give possibility to dynamically select region for writing
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
