If you want to create a custom S3ClientBuilderFactory, you just need to ensure your class has a zero-args constructor implements S3ClientBuilderFactory. What was the Jackson error you saw?
If you only care about modifying some of the ClientConfiguration parameters, you can open a pull request and add all the parameters that you want in ClientConfiguration[1] to the ClientConfigurationMixin[2]. It looks like the original authors only added a few. Do you know of a good way to do this for all client configuration parameters without needing to use an interface marked with @JsonProperty for the relevant methods since I didn't see a way to enumerate/set options without calling all the individual methods on the ClientConfiguration object. Also, what do you want to do with the parameters which aren't simple properties but complex types like. 1: https://docs.aws.amazon.com/AWSJavaSDK/latest/javadoc/com/amazonaws/ClientConfiguration.html 2: https://github.com/apache/beam/blob/001e16587e3ca72c49bb22284d00c6faf8468836/sdks/java/io/amazon-web-services/src/main/java/org/apache/beam/sdk/io/aws/options/AwsModule.java#L247 On Tue, Mar 17, 2020 at 2:36 PM Akshay Iyangar <aiyan...@godaddy.com> wrote: > Hi here, > > > > So I was interested in setting some Aws Client Configuraiton options for > S3 to tune some of its client parameters according to my liking. > > > > I was going through the code and found that there is something called > DefaultS3ClientBuilderFactory available and was thinking of using > something like that by writing my own CustomS3ClientBuilderFactory. > > But I wanted to pass this as a command line pipeline option rather than > setting it in the code itself .. > > I tried passing it as a command line option and immediately saw it fail > with a Jackson error. > > > > Then I saw there are ways to do that and fellow beam committers used auto > service to register their respective options. For instance if you have a > look at the AwsModule class you’ll understand what I mean. Then that allows > them to pass their respective pipeline option from command line . For > example if you have a look at AwsOptions . getClientConfiguration you’ll > see how the options are passed. > > > > So I can do something like this but the only issue with this is I’ll have > to have a fix contract. I can only pass parameters for which I allow the > registration to happen. > > I was wondering if there is a better way to do this? > > > > Something like we pass the ClientConfiguration and during runtime we > instantiate it and attach it to the respective Aws<SERVICE>Builder where > service could be (SQS,SNS,Kinesis,S3). > > > > Let me know what you’ll think and also if there is something like that > already present and I just missed it > > > > Thanks > > Akshay I >
