Hi David, Thank you for your feedback, sounds totally reasonable to me. I agree that before deprecation any of AWS V1 connectors we have to make sure that V2 version may completely substitute the previous one.
> On 17 Sep 2020, at 19:19, David Hollands <[email protected]> wrote: > > Hi Alexey – > > As relatively new users of Beam, we recently selected v1 over v2 because we > didn’t think v2 currently (as of 2.24.0-snapshot) had feature parity > especially the lack of a v2 based S3FileSystem and KinesisIO.Write. > > Ideally we would have selected v2. > > On a related note, and not really Beam’s problem, but if I remember rightly, > we also had a bit of trouble creating some LocalStack testcontainers based > integration tests with v2… > > Cheers, David > > David Hollands > Audience Platform – Audience Data Engineering > [email protected] <mailto:[email protected]> > BC5 C5, BBC Broadcast Centre, London, W12 7TQ > > From: Alexey Romanenko <[email protected] > <mailto:[email protected]>> > Reply to: "[email protected] <mailto:[email protected]>" > <[email protected] <mailto:[email protected]>> > Date: Tuesday, 15 September 2020 at 17:06 > To: "[email protected]" <[email protected]> > Subject: Re: [DISCUSS] Deprecation of AWS SDK v2 IO connectors > > I just want to cross-post it on users@ to find out which version of AWS SDK > connectors is mostly used in user applications and if there are any strong > objections to switch mostly to AWS SDK v2? > > Thank you for any feedback in advance. > > > On 11 Sep 2020, at 19:13, Alexey Romanenko <[email protected] > <mailto:[email protected]>> wrote: > > Hello, > > In Beam, there are two versions of AWS IO connectors for Java SDK - based on > AWS SDK v1 [1] and v2 [2]. For now, they are pretty equal in terms of > functionality, but since AWS SDK v2 is more modern (it's a major rewrite of > the version 1.x code base, it’s built on top of Java 8+ and adds more > features [3]), then it would be more logical to use only V2. Also, it’s not > reasonable to support two versions of similar connectors, since it’s a big > pain for us, and it will be more clear for users which package of AWS > connectors to use . > > According to this, I’d propose to deprecate all Java AWS IO connectors V1 (+ > KinesisIO which is in a different package for now) starting from Beam 2.25 > and then add new features only to V2 connectors. Bug fixes should be applied > to V2 connectors in the first order, and to V1 connectors if it’s only > necessary. > > What are the community thoughts on this? Any pros and cons that I'm missing? > > > [1] > https://github.com/apache/beam/tree/master/sdks/java/io/amazon-web-services > <https://github.com/apache/beam/tree/master/sdks/java/io/amazon-web-services> > [2] > https://github.com/apache/beam/tree/master/sdks/java/io/amazon-web-services2 > <https://github.com/apache/beam/tree/master/sdks/java/io/amazon-web-services2> > [3] https://docs.aws.amazon.com/sdk-for-java/v2/developer-guide/welcome.html > <https://docs.aws.amazon.com/sdk-for-java/v2/developer-guide/welcome.html>
