Re: [External] Re: DISCUSS: Sorted MapState API

2020-09-15 Thread Reuven Lax
Hi, Currently we only support removing a timestamp range. You can remove a single timestamp of course by removing [ts, ts+1), however if there are multiple elements with the same timestamp this will remove all of those elements. Does this fit your use case? If not, I wonder if MapState is closer

[RELEASE VOTE RESULT] Release 2.24.0, candidate #3

2020-09-15 Thread Daniel Oliveira
I'm happy to announce that we have approved the 2.24.0 release. There are 5 approving votes, 3 of which are binding: * Ahmet Altay * Robert Bradshaw * Thomas Weise Thanks everyone for your help to prepare the release. I'm going to finalize the release and send out the official release

Re: [External] Re: DISCUSS: Sorted MapState API

2020-09-15 Thread Tyson Hamilton
Hi Reuven, I noticed that there was an implementation of the in-memory OrderedListState introduced [1]. Where can I find out more regarding the plan and design? Is there a design doc? I'd like to know more details about the implementation to see if it fits my use case. I was hoping it would

Re: [DISCUSS] Deprecation of AWS SDK v2 IO connectors

2020-09-15 Thread Robert Bradshaw
Thanks for clarifying the state of things. +1 to deprecating once we have parity. If the v2 ones are better, perhaps a blog post would be a good way to advertise (and document) their existence and advantages too. On Tue, Sep 15, 2020 at 2:15 PM Ismaël Mejía wrote: > The reason why most people

Can someone review my pull request https://github.com/apache/beam/pull/12818 ?

2020-09-15 Thread terry xian
Hi, In case that the reviewer I added in the PR is OOO, I would like to ask another committee member to review and approve the PR. Can someone take a look? Thanks! | | | | | | | | | | | [BEAM-10875] Support NUMERIC in spanner schema parser by terryxian78 · P... R: @chamikaramj,

Re: [DISCUSS] Move Avro dependency out of core Beam

2020-09-15 Thread Ismaël Mejía
Avro differences in our implementation are pretty minimal if you look at the PR, to the point that an Adapter should be really tiny if even needed. The big backwards incompatible changes in Avro > 1.8 were to remove external libraries from the public APIs e.g. guava, jackson and joda-time. Of

Re: [DISCUSS] Deprecation of AWS SDK v2 IO connectors

2020-09-15 Thread Ismaël Mejía
The reason why most people are using AWSv1 IOs is probably because they are in Beam since 2017 instead of just added in the last year which is the case of the AWSv2 ones. Alexey mentions that maintaining both versions is becoming painful and I would like to expand on that because we have now

Re: Infra Improvements

2020-09-15 Thread Tyson Hamilton
There exists a few that could be used and seem adequate to cover infra, it may have been that I just didn't know about them all. Looking through the component list now in Jira some that make sense: * build-system: Build, CI, release systems and processes (519 issues) * testing: Testing: general

Re: Infra Improvements

2020-09-15 Thread Ahmet Altay
Thank you Tyson for the update. Thank you Damian, Tobiasz for the improvements. On Mon, Sep 14, 2020 at 3:29 PM Kenneth Knowles wrote: > I think the big bucket makes sense and is pretty self-documenting. I think > we'll hit some name conflicts between our infra component and ASF INFRA. > They

Re: [DISCUSS] Deprecation of AWS SDK v2 IO connectors

2020-09-15 Thread Robert Bradshaw
The 10x-100x ratio looks like an answer right there about (non-)suitability for deprecation. The new question would be *why* people are using the v1 APIs. Is it because it was the original, or that it's been around longer, or it has more features?

Re: Adding Twister2 documentation

2020-09-15 Thread Ismaël Mejía
Hi, both PRs were reviewed/merged yesterday, thanks for adding more docs and keep improving the runner! I would also like to ask the other members of the community for help in general in circumstances like this one. One argument to not help with these reviews is the lack of knowledge of the

Re: [DISCUSS] Deprecation of AWS SDK v2 IO connectors

2020-09-15 Thread Alexey Romanenko
Well, on IO transforms page [1] we provide the links to both versions but yes, you have a point, and I guess that users just choose a first one. Therefore, I think it will worth to ask it on user@ [1] https://beam.apache.org/documentation/io/built-in/ > On 14 Sep 2020, at 23:51, Kenneth Knowles

Re: [ANNOUNCE] New committer: Reza Ardeshir Rokni

2020-09-15 Thread Ismaël Mejía
Congratulations Reza, well done ! On Mon, Sep 14, 2020 at 10:10 AM Katarzyna Kucharczyk wrote: > > Congratulations Reza! :) > > On Mon, Sep 14, 2020 at 10:05 AM Alexey Romanenko > wrote: >> >> Congratulations! Thanks Reza for your contributions! >> >> On 12 Sep 2020, at 10:00, Jan Lukavský