New Contributor

Hi All, I'd like to contribute an improvement to the Kafka.io module, specifically this issue BEAM-11851 . Could I be added as a contributor to the Beam issue tracker please? My username is: tunderhill Many Thanks! Tom.

Re: Proposal: Generalize S3FileSystem

This is a random idea, but the whole file IO system inside Beam would actually be awesome to extract into its own project. IIRC, it’s not particularly tied to Beam. I’m not saying this should be done now, but it’s be nice to keep it mind for a future goal. -chad On Wed, May 19, 2021 at 10:23

Beam Website Feedback

No, I couldn't find anything about non-file based sinks. I need to make a sink for a GCP service that's only available as an API right now. We have to send POST requests to it.

Proposal: Generalize S3FileSystem

Hi, This is a quick sketch of a proposal - I wanted to get a sense of whether there's general support for this idea before fleshing it out further, getting internal approvals, etc. I'm working with multiple storage systems that speak the S3 api. I would like to support FileIO operations for

Re: Question about SplittableDoFn

Thanks for sharing. I'll comment on the PR. On Tue, May 18, 2021 at 3:44 PM Miguel Anzo Palomo wrote: > Boyuan Zhang, It's about this issue > , the code can be found > here > > On Tue, May 18,

Re: Proposal: Generalize S3FileSystem

That would be great to add, Matt. Of course it's important to make this backwards compatible, but other than that, the addition would be very welcome. On Wed, May 19, 2021 at 9:41 AM Matt Rudary wrote: > Hi, > > > > This is a quick sketch of a proposal – I wanted to get a sense of whether >

Re: Need maintainer permission of PyPI apache-beam package

I've sent you an invite to be a project maintainer. Let me know if that works. Best -P. On Tue, May 18, 2021 at 6:28 PM Heejong Lee wrote: > Hi, > > I'm currently working on Beam 2.30.0 release and need help adding myself > to the maintainer group of PyPI apache-beam package. My PyPI username

Re: [VOTE] Vendored Dependencies Release Byte Buddy 1.11.0

Kind reminder, the vote is ongoing On Mon, May 17, 2021 at 5:32 PM Ismaël Mejía wrote: > Please review the release of the following artifacts that we vendor: > * beam-vendor-bytebuddy-1_11_0 > > Hi everyone, > Please review and vote on the release candidate #1 for the version 0.1, as >

Re: [VOTE] Vendored Dependencies Release Byte Buddy 1.11.0

I'd like to help, but I don't know how to determine whether this upgrade is going to cause problems or not. Are there tests I should look at, or some validation I should perform? On Wed, May 19, 2021 at 11:29 AM Ismaël Mejía wrote: > Kind reminder, the vote is ongoing > > On Mon, May 17, 2021

Re: [VOTE] Vendored Dependencies Release Byte Buddy 1.11.0

+1 (binding) I've verified the hash sums. Best -P. On Wed, May 19, 2021 at 11:45 AM Tyson Hamilton wrote: > I'd like to help, but I don't know how to determine whether this upgrade > is going to cause problems or not. Are there tests I should look at, or > some validation I should perform? > >

Re: [VOTE] Vendored Dependencies Release Byte Buddy 1.11.0

This release is only to publish the vendored dependency artifacts. We need those to integrate it and be able to verify if it causes problems or not. The PR for this is already opened but it needs the artifacts of this vote to be ran. https://github.com/apache/beam/pull/14824 For ref there is a

Re: [VOTE] Vendored Dependencies Release Byte Buddy 1.11.0

+1 All classes in the JAR are shaded correctly under org/apache/beam/vendor On Wed, May 19, 2021 at 2:54 PM Ismaël Mejía wrote: > This release is only to publish the vendored dependency artifacts. We need > those to integrate it and be able to verify if it causes problems or not. > The PR for

Re: [VOTE] Vendored Dependencies Release Byte Buddy 1.11.0

+1 (non-binding) verified correct shading.  Jan On 5/19/21 8:53 PM, Ismaël Mejía wrote: This release is only to publish the vendored dependency artifacts. We need those to integrate it and be able to verify if it causes problems or not. The PR for this is already opened but it needs the

Re: Compatibility Check Badges need attention

I have not worked on this project for quite some time. +Colin Nelson On Tue, May 18, 2021 at 4:00 PM Valentyn Tymofieiev wrote: > Looks like they were added in https://github.com/apache/beam/pull/8791. > +Lily Li do you know by chance if this tooling is > still maintained? > > > On Tue, May

Re: [VOTE] Vendored Dependencies Release Byte Buddy 1.11.0

+1 (non-binding) On Wed, May 19, 2021 at 12:23 PM Jan Lukavský wrote: > +1 (non-binding) > > verified correct shading. > > Jan > On 5/19/21 8:53 PM, Ismaël Mejía wrote: > > This release is only to publish the vendored dependency artifacts. We need > those to integrate it and be able to verify

Re: Need maintainer permission of PyPI apache-beam package

It's done. Thanks Pablo! On Wed, May 19, 2021 at 11:21 AM Pablo Estrada wrote: > I've sent you an invite to be a project maintainer. Let me know if that > works. > Best > -P. > > On Tue, May 18, 2021 at 6:28 PM Heejong Lee wrote: > >> Hi, >> >> I'm currently working on Beam 2.30.0 release and

Re: Some questions around GroupIntoBatches

Reading over the other thread, there was consensus to implement. Reading commentary on the PR, there were good questions raised about the semantics. Questions which I feel able to have an opinion about :-) The questions surrounded bundling and timers in the same bundling clearing each other.

P1 issues report

This is your daily summary of Beam's current P1 issues, not including flaky tests. See https://beam.apache.org/contribute/jira-priorities/#p1-critical for the meaning and expectations around P1 issues. BEAM-12352: GcsIOIntegrationTest.test_copy_rewrite_token and

Flaky test issue report

This is your daily summary of Beam's current flaky tests. These are P1 issues because they have a major negative impact on the community and make it hard to determine the quality of the software. BEAM-12322: FnApiRunnerTestWithGrpcAndMultiWorkers flaky (py precommit)

Re: Some questions around GroupIntoBatches

+1. It was my understanding as well that consensus was that timers must be delivered in timestamp order, and "within bundle" resetting/clearing of timers should be respected (as if each timer was in its own bundle). On Wed, May 19, 2021 at 3:01 PM Kenneth Knowles wrote: > > Reading over the

Re: Some questions around GroupIntoBatches

100% - the contract should not change because things are in a bundle. IIRC there are some open bugs in Beam around this that really should be fixed. My issue with GroupIntoBatches is different. This transform works as follows: if (this is the first element in batch - checked by reading a count

Re: Upgrading vendored gRPC from 1.26.0 to 1.36.0

Update: I just merged Kiley's https://github.com/apache/beam/pull/14833, in which I tried several "Run Java Precommit" and didn't observe the logging test (BeamFnLoggingServiceTest) failures. Let's see how the builds go. Kenn, Ismaël, and Kiley, Thank you for the help and follow-up! On Thu,

Re: LGPL-2.1 in beam-vendor-grpc

As I sent in the separate email about https://github.com/apache/beam/pull/14833, now the Beam's master branch depends on the newer version of the vendored gRPC, which does not include the problematic jboss artifacts. On Tue, May 11, 2021 at 12:39 PM Kenneth Knowles wrote: > +1 > > It seems we

Re: New Contributor

Hi Tom, Thanks for working on this! I added you as a contributor, welcome to Beam! Feel free to ping me for review once it will be ready. — Alexey > On 19 May 2021, at 11:51, Tom Underhill wrote: > > Hi All, > > I'd like to contribute an improvement to the Kafka.io module, specifically >