Re: [VOTE] Vendored Dependencies Release gRPC 1.26.0 v0.2 for BEAM-9252

2020-02-21 Thread Kai Jiang
+1 (non-binding) On Fri, Feb 21, 2020 at 5:03 PM Robin Qiu wrote: > +1 (verified) > > On Fri, Feb 21, 2020 at 4:55 PM Robert Bradshaw > wrote: > >> +1 (binding) >> >> >> On Fri, Feb 21, 2020 at 4:48 PM Ahmet Altay wrote: >> > >> > +1 >> > >> > On Fri, Feb 21, 2020 at 4:39 PM Luke Cwik wrote:

Bay Area Beam Meetup 19 Feb (Last Wednesday).

2020-02-21 Thread Austin Bennett
Hi All, We had a meetup @Sentry.io on Wednesday -- with a solid 40+ engaged attendees. Thanks for those that joined in person, and for those that were unable, talks can be found online --> Syd's talk (real time data warehouse): https://youtu.be/rFK6drAWN40 Mike's talk (beam in production):

Re: Schema Convert transform fails on type metadata

2020-02-21 Thread Reuven Lax
I think that it's incorrect - we shouldn't enforce that the metadata match. On Fri, Feb 21, 2020 at 10:38 AM Jozef Vilcek wrote: > Hi, > > I am playing with Schemas in Beam and conversion between types. I am > experimenting with Convert transform to convert PCollection of POJOs to > Protobufs,

Re: [VOTE] Vendored Dependencies Release gRPC 1.26.0 v0.2 for BEAM-9252

2020-02-21 Thread Robin Qiu
+1 (verified) On Fri, Feb 21, 2020 at 4:55 PM Robert Bradshaw wrote: > +1 (binding) > > > On Fri, Feb 21, 2020 at 4:48 PM Ahmet Altay wrote: > > > > +1 > > > > On Fri, Feb 21, 2020 at 4:39 PM Luke Cwik wrote: > >> > >> +1 (binding) > >> I diffed the binary contents of the 0.1 jar and 0.2 jar

Re: [VOTE] Vendored Dependencies Release gRPC 1.26.0 v0.2 for BEAM-9252

2020-02-21 Thread Robert Bradshaw
+1 (binding) On Fri, Feb 21, 2020 at 4:48 PM Ahmet Altay wrote: > > +1 > > On Fri, Feb 21, 2020 at 4:39 PM Luke Cwik wrote: >> >> +1 (binding) >> I diffed the binary contents of the 0.1 jar and 0.2 jar with no changes to >> the contents of the files and can confirm that module-info.class the

Re: [VOTE] Vendored Dependencies Release gRPC 1.26.0 v0.2 for BEAM-9252

2020-02-21 Thread Ahmet Altay
+1 On Fri, Feb 21, 2020 at 4:39 PM Luke Cwik wrote: > +1 (binding) > I diffed the binary contents of the 0.1 jar and 0.2 jar with no changes to > the contents of the files and can confirm that module-info.class the > offending Main.class and Main$1.class have been removed as well. > > On Fri,

Re: [VOTE] Vendored Dependencies Release gRPC 1.26.0 v0.2 for BEAM-9252

2020-02-21 Thread Luke Cwik
+1 (binding) I diffed the binary contents of the 0.1 jar and 0.2 jar with no changes to the contents of the files and can confirm that module-info.class the offending Main.class and Main$1.class have been removed as well. On Fri, Feb 21, 2020 at 4:38 PM Luke Cwik wrote: > Please review the

[VOTE] Vendored Dependencies Release gRPC 1.26.0 v0.2 for BEAM-9252

2020-02-21 Thread Luke Cwik
Please review the release of the following artifacts that we vendor: * beam-vendor-grpc-1_26_0 Hi everyone, Please review and vote on the release candidate #1 for the version 0.2, as follows: [ ] +1, Approve the release [ ] -1, Do not approve the release (please provide specific comments) The

Re: Help needed on a problem statement

2020-02-21 Thread Luke Cwik
There is the TextIO.readAll[1] that you could use which watches a file pattern and will automatically read the files that appear there. Note that Apache Beam only has TextIO and no CsvIO or JsonIO. TextIO does allow skipping so many lines from the header so you could still use it for csv files

Re: [VOTE][BIP-1] Beam Schema Options

2020-02-21 Thread Brian Hulette
+1 (non-binding) thanks for all your work on this Alex :) On Fri, Feb 21, 2020 at 6:50 AM Alex Van Boxel wrote: > +1 (non-binding) > > I assume I can vote on my own proposal :-) > > _/ > _/ Alex Van Boxel > > > On Fri, Feb 21, 2020 at 6:36 AM Jean-Baptiste Onofre > wrote: > >> +1 (binding) >>

Re: Docker images are migrated to Apache org.

2020-02-21 Thread Ahmet Altay
Thank you, Hannah! This is great. On Fri, Feb 21, 2020 at 11:24 AM Hannah Jiang wrote: > Hello team > > Docker SDK images (Python, Java, Go) and Flink job server images are > migrated to Apache org[1]. > I confirmed digests of all the images of the two repos are exactly the > same. > In

Docker images are migrated to Apache org.

2020-02-21 Thread Hannah Jiang
Hello team Docker SDK images (Python, Java, Go) and Flink job server images are migrated to Apache org[1]. I confirmed digests of all the images of the two repos are exactly the same. In addition, I updated readme pages of the new repos. The readma pages match to the ones at github. New images

Schema Convert transform fails on type metadata

2020-02-21 Thread Jozef Vilcek
Hi, I am playing with Schemas in Beam and conversion between types. I am experimenting with Convert transform to convert PCollection of POJOs to Protobufs, but get a failure about schemas not being compatible. The root cause is that FieldType are not passing `equivalent()` check because of

Re: [VOTE][BIP-1] Beam Schema Options

2020-02-21 Thread Alex Van Boxel
+1 (non-binding) I assume I can vote on my own proposal :-) _/ _/ Alex Van Boxel On Fri, Feb 21, 2020 at 6:36 AM Jean-Baptiste Onofre wrote: > +1 (binding) > > Very interesting. It remembers me when we start the discussion about > schema support ;) > > Regards > JB > > Le 20 févr. 2020 à

Re: Discrete Transforms vs One Single transform

2020-02-21 Thread Jeff Klukas
Also note that runners in many cases will fuse discrete transforms together for efficiency, so while you might worry about performance degradation from breaking your logic into many discrete transforms, that likely won't be an issue in practice. Also note that you have the option of defining

Re: FnAPI proto backwards compatibility

2020-02-21 Thread Jan Lukavský
This is not exactly a problem. Having the runner to explicitly declare its capabilities is of course a possibility. But do we want to modify each runner, in case all the functionality is actually provided by a common library? This is about usability. I can imagine (and easier if we can provide