Re: [ANNOUNCE] New PMC Member: Alex Van Boxel

2023-10-14 Thread Alex Van Boxel
Thank you everyone. I'm excited and... well, pretty new to the PMC. So I'll be in observability mode for a bit :-) _/ _/ Alex Van Boxel On Thu, Oct 5, 2023 at 10:07 AM Ismaël Mejía wrote: > Congratulations Alex, well deserved! > > On Wed, Oct 4, 2023 at 11:59 PM Chamikara Jayalath

Re: Dataflow dependencies require non-maven central dependencies (confluent kafka)

2021-07-12 Thread Alex Van Boxel
That's not the problem, the example Gradle build excludes it, the Dataflow runner fails with the error stated in the original post. _/ _/ Alex Van Boxel On Mon, Jul 12, 2021 at 6:27 PM Alexey Romanenko wrote: > I agree that we should make this optional. What would be the best

Re: Dataflow dependencies require non-maven central dependencies (confluent kafka)

2021-07-11 Thread Alex Van Boxel
y-proto:1.3.0-alpha") compile("com.microsoft.sqlserver:mssql-jdbc:9.1.0.jre8-preview") compile("io.swagger.parser.v3:swagger-parser:2.0.24") testCompile("org.hamcrest:hamcrest-all:1.3") testCompile("org.assertj:assertj-core:3.4.1") testCompile("junit:junit:4.12&quo

Dataflow dependencies require non-maven central dependencies (confluent kafka)

2021-07-09 Thread Alex Van Boxel
(DataflowRunner.java:196) This happens from version 2.30 onwards. Is this intended?! _/ _/ Alex Van Boxel

Re: [RESULT][VOTE] Accept the Firefly design donation as Beam Mascot - Deadline Mon April 6

2020-04-09 Thread Alex Van Boxel
We forgot something ... ... it/she/he needs a *name*! _/ _/ Alex Van Boxel On Fri, Apr 10, 2020 at 6:19 AM Kenneth Knowles wrote: > Looking forward to the guide. I enjoy doing (bad) drawings as a way to > relax. And I want them to be properly on brand :-) > > Kenn >

Re: [VOTE] Accept the Firefly design donation as Beam Mascot - Deadline Mon April 6

2020-04-02 Thread Alex Van Boxel
Thanks for clearing this up Aizhamal. +1 (non binding) _/ _/ Alex Van Boxel On Thu, Apr 2, 2020 at 8:14 PM Aizhamal Nurmamat kyzy wrote: > Good point, Alex. Actually Julian and I have talked about producing this > kind of guide. It will be delivered as an additional contri

Re: [VOTE] Accept the Firefly design donation as Beam Mascot - Deadline Mon April 6

2020-04-02 Thread Alex Van Boxel
and match. Without this it will never reach the potential of the Go gopher or gRPC Pancakes. Note this is *not* a negative vote but I'm afraid that the use without a guide will be fairly limited as most of use are not designers. Just a concern. _/ _/ Alex Van Boxel On Thu, Apr 2, 2020 at 7:27 PM

Re: [VOTE + INPUT] Beam Mascot Designs, 3rd iteration - Deadline Wednesday, April 1

2020-03-31 Thread Alex Van Boxel
Nooo stripes _/ _/ Alex Van Boxel On Tue, Mar 31, 2020 at 6:06 AM Joshua B. Harrison wrote: > > > On Mon, Mar 30, 2020 at 8:09 PM Julian Bruno > wrote: > >> Hello Apache Beam Community, >> >> We need a third input from the community to finish the design. Pl

Re: [VOTE + INPUT] Beam Mascot Designs, 1st iteration - Deadline Wed March 18

2020-03-17 Thread Alex Van Boxel
;-). _/ _/ Alex Van Boxel On Fri, Mar 13, 2020 at 8:25 PM Julian Bruno wrote: > Hello Apache Beam Community, > > > Together with Aizhamal and her team, we have been working on the design of > the Apache Beam mascot. > > We now need input from the community to co

Re: Contributing Twister2 runner to Apache Beam

2020-03-10 Thread Alex Van Boxel
One last thing, for any runner after this one... wouldn't it be a good acceptance criteria to only accept portable implementations anymore? _/ _/ Alex Van Boxel On Mon, Mar 9, 2020 at 10:42 PM Ismaël Mejía wrote: > Good points Kenn. I think we mostly agree on what has been discus

Re: [VOTE] Upgrade gradle to 6.2

2020-03-05 Thread Alex Van Boxel
https://issues.apache.org/jira/browse/BEAM-9456 I'll take it step-by-step, expect slow proces as till now haven't focussed on Python, Go and other runners. So be patient. _/ _/ Alex Van Boxel On Thu, Mar 5, 2020 at 9:13 PM Jean-Baptiste Onofre wrote: > Fair enough, we have the consen

Re: [VOTE] Upgrade gradle to 6.2

2020-03-05 Thread Alex Van Boxel
I will _/ _/ Alex Van Boxel On Thu, Mar 5, 2020 at 8:17 PM Ismaël Mejía wrote: > Looks like we have consensus on this one. Can you create a JIRA to > track this Alex. > I found this interesting presentation and associated repo, for the > interested on new improvement

Re: Proposal: Beam to use GCP Libraries BOM

2020-03-04 Thread Alex Van Boxel
+1, I can remember the countless hours that we fought with Google dependencies. On Thu, Mar 5, 2020, 04:07 Chamikara Jayalath wrote: > +1 for this. > > This will make life easy for many of our users and will help us keep GCP > related dependencies compatible (which has not been easy in the

Re: [VOTE][BIP-1] Beam Schema Options

2020-03-02 Thread Alex Van Boxel
Anyone keen to review this PR: https://github.com/apache/beam/pull/10413 without this foundation I can't continue with the rest. _/ _/ Alex Van Boxel On Fri, Feb 28, 2020 at 11:40 PM Alex Van Boxel wrote: > Thank you everyone for voting: Accepted by majority vote +1 (7 votes, 3 > b

Re: [VOTE][BIP-1] Beam Schema Options

2020-02-28 Thread Alex Van Boxel
of the interface in, then I can focus on the different implementations: https://github.com/apache/beam/pull/10413 _/ _/ Alex Van Boxel On Wed, Feb 26, 2020 at 9:25 PM Maximilian Michels wrote: > +1 (binding) > > Looks like a useful feature to have in schemas and fields. Thank you for > the g

Re: [VOTE][BIP-1] Beam Schema Options

2020-02-26 Thread Alex Van Boxel
Nico, thanks for the addition to SQL/MED. I could use another PMC vote to conclude the vote. _/ _/ Alex Van Boxel On Mon, Feb 24, 2020 at 11:25 PM Kenneth Knowles wrote: > +1 (binding) > > Added a link to the wiki proposal for SQL/MED (Management of External > Data) which

Jenkins problems: javaPreCommitPortabilityApiJava11 and No Space left

2020-02-26 Thread Alex Van Boxel
ple: https://builds.apache.org/job/beam_PreCommit_Portable_Python_Commit/9287/console _/ _/ Alex Van Boxel

Re: [ANNOUNCE] New committer: Jincheng Sun

2020-02-24 Thread Alex Van Boxel
Congrats! _/ _/ Alex Van Boxel On Mon, Feb 24, 2020 at 8:13 PM Kyle Weaver wrote: > Thanks Jincheng for all your work on Beam and Flink integration. > > On Mon, Feb 24, 2020 at 11:02 AM Yichi Zhang wrote: > >> Congrats, Jincheng! >> >> On Mon, Feb 24, 2020

Re: [ANNOUNCE] New committer: Chad Dombrova

2020-02-24 Thread Alex Van Boxel
Congrats! _/ _/ Alex Van Boxel On Tue, Feb 25, 2020 at 6:21 AM Thomas Weise wrote: > Congratulations! > > > On Mon, Feb 24, 2020, 3:38 PM Ankur Goenka wrote: > >> Congratulations Chad! >> >> On Mon, Feb 24, 2020 at 3:34 PM Ahmet Altay wrote: >> >

Re: [VOTE] Upgrade gradle to 6.2

2020-02-24 Thread Alex Van Boxel
OK, great. I know someone that works at gradle, so I can ping them when I have some problems. Any other know pitfalls I can expect, let me know :-) _/ _/ Alex Van Boxel On Tue, Feb 25, 2020 at 7:20 AM Jean-Baptiste Onofré wrote: > +1 > > It makes sense. > > Thanks. > Regar

[VOTE] Upgrade gradle to 6.2

2020-02-24 Thread Alex Van Boxel
Anyone objections that I upgrade gradle to 6.2. If ok this will be done over several commits where I will: - Upgrade plugins - Upgrade gradle to 6.2 - See where we can use some of the new features _/ _/ Alex Van Boxel

Re: [PROPOSAL] Vendored bytebuddy dependency release

2020-02-22 Thread Alex Van Boxel
+1 _/ _/ Alex Van Boxel On Sat, Feb 22, 2020 at 5:45 PM Jean-Baptiste Onofré wrote: > +1 > > Regards > JB > > Le sam. 22 f?vr. 2020 ? 15:02, Isma?l Mej?a a ?crit : > >> The version of bytebuddy Beam is vendoring (1.9.3) is more than 16 months >> old. I

Re: Schema Convert transform fails on type metadata

2020-02-22 Thread Alex Van Boxel
I've assigned it too you. If you create a PR you can add me as reviewer. _/ _/ Alex Van Boxel On Sat, Feb 22, 2020 at 9:14 AM Jozef Vilcek wrote: > I have filed https://issues.apache.org/jira/browse/BEAM-9360 for this. > > Seems like I am not able to self assign this, can I get rig

Re: [VOTE][BIP-1] Beam Schema Options

2020-02-21 Thread Alex Van Boxel
+1 (non-binding) I assume I can vote on my own proposal :-) _/ _/ Alex Van Boxel On Fri, Feb 21, 2020 at 6:36 AM Jean-Baptiste Onofre wrote: > +1 (binding) > > Very interesting. It remembers me when we start the discussion about > schema support ;) > > Regards > JB >

[VOTE][BIP-1] Beam Schema Options

2020-02-19 Thread Alex Van Boxel
votes. _/ _/ Alex Van Boxel

Re: Custom 2.20 failing on Dataflow: what am I doing wrong?

2020-02-19 Thread Alex Van Boxel
mpression that this will benefit us all at some point. > > Thanks, > > > > On Tue, Feb 18, 2020 at 9:46 AM Alex Van Boxel wrote: > >> Thanks everyone. This really helped a lot. I used Gleb's tip to make it >> work. Successfully validated my Pull Requests against Dataflow! >>

DynamicMessage (proto) schema support PR review please

2020-02-19 Thread Alex Van Boxel
for going back from Row to Proto). I like to make it for the 2.20. Don't mind the failed test as it's again a flaky test that fails. Thanks. _/ _/ Alex Van Boxel

Re: [DISCUSS] BIP reloaded

2020-02-18 Thread Alex Van Boxel
So how should we do the voting now? Ask for a vote on BIP-1, but what of open issues... could people add open issues, or should this be done before the vote? _/ _/ Alex Van Boxel On Mon, Feb 17, 2020 at 4:29 PM Jan Lukavský wrote: > Hi Alex, > > +1, I think that the current structur

thrift command missing on Jenkins?

2020-02-18 Thread Alex Van Boxel
I think the Thrift PR broke master. I rebased (and see other new PR's) fail with: * What went wrong:*15:30:40* Execution failed for task ':sdks:java:io:thrift:compileThrift'.*15:30:40* > A problem occurred starting process 'command 'thrift'' _/ _/ Alex Van Boxel

Re: [ANNOUNCE] New committer: Alex Van Boxel

2020-02-18 Thread Alex Van Boxel
Thank you everyone! _/ _/ Alex Van Boxel On Tue, Feb 18, 2020 at 7:05 PM wrote: > Congrats Alex! > Jan > > > Dne 18. 2. 2020 18:46 napsal uživatel Thomas Weise : > > Congratulations! > > > On Tue, Feb 18, 2020 at 8:33 AM Ismaël Mejía wrote: > > Congrats

Re: Custom 2.20 failing on Dataflow: what am I doing wrong?

2020-02-18 Thread Alex Van Boxel
Thanks everyone. This really helped a lot. I used Gleb's tip to make it work. Successfully validated my Pull Requests against Dataflow! _/ _/ Alex Van Boxel On Mon, Feb 17, 2020 at 11:55 PM Brian Hulette wrote: > I think if you update past [1] this will go away. We had to build a

Re: Custom 2.20 failing on Dataflow: what am I doing wrong?

2020-02-16 Thread Alex Van Boxel
Yes, running it manually with the normal parameters as I do for production Dataflow. I'm probably a bit ignorant on that, and I probably need to provide my own worker. Thanks for the hint... I'll dive into that. _/ _/ Alex Van Boxel On Mon, Feb 17, 2020 at 8:16 AM Reuven Lax wrote

Re: Custom 2.20 failing on Dataflow: what am I doing wrong?

2020-02-16 Thread Alex Van Boxel
Digging further in the traces, it seems like a result of changes to the model: Caused by: java.lang.ClassNotFoundException: org.apache.beam.model.pipeline.v1.StandardWindowFns$SessionsPayload$Enum I see changes by Lukasz Cwik. Will this be a problem for the release? _/ _/ Alex Van Boxel

Custom 2.20 failing on Dataflow: what am I doing wrong?

2020-02-16 Thread Alex Van Boxel
org.apache.beam.runners.dataflow.worker.repackaged.org.apache.beam.runners.core.construction.WindowingStrategyTranslation 1. Am I doing something wrong? _/ _/ Alex Van Boxel

Re: [DISCUSS] BIP reloaded

2020-02-16 Thread Alex Van Boxel
natives", how it would look like, > so that one can make a decision even when not being involved with schema on > a daily basis. Your explanation is probably well understood by people who > are experts in the area, but maybe might somewhat limit the audience. > > What do you think? &

Re: Labels on PR

2020-02-15 Thread Alex Van Boxel
gt;>> For an example see the attachment image (hope it works). >>>> [image: Peek 2020-02-15 00-30.gif] >>>> >>>> >>>> >>>> >>>> On Fri, Feb 14, 2020 at 3:40 AM Kyle Weaver >>>> wrote: >>>> >>>&g

Re: Labels on PR

2020-02-12 Thread Alex Van Boxel
was even thinking of adding more text in the description as it could help new contributors to identify something they could help with. It's also nice when you hover over the label. So, could you exactly pinpoint where you see a problem? _/ _/ Alex Van Boxel On Wed, Feb 12, 2020 at 10:22 PM Ismaël

Re: Labels on PR

2020-02-12 Thread Alex Van Boxel
Ismael, I saw that you removed the prefix. I still want to have some grouping between the subtypes, so I color coded them. I also added all the labels in the file. We now have 62 labels. _/ _/ Alex Van Boxel On Wed, Feb 12, 2020 at 12:03 PM Ismaël Mejía wrote: > Forgot to mention, older

Re: Labels on PR

2020-02-11 Thread Alex Van Boxel
are useful during the life-cycle of a PR. _/ _/ Alex Van Boxel On Tue, Feb 11, 2020 at 9:46 AM Ismaël Mejía wrote: > Let some comments too, let's keep the discussion on refinements in the PR. > > On Tue, Feb 11, 2020 at 9:13 AM jincheng sun > wrote: > >> I left com

Re: Labels on PR

2020-02-10 Thread Alex Van Boxel
I've opened a PR and a ticket with INFRA. PR: https://github.com/apache/beam/pull/10824 _/ _/ Alex Van Boxel On Tue, Feb 11, 2020 at 6:57 AM jincheng sun wrote: > +1. Autolabeler seems really cool and it seems that it's simple to > configure and set up. > > Best, > Jinch

Re: Labels on PR

2020-02-10 Thread Alex Van Boxel
Ha, cool. I'll have a look at the autolabeler. The infra stuff is not something I've looked at... I'll dive into that. _/ _/ Alex Van Boxel On Mon, Feb 10, 2020 at 11:49 AM Ismaël Mejía wrote: > +1 > > You don't need to write your own action, there is already one autolabeler &g

Re: [DISCUSS] BIP reloaded

2020-02-09 Thread Alex Van Boxel
a = motivation b => *added current state in Beam* c = alternatives d = implementation *(I prefer this to define before the alternatives)* e = *rest of document?* _/ _/ Alex Van Boxel On Sun, Feb 9, 2020 at 7:50 PM Jan Lukavský wrote: > It's absolutely fine. :-) I think that the

Re: [DISCUSS] BIP reloaded

2020-02-09 Thread Alex Van Boxel
I'm sorry, I stole the number 1 from you. Feel free to give suggestions on the form, so we can get a good template for further BIPs _/ _/ Alex Van Boxel On Sun, Feb 9, 2020 at 6:43 PM Jan Lukavský wrote: > Hi Alex, > > this is cool! Thanks for pushing this topic forward! > >

Re: [DISCUSS] BIP reloaded

2020-02-09 Thread Alex Van Boxel
BIP-1 is available here: https://cwiki.apache.org/confluence/display/BEAM/%5BBIP-1%5D+Beam+Schema+Options _/ _/ Alex Van Boxel On Sat, Feb 1, 2020 at 9:11 PM Kenneth Knowles wrote: > Sounds great. If you scrape recent dev@ for proposals that are not yet > implemented, I think you wil

Re: big data blog

2020-02-08 Thread Alex Van Boxel
Great, we need more Beam articles! Feel free to keep posting updates to the mailing list. :) On Fri, Feb 7, 2020, 16:24 Etienne Chauchot wrote: > Hi all, > > FYI, I just started a blog around big data technologies and for now it > is focused on Beam. > > https://echauchot.blogspot.com/ > > Feel

Re: [PROPOSAL] Beam Schema Options

2020-02-08 Thread Alex Van Boxel
>>> schema-aware Beam IOs are like Tables in a database, other PCollections >>>>> are >>>>> like intermediate results in a query. In a database, Tables can be defined >>>>> with some DDL and have schema-level or column-level options that change &g

Re: Schema Aware PCollection stabilization

2020-02-05 Thread Alex Van Boxel
Oh, I thought it was. I've set anyone with a link can edit... as long as there is no spam detected. Added you explicitly. _/ _/ Alex Van Boxel On Thu, Feb 6, 2020 at 2:01 AM Brian Hulette wrote: > Can you open up the document for commenting or editing? > > On Wed, Feb 5, 2020 at 4:4

Schema Aware PCollection stabilization

2020-02-05 Thread Alex Van Boxel
cument with those tickets as well. Thanks. _/ _/ Alex Van Boxel

Re: [PROPOSAL] Beam Schema Options

2020-02-05 Thread Alex Van Boxel
powerful as it is not limited on fields. _/ _/ Alex Van Boxel On Wed, Jan 29, 2020 at 4:55 AM Kenneth Knowles wrote: > Using schema types for the metadata values is a nice touch. > > Are the options expected to be common across many fields? Perhaps the name > should be a URN to m

What's up with the PreCommit_Java_Examples_Dataflow_Phrase?

2020-01-31 Thread Alex Van Boxel
It seems to be in an "terminated in state UNRECOGNIZED but did not return a failure reason" https://builds.apache.org/job/beam_PreCommit_Java_Examples_Dataflow_Phrase/ _/ _/ Alex Van Boxel

[PROPOSAL] Beam Schema Options

2020-01-26 Thread Alex Van Boxel
points to 2 pull requests with the implementation, one for the API another for translating proto options to beam options. Thanks to Brian Hulette and Reuven Lax for the initial feedback. All feedback is welcome. _/ _/ Alex Van Boxel

Re: [DISCUSS] Integrate Google Cloud AI functionalities

2020-01-14 Thread Alex Van Boxel
. Also note that although gRPC is vendored, the google IO's do still have their own dependency on gRPC and this is the biggest reason for trouble. _/ _/ Alex Van Boxel On Wed, Jan 15, 2020 at 1:18 AM Luke Cwik wrote: > It depends on what language the client libraries are expo

Re: [RESULT] [VOTE] Beam's Mascot will be the Firefly (Lampyridae)

2020-01-14 Thread Alex Van Boxel
I hope for the mascot will be simple enough so everyone can draw it and adapt. The mascot will be license free right... so you don't need to pay the graphic artist for every use of the mascot? _/ _/ Alex Van Boxel On Tue, Jan 14, 2020 at 8:26 PM Aizhamal Nurmamat kyzy wrote: > Thanks K

Re: master on Dataflow with schema aware PCollections stuck

2020-01-13 Thread Alex Van Boxel
It's indeed the first Logical identifier with Row base type. The UUID is generated from the name of the class, but doing it in code (from a string) you need to create bytes from the string, then a UUID. _/ _/ Alex Van Boxel On Mon, Jan 13, 2020 at 10:40 PM Brian Hulette wrote: > I gu

Re: master on Dataflow with schema aware PCollections stuck

2020-01-13 Thread Alex Van Boxel
://github.com/apache/beam/pull/10486 _/ _/ Alex Van Boxel On Mon, Jan 13, 2020 at 8:40 PM Alex Van Boxel wrote: > So I think the following happens: > >1. the schema tree is initialized at construction time. The tree get >serialized and send to the workers >2. the work

Re: master on Dataflow with schema aware PCollections stuck

2020-01-13 Thread Alex Van Boxel
-initialized without the UUID* (as it was never serialized) 3. this is why setting a fixed UUID at static initialization works So solution is - as tested, se a fixed UUID - make the schema not static in the logical type _/ _/ Alex Van Boxel On Mon, Jan 13, 2020 at 8:08 PM Reuven Lax wrote

Re: master on Dataflow with schema aware PCollections stuck

2020-01-13 Thread Alex Van Boxel
for LogicalType. Setting it to a fixed value makes it run on Dataflow. Note that the pipeline works perfect on DirectRunner. _/ _/ Alex Van Boxel On Mon, Jan 13, 2020 at 6:06 PM Reuven Lax wrote: > I don't think that should be the case. Also SchemaCoder will automatically > set th

Re: master on Dataflow with schema aware PCollections stuck

2020-01-13 Thread Alex Van Boxel
OK, I've rechecked everything and eventually found the problem. The problem is when you use a LogicalType backed back a Row, then the UUID needs to be set to make it work. (this is the case for Proto based Timestamps). I'll create a fix. _/ _/ Alex Van Boxel On Mon, Jan 13, 2020 at 8:36 AM

Re: master on Dataflow with schema aware PCollections stuck

2020-01-12 Thread Alex Van Boxel
BTW. This is not a support ticket, I'm wondering if we are aware and we're missing schema aware integration tests as well. _/ _/ Alex Van Boxel On Sun, Jan 12, 2020 at 10:43 AM Alex Van Boxel wrote: > Hey all, > > anyone tried master with a *schema aware pipeline* on Dataflow? I'm

master on Dataflow with schema aware PCollections stuck

2020-01-12 Thread Alex Van Boxel
can get help with Cloud Dataflow at because I got this I wanted to see if master (without my changes) also have the same behaviour. It's easy to simulate: Just read for BigQuery with: BigQueryIO.readTableRowsWithSchema() it works with the classic: readTableRows(). _/ _/ Alex Van Boxel

Re: [DISCUSS] BIP reloaded

2020-01-09 Thread Alex Van Boxel
, maybe an amendment needs to be made (or correct). Corrections could be added to a *changelog* in the beginning. - If a new proposal supersedes on, both should be linked - Most importantly: Who can act as *owner* end reviewers for this proposal. _/ _/ Alex Van Boxel On Thu, Jan 9, 2020

Re: Flaky Java warning/error inventory (cannot find symbol)

2020-01-05 Thread Alex Van Boxel
/beam_PreCommit_Java_Phrase/1595/ https://builds.apache.org/job/beam_PreCommit_Java_Phrase/1594/ https://builds.apache.org/job/beam_PreCommit_Java_Phrase/1588/ _/ _/ Alex Van Boxel On Mon, Jan 6, 2020 at 12:56 AM Tomo Suzuki wrote: > I’m interested in that problem. “Cannot find symbol” problem is usually >

Flaky Java warning/error inventory (cannot find symbol)

2020-01-05 Thread Alex Van Boxel
ant to get a green build to call out reviewers, but it's hard with these random failures. _/ _/ Alex Van Boxel

Re: [VOTE] Beam's Mascot will be the Firefly (Lampyridae)

2019-12-13 Thread Alex Van Boxel
+1 On Fri, Dec 13, 2019, 05:58 Kenneth Knowles wrote: > Please vote on the proposal for Beam's mascot to be the Firefly. This > encompasses the Lampyridae family of insects, without specifying a genus or > species. > > [ ] +1, Approve Firefly being the mascot > [ ] -1, Disapprove Firefly being

Re: Version Beam Website Documentation

2019-12-05 Thread Alex Van Boxel
It seems also be too complex for the Google Crawler as well. A lot of times I arrived on documentation on an older version of a product when I search (aka Google) for something. _/ _/ Alex Van Boxel On Fri, Dec 6, 2019 at 6:20 AM Kenneth Knowles wrote: > Since we are not making break

Re: [VOTE] Beam Mascot animal choice: vote for as many as you want

2019-11-19 Thread Alex Van Boxel
[ ] Beaver [ ] Hedgehog [ ] Lemur [ ] Owl [ ] Salmon [ ] Trout [ ] Robot dinosaur [ X] Firefly [ ] Cuttlefish [ ] Dumbo Octopus [ X] Angler fish _/ _/ Alex Van Boxel On Wed, Nov 20, 2019 at 3:57 AM Reza Rokni wrote: > [ ] Beaver > [ ] Hedgehog > [ ] Lemur > [ ] Owl > [X] Sal

Re: [PROPOSAL] Add support for writing flattened schemas to pubsub

2019-11-17 Thread Alex Van Boxel
ot;)*, name VARCHAR, age INTEGER ) TYPE 'pubsub' LOCATION 'projects/my-project/topics/my-topic' _/ _/ Alex Van Boxel On Sat, Nov 16, 2019 at 7:58 PM Kenneth Knowles wrote: > Big +1 from me. > > Nice explanation. This makes a lot of sense. Much simpler to understand &g

Re: [Discuss] Beam mascot

2019-11-15 Thread Alex Van Boxel
I like the "Angler fish" as well... it certainly faster then a cuttlefish.... _/ _/ Alex Van Boxel On Fri, Nov 15, 2019 at 9:57 PM Kirill Kozlov wrote: > Angler fish? Found a few animated examples that may look interesting [1, > 2, 3, 4]. > [1] https://ww

Re: [Discuss] Beam Summit 2020 Dates & locations

2019-11-07 Thread Alex Van Boxel
For date wise, I'm wondering why we should switching the Europe and NA one, this would mean that the Berlin and the new EU summit would be almost 1.5 years apart. _/ _/ Alex Van Boxel On Thu, Nov 7, 2019 at 8:43 PM Ahmet Altay wrote: > I prefer bay are for NA summit. My reason

Re: Apache Pulsar connector for Beam

2019-10-26 Thread Alex Van Boxel
Hey Taher, do you have a Jira account? Then I will assign the ticket to you. I made the ticket because we should have one, feel free to take the lead on this one. _/ _/ Alex Van Boxel On Fri, Oct 25, 2019 at 9:35 PM Taher Koitawala wrote: > I would be interested in contributing to the Pul

Need help reviewing Protobuf enhancement

2019-09-04 Thread Alex Van Boxel
PR's are aligned. Thanks for the help. _/ _/ Alex Van Boxel

Re: Cassandra flaky on Jenkins?

2019-09-04 Thread Alex Van Boxel
Thanks, got my PR green now, _/ _/ Alex Van Boxel On Wed, Sep 4, 2019 at 3:02 PM David Morávek wrote: > Hi, temporarily disabling the test > <https://github.com/apache/beam/pull/9470>, until BEAM-8025 > <https://jira.apache.org/jira/browse/BEAM-8025> is resolved (ma

Cassandra flaky on Jenkins?

2019-09-03 Thread Alex Van Boxel
nit/org.apache.beam.sdk.io.cassandra/CassandraIOTest/classMethod/> _/ _/ Alex Van Boxel

RowWithGetters, FieldValueGetter being Serializable?

2019-08-28 Thread Alex Van Boxel
blocking my implementation of Protobuf (as it wont pass SpotBug complaining on non serializable fields in FieldValueGetter). _/ _/ Alex Van Boxel

Re: Is it too late to switch to Java 8 time for the schema aware Row and Beam SQL?

2019-08-28 Thread Alex Van Boxel
Thanks, how will ZetaSQL support higher precision as the input in general will be Instant anyway. Will it rely on the "pending" standardized logical types? _/ _/ Alex Van Boxel On Mon, Aug 19, 2019 at 7:02 AM Rui Wang wrote: > However, more challengings come from: > >

Re: Support ZetaSQL as a new SQL dialect in BeamSQL

2019-08-22 Thread Alex Van Boxel
This is a very informative thread. I would love that a lot of this information and reasoning end up in the documentation. _/ _/ Alex Van Boxel On Wed, Aug 21, 2019 at 9:17 PM Rui Wang wrote: > Thanks everyone! Now Beam ZetaSQL is merged into Beam repo! > > > -Rui > > On

Is it too late to switch to Java 8 time for the schema aware Row and Beam SQL?

2019-08-17 Thread Alex Van Boxel
(nanoseconds), but I also notice it with BigQuery (milliseconds). Suggestions? _/ _/ Alex Van Boxel

Re: Help triaging Jira issues

2019-06-05 Thread Alex Van Boxel
Hey Kenneth, I help out. I'm planning to contribute more on Beam and it seems to be ideal to keep up-to-date with the project. _/ _/ Alex Van Boxel On Wed, Jun 5, 2019 at 6:46 PM Kenneth Knowles wrote: > Hi all, > > I am requesting help in triaging incoming issues. I made a se

Re: Timer support in Flink

2019-05-30 Thread Alex Van Boxel
Oh... you can expand the matrix. Never saw that, this could indeed be better. So it isn't you. _/ _/ Alex Van Boxel On Thu, May 30, 2019 at 12:24 PM Reza Rokni wrote: > PS, until it was just pointed out to me by Max, I had missed the (expand > details) clickable link in the capability

Re: Schema is not final... is it allowed to override

2019-05-16 Thread Alex Van Boxel
Everything will be ready for the Beam Summit :-) _/ _/ Alex Van Boxel On Wed, May 15, 2019 at 3:26 AM Reuven Lax wrote: > > > *From: *Alex Van Boxel > *Date: *Tue, May 14, 2019 at 3:38 PM > *To: *ML Beam/Dev > > ProtoBuf and certainly the Descriptor is a challenging beast, an

Re: [VOTE] Remove deprecated Java Reference Runner code from repository.

2019-05-15 Thread Alex Van Boxel
+1 (best commits are the once that remove code :-) _/ _/ Alex Van Boxel On Wed, May 15, 2019 at 2:04 PM Manu Zhang wrote: > +1 > > On Wed, May 15, 2019 at 7:57 PM Maximilian Michels wrote: > >> +1 >> >> On 15.05.19 13:19, Robert Bradshaw wrote: >&

Re: Schema is not final... is it allowed to override

2019-05-14 Thread Alex Van Boxel
was allowed to be extensible... I'll add a PR to make Schema final, should be a no brainer. _/ _/ Alex Van Boxel On Tue, May 14, 2019 at 8:05 PM Reuven Lax wrote: > Can you explain what you're trying to do? I don't think that embedding the > proto descriptor in the schema is a great way

Schema is not final... is it allowed to override

2019-05-14 Thread Alex Van Boxel
would be able to store it in Schema I can reuse it in to/fromRow. Thoughts? _/ _/ Alex Van Boxel

Help reviewing DynamicMessage protobuf support PR

2019-05-10 Thread Alex Van Boxel
. Thanks. _/ _/ Alex Van Boxel

Re: [DISCUSS] Portability representation of schemas

2019-05-09 Thread Alex Van Boxel
OK, fair. This is parallel how timestamp are implemented in protobuf. Then it's important (and I'll join the design doc) that we have a list of standard logical types. _/ _/ Alex Van Boxel On Thu, May 9, 2019 at 4:11 PM Reuven Lax wrote: > > > On Thu, May 9, 2019 at 6:34 AM Alex

Re: [DISCUSS] Portability representation of schemas

2019-05-09 Thread Alex Van Boxel
y protobuf implementation. _/ _/ Alex Van Boxel On Thu, May 9, 2019 at 10:05 AM Kenneth Knowles wrote: > This is a huge development. Top posting because I can be more compact. > > I really think after the initial idea converges this needs a design doc > with goals and alternatives. It

Request wiki access

2019-04-28 Thread Alex Van Boxel
it seems the IntelliJ guide is a bit behind, I like to correct some small changes. Can somebody give me access to the wiki. I've linked my JIRA account with the wiki one. This is my login. Thanks: username: *alexvanboxel* _/ _/ Alex Van Boxel

Re: The full list of proposals / prototype documents

2019-01-18 Thread Alex Van Boxel
that I had access to, from the past I don't know anymore of course. _/ _/ Alex Van Boxel On Fri, Jan 18, 2019 at 6:13 PM Alexey Romanenko wrote: > Hi Alex, > > Hmm, afaik, this is mostly google docs file which shared with anyone who > knows the link. > Could you send here the names of

Re: The full list of proposals / prototype documents

2019-01-18 Thread Alex Van Boxel
Hey Alexey, I see that a lot (well, I tried 2) proposals require access approval. Should that be the case? _/ _/ Alex Van Boxel On Fri, Jan 18, 2019 at 4:51 PM Alexey Romanenko wrote: > I’m sorry but I forgot to mention that the whole list could be found here: > https://beam.apac

Re: AvroIO read from unknown schema to generic record.

2019-01-12 Thread Alex Van Boxel
to decode without the schema. So if you really want todo flexible messages, I would read it binary, message per message and handle your schema switching into a DoFn. _/ _/ Alex Van Boxel On Sat, Jan 12, 2019 at 7:44 PM Niel Markwick wrote: > Is there a reason why don't we have an AvroIO rea

Re: Beam Summit community feedback

2018-10-07 Thread Alex Van Boxel
to collaborate on that topic let me know. _/ _/ Alex Van Boxel On Fri, Oct 5, 2018 at 4:05 PM Maximilian Michels wrote: > Hi, > > What do you think about collecting some of the feedback from the > community at Beam Summit last week? Here's what I've come across: > > > * The Kube

Re: SplittableDoFn

2018-10-03 Thread Alex Van Boxel
Yes, but we need at least Mongo 4.0 to make it production ready. I wouldn't let anyone work with anything less because you can't checkpoint). I'm waiting till our test cluster is 4.0 to continue on this. _/ _/ Alex Van Boxel On Wed, Oct 3, 2018 at 9:43 AM Ismaël Mejía wrote: > Hello A

Re: SplittableDoFn

2018-10-02 Thread Alex Van Boxel
/presentation/d/1dSc6oKh5pZItQPB_QiUyEoLT2TebMnj-pmdGipkVFPk/edit?usp=sharing (quite proud of the animations though ;-) _/ _/ Alex Van Boxel On Thu, Sep 27, 2018 at 12:04 AM Lukasz Cwik wrote: > Reuven, just inside the restriction tracker itself which is scoped per > executing SplittableDoFn.