Re: [VOTE] Code Review Response-time SLO

Agree. Having a best effort is fine but not a strong commitment, because, like in any Apache project, lot of people works on the project in their spare time, or even during business time, with lower priority. Regards JB On 07/06/2018 16:11, Thomas Weise wrote: > I like the idea of speeding up

Re: [VOTE] Apache Beam, version 2.5.0, release candidate #1

Thanks for the details Etienne ! The good news is that the artifacts seem OK and the overall Nexmark results are consistent with the 2.4.0 release ones. I'm starting a complete review using the beam-samples as well. Regards JB On 07/06/2018 16:14, Etienne Chauchot wrote: > Hi, > I've just run

Re: [VOTE] Code Review Response-time SLO

Yeah, that sounds reasonable to me. We just have to provide an update about the review status, not necessary the review itself ;) Regards JB On 07/06/2018 16:25, Kenneth Knowles wrote: > Yea, this was what I mean by my question "What is the action when an SLO > is missed?" > > For example, the

Re: [VOTE] Code Review Response-time SLO

Yea, this was what I mean by my question "What is the action when an SLO is missed?" For example, the way we implement could just be a line in the contribution guide like this: "We aspire to respond quickly to all pull requests, even if it can take some time to review them. If you have not heard

Re: [Proposal] Apache Beam's Public Project Roadmap

Yea, what you call overview/list style page is what I want to add. Just a few, and focused on modules that have an interesting long-term trajectory, like SDKs, DSLs, runners, or meaningful collections of connectors. I want it to be reasonably consumable by users as well as contributors. To me that

Re: [VOTE] Apache Beam, version 2.5.0, release candidate #1

Hi,I've just run the nexmark queries on v2.5.0-RC1 tagWhat we can notice: - query 3 (exercises CoGroupByKey, state and timer) shows different output with DR between batch and streaming and with the other runners => I compared with v2.4 there were still these differences but with different output

Jenkins build is back to normal : beam_SeedJob #1907

See

Re: [Proposal] Apache Beam's Public Project Roadmap

I like the idea of presenting strategic and user impacting initiatives (ongoing and future) in an easy to consume format, but also think that "roadmap" sounds a bit too corporate... Other projects, including Flink, Kafka and Spark have the concept of "Improvement Proposal" that could serve this

Re: [Proposal] Apache Beam's Public Project Roadmap

OK, I have one note: what about longer-term "roadmap", in quotes because that just means it is "ideas for the future". Like the Spark runner could have a written roadmap that includes lots of TODOs that extend far into the future. It gives a menu of things to think about when filling this out each

Re: [VOTE] Policies for managing Beam dependencies

+1 to these. Thanks for clarifying! Kenn On Wed, Jun 6, 2018 at 10:40 PM Chamikara Jayalath wrote: > Hi Kenn, > > On Wed, Jun 6, 2018 at 8:14 PM Kenneth Knowles wrote: > >> +0.5 >> >> I like the spirit of these policies. I think they need a little wording >> work. Comments inline. >> >> On

Re: Beam SQL Pipeline Options

Hi Arun, If you are looking to connect to a SQL database from Beam Java code, then I you might want JdbcIO [1] with a PostgreSQL JDBC driver [2]. This is not currently wired up to Beam SQL specially; you have to connect in Java code. It would be a nice contribution. Kenn [1]

Build failed in Jenkins: beam_SeedJob #1906

See -- GitHub pull request #5576 of commit 2a39884772252f45fbdf9121c5e3fbc4fb5a0730, no merge conflicts. Setting status of 2a39884772252f45fbdf9121c5e3fbc4fb5a0730 to PENDING with url

Re: [VOTE] Code Review Response-time SLO

Yes, I agree with that, I thought it was the delay to start the review. +1 then on the delay to answer Etienne Le jeudi 07 juin 2018 à 11:02 +0100, Reuven Lax a écrit : > However I think it's reasonable to expect a response within 3 days, even if > that response is not an actual review. For >

Re: [VOTE] Code Review Response-time SLO

I like the idea of speeding up code review and promptly responding to PRs, since this is more motivating to the contributors, especially those that are not committers. However, the notion of SLO or target response time is something that works within a company with dedicated resources. It may not

Re: [VOTE] Policies for managing Beam dependencies

+1 On Thu, Jun 7, 2018 at 5:18 AM Kenneth Knowles wrote: > +1 to these. Thanks for clarifying! > > Kenn > > On Wed, Jun 6, 2018 at 10:40 PM Chamikara Jayalath > wrote: > >> Hi Kenn, >> >> On Wed, Jun 6, 2018 at 8:14 PM Kenneth Knowles wrote: >> >>> +0.5 >>> >>> I like the spirit of these

Re: [Proposal] Apache Beam's Public Project Roadmap

> > Just to unwrap BIP: How would it differ from (a) JIRA ticket and (b) a > proposal gdoc. Just for history, when Beam started, the (much smaller, and > now very different) community chose JIRA instead of BIP. How would it > relate to (c) the overview/list style page and (d) the quarterly

Re: Proposal: keeping precommit times fast

Would I need a vote on installing this plugin, or can I just open a ticket to infra? On Wed, Jun 6, 2018, 16:18 Robert Bradshaw wrote: > Even if it's not perfect, seems like it'd surely be a net win (and > probably a large one). Also, the build cache should look back at more than > just the

Build failed in Jenkins: beam_SeedJob #1910

See -- GitHub pull request #5576 of commit 624fdae40b7242500e945e5aa2eafb19c534b713, no merge conflicts. Setting status of 624fdae40b7242500e945e5aa2eafb19c534b713 to PENDING with url

Jenkins build is back to normal : beam_SeedJob #1912

See

Build failed in Jenkins: beam_SeedJob #1911

See -- GitHub pull request #5576 of commit c85bfd200b8ee843608a2a28bfc429c52f2fc384, no merge conflicts. Setting status of c85bfd200b8ee843608a2a28bfc429c52f2fc384 to PENDING with url

Re: Proposal: keeping precommit times fast

I don't see any downside to enabling this plugin; we previously filed BEAM-4400 [1] for this work. I believe the next steps would be working with Infra on enabling it. [1] https://issues.apache.org/jira/browse/BEAM-4400 On Thu, Jun 7, 2018 at 9:29 AM Udi Meiri wrote: > Would I need a vote on

Re: Design Proposal: Beam-Site Automation Reliability

Sounds good; I'm really excited about these changes Scott. Thanks for taking this on! On Tue, Jun 5, 2018 at 4:00 PM Scott Wegner wrote: > Thanks everyone; I've responded to feedback in the doc [1] and I believe > we've reached consensus. I've added implementation tasks in JIRA > under

Re: Proposal: keeping precommit times fast

No, this isn't the kind of thing that should require a vote (unless someone really wants a vote). On Thu, Jun 7, 2018 at 9:29 AM Udi Meiri wrote: > Would I need a vote on installing this plugin, or can I just open a ticket > to infra? > > On Wed, Jun 6, 2018, 16:18 Robert Bradshaw wrote: > >>

Beam Cookbook?

I'm looking at assembling a physical book along the lines of "Apache Beam Cookbook", though might take a different approach to topic (if realize there is a better hole to fill or something that needs more attention before that). I believe many could benefit from more substantive write-ups and

Re: [Proposal] Apache Beam's Public Project Roadmap

I got asked this offline, so clarifying: FLIP = Flink Improvement Proposal [1] KIP = Kafka Improvement Proposal [2] SPIP = Spark Improvement Proposal [3] ONIP = OpenNLP Improvement Proposal [4] Zeppelin Improvement Proposal [5] inexplicably exists but did not use the sweet acronym And that's all

[DISCUSS] Use Confluence wiki for non-user-facing stuff

Hi all, I've been in half a dozen conversations recently about whether to have a wiki and what to use it for. Some things I've heard: - "why is all this stuff that users don't care about here?" - "can we have a lighter weight place to put technical references for contributors" So I want to

Re: Announcement & Proposal: HDFS tests on large cluster.

Done. Changed the size of the io-datastores kubernetes cluster in apache-beam-testing to 3 nodes. On Thu, Jun 7, 2018 at 1:45 AM Kamil Szewczyk wrote: > Hi, > > the node pool size of io-datastores kubernetes cluster in > apache-beam-testing project must be changed from 1 -> 3 (or other value).

Re: [Proposal] Apache Beam's Public Project Roadmap

On Thu, Jun 7, 2018 at 1:11 PM, Kenneth Knowles wrote: > I got asked this offline, so clarifying: > > FLIP = Flink Improvement Proposal [1] > KIP = Kafka Improvement Proposal [2] > SPIP = Spark Improvement Proposal [3] > ONIP = OpenNLP Improvement Proposal [4] > Zeppelin Improvement Proposal [5]

Re: Announcement & Proposal: HDFS tests on large cluster.

We still use Jenkins machines to execute the test but data stores are hosted in Kubernetes. On Thu, Jun 7, 2018 at 1:35 PM Pablo Estrada wrote: > Just out of curiosity: This does not use the Jenkins machines then? > -P. > > On Thu, Jun 7, 2018 at 1:33 PM Alan Myrvold wrote: > >> Done. Changed

Re: Announcement & Proposal: HDFS tests on large cluster.

Just out of curiosity: This does not use the Jenkins machines then? -P. On Thu, Jun 7, 2018 at 1:33 PM Alan Myrvold wrote: > Done. Changed the size of the io-datastores kubernetes cluster in > apache-beam-testing to 3 nodes. > > On Thu, Jun 7, 2018 at 1:45 AM Kamil Szewczyk wrote: > >> Hi, >>

Re: [Proposal] Apache Beam's Public Project Roadmap

Isn't that a bib? On Thu, Jun 7, 2018 at 1:33 PM Thomas Weise wrote: > > > On Thu, Jun 7, 2018 at 1:11 PM, Kenneth Knowles wrote: > >> I got asked this offline, so clarifying: >> >> FLIP = Flink Improvement Proposal [1] >> KIP = Kafka Improvement Proposal [2] >> SPIP = Spark Improvement

Re: [DISCUSS] Use Confluence wiki for non-user-facing stuff

+1. It would be very helpful to have dev-facing walkthroughs / technical documentation for relevant aspects of the codebase that aren't user-facing. On Thu, Jun 7, 2018, 1:23 PM Kenneth Knowles wrote: > Hi all, > > I've been in half a dozen conversations recently about whether to have a > wiki

Re: Announcement & Proposal: HDFS tests on large cluster.

Hi, the node pool size of io-datastores kubernetes cluster in apache-beam-testing project must be changed from 1 -> 3 (or other value). @Alan Myrvold was already helpful with kubernetes cluster settings so far, but I am not aware who made decisions regarding that as this will increase monthly

Re: [VOTE] Code Review Response-time SLO

+1 on the purpose, but I think 3 business days for proposal 1 are over-optimistic. As an example I saw PRs taking weeks before the review started. IMHO, I think you should consider raising up the 3 days bar a bit. BestEtienne Le mardi 05 juin 2018 à 15:04 -0700, Andrew Pilloud a écrit : > The

Re: [VOTE] Code Review Response-time SLO

However I think it's reasonable to expect a response within 3 days, even if that response is not an actual review. For instance, the response could be "I'm extremely busy right now, and it will take another week for me to get to this." On Thu, Jun 7, 2018, 10:40 AM Etienne Chauchot wrote: > +1