Re: Filtered Pre-commit triggering is BACK!

It failed again. I'm pinging infra a new time. Sorry about that. Regards JB Le 29 juin 2018 à 08:52, à 08:52, Reuven Lax a écrit: >"retest this please" also seems to not be working right now. > >On Thu, Jun 28, 2018 at 5:49 PM Reuven Lax wrote: > >> I've pushed changes that definitely affect

Re: Proposing interactive beam runner

It seems there is support in the community -- we have a PR out for review ( https://github.com/apache/beam/pull/5818), which provides a baseline InteractiveRunner on which we can build further. PTAL. Thanks, ~Harsh. On Fri, Jun 8, 2018 at 5:57 PM Sindy Li wrote: > Hello, > > We were exploring

Re: Filtered Pre-commit triggering is BACK!

My PR with only changes in Java also triggered Python ("Run Python PreCommit") and Go ("Run Go PreCommit") tests. Is that the expected behavior? https://github.com/apache/beam/pull/5787 On Wed, Jun 27, 2018 at 5:04 PM Ahmet Altay wrote: > Two of my PRs with python changes did not trigger any

Re: Python SDK: pytest vs nose

Huge +1 to move off from Nose. The last commit to Nose is two years ago which leaves a lot of holes that makes supporting some Beam infra features painful. Vote on pytest since it's more actively maintained than Nose2 (release once a month vs only one for Nose2 this year) and rich of user

Re: Python SDK: pytest vs nose

I support trying to move away from nose, as it is no longer maintained. I don't know the state of other libraries like pytest or nose2. Best -P On Thu, Jun 28, 2018 at 3:37 PM Udi Meiri wrote: > https://issues.apache.org/jira/browse/BEAM-3713 > > On Thu, Jun 28, 2018 at 3:35 PM Udi Meiri

Re: Python SDK: pytest vs nose

https://issues.apache.org/jira/browse/BEAM-3713 On Thu, Jun 28, 2018 at 3:35 PM Udi Meiri wrote: > Hi, > I'm currently leaning towards migrating us to pytest, since it has > features like --last-failed, is actively maintained, and seems more > polished in general. > > (Pytest is supposed to be

Python SDK: pytest vs nose

Hi, I'm currently leaning towards migrating us to pytest, since it has features like --last-failed, is actively maintained, and seems more polished in general. (Pytest is supposed to be able to run nose tests, but not all our tests work with it out of the box.) Does anyone have a preference for

Re: [Design Proposal] Improving Beam code review

The main limitation is that only members of the "apache" GitHub organization can be assigned to these fields. Otherwise they would be perfect for tracking both who is doing the review and whose turn it is to take action! I don't know how Github CODEOWNERS behaves if you put a non-member in it.

New JIRA component for GCP Quota issues: gcp-quota

Hi all, Rafael has volunteered to be the point of contact for resolving GCP Quota issues. I've created a JIRA component gcp-quota where he'll be auto-assigned issues. FYI, that's where to send them. And thank, Rafael! Kenn

Re: [Design Proposal] Improving Beam code review

That's a good document. I have a general question: Is there a reason why we do not assign reviewer/assignee/labels to PRs? I see that we add @reviewer comments, but never actually assign reviewers. Those are good tools that Github can use as filters for you. --Mikhail Have feedback

Re: [Design Proposal] Improving Beam code review

Thank you Huygaa. This document looks good to me. I think auto-assigning PRs could significantly help especially with first time contributors. It could also give us a chance to distribute reviews in a more balanced way across committers. On Thu, Jun 28, 2018 at 7:37 AM, Alexey Romanenko wrote:

Re: Using user developped source in streamline python

Hi I'm completly fine to contribute, let's start with Cloud SQL and Firestore. Get me right: I've already written a working writer and sink. So how to make them native so they will be able to be supported in the streamline pipeline? Just inheritance from the native_io? My sinks inherit from

Re: Help! Beam SQL needs more committer support

I am also happy to happy to help and interested in learning more about Beam SQL. On Thu, Jun 28, 2018 at 11:53 AM, Kai Jiang wrote: > I am very interested in supporting SQL PRs to help you reduce your > bandwidths. > > Best, > Kai > > > On Thu, Jun 28, 2018, 11:52 Austin Bennett > wrote: > >>

Re: Samza runner committer support

I would be happy to help with this. I can prioritize python related changes. For portability related changes I can try to help but I may not be the best person. On Thu, Jun 28, 2018 at 12:33 PM, Xinyu Liu wrote: > Hi, All, > > Our Samza runner has recently been merged to master, and Kenn has

Samza runner committer support

Hi, All, Our Samza runner has recently been merged to master, and Kenn has been extremely instrumental during the whole process, e.g. design decisions, feature requests and code reviews. We would like thank him for all the support he has been given to us! Given Kenn is going to be on leave soon,

Re: Help! Beam SQL needs more committer support

I am very interested in supporting SQL PRs to help you reduce your bandwidths. Best, Kai On Thu, Jun 28, 2018, 11:52 Austin Bennett wrote: > Same as Pablo - you can count me in, though am vastly less experience in > the Beam way, as him, etc. > > > On Thu, Jun 28, 2018 at 11:36 AM, Pablo

Re: Help! Beam SQL needs more committer support

Same as Pablo - you can count me in, though am vastly less experience in the Beam way, as him, etc. On Thu, Jun 28, 2018 at 11:36 AM, Pablo Estrada wrote: > I am always willing to help : ) > I'd say I can look at Python changes, and be a second-place option for > Java changes (some experience

Re: Help! Beam SQL needs more committer support

I am always willing to help : ) I'd say I can look at Python changes, and be a second-place option for Java changes (some experience but not as much on the Java side). Best -P On Thu, Jun 28, 2018 at 10:50 AM Kenneth Knowles wrote: > Hi all, > > I have been reviewing a ton of PRs from Andrew

Help! Beam SQL needs more committer support

Hi all, I have been reviewing a ton of PRs from Andrew (@apilloud), Anton (@akedin), Rui (@amaliujia), and Kai (@vectorijk) and so has Mingmin (@XuMingmin). Beam SQL is really moving!* But I worry that when I am on leave there will be a shortage of committer review availability. I don't think

Re: [DISCUSS] Automation for Java code formatting

It is done. In the same PR I wiped out checkstyle rules that became redundant. There are still quite a few. On Thu, Jun 28, 2018 at 10:02 AM Udi Meiri wrote: > Python already has lint checks: :beam-sdks-python:lint > There are autoformatting tools, which we don't use AFAIK: >

Re: [DISCUSS] Automation for Java code formatting

Python already has lint checks: :beam-sdks-python:lint There are autoformatting tools, which we don't use AFAIK: https://github.com/myint/autoflake https://github.com/hhatto/autopep8 On Thu, Jun 28, 2018 at 12:31 AM Daniel Kulp wrote: > > > On Jun 28, 2018, at 6:01 AM, Kenneth Knowles wrote:

Re: [DISCUSS] Automation for Java code formatting

I don't think so. Checkstyle checks for a number of things beyond simple formatting issues, (i.e. redundant imports). On Thu, Jun 28, 2018 at 12:31 AM Daniel Kulp wrote: > > > On Jun 28, 2018, at 6:01 AM, Kenneth Knowles wrote: > > So, there are a few modes of checkstyle failure that can be

Re: [Design Proposal] Improving Beam code review

Strongly agree with auto assigning code reviewers, I guess this is one of the main issue for first-starters to whom address their PR. Also, I’m totally pro for having review style guide which definitively should help to unify review process and make it more transparent for all. Thanks to last

Jenkins build is back to normal : beam_SeedJob #2109

See

Re: Beam Dependency Ownership

Thanks for this Yifan, I've added my name to all Hadoop related dependencies, solr, along with es. On Thu, Jun 28, 2018 at 3:28 PM, Etienne Chauchot wrote: > I've added myself and @Tim Robertson on elasticsearchIO related deps. > > Etienne > > Le mercredi 27 juin 2018 à 14:05 -0700, Chamikara

Build failed in Jenkins: beam_SeedJob #2108

See -- GitHub pull request #4976 of commit a34998bba3c4b1f2c96c4654c75d7d05ab8cd5ee, no merge conflicts. Setting status of a34998bba3c4b1f2c96c4654c75d7d05ab8cd5ee to PENDING with url

Jenkins build is back to normal : beam_SeedJob #2107

See

Re: Beam Dependency Ownership

I've added myself and @Tim Robertson on elasticsearchIO related deps. Etienne Le mercredi 27 juin 2018 à 14:05 -0700, Chamikara Jayalath a écrit : > It's mentioned under "Dependency declarations may identify owners that are > responsible for upgrading respective > dependencies". Feel free to

Build failed in Jenkins: beam_SeedJob #2106

See -- GitHub pull request #4976 of commit 80fb70b3798f1654ac648cf8dcada10122e351ab, no merge conflicts. Setting status of 80fb70b3798f1654ac648cf8dcada10122e351ab to PENDING with url

Build failed in Jenkins: beam_SeedJob #2105

See -- GitHub pull request #4976 of commit 32af985b2826410af239dbfb592d18bd3e37f3a2, no merge conflicts. Setting status of 32af985b2826410af239dbfb592d18bd3e37f3a2 to PENDING with url

Build failed in Jenkins: beam_SeedJob #2104

See -- GitHub pull request #4976 of commit 2ea0bfd326fe932abf5d0e2fb6c1b7fb2e497fdf, no merge conflicts. Setting status of 2ea0bfd326fe932abf5d0e2fb6c1b7fb2e497fdf to PENDING with url

Re: Unbounded source translation for portable pipelines

Ankur/Eugene, When you have a chance, please also update the Flink section of: https://docs.google.com/spreadsheets/d/1KDa_FGn1ShjomGd-UUDOhuh2q73de2tPz6BqHpzqvNI/edit#gid=0 Thanks! On Thu, Jun 28, 2018 at 10:29 AM Thomas Weise wrote: > The command to run the job server appears to be:

Re: Unbounded source translation for portable pipelines

The command to run the job server appears to be: ./gradlew -p runners/flink/job-server runShadow Can you please provide the equivalent of the super basic Python example from the prototype: https://github.com/bsidhom/beam/blob/hacking-job-server/sdks/python/flink-example.py Looks as if the

Re: [Design Proposal] Improving Beam code review

Hi, Thanks for that ! I left comments in the doc, mostly agreements and also a comment about public communication. Etienne Le mercredi 27 juin 2018 à 15:29 -0700, Robert Bradshaw a écrit : > Thanks for writing this up! I especially like the idea ofautomatically > assigning code reviewers, e.g.

Re: [DISCUSS] Automation for Java code formatting

> On Jun 28, 2018, at 6:01 AM, Kenneth Knowles wrote: > > So, there are a few modes of checkstyle failure that can be induced by this: > > - lines get too long because of wrap & indent logic > - a lot of our HTML is actually already broken and doesn't have tags > where it should; spotless