Python incremental testing

2018-04-19 Thread Udi Meiri
I cooked up a quick Bazel proof of concept to run Python tests. Bazel's advantage over Nosetests is that it caches results and does dependency tracking on tests to know when to rerun them. (I am not trying to replace Gradle.) The config is almost working (121 out of 123 tests files work, 2 have

Re: The Go SDK got accidentally merged - options to deal with the pain

2018-04-19 Thread Henning Rohde
Hi everyone, Thank you all for your patience. The last major identified feature (Go windowing) is now in review: https://github.com/apache/beam/pull/5179. The remaining work listed under https://issues.apache.org/jira/browse/BEAM-2083 is integration tests and documentation (quickstart,

Jenkins build is back to normal : beam_SeedJob #1541

2018-04-19 Thread Apache Jenkins Server
See

Re: Updated [Proposal] Apache Beam Fn API : Defining and adding SDK Metrics

2018-04-19 Thread Alex Amato
Hello, I have rewritten most of the proposal. Though I think that there is some more research that needs to be done to get the Metric specification perfect. I plan to do more research, and would like to ask you all for more help to make this proposal better. In particular, now that the metrics

Re: Go SDK build failures with maven

2018-04-19 Thread Henning Rohde
Hi Colm, The extra pubsub dependency broke the Maven build. The warning you see in gradle seems to be a (non-breaking) linter check that the format string, which is interesting given that the function called is an internal one -- but the check happens to be correct. Sent you

Re: [Go SDK] Proposal: Set up a Vanity Import Path

2018-04-19 Thread Jason Kuster
+1 this will be very convenient for the go community. On Thu, Apr 19, 2018 at 2:21 PM Henning Rohde wrote: > +1 Great proposal! > > On Wed, Apr 18, 2018 at 1:37 PM Ismaël Mejía wrote: > >> Well it is not really a rule for a proposal but notice that there

Re: Plan for a Parquet new release and writing Parquet file with outputstream

2018-04-19 Thread Eugene Kirpichov
Very cool! JB, time to update your PR? On Thu, Apr 19, 2018 at 9:17 AM Alexey Romanenko wrote: > FYI: Apache Parquet 1.10.0 was release recently. > It contains *org.apache.parquet.io.OutputFile *and updated > *org.apache.parquet.hadoop.ParquetFileWriter* > > WBR, >

Re: [Go SDK] Proposal: Set up a Vanity Import Path

2018-04-19 Thread Henning Rohde
+1 Great proposal! On Wed, Apr 18, 2018 at 1:37 PM Ismaël Mejía wrote: > Well it is not really a rule for a proposal but notice that there are > people in different time zones or people that for different reasons > cannot answer immediately, so a longer period could give them

Re: Go SDK build failures with maven

2018-04-19 Thread Henning Rohde
The Go build works best with Gradle and that's what you should use (other than possibly manually running go build etc). It looks like the build might be broken due to the Go streaming PR independently of the build tooling. Let me take a look. On Thu, Apr 19, 2018 at 7:09 AM Colm O hEigeartaigh

Re: Running Nexmark with PubSub issue

2018-04-19 Thread Alexey Romanenko
Hi Raghu, Thank you, I’ll take a look on your changes. WBR, Alexey > On 13 Apr 2018, at 20:14, Raghu Angadi wrote: > > Hi Alexey, > > I am not sure if I can get to it soon. If you are planning to run nexmark > with Kafka, would you mind porting changes from my branch to

Re: Plan for a Parquet new release and writing Parquet file with outputstream

2018-04-19 Thread Alexey Romanenko
FYI: Apache Parquet 1.10.0 was release recently. It contains org.apache.parquet.io.OutputFile and updated org.apache.parquet.hadoop.ParquetFileWriter WBR, Alexey > On 14 Feb 2018, at 20:10, Jean-Baptiste Onofré wrote: > > Great !! > > In the mean time, I started to PoC

Jenkins build is back to normal : beam_SeedJob #1533

2018-04-19 Thread Apache Jenkins Server
See

Go SDK build failures with maven

2018-04-19 Thread Colm O hEigeartaigh
Hi all, Is the Apache Maven build still maintained? I'm seeing some recent failures in the Go SDK: [INFO] --- mvn-golang-wrapper:2.1.7:test (go-test) @ beam-sdks-go --- [INFO] Prepared command line : bin/go test ./... [ERROR] [ERROR] -Exec.Err- [ERROR]

Jenkins build is back to normal : beam_SeedJob #1529

2018-04-19 Thread Apache Jenkins Server
See

Jenkins build is back to normal : beam_Release_Gradle_NightlySnapshot #13

2018-04-19 Thread Apache Jenkins Server
See

Re: [BEAM-4088] Test isolation differences in Gradle

2018-04-19 Thread Romain Manni-Bucau
No, otherwise it wouldnt pass in idea and maven. Le 19 avr. 2018 00:57, "Kenneth Knowles" a écrit : Does the test think a thread leaked when no thread leaked? On Wed, Apr 18, 2018 at 1:51 PM Romain Manni-Bucau wrote: > The leaking threadq are already

Build failed in Jenkins: beam_SeedJob #1528

2018-04-19 Thread Apache Jenkins Server
See -- Started by timer [EnvInject] - Loading node environment variables. Building remotely on beam5 (beam) in workspace /home/jenkins/jenkins-slave/workspace/beam_SeedJob FATAL: