Build failed in Jenkins: beam_PostRelease_NightlySnapshot #24

See Changes: [robertwb] [BEAM-3625] Enable DoFn params in Map, Filter, etc. [apilloud] [BEAM-410] Sort PriorityQueue with explicit comparator [robertwb] Disable verbose typecoder warnings.

[CANCEL][VOTE] Release 2.3.0, release candidate #2

Hi guys, due to BEAM-3519 [1], I cancel RC2. We have to shade netty version in the IOs to avoid issue with netty version in runners (Spark in that case). We gonna work on a fix (I should be pretty fast), and I will cut a RC3 when cherry-pick on release-2.3.0 branch. Regards JB [1]

[INFO] Gradle build is flaky on Jenkins

Hi guys, I noticed that the Gradle build on Jenkins is flaky: it almost always fails for different reason (I can't download pentaho artifact sometime, interrupted other times). Jenkins is doing: Jenkins: ./gradlew --continue --rerun-tasks :javaPreCommit I gonna investigate how to improve the

Re: [INFO] Gradle build is flaky on Jenkins

Yes, I was just about to write about this as well. In my recent PRs this always failed for different reasons. Thanks for looking into this! > On 9. Feb 2018, at 11:35, Jean-Baptiste Onofré wrote: > > Hi guys, > > I noticed that the Gradle build on Jenkins is flaky: it

Re: [jira] [Commented] (BEAM-3519) GCP IO exposes netty on its API surface, causing conflicts with runners

Shading netty everywhere would prevent conflicts with user supplied versions of netty. On Fri, Feb 9, 2018 at 1:03 AM, Ismaël Mejía (JIRA) wrote: > > [ https://issues.apache.org/jira/browse/BEAM-3519?page= > com.atlassian.jira.plugin.system.issuetabpanels:comment- >

Re: [INFO] Gradle build is flaky on Jenkins

I have been hitting these issues and investigating them. - HCatalog pentaho dependency: https://issues.apache.org/jira/browse/BEAM-3621 - We rerun way way more than we need to: https://issues.apache.org/jira/browse/BEAM-3253 but dependency stuff occurs during config, not tasks, I think The

Re: [INFO] Gradle build is flaky on Jenkins

Separately, we have many tests that seem to have been passing by luck and Gradle exposes some of their weaknesses by its increased parallelism. They are filed under the "flake" and later the "sickbay" tags. I've put together a saved search: https://issues.apache.org/jira/issues/?filter=12343195

Re: [jira] [Commented] (BEAM-2591) Python shim for submitting to FlinkRunner

Just seeing this. I'm not sure if a Jira integration has regressed, but these emailed replies didn't make it to the ticket. Kenn On Thu, Feb 8, 2018 at 4:31 PM, Robert Bradshaw wrote: > From the Python point of view, I think it makes sense to configure the > "shim" runner

Re: [jira] [Commented] (BEAM-3519) GCP IO exposes netty on its API surface, causing conflicts with runners

I think we should take it case-by-case. Guava universal shading is already quite a mess - build super slow, copies of Guava in jars that don't even depend on Guava, things shaded that break API surfaces. In this case there's no mystery what needs to happen, is there? Kenn On Fri, Feb 9, 2018 at

Re: [CANCEL][VOTE] Release 2.3.0, release candidate #2

Late to respond, but +1 to this decision. On Fri, Feb 9, 2018 at 2:26 AM, Jean-Baptiste Onofré wrote: > Hi guys, > > due to BEAM-3519 [1], I cancel RC2. > > We have to shade netty version in the IOs to avoid issue with netty > version in > runners (Spark in that case). > > We

Re: [CANCEL][VOTE] Release 2.3.0, release candidate #2

https://github.com/apache/beam/pull/4651 is in review. On Fri, Feb 9, 2018 at 1:38 PM Reuven Lax wrote: > Late to respond, but +1 to this decision. > > On Fri, Feb 9, 2018 at 2:26 AM, Jean-Baptiste Onofré > wrote: > >> Hi guys, >> >> due to BEAM-3519 [1], I

Re: [SQL] Reject unsupported inputs to Joins

Huge +1 to only allowing single-firing triggers through for sane semantics in this case. Commented on the doc. On Fri, Feb 9, 2018 at 5:08 PM, Anton Kedin wrote: > Hi, > > If you're not using Beam SQL JOINs, you're not affected. > > In Short > > Beam SQL JOIN operation does not

[SQL] Reject unsupported inputs to Joins

Hi, If you're not using Beam SQL JOINs, you're not affected. In Short Beam SQL JOIN operation does not work well with multiple trigger firings. More Detail Beam SQL JOIN is a CoGBK under the hood. It joins available elements per-pane. This means that: - in discarding mode we're joining

Re: [CANCEL][VOTE] Release 2.3.0, release candidate #2

Hi guys Quick update for RC3: BEAM-3668 & BEAM-3519 have a PR in review (shading netty in GCP IO). Hopefully, the PR will be merge later today, then it will be cherry-pick on release-2.3.0 branch. As soon as it's done, I will cut a RC3 (hopefully tonight or tomorrow). Regards JB On 02/10/2018

Re: [VOTE] Release 2.3.0, release candidate #2

+1 for option 1 (fix and create a new +RC3). This not only breaks examples, this means that users of the spark runner won't have any guarantee about the fact that the Google IOs work or not because of possible missing methods or conflicts between netty versions. On Fri, Feb 9, 2018 at 7:31 AM,

Re: [INFO] Gradle build is flaky on Jenkins

Hi Romain, that's actually part of my investigations ;) Regards JB On 02/09/2018 02:26 PM, Romain Manni-Bucau wrote: > did you check the concurrency? For me it is plain wrong since it is hardcoded > in > the build file and doesnt let me or the tool customize it. This means it just > blocks my

Re: [INFO] Gradle build is flaky on Jenkins

did you check the concurrency? For me it is plain wrong since it is hardcoded in the build file and doesnt let me or the tool customize it. This means it just blocks my computer in general and makes some timeout related tests fail easily. (this kind of config should always be customized on the CI