[jira] [Commented] (BEAM-1067) apex.examples.WordCountTest.testWordCountExample is be flaky
[
https://issues.apache.org/jira/browse/BEAM-1067?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15765632#comment-15765632
]
Jason Kuster commented on BEAM-1067:
https://builds.apache.org/job/beam_PostCommit_Java_MavenInstall/2185/ just now
> apex.examples.WordCountTest.testWordCountExample is be flaky
>
>
> Key: BEAM-1067
> URL: https://issues.apache.org/jira/browse/BEAM-1067
> Project: Beam
> Issue Type: Bug
> Components: runner-apex
>Reporter: Stas Levin
>Assignee: Thomas Weise
>
> Seems that
> {{org.apache.beam.runners.apex.examples.WordCountTest.testWordCountExample}}
> is flaky.
> For example,
> [this|https://builds.apache.org/job/beam_PreCommit_Java_MavenInstall/5408/org.apache.beam$beam-runners-apex/testReport/org.apache.beam.runners.apex.examples/WordCountTest/testWordCountExample/
> ] run failed although no changes were made in {{runner-apex}}.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (BEAM-1089) Jenkins comments on PRs are too many & too large
[ https://issues.apache.org/jira/browse/BEAM-1089?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15762985#comment-15762985 ] Jason Kuster commented on BEAM-1089: Example of a PR with a lot of spam: https://github.com/apache/incubator-beam/pull/1536 > Jenkins comments on PRs are too many & too large > > > Key: BEAM-1089 > URL: https://issues.apache.org/jira/browse/BEAM-1089 > Project: Beam > Issue Type: Bug > Components: testing >Reporter: Kenneth Knowles >Assignee: Jason Kuster > > Lately, I've been finding review comments somewhat drowned out by asfbot > copying build results onto a PR. It also generates a lot of needless email. I > have not yet tried to devise just the right filter, hoping we can just return > to the normal practice of leaving just a commit status. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (BEAM-1131) Consider japicmp for testing backwards-compatibility
[ https://issues.apache.org/jira/browse/BEAM-1131?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15762982#comment-15762982 ] Jason Kuster commented on BEAM-1131: Repo here: https://siom79.github.io/japicmp/ > Consider japicmp for testing backwards-compatibility > > > Key: BEAM-1131 > URL: https://issues.apache.org/jira/browse/BEAM-1131 > Project: Beam > Issue Type: New Feature > Components: testing >Reporter: Kenneth Knowles >Assignee: Jason Kuster > > japicmp is what Flink uses to test backwards-compatibility, configuration > here: > https://github.com/apache/flink/blob/41d5875bfc272f2cd5c7e8c8523036684865c1ce/pom.xml#L1184 > At our first stable release, we should activate this plugin too! > (configuration to be determined by community discussion). We should have it > installed and configured beforehand, informational-only, so that we have a > sense of it. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Resolved] (BEAM-1141) Beam Quickstart doesn't work on DataflowRunner
[
https://issues.apache.org/jira/browse/BEAM-1141?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jason Kuster resolved BEAM-1141.
Resolution: Fixed
Fix Version/s: Not applicable
Fixed in https://github.com/apache/incubator-beam-site/pull/107
> Beam Quickstart doesn't work on DataflowRunner
> --
>
> Key: BEAM-1141
> URL: https://issues.apache.org/jira/browse/BEAM-1141
> Project: Beam
> Issue Type: Bug
> Components: website
>Reporter: Jason Kuster
>Assignee: Jason Kuster
> Fix For: Not applicable
>
>
> Repro steps:
> In new dir, run current archetype:generate command:
> {code}
> mvn archetype:generate \
>
> -DarchetypeRepository=https://repository.apache.org/content/groups/snapshots \
> -DarchetypeGroupId=org.apache.beam \
> -DarchetypeArtifactId=beam-sdks-java-maven-archetypes-examples \
> -DarchetypeVersion=LATEST \
> -DgroupId=org.example \
> -DartifactId=word-count-beam \
> -Dversion="0.1" \
> -Dpackage=org.apache.beam.examples \
> -DinteractiveMode=false
> {code}
> Then run the sample command for DataflowRunner:
> {code}
> mvn compile exec:java -Dexec.mainClass=org.apache.beam.examples.WordCount \
>-Dexec.args="--runner=DataflowRunner
> --gcpTempLocation=gs:///tmp \
> --inputFile=gs://apache-beam-samples/shakespeare/*
> --output=gs:///counts"
> {code}
> Expected outcome: WordCount runs on Dataflow
> Actual outcome:
> {code}
> [INFO] --- exec-maven-plugin:1.4.0:java (default-cli) @ word-count-beam ---
> [WARNING]
> java.lang.reflect.InvocationTargetException
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498)
> at org.codehaus.mojo.exec.ExecJavaMojo$1.run(ExecJavaMojo.java:293)
> at java.lang.Thread.run(Thread.java:745)
> Caused by: java.lang.IllegalArgumentException: Unknown 'runner' specified
> 'DataflowRunner', supported pipeline runners [DirectRunner]
> at
> org.apache.beam.sdk.options.PipelineOptionsFactory.parseObjects(PipelineOptionsFactory.java:1610)
> at
> org.apache.beam.sdk.options.PipelineOptionsFactory.access$400(PipelineOptionsFactory.java:105)
> at
> org.apache.beam.sdk.options.PipelineOptionsFactory$Builder.as(PipelineOptionsFactory.java:290)
> at org.apache.beam.examples.WordCount.main(WordCount.java:174)
> ... 6 more
> Caused by: java.lang.ClassNotFoundException: DataflowRunner
> at java.net.URLClassLoader.findClass(URLClassLoader.java:381)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
> at java.lang.Class.forName0(Native Method)
> at java.lang.Class.forName(Class.java:264)
> at
> org.apache.beam.sdk.options.PipelineOptionsFactory.parseObjects(PipelineOptionsFactory.java:1596)
> ... 9 more
> [INFO]
>
> [INFO] BUILD FAILURE
> [INFO]
>
> [INFO] Total time: 17.916 s
> [INFO] Finished at: 2016-12-12T16:38:55-08:00
> [INFO] Final Memory: 29M/266M
> [INFO]
>
> [ERROR] Failed to execute goal org.codehaus.mojo:exec-maven-plugin:1.4.0:java
> (default-cli) on project word-count-beam: An exception occured while
> executing the Java class. null: InvocationTargetException: Unknown 'runner'
> specified 'DataflowRunner', supported pipeline runners [DirectRunner] ->
> [Help 1]
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (BEAM-1141) Beam Quickstart doesn't work on DataflowRunner
[
https://issues.apache.org/jira/browse/BEAM-1141?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jason Kuster updated BEAM-1141:
---
Assignee: Jason Kuster (was: Davor Bonaci)
> Beam Quickstart doesn't work on DataflowRunner
> --
>
> Key: BEAM-1141
> URL: https://issues.apache.org/jira/browse/BEAM-1141
> Project: Beam
> Issue Type: Bug
> Components: website
>Reporter: Jason Kuster
>Assignee: Jason Kuster
>
> Repro steps:
> In new dir, run current archetype:generate command:
> {code}
> mvn archetype:generate \
>
> -DarchetypeRepository=https://repository.apache.org/content/groups/snapshots \
> -DarchetypeGroupId=org.apache.beam \
> -DarchetypeArtifactId=beam-sdks-java-maven-archetypes-examples \
> -DarchetypeVersion=LATEST \
> -DgroupId=org.example \
> -DartifactId=word-count-beam \
> -Dversion="0.1" \
> -Dpackage=org.apache.beam.examples \
> -DinteractiveMode=false
> {code}
> Then run the sample command for DataflowRunner:
> {code}
> mvn compile exec:java -Dexec.mainClass=org.apache.beam.examples.WordCount \
>-Dexec.args="--runner=DataflowRunner
> --gcpTempLocation=gs:///tmp \
> --inputFile=gs://apache-beam-samples/shakespeare/*
> --output=gs:///counts"
> {code}
> Expected outcome: WordCount runs on Dataflow
> Actual outcome:
> {code}
> [INFO] --- exec-maven-plugin:1.4.0:java (default-cli) @ word-count-beam ---
> [WARNING]
> java.lang.reflect.InvocationTargetException
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498)
> at org.codehaus.mojo.exec.ExecJavaMojo$1.run(ExecJavaMojo.java:293)
> at java.lang.Thread.run(Thread.java:745)
> Caused by: java.lang.IllegalArgumentException: Unknown 'runner' specified
> 'DataflowRunner', supported pipeline runners [DirectRunner]
> at
> org.apache.beam.sdk.options.PipelineOptionsFactory.parseObjects(PipelineOptionsFactory.java:1610)
> at
> org.apache.beam.sdk.options.PipelineOptionsFactory.access$400(PipelineOptionsFactory.java:105)
> at
> org.apache.beam.sdk.options.PipelineOptionsFactory$Builder.as(PipelineOptionsFactory.java:290)
> at org.apache.beam.examples.WordCount.main(WordCount.java:174)
> ... 6 more
> Caused by: java.lang.ClassNotFoundException: DataflowRunner
> at java.net.URLClassLoader.findClass(URLClassLoader.java:381)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
> at java.lang.Class.forName0(Native Method)
> at java.lang.Class.forName(Class.java:264)
> at
> org.apache.beam.sdk.options.PipelineOptionsFactory.parseObjects(PipelineOptionsFactory.java:1596)
> ... 9 more
> [INFO]
>
> [INFO] BUILD FAILURE
> [INFO]
>
> [INFO] Total time: 17.916 s
> [INFO] Finished at: 2016-12-12T16:38:55-08:00
> [INFO] Final Memory: 29M/266M
> [INFO]
>
> [ERROR] Failed to execute goal org.codehaus.mojo:exec-maven-plugin:1.4.0:java
> (default-cli) on project word-count-beam: An exception occured while
> executing the Java class. null: InvocationTargetException: Unknown 'runner'
> specified 'DataflowRunner', supported pipeline runners [DirectRunner] ->
> [Help 1]
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Created] (BEAM-1141) Beam Quickstart doesn't work on DataflowRunner
Jason Kuster created BEAM-1141:
--
Summary: Beam Quickstart doesn't work on DataflowRunner
Key: BEAM-1141
URL: https://issues.apache.org/jira/browse/BEAM-1141
Project: Beam
Issue Type: Bug
Reporter: Jason Kuster
Assignee: Davor Bonaci
Repro steps:
In new dir, run current archetype:generate command:
{code}
mvn archetype:generate \
-DarchetypeRepository=https://repository.apache.org/content/groups/snapshots \
-DarchetypeGroupId=org.apache.beam \
-DarchetypeArtifactId=beam-sdks-java-maven-archetypes-examples \
-DarchetypeVersion=LATEST \
-DgroupId=org.example \
-DartifactId=word-count-beam \
-Dversion="0.1" \
-Dpackage=org.apache.beam.examples \
-DinteractiveMode=false
{code}
Then run the sample command for DataflowRunner:
{code}
mvn compile exec:java -Dexec.mainClass=org.apache.beam.examples.WordCount \
-Dexec.args="--runner=DataflowRunner
--gcpTempLocation=gs:///tmp \
--inputFile=gs://apache-beam-samples/shakespeare/*
--output=gs:///counts"
{code}
Expected outcome: WordCount runs on Dataflow
Actual outcome:
{code}
[INFO] --- exec-maven-plugin:1.4.0:java (default-cli) @ word-count-beam ---
[WARNING]
java.lang.reflect.InvocationTargetException
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.codehaus.mojo.exec.ExecJavaMojo$1.run(ExecJavaMojo.java:293)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.IllegalArgumentException: Unknown 'runner' specified
'DataflowRunner', supported pipeline runners [DirectRunner]
at
org.apache.beam.sdk.options.PipelineOptionsFactory.parseObjects(PipelineOptionsFactory.java:1610)
at
org.apache.beam.sdk.options.PipelineOptionsFactory.access$400(PipelineOptionsFactory.java:105)
at
org.apache.beam.sdk.options.PipelineOptionsFactory$Builder.as(PipelineOptionsFactory.java:290)
at org.apache.beam.examples.WordCount.main(WordCount.java:174)
... 6 more
Caused by: java.lang.ClassNotFoundException: DataflowRunner
at java.net.URLClassLoader.findClass(URLClassLoader.java:381)
at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
at java.lang.Class.forName0(Native Method)
at java.lang.Class.forName(Class.java:264)
at
org.apache.beam.sdk.options.PipelineOptionsFactory.parseObjects(PipelineOptionsFactory.java:1596)
... 9 more
[INFO]
[INFO] BUILD FAILURE
[INFO]
[INFO] Total time: 17.916 s
[INFO] Finished at: 2016-12-12T16:38:55-08:00
[INFO] Final Memory: 29M/266M
[INFO]
[ERROR] Failed to execute goal org.codehaus.mojo:exec-maven-plugin:1.4.0:java
(default-cli) on project word-count-beam: An exception occured while executing
the Java class. null: InvocationTargetException: Unknown 'runner' specified
'DataflowRunner', supported pipeline runners [DirectRunner] -> [Help 1]
{code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Created] (BEAM-1130) SparkRunner ResumeFromCheckpointStreamingTest Failing
Jason Kuster created BEAM-1130:
--
Summary: SparkRunner ResumeFromCheckpointStreamingTest Failing
Key: BEAM-1130
URL: https://issues.apache.org/jira/browse/BEAM-1130
Project: Beam
Issue Type: Bug
Reporter: Jason Kuster
Assignee: Aviem Zur
Failing in (at least) suites:
https://builds.apache.org/job/beam_PostCommit_Java_RunnableOnService_Spark/
381 - 385
https://builds.apache.org/job/beam_PostCommit_Java_MavenInstall/
2063 - 2066
Culprit change seems to be:
https://github.com/apache/incubator-beam/commit/aba40e2de9ba058f33086eb6a913fa583a82b058
Error:
{code}
Success aggregator should be greater than zero.
Expected: not <0>
but: was <0>
{code}
Sample output:
https://builds.apache.org/job/beam_PostCommit_Java_RunnableOnService_Spark/lastCompletedBuild/org.apache.beam$beam-runners-spark/testReport/org.apache.beam.runners.spark.translation.streaming/ResumeFromCheckpointStreamingTest/testRun/
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (BEAM-1064) Convert Jenkins jobs to DSL
[ https://issues.apache.org/jira/browse/BEAM-1064?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15723493#comment-15723493 ] Jason Kuster commented on BEAM-1064: Everything done and committed except for Coveralls. Problem is that the currently-installed version of the credentials plugin doesn't mask the credentials in the log, which is a blocker. According to their release log it should be in the latest version; need to check which version is currently installed on our jenkins. > Convert Jenkins jobs to DSL > --- > > Key: BEAM-1064 > URL: https://issues.apache.org/jira/browse/BEAM-1064 > Project: Beam > Issue Type: Improvement > Components: testing >Reporter: Jason Kuster >Assignee: Jason Kuster > Labels: jenkins > > Move Jenkins jobs to DSL. PR is here: > https://github.com/apache/incubator-beam/pull/1390 -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (BEAM-1064) Convert Jenkins jobs to DSL
[ https://issues.apache.org/jira/browse/BEAM-1064?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15710476#comment-15710476 ] Jason Kuster commented on BEAM-1064: I've updated all of the jobs based on the pull request and am tracking down any failures. So far, the Coveralls token got trampled. Re-adding it, and updating the PR to fix the token getting removed. > Convert Jenkins jobs to DSL > --- > > Key: BEAM-1064 > URL: https://issues.apache.org/jira/browse/BEAM-1064 > Project: Beam > Issue Type: Improvement >Reporter: Jason Kuster >Assignee: Jason Kuster > Labels: jenkins > > Move Jenkins jobs to DSL. PR is here: > https://github.com/apache/incubator-beam/pull/1390 -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (BEAM-1064) Convert Jenkins jobs to DSL
Jason Kuster created BEAM-1064: -- Summary: Convert Jenkins jobs to DSL Key: BEAM-1064 URL: https://issues.apache.org/jira/browse/BEAM-1064 Project: Beam Issue Type: Improvement Reporter: Jason Kuster Assignee: Jason Kuster Move Jenkins jobs to DSL. PR is here: https://github.com/apache/incubator-beam/pull/1390 -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (BEAM-983) runners/spark/translation/streaming/utils/TestPipelineOptions.java missing Apache license header.
Jason Kuster created BEAM-983: - Summary: runners/spark/translation/streaming/utils/TestPipelineOptions.java missing Apache license header. Key: BEAM-983 URL: https://issues.apache.org/jira/browse/BEAM-983 Project: Beam Issue Type: Bug Reporter: Jason Kuster Assignee: Stas Levin Pull request #1332 (https://github.com/apache/incubator-beam/pull/1332) is failing in beam_PostCommit_MavenVerify with an Apache Rat failure -- https://builds.apache.org/job/beam_PostCommit_MavenVerify/1827/ -- Rat output points to runners/spark/src/test/java/org/apache/beam/runners/spark/translation/streaming/utils/TestPipelineOptions.java as the culprit. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (BEAM-938) Break in direct.UnboundedReadEvaluatorFactoryTest.noElementsAvailableReaderIncludedInResidual
Jason Kuster created BEAM-938: - Summary: Break in direct.UnboundedReadEvaluatorFactoryTest.noElementsAvailableReaderIncludedInResidual Key: BEAM-938 URL: https://issues.apache.org/jira/browse/BEAM-938 Project: Beam Issue Type: Bug Reporter: Jason Kuster Assignee: Thomas Groh Likely culprit: https://github.com/apache/incubator-beam/commit/ff7fe07be96de393b763e7b3d213734040aa3795 -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (BEAM-795) org.apache.beam.sdk.io.ReadTest.testStreamingPrimitiveDisplayData failing in Dataflow RunnableOnService
Jason Kuster created BEAM-795: - Summary: org.apache.beam.sdk.io.ReadTest.testStreamingPrimitiveDisplayData failing in Dataflow RunnableOnService Key: BEAM-795 URL: https://issues.apache.org/jira/browse/BEAM-795 Project: Beam Issue Type: Bug Reporter: Jason Kuster Assignee: Scott Wegner Priority: Critical Test is failing in beam_PostCommit_RunnableOnService_GoogleCloudDataflow since yesterday. Culprit change seems to be https://github.com/apache/incubator-beam/commit/ad03d07ae783f054a31e8b2e14100afff8cdf747 Affected Jenkins runs: https://builds.apache.org/job/beam_PostCommit_RunnableOnService_GoogleCloudDataflow/ 1381 - 1386, 1388 as of bug creation. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (BEAM-769) Spark translation tests failing with NullPointerException
[
https://issues.apache.org/jira/browse/BEAM-769?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15586248#comment-15586248
]
Jason Kuster commented on BEAM-769:
---
This also seems to have caused a postcommit failure:
https://builds.apache.org/job/beam_PostCommit_MavenVerify/1567/
> Spark translation tests failing with NullPointerException
> -
>
> Key: BEAM-769
> URL: https://issues.apache.org/jira/browse/BEAM-769
> Project: Beam
> Issue Type: Bug
> Components: runner-spark
>Affects Versions: Not applicable
>Reporter: Daniel Halperin
>Assignee: Amit Sela
>
> https://builds.apache.org/job/beam_PreCommit_MavenVerify/4071/
> {code}
> org.apache.beam.runners.spark.translation.streaming.EmptyStreamAssertionTest.testFixedWindows
> org.apache.beam.runners.spark.translation.streaming.FlattenStreamingTest.testFlattenUnbounded
> org.apache.beam.runners.spark.translation.streaming.KafkaStreamingTest.testRun
> org.apache.beam.runners.spark.translation.streaming.SimpleStreamingWordCountTest.testFixedWindows
> {code}
> Amit, can you please take a look?
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Created] (BEAM-754) WordCountIT Flake -- Incorrect checksum
Jason Kuster created BEAM-754: - Summary: WordCountIT Flake -- Incorrect checksum Key: BEAM-754 URL: https://issues.apache.org/jira/browse/BEAM-754 Project: Beam Issue Type: Bug Reporter: Jason Kuster Assignee: Mark Liu Priority: Minor WordCountIT flaked in Jenkins PostCommit -- https://builds.apache.org/job/beam_PostCommit_MavenVerify/org.apache.beam$beam-examples-java/1532/testReport/junit/org.apache.beam.examples/WordCountIT/ -- due to an incorrect checksum. Can we add some additional debug output so we can understand what happened in these cases? -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (BEAM-736) BigQueryTornadoesIT broken, blocking nightly release.
Jason Kuster created BEAM-736: - Summary: BigQueryTornadoesIT broken, blocking nightly release. Key: BEAM-736 URL: https://issues.apache.org/jira/browse/BEAM-736 Project: Beam Issue Type: Bug Components: testing Reporter: Jason Kuster Assignee: Pei He Build break begins here: https://builds.apache.org/job/beam_PostCommit_MavenVerify/1471/ listing 3 potential culprit commits. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Closed] (BEAM-493) All Runners Run WordCount in Precommit
[ https://issues.apache.org/jira/browse/BEAM-493?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jason Kuster closed BEAM-493. - Resolution: Fixed Fix Version/s: 0.3.0-incubating All mainline runners now run wordcount in precommit > All Runners Run WordCount in Precommit > -- > > Key: BEAM-493 > URL: https://issues.apache.org/jira/browse/BEAM-493 > Project: Beam > Issue Type: Improvement >Reporter: Jason Kuster >Assignee: Jason Kuster > Fix For: 0.3.0-incubating > > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (BEAM-655) Rename @RunnableonService to something more descriptive
Jason Kuster created BEAM-655: - Summary: Rename @RunnableonService to something more descriptive Key: BEAM-655 URL: https://issues.apache.org/jira/browse/BEAM-655 Project: Beam Issue Type: Bug Reporter: Jason Kuster Assignee: Jason Kuster -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (BEAM-561) WindowedWordCountIT for Beam
Jason Kuster created BEAM-561: - Summary: WindowedWordCountIT for Beam Key: BEAM-561 URL: https://issues.apache.org/jira/browse/BEAM-561 Project: Beam Issue Type: Bug Reporter: Jason Kuster -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (BEAM-561) Add WindowedWordCountIT
[ https://issues.apache.org/jira/browse/BEAM-561?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jason Kuster updated BEAM-561: -- Summary: Add WindowedWordCountIT (was: WindowedWordCountIT for Beam) > Add WindowedWordCountIT > --- > > Key: BEAM-561 > URL: https://issues.apache.org/jira/browse/BEAM-561 > Project: Beam > Issue Type: Bug >Reporter: Jason Kuster > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (BEAM-493) All Runners Run WordCount in Precommit
[ https://issues.apache.org/jira/browse/BEAM-493?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jason Kuster updated BEAM-493: -- Summary: All Runners Run WordCount in Precommit (was: All Runners Run WordCount in Presubmit) > All Runners Run WordCount in Precommit > -- > > Key: BEAM-493 > URL: https://issues.apache.org/jira/browse/BEAM-493 > Project: Beam > Issue Type: Improvement >Reporter: Jason Kuster >Assignee: Jason Kuster > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Resolved] (BEAM-256) Add lifecycle event verifiers for Beam pipelines.
[ https://issues.apache.org/jira/browse/BEAM-256?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jason Kuster resolved BEAM-256. --- Resolution: Fixed Fix Version/s: 0.1.0-incubating > Add lifecycle event verifiers for Beam pipelines. > - > > Key: BEAM-256 > URL: https://issues.apache.org/jira/browse/BEAM-256 > Project: Beam > Issue Type: New Feature > Components: testing >Reporter: Jason Kuster >Assignee: Jason Kuster > Fix For: 0.1.0-incubating > > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (BEAM-494) All E2E Tests Run Against All Runners In Postcommit
Jason Kuster created BEAM-494: - Summary: All E2E Tests Run Against All Runners In Postcommit Key: BEAM-494 URL: https://issues.apache.org/jira/browse/BEAM-494 Project: Beam Issue Type: Improvement Reporter: Jason Kuster Assignee: Jason Kuster -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (BEAM-493) All Runners Run WordCount in Presubmit
Jason Kuster created BEAM-493: - Summary: All Runners Run WordCount in Presubmit Key: BEAM-493 URL: https://issues.apache.org/jira/browse/BEAM-493 Project: Beam Issue Type: Improvement Reporter: Jason Kuster Assignee: Jason Kuster -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (BEAM-124) Testing -- End to End WordCount Batch and Streaming Tests
[ https://issues.apache.org/jira/browse/BEAM-124?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15366643#comment-15366643 ] Jason Kuster commented on BEAM-124: --- Update: Batch WordCount E2E has been in for a while. Verifiers are coming in a pull request soon, and then we'll look at expanding to a streaming test as well. > Testing -- End to End WordCount Batch and Streaming Tests > - > > Key: BEAM-124 > URL: https://issues.apache.org/jira/browse/BEAM-124 > Project: Beam > Issue Type: New Feature > Components: testing >Reporter: Steve Wheeler >Assignee: Jason Kuster > > Set up testing infrastructure so that an end to end test for WordCount (both > batch and streaming) will be run periodically. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Comment Edited] (BEAM-124) Testing -- End to End WordCount Batch and Streaming Tests
[ https://issues.apache.org/jira/browse/BEAM-124?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15366643#comment-15366643 ] Jason Kuster edited comment on BEAM-124 at 7/7/16 7:28 PM: --- Update: Batch WordCount E2E has been in for a while. Verifiers are coming in a pull request soon, and then we'll look at expanding to a streaming test as well. Work is in-progress to get them running on all runners. was (Author: jasonkuster): Update: Batch WordCount E2E has been in for a while. Verifiers are coming in a pull request soon, and then we'll look at expanding to a streaming test as well. > Testing -- End to End WordCount Batch and Streaming Tests > - > > Key: BEAM-124 > URL: https://issues.apache.org/jira/browse/BEAM-124 > Project: Beam > Issue Type: New Feature > Components: testing >Reporter: Steve Wheeler >Assignee: Jason Kuster > > Set up testing infrastructure so that an end to end test for WordCount (both > batch and streaming) will be run periodically. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Closed] (BEAM-228) Create a merge bot for Beam
[ https://issues.apache.org/jira/browse/BEAM-228?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jason Kuster closed BEAM-228. - Resolution: Won't Fix Fix Version/s: Not applicable Mergebot is being delivered to apache infra instead - closing as obsolete. > Create a merge bot for Beam > --- > > Key: BEAM-228 > URL: https://issues.apache.org/jira/browse/BEAM-228 > Project: Beam > Issue Type: New Feature > Components: project-management >Reporter: Jason Kuster >Assignee: Jason Kuster > Fix For: Not applicable > > > This issue tracks the creation of a merge bot for Beam. This merge bot should > watch the Beam github repository and queue and merge pull requests which are > marked LGTM and good for merge by an approved Beam committer. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (BEAM-124) Testing -- End to End WordCount Batch and Streaming Tests
[ https://issues.apache.org/jira/browse/BEAM-124?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15287652#comment-15287652 ] Jason Kuster commented on BEAM-124: --- Pull request https://github.com/apache/incubator-beam/pull/345 out which allows this. > Testing -- End to End WordCount Batch and Streaming Tests > - > > Key: BEAM-124 > URL: https://issues.apache.org/jira/browse/BEAM-124 > Project: Beam > Issue Type: New Feature > Components: testing >Reporter: Steve Wheeler >Assignee: Steve Wheeler > > Set up testing infrastructure so that an end to end test for WordCount (both > batch and streaming) will be run periodically. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (BEAM-133) Test flakiness in the Spark runner
[
https://issues.apache.org/jira/browse/BEAM-133?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15202273#comment-15202273
]
Jason Kuster commented on BEAM-133:
---
We've changed the Jenkins projects to use a Maven repository local to the
workspace. This should fix the problem; please let us know if it does not.
> Test flakiness in the Spark runner
> --
>
> Key: BEAM-133
> URL: https://issues.apache.org/jira/browse/BEAM-133
> Project: Beam
> Issue Type: Bug
> Components: project-management
>Reporter: Davor Bonaci
>Assignee: Davor Bonaci
>
> Jenkins shows some flakiness in the Spark runner in the context of an
> unrelated pre-commit test.
> {code}
> Results :
> Tests in error:
> AvroPipelineTest.testGeneric:75 » Runtime java.io.IOException: Could
> not creat...
> NumShardsTest.testText:77 » Runtime java.io.IOException: Could not
> create File...
> HadoopFileFormatPipelineTest.testSequenceFile:83 » Runtime
> java.io.IOException...
>
> TransformTranslatorTest.testTextIOReadAndWriteTransforms:76->runPipeline:96 »
> Runtime
> KafkaStreamingTest.testRun:121 » Runtime java.io.IOException: failure
> to login...
> Tests run: 27, Failures: 0, Errors: 5, Skipped: 0
> [ERROR] There are test failures.
> {code}
> https://builds.apache.org/job/beam_PreCommit/98/console
> Amit, does this sounds like a test code issue or the infrastructure issue?
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
