See <https://builds.apache.org/job/beam_PerformanceTests_Python/1194/display/redirect?page=changes>
Changes: [tgroh] Add OutputDeduplicator [tgroh] Update ExecutableStageMatcher to accept Matchers [tgroh] Use OutputDeduplicator in GreedyPipelineFuser [swegner] Upgrade Gradle to 4.7 [swegner] Upgrade Gradle protobuf plugin to latest. [tgroh] fixup! Add OutputDeduplicator ------------------------------------------ [...truncated 816.79 KB...] [INFO] Apache Beam :: Model ............................... SUCCESS [ 0.035 s] [INFO] Apache Beam :: Model :: Pipeline ................... SUCCESS [ 8.533 s] [INFO] Apache Beam :: Model :: Job Management ............. SUCCESS [ 3.907 s] [INFO] Apache Beam :: Model :: Fn Execution ............... SUCCESS [ 3.906 s] [INFO] Apache Beam :: SDKs ................................ SUCCESS [ 0.132 s] [INFO] Apache Beam :: SDKs :: Go .......................... SUCCESS [ 31.284 s] [INFO] Apache Beam :: SDKs :: Go :: Container ............. SUCCESS [ 22.075 s] [INFO] Apache Beam :: SDKs :: Java ........................ SUCCESS [ 0.154 s] [INFO] Apache Beam :: SDKs :: Java :: Core ................ SUCCESS [ 26.405 s] [INFO] Apache Beam :: SDKs :: Java :: Fn Execution ........ SUCCESS [ 3.083 s] [INFO] Apache Beam :: SDKs :: Java :: Extensions .......... SUCCESS [ 0.050 s] [INFO] Apache Beam :: SDKs :: Java :: Extensions :: Google Cloud Platform Core SUCCESS [ 3.428 s] [INFO] Apache Beam :: Runners ............................. SUCCESS [ 0.060 s] [INFO] Apache Beam :: Runners :: Core Construction Java ... SUCCESS [ 4.812 s] [INFO] Apache Beam :: Runners :: Core Java ................ SUCCESS [ 4.796 s] [INFO] Apache Beam :: SDKs :: Java :: Harness ............. SUCCESS [ 8.793 s] [INFO] Apache Beam :: SDKs :: Java :: Container ........... SUCCESS [ 8.932 s] [INFO] Apache Beam :: SDKs :: Java :: IO .................. SUCCESS [ 0.054 s] [INFO] Apache Beam :: SDKs :: Java :: IO :: Amazon Web Services SUCCESS [ 2.953 s] [INFO] Apache Beam :: Runners :: Local Java Core .......... SUCCESS [ 1.606 s] [INFO] Apache Beam :: Runners :: Direct Java .............. SUCCESS [ 11.405 s] [INFO] Apache Beam :: SDKs :: Java :: IO :: AMQP .......... SUCCESS [ 2.618 s] [INFO] Apache Beam :: SDKs :: Java :: IO :: Common ........ SUCCESS [ 1.865 s] [INFO] Apache Beam :: SDKs :: Java :: IO :: Cassandra ..... SUCCESS [ 2.769 s] [INFO] Apache Beam :: SDKs :: Java :: IO :: Elasticsearch . SUCCESS [ 1.959 s] [INFO] Apache Beam :: SDKs :: Java :: IO :: Elasticsearch-Tests SUCCESS [ 0.553 s] [INFO] Apache Beam :: SDKs :: Java :: IO :: Elasticsearch-Tests :: Common SUCCESS [ 0.638 s] [INFO] Apache Beam :: SDKs :: Java :: IO :: Elasticsearch-Tests :: 2.x SUCCESS [ 1.601 s] [INFO] Apache Beam :: SDKs :: Java :: IO :: Elasticsearch-Tests :: 5.x SUCCESS [ 1.990 s] [INFO] Apache Beam :: SDKs :: Java :: IO :: XML ........... SUCCESS [ 4.223 s] [INFO] Apache Beam :: SDKs :: Java :: Extensions :: Protobuf SUCCESS [ 2.117 s] [INFO] Apache Beam :: SDKs :: Java :: IO :: Google Cloud Platform SUCCESS [ 4.934 s] [INFO] Apache Beam :: Runners :: Google Cloud Dataflow .... SUCCESS [ 11.940 s] [INFO] Apache Beam :: SDKs :: Java :: IO :: File-based-io-tests SUCCESS [ 2.123 s] [INFO] Apache Beam :: SDKs :: Java :: IO :: Hadoop Common . SUCCESS [ 3.187 s] [INFO] Apache Beam :: SDKs :: Java :: IO :: Hadoop File System SUCCESS [ 3.302 s] [INFO] Apache Beam :: SDKs :: Java :: IO :: JDBC .......... SUCCESS [ 4.767 s] [INFO] Apache Beam :: SDKs :: Java :: IO :: Hadoop Input Format SUCCESS [ 8.049 s] [INFO] Apache Beam :: SDKs :: Java :: IO :: HBase ......... SUCCESS [ 6.612 s] [INFO] Apache Beam :: SDKs :: Java :: IO :: HCatalog ...... SUCCESS [ 7.833 s] [INFO] Apache Beam :: SDKs :: Java :: IO :: JMS ........... SUCCESS [ 3.272 s] [INFO] Apache Beam :: SDKs :: Java :: IO :: Kafka ......... SUCCESS [ 2.675 s] [INFO] Apache Beam :: SDKs :: Java :: IO :: Kinesis ....... SUCCESS [ 1.614 s] [INFO] Apache Beam :: SDKs :: Java :: IO :: MongoDB ....... SUCCESS [ 3.357 s] [INFO] Apache Beam :: SDKs :: Java :: IO :: MQTT .......... SUCCESS [ 2.192 s] [INFO] Apache Beam :: SDKs :: Java :: IO :: Redis ......... SUCCESS [ 2.052 s] [INFO] Apache Beam :: SDKs :: Java :: IO :: Solr .......... SUCCESS [ 4.998 s] [INFO] Apache Beam :: SDKs :: Java :: IO :: Tika .......... SUCCESS [ 3.233 s] [INFO] Apache Beam :: SDKs :: Java :: Maven Archetypes .... SUCCESS [ 0.028 s] [INFO] Apache Beam :: SDKs :: Java :: Maven Archetypes :: Starter SUCCESS [ 6.954 s] [INFO] Apache Beam :: Examples ............................ SUCCESS [ 0.028 s] [INFO] Apache Beam :: Examples :: Java .................... SUCCESS [ 3.841 s] [INFO] Apache Beam :: SDKs :: Java :: Maven Archetypes :: Examples SUCCESS [ 28.156 s] [INFO] Apache Beam :: SDKs :: Java :: Extensions :: Jackson SUCCESS [ 1.933 s] [INFO] Apache Beam :: SDKs :: Java :: Extensions :: Join library SUCCESS [ 2.213 s] [INFO] Apache Beam :: SDKs :: Java :: Extensions :: Sketching SUCCESS [ 2.786 s] [INFO] Apache Beam :: SDKs :: Java :: Extensions :: Sorter SUCCESS [ 2.653 s] [INFO] Apache Beam :: SDKs :: Java :: Extensions :: SQL ... SUCCESS [ 14.327 s] [INFO] Apache Beam :: SDKs :: Java :: Nexmark ............. SUCCESS [ 17.786 s] [INFO] Apache Beam :: SDKs :: Python ...................... SUCCESS [ 12.597 s] [INFO] Apache Beam :: SDKs :: Python :: Container ......... SUCCESS [ 7.331 s] [INFO] Apache Beam :: Runners :: Java Fn Execution ........ SUCCESS [ 3.937 s] [INFO] Apache Beam :: Runners :: Java Local Artifact Service SUCCESS [ 2.059 s] [INFO] Apache Beam :: Runners :: Reference ................ SUCCESS [ 0.022 s] [INFO] Apache Beam :: Runners :: Reference :: Java ........ FAILURE [ 0.551 s] [INFO] Apache Beam :: Runners :: Reference :: Job Orchestrator SKIPPED [INFO] Apache Beam :: Runners :: Flink .................... SKIPPED [INFO] Apache Beam :: Runners :: Gearpump ................. SKIPPED [INFO] Apache Beam :: Runners :: Spark .................... SKIPPED [INFO] Apache Beam :: Runners :: Apex ..................... SKIPPED [INFO] Apache Beam :: Runners :: Google Cloud Platform .... SKIPPED [INFO] Apache Beam :: Runners :: Google Cloud Platform :: GCE metadata provisioning SKIPPED [INFO] Apache Beam :: Runners :: Google Cloud Platform :: GCS artifact proxy SKIPPED [INFO] Apache Beam :: Runners :: Extensions Java .......... SKIPPED [INFO] Apache Beam :: Runners :: Extensions Java :: Metrics SKIPPED [INFO] Apache Beam :: SDKs :: Java :: Aggregated Javadoc .. SKIPPED [INFO] ------------------------------------------------------------------------ [INFO] BUILD FAILURE [INFO] ------------------------------------------------------------------------ [INFO] Total time: 05:57 min [INFO] Finished at: 2018-04-26T06:07:40Z [INFO] Final Memory: 644M/9607M [INFO] ------------------------------------------------------------------------ [ERROR] Failed to execute goal org.apache.maven.plugins:maven-compiler-plugin:3.7.0:compile (default-compile) on project beam-runners-reference-java: Compilation failure: Compilation failure: [ERROR] <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/e6f070a9/beam/runners/reference/java/src/main/java/org/apache/beam/runners/reference/testing/InProcessManagedChannelFactory.java>:[23,38] package org.apache.beam.sdk.fn.channel does not exist [ERROR] <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/e6f070a9/beam/runners/reference/java/src/main/java/org/apache/beam/runners/reference/testing/InProcessManagedChannelFactory.java>:[30,53] cannot find symbol [ERROR] symbol: class ManagedChannelFactory [ERROR] <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/e6f070a9/beam/runners/reference/java/src/main/java/org/apache/beam/runners/reference/PortableRunner.java>:[49,38] package org.apache.beam.sdk.fn.channel does not exist [ERROR] <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/e6f070a9/beam/runners/reference/java/src/main/java/org/apache/beam/runners/reference/PortableRunner.java>:[69,17] cannot find symbol [ERROR] symbol: class ManagedChannelFactory [ERROR] location: class org.apache.beam.runners.reference.PortableRunner [ERROR] <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/e6f070a9/beam/runners/reference/java/src/main/java/org/apache/beam/runners/reference/PortableRunner.java>:[82,57] cannot find symbol [ERROR] symbol: class ManagedChannelFactory [ERROR] location: class org.apache.beam.runners.reference.PortableRunner [ERROR] <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/e6f070a9/beam/runners/reference/java/src/main/java/org/apache/beam/runners/reference/PortableRunner.java>:[128,7] cannot find symbol [ERROR] symbol: class ManagedChannelFactory [ERROR] location: class org.apache.beam.runners.reference.PortableRunner [ERROR] -> [Help 1] [ERROR] [ERROR] To see the full stack trace of the errors, re-run Maven with the -e switch. [ERROR] Re-run Maven using the -X switch to enable full debug logging. [ERROR] [ERROR] For more information about the errors and possible solutions, please read the following articles: [ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/MojoFailureException [ERROR] [ERROR] After correcting the problems, you can resume the build with the command [ERROR] mvn <goals> -rf :beam-runners-reference-java STDERR: log4j:WARN No appenders could be found for logger (freemarker.cache). log4j:WARN Please initialize the log4j system properly. Warning: Lookahead adequacy checking not being performed since option LOOKAHEAD is more than 1. Set option FORCE_LA_CHECK to true to force checking. /usr/local/lib/python2.7/dist-packages/setuptools/dist.py:397: UserWarning: Normalizing '2.5.0.dev' to '2.5.0.dev0' normalized_version, /usr/local/lib/python2.7/dist-packages/setuptools/dist.py:397: UserWarning: Normalizing '2.5.0.dev' to '2.5.0.dev0' normalized_version, <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/e6f070a9/beam/sdks/python/gen_protos.py>:49: UserWarning: Installing grpcio-tools is recommended for development. warnings.warn('Installing grpcio-tools is recommended for development.') WARNING:root:Installing grpcio-tools into <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/e6f070a9/beam/sdks/python/.eggs/grpcio-wheels> WARNING:root:Installing grpcio-tools took 3.74 seconds. The script virtualenv is installed in '<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/e6f070a9/beam/sdks/python/target/python/bin'> which is not on PATH. Consider adding this directory to PATH or, if you prefer to suppress this warning, use --no-warn-script-location. The scripts tox and tox-quickstart are installed in '<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/e6f070a9/beam/sdks/python/target/python/bin'> which is not on PATH. Consider adding this directory to PATH or, if you prefer to suppress this warning, use --no-warn-script-location. <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/e6f070a9/beam/sdks/python/target/python/lib/python2.7/site-packages/setuptools/dist.py>:397: UserWarning: Normalizing '2.5.0.dev' to '2.5.0.dev0' normalized_version, 2018-04-26 06:07:42,374 e6f070a9 MainThread beam_integration_benchmark(1/1) INFO Running benchmark beam_integration_benchmark 2018-04-26 06:07:42,396 e6f070a9 MainThread beam_integration_benchmark(1/1) INFO Running: python setup.py nosetests --tests=apache_beam.examples.wordcount_it_test --attr=IT --test-pipeline-options="--project=apache-beam-testing" "--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it" "--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it" "--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output" --runner=TestDataflowRunner --sdk_location=<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/e6f070a9/beam/sdks/python/target/apache-beam-2.5.0.dev0.tar.gz> 2018-04-26 06:07:42,666 e6f070a9 MainThread beam_integration_benchmark(1/1) INFO Ran: {python setup.py nosetests --tests=apache_beam.examples.wordcount_it_test --attr=IT --test-pipeline-options="--project=apache-beam-testing" "--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it" "--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it" "--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output" --runner=TestDataflowRunner --sdk_location=<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/e6f070a9/beam/sdks/python/target/apache-beam-2.5.0.dev0.tar.gz}> ReturnCode:1 STDOUT: STDERR: /usr/local/lib/python2.7/dist-packages/setuptools/dist.py:397: UserWarning: Normalizing '2.5.0.dev' to '2.5.0.dev0' normalized_version, usage: setup.py [global_opts] cmd1 [cmd1_opts] [cmd2 [cmd2_opts] ...] or: setup.py --help [cmd1 cmd2 ...] or: setup.py --help-commands or: setup.py cmd --help error: invalid command 'nosetests' 2018-04-26 06:07:42,667 e6f070a9 MainThread beam_integration_benchmark(1/1) ERROR Error during benchmark beam_integration_benchmark Traceback (most recent call last): File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py";,> line 646, in RunBenchmark DoRunPhase(spec, collector, detailed_timer) File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py";,> line 526, in DoRunPhase samples = spec.BenchmarkRun(spec) File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/linux_benchmarks/beam_integration_benchmark.py";,> line 159, in Run job_type=job_type) File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/providers/gcp/gcp_dpb_dataflow.py";,> line 90, in SubmitJob assert retcode == 0, "Integration Test Failed." AssertionError: Integration Test Failed. 2018-04-26 06:07:42,668 e6f070a9 MainThread beam_integration_benchmark(1/1) INFO Cleaning up benchmark beam_integration_benchmark 2018-04-26 06:07:42,671 e6f070a9 MainThread beam_integration_benchmark(1/1) ERROR Exception running benchmark Traceback (most recent call last): File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py";,> line 780, in RunBenchmarkTask RunBenchmark(spec, collector) File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py";,> line 646, in RunBenchmark DoRunPhase(spec, collector, detailed_timer) File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py";,> line 526, in DoRunPhase samples = spec.BenchmarkRun(spec) File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/linux_benchmarks/beam_integration_benchmark.py";,> line 159, in Run job_type=job_type) File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/providers/gcp/gcp_dpb_dataflow.py";,> line 90, in SubmitJob assert retcode == 0, "Integration Test Failed." AssertionError: Integration Test Failed. 2018-04-26 06:07:42,672 e6f070a9 MainThread beam_integration_benchmark(1/1) ERROR Benchmark 1/1 beam_integration_benchmark (UID: beam_integration_benchmark0) failed. Execution will continue. 2018-04-26 06:07:42,725 e6f070a9 MainThread INFO Benchmark run statuses: --------------------------------------------------------------------------------- Name UID Status Failed Substatus --------------------------------------------------------------------------------- beam_integration_benchmark beam_integration_benchmark0 FAILED --------------------------------------------------------------------------------- Success rate: 0.00% (0/1) 2018-04-26 06:07:42,726 e6f070a9 MainThread INFO Complete logs can be found at: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/e6f070a9/pkb.log> 2018-04-26 06:07:42,726 e6f070a9 MainThread INFO Completion statuses can be found at: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/e6f070a9/completion_statuses.json> Build step 'Execute shell' marked build as failure Not sending mail to unregistered user [email protected] Not sending mail to unregistered user [email protected] Not sending mail to unregistered user [email protected] Not sending mail to unregistered user [email protected] Not sending mail to unregistered user [email protected] Not sending mail to unregistered user [email protected] Not sending mail to unregistered user [email protected] Not sending mail to unregistered user [email protected] Not sending mail to unregistered user [email protected] Not sending mail to unregistered user [email protected] Not sending mail to unregistered user [email protected] Not sending mail to unregistered user [email protected] Not sending mail to unregistered user [email protected] Not sending mail to unregistered user [email protected] Not sending mail to unregistered user [email protected] Not sending mail to unregistered user [email protected] Not sending mail to unregistered user [email protected] Not sending mail to unregistered user [email protected] Not sending mail to unregistered user [email protected] Not sending mail to unregistered user [email protected] Not sending mail to unregistered user [email protected] Not sending mail to unregistered user [email protected] Not sending mail to unregistered user [email protected] Not sending mail to unregistered user [email protected] Not sending mail to unregistered user [email protected] Not sending mail to unregistered user [email protected] Not sending mail to unregistered user [email protected] Not sending mail to unregistered user [email protected] Not sending mail to unregistered user [email protected] Not sending mail to unregistered user [email protected] Not sending mail to unregistered user [email protected] Not sending mail to unregistered user [email protected] Not sending mail to unregistered user [email protected] Not sending mail to unregistered user [email protected] Not sending mail to unregistered user [email protected] Not sending mail to unregistered user [email protected] Not sending mail to unregistered user [email protected] Not sending mail to unregistered user [email protected] Not sending mail to unregistered user [email protected] Not sending mail to unregistered user [email protected] Not sending mail to unregistered user [email protected] Not sending mail to unregistered user [email protected] Not sending mail to unregistered user [email protected] Not sending mail to unregistered user [email protected] Not sending mail to unregistered user [email protected] Not sending mail to unregistered user [email protected] Not sending mail to unregistered user [email protected] Not sending mail to unregistered user [email protected] Not sending mail to unregistered user [email protected] Not sending mail to unregistered user [email protected] Not sending mail to unregistered user [email protected] Not sending mail to unregistered user [email protected] Not sending mail to unregistered user [email protected] Not sending mail to unregistered user [email protected] Not sending mail to unregistered user [email protected] Not sending mail to unregistered user [email protected] Not sending mail to unregistered user [email protected] Not sending mail to unregistered user [email protected] Not sending mail to unregistered user [email protected] Not sending mail to unregistered user [email protected] Not sending mail to unregistered user [email protected] Not sending mail to unregistered user [email protected] Not sending mail to unregistered user [email protected] Not sending mail to unregistered user [email protected] Not sending mail to unregistered user [email protected] Not sending mail to unregistered user [email protected] Not sending mail to unregistered user [email protected] Not sending mail to unregistered user [email protected] Not sending mail to unregistered user [email protected] Not sending mail to unregistered user [email protected] Not sending mail to unregistered user [email protected] Not sending mail to unregistered user [email protected] Not sending mail to unregistered user [email protected] Not sending mail to unregistered user [email protected]
