See <https://builds.apache.org/job/beam_LoadTests_Python_GBK_Flink_Batch/42/display/redirect?page=changes>
Changes: [lukasz.gajowy] [BEAM-7775] Remove job_PerformanceTests_Spark [lukasz.gajowy] [BEAM-7775] Remove job_PerformanceTests_Dataflow [je.ik] [BEAM-7785] synchronize watermark update with bundle processing [github] [BEAM-7747] Close the file handle owned by fastavro.write.Writer in [yanzhi.wyl] [BEAM-7696]prepare files to stage also in local master of spark runner. [gleb] Add setFailOnNoMatchingTests(true) [cmach] [BEAM-7794] DynamoDBIOTest is blocking forever [robertwb] Revert "Revert "[BEAM-7060] Migrate to native typing types where [robertwb] [BEAM-7798] Unit test for BundleBasedRunner type hints. [robertwb] [BEAM-7798] Typehints fix for bundle based runner. [robertwb] [BEAM-7801] Fix wordcount_xlang postcommit. [gleb] [BEAM-6783] byte[] breaks in BeamSQL codegen [gleb] [SQL] Annotate BuiltinStringFunctions with Strict [relax] Merge pull request #8943: Schema conversion cleanup [lostluck] [BEAM-7781] Update from Go Version 1.10 to 1.12 [lostluck] [BEAM-7781] Use correct Go gradle tasks [lostluck] [BEAM-7781] Update to latest GoGradle [lostluck] [BEAM-7726] Add tests for DataSource protocol [lostluck] [BEAM-7726] Refactor to prepare for state backing. [lostluck] [BEAM-7726] Rename some SideInput code to State [lostluck] [BEAM-7726] Handle State Backed iterables [pabloem] [BEAM-7680] Removed flaky assertion and separated unit tests in [ehudm] [BEAM-7650] Add docker system prune to inventory job [iemejia] [BEAM-7794] Ignore blocking DynamoDBIOtest class ------------------------------------------ [...truncated 132.27 KB...] reading manifest template 'MANIFEST.in' warning: no files found matching 'README.md' warning: no files found matching 'NOTICE' warning: no files found matching 'LICENSE' writing manifest file 'apache_beam.egg-info/SOURCES.txt' testGroupByKey (apache_beam.testing.load_tests.group_by_key_test.GroupByKeyTest) ... ok ---------------------------------------------------------------------- XML: <https://builds.apache.org/job/beam_LoadTests_Python_GBK_Flink_Batch/ws/src/sdks/python/nosetests.xml> ---------------------------------------------------------------------- Ran 1 test in 84.280s OK Deprecated Gradle features were used in this build, making it incompatible with Gradle 6.0. Use '--warning-mode all' to show the individual deprecation warnings. See https://docs.gradle.org/5.2.1/userguide/command_line_interface.html#sec:command_line_warnings BUILD SUCCESSFUL in 1m 32s 3 actionable tasks: 2 executed, 1 up-to-date Publishing build scan... https://gradle.com/s/josclg5fqhkxs [beam_LoadTests_Python_GBK_Flink_Batch] $ /bin/bash -xe /tmp/jenkins4076652661917238798.sh + echo Changing number of workers to 6 Changing number of workers to 6 [beam_LoadTests_Python_GBK_Flink_Batch] $ /bin/bash -xe /tmp/jenkins8315995938842541961.sh + gcloud dataproc clusters update beam-loadtests-python-gbk-flink-batch-42 --num-workers=6 --quiet Waiting on operation [projects/apache-beam-testing/regions/global/operations/3c5acb8a-9438-3f81-8448-f8192e99fd62]. Waiting for cluster update operation... .....WARNING: Cluster has active YARN containers. If any container is actively writing to HDFS then the downsize operation may block until all writers are stopped. .............................................................................................................................................................................................................................done. Updated [https://dataproc.googleapis.com/v1/projects/apache-beam-testing/regions/global/clusters/beam-loadtests-python-gbk-flink-batch-42]. [beam_LoadTests_Python_GBK_Flink_Batch] $ /bin/bash -xe /tmp/jenkins5053885836945730633.sh + echo src Load test: 2GB of 10B records src src Load test: 2GB of 10B records src [Gradle] - Launching build. [src] $ <https://builds.apache.org/job/beam_LoadTests_Python_GBK_Flink_Batch/ws/src/gradlew> --continue --max-workers=12 -Dorg.gradle.jvmargs=-Xms2g -Dorg.gradle.jvmargs=-Xmx4g -PloadTest.mainClass=apache_beam.testing.load_tests.group_by_key_test:GroupByKeyTest.testGroupByKey -Prunner=PortableRunner '-PloadTest.args=--job_name=load_tests_Python_Flink_Batch_GBK_1_0724124552 --publish_to_big_query=false --project=apache-beam-testing --metrics_dataset=load_test --metrics_table=python_flink_batch_GBK_1 --input_options='{"num_records": 200000000,"key_size": 1,"value_size":9}' --iterations=1 --fanout=1 --parallelism=5 --job_endpoint=localhost:8099 --environment_config=gcr.io/apache-beam-testing/beam_portability/python:latest --environment_type=DOCKER --runner=PortableRunner' :sdks:python:apache_beam:testing:load_tests:run > Task :buildSrc:compileJava NO-SOURCE > Task :buildSrc:compileGroovy UP-TO-DATE > Task :buildSrc:pluginDescriptors UP-TO-DATE > Task :buildSrc:processResources UP-TO-DATE > Task :buildSrc:classes UP-TO-DATE > Task :buildSrc:jar UP-TO-DATE > Task :buildSrc:assemble UP-TO-DATE > Task :buildSrc:spotlessGroovy UP-TO-DATE > Task :buildSrc:spotlessGroovyCheck UP-TO-DATE > Task :buildSrc:spotlessGroovyGradle UP-TO-DATE > Task :buildSrc:spotlessGroovyGradleCheck UP-TO-DATE > Task :buildSrc:spotlessCheck UP-TO-DATE > Task :buildSrc:pluginUnderTestMetadata UP-TO-DATE > Task :buildSrc:compileTestJava NO-SOURCE > Task :buildSrc:compileTestGroovy NO-SOURCE > Task :buildSrc:processTestResources NO-SOURCE > Task :buildSrc:testClasses UP-TO-DATE > Task :buildSrc:test NO-SOURCE > Task :buildSrc:validateTaskProperties UP-TO-DATE > Task :buildSrc:check UP-TO-DATE > Task :buildSrc:build UP-TO-DATE Configuration on demand is an incubating feature. > Task :sdks:python:apache_beam:testing:load_tests:setupVirtualenv UP-TO-DATE > Task :sdks:python:apache_beam:testing:load_tests:installGcpTest DEPRECATION: Python 2.7 will reach the end of its life on January 1st, 2020. Please upgrade your Python as Python 2.7 won't be maintained after that date. A future version of pip will drop support for Python 2.7. More details about Python 2 support in pip, can be found at https://pip.pypa.io/en/latest/development/release-process/#python-2-support Obtaining file://<https://builds.apache.org/job/beam_LoadTests_Python_GBK_Flink_Batch/ws/src/sdks/python> Requirement already satisfied: crcmod<2.0,>=1.7 in <https://builds.apache.org/job/beam_LoadTests_Python_GBK_Flink_Batch/ws/src/build/gradleenv/1329484227/lib/python2.7/site-packages> (from apache-beam==2.15.0.dev0) (1.7) Requirement already satisfied: dill<0.2.10,>=0.2.9 in <https://builds.apache.org/job/beam_LoadTests_Python_GBK_Flink_Batch/ws/src/build/gradleenv/1329484227/lib/python2.7/site-packages> (from apache-beam==2.15.0.dev0) (0.2.9) Requirement already satisfied: fastavro<0.22,>=0.21.4 in <https://builds.apache.org/job/beam_LoadTests_Python_GBK_Flink_Batch/ws/src/build/gradleenv/1329484227/lib/python2.7/site-packages> (from apache-beam==2.15.0.dev0) (0.21.24) Requirement already satisfied: future<1.0.0,>=0.16.0 in <https://builds.apache.org/job/beam_LoadTests_Python_GBK_Flink_Batch/ws/src/build/gradleenv/1329484227/lib/python2.7/site-packages> (from apache-beam==2.15.0.dev0) (0.16.0) Requirement already satisfied: grpcio<2,>=1.8 in <https://builds.apache.org/job/beam_LoadTests_Python_GBK_Flink_Batch/ws/src/build/gradleenv/1329484227/lib/python2.7/site-packages> (from apache-beam==2.15.0.dev0) (1.22.0) Requirement already satisfied: hdfs<3.0.0,>=2.1.0 in <https://builds.apache.org/job/beam_LoadTests_Python_GBK_Flink_Batch/ws/src/build/gradleenv/1329484227/lib/python2.7/site-packages> (from apache-beam==2.15.0.dev0) (2.5.8) Requirement already satisfied: httplib2<=0.12.0,>=0.8 in <https://builds.apache.org/job/beam_LoadTests_Python_GBK_Flink_Batch/ws/src/build/gradleenv/1329484227/lib/python2.7/site-packages> (from apache-beam==2.15.0.dev0) (0.12.0) Requirement already satisfied: mock<3.0.0,>=1.0.1 in <https://builds.apache.org/job/beam_LoadTests_Python_GBK_Flink_Batch/ws/src/build/gradleenv/1329484227/lib/python2.7/site-packages> (from apache-beam==2.15.0.dev0) (2.0.0) Requirement already satisfied: pymongo<4.0.0,>=3.8.0 in <https://builds.apache.org/job/beam_LoadTests_Python_GBK_Flink_Batch/ws/src/build/gradleenv/1329484227/lib/python2.7/site-packages> (from apache-beam==2.15.0.dev0) (3.8.0) Requirement already satisfied: oauth2client<4,>=2.0.1 in <https://builds.apache.org/job/beam_LoadTests_Python_GBK_Flink_Batch/ws/src/build/gradleenv/1329484227/lib/python2.7/site-packages> (from apache-beam==2.15.0.dev0) (3.0.0) Requirement already satisfied: protobuf<4,>=3.5.0.post1 in <https://builds.apache.org/job/beam_LoadTests_Python_GBK_Flink_Batch/ws/src/build/gradleenv/1329484227/lib/python2.7/site-packages> (from apache-beam==2.15.0.dev0) (3.9.0) Requirement already satisfied: pydot<2,>=1.2.0 in <https://builds.apache.org/job/beam_LoadTests_Python_GBK_Flink_Batch/ws/src/build/gradleenv/1329484227/lib/python2.7/site-packages> (from apache-beam==2.15.0.dev0) (1.4.1) Requirement already satisfied: pytz>=2018.3 in <https://builds.apache.org/job/beam_LoadTests_Python_GBK_Flink_Batch/ws/src/build/gradleenv/1329484227/lib/python2.7/site-packages> (from apache-beam==2.15.0.dev0) (2019.1) Requirement already satisfied: pyyaml<4.0.0,>=3.12 in <https://builds.apache.org/job/beam_LoadTests_Python_GBK_Flink_Batch/ws/src/build/gradleenv/1329484227/lib/python2.7/site-packages> (from apache-beam==2.15.0.dev0) (3.13) Requirement already satisfied: avro<2.0.0,>=1.8.1 in <https://builds.apache.org/job/beam_LoadTests_Python_GBK_Flink_Batch/ws/src/build/gradleenv/1329484227/lib/python2.7/site-packages> (from apache-beam==2.15.0.dev0) (1.9.0) Requirement already satisfied: futures<4.0.0,>=3.2.0 in <https://builds.apache.org/job/beam_LoadTests_Python_GBK_Flink_Batch/ws/src/build/gradleenv/1329484227/lib/python2.7/site-packages> (from apache-beam==2.15.0.dev0) (3.3.0) Requirement already satisfied: pyvcf<0.7.0,>=0.6.8 in <https://builds.apache.org/job/beam_LoadTests_Python_GBK_Flink_Batch/ws/src/build/gradleenv/1329484227/lib/python2.7/site-packages> (from apache-beam==2.15.0.dev0) (0.6.8) Requirement already satisfied: typing<3.7.0,>=3.6.0 in <https://builds.apache.org/job/beam_LoadTests_Python_GBK_Flink_Batch/ws/src/build/gradleenv/1329484227/lib/python2.7/site-packages> (from apache-beam==2.15.0.dev0) (3.6.6) Requirement already satisfied: pyarrow<0.15.0,>=0.11.1 in <https://builds.apache.org/job/beam_LoadTests_Python_GBK_Flink_Batch/ws/src/build/gradleenv/1329484227/lib/python2.7/site-packages> (from apache-beam==2.15.0.dev0) (0.14.1) Requirement already satisfied: cachetools<4,>=3.1.0 in <https://builds.apache.org/job/beam_LoadTests_Python_GBK_Flink_Batch/ws/src/build/gradleenv/1329484227/lib/python2.7/site-packages> (from apache-beam==2.15.0.dev0) (3.1.1) Requirement already satisfied: google-apitools<0.5.29,>=0.5.28 in <https://builds.apache.org/job/beam_LoadTests_Python_GBK_Flink_Batch/ws/src/build/gradleenv/1329484227/lib/python2.7/site-packages> (from apache-beam==2.15.0.dev0) (0.5.28) Requirement already satisfied: google-cloud-datastore<1.8.0,>=1.7.1 in <https://builds.apache.org/job/beam_LoadTests_Python_GBK_Flink_Batch/ws/src/build/gradleenv/1329484227/lib/python2.7/site-packages> (from apache-beam==2.15.0.dev0) (1.7.4) Requirement already satisfied: google-cloud-pubsub<0.40.0,>=0.39.0 in <https://builds.apache.org/job/beam_LoadTests_Python_GBK_Flink_Batch/ws/src/build/gradleenv/1329484227/lib/python2.7/site-packages> (from apache-beam==2.15.0.dev0) (0.39.1) Requirement already satisfied: google-cloud-bigquery<1.7.0,>=1.6.0 in <https://builds.apache.org/job/beam_LoadTests_Python_GBK_Flink_Batch/ws/src/build/gradleenv/1329484227/lib/python2.7/site-packages> (from apache-beam==2.15.0.dev0) (1.6.1) Requirement already satisfied: google-cloud-core<0.30.0,>=0.28.1 in <https://builds.apache.org/job/beam_LoadTests_Python_GBK_Flink_Batch/ws/src/build/gradleenv/1329484227/lib/python2.7/site-packages> (from apache-beam==2.15.0.dev0) (0.29.1) Requirement already satisfied: google-cloud-bigtable<0.33.0,>=0.31.1 in <https://builds.apache.org/job/beam_LoadTests_Python_GBK_Flink_Batch/ws/src/build/gradleenv/1329484227/lib/python2.7/site-packages> (from apache-beam==2.15.0.dev0) (0.32.2) Requirement already satisfied: proto-google-cloud-datastore-v1<=0.90.4,>=0.90.0 in <https://builds.apache.org/job/beam_LoadTests_Python_GBK_Flink_Batch/ws/src/build/gradleenv/1329484227/lib/python2.7/site-packages> (from apache-beam==2.15.0.dev0) (0.90.4) Requirement already satisfied: googledatastore<7.1,>=7.0.1 in <https://builds.apache.org/job/beam_LoadTests_Python_GBK_Flink_Batch/ws/src/build/gradleenv/1329484227/lib/python2.7/site-packages> (from apache-beam==2.15.0.dev0) (7.0.2) Requirement already satisfied: nose>=1.3.7 in <https://builds.apache.org/job/beam_LoadTests_Python_GBK_Flink_Batch/ws/src/build/gradleenv/1329484227/lib/python2.7/site-packages> (from apache-beam==2.15.0.dev0) (1.3.7) Requirement already satisfied: nose_xunitmp>=0.4.1 in <https://builds.apache.org/job/beam_LoadTests_Python_GBK_Flink_Batch/ws/src/build/gradleenv/1329484227/lib/python2.7/site-packages> (from apache-beam==2.15.0.dev0) (0.4.1) Requirement already satisfied: numpy<2,>=1.14.3 in <https://builds.apache.org/job/beam_LoadTests_Python_GBK_Flink_Batch/ws/src/build/gradleenv/1329484227/lib/python2.7/site-packages> (from apache-beam==2.15.0.dev0) (1.16.4) Requirement already satisfied: pandas<0.24,>=0.23.4 in <https://builds.apache.org/job/beam_LoadTests_Python_GBK_Flink_Batch/ws/src/build/gradleenv/1329484227/lib/python2.7/site-packages> (from apache-beam==2.15.0.dev0) (0.23.4) Requirement already satisfied: parameterized<0.7.0,>=0.6.0 in <https://builds.apache.org/job/beam_LoadTests_Python_GBK_Flink_Batch/ws/src/build/gradleenv/1329484227/lib/python2.7/site-packages> (from apache-beam==2.15.0.dev0) (0.6.3) Requirement already satisfied: pyhamcrest<2.0,>=1.9 in <https://builds.apache.org/job/beam_LoadTests_Python_GBK_Flink_Batch/ws/src/build/gradleenv/1329484227/lib/python2.7/site-packages> (from apache-beam==2.15.0.dev0) (1.9.0) Requirement already satisfied: tenacity<6.0,>=5.0.2 in <https://builds.apache.org/job/beam_LoadTests_Python_GBK_Flink_Batch/ws/src/build/gradleenv/1329484227/lib/python2.7/site-packages> (from apache-beam==2.15.0.dev0) (5.0.4) Requirement already satisfied: six>=1.5.2 in <https://builds.apache.org/job/beam_LoadTests_Python_GBK_Flink_Batch/ws/src/build/gradleenv/1329484227/lib/python2.7/site-packages> (from grpcio<2,>=1.8->apache-beam==2.15.0.dev0) (1.12.0) Requirement already satisfied: enum34>=1.0.4; python_version < "3.4" in <https://builds.apache.org/job/beam_LoadTests_Python_GBK_Flink_Batch/ws/src/build/gradleenv/1329484227/lib/python2.7/site-packages> (from grpcio<2,>=1.8->apache-beam==2.15.0.dev0) (1.1.6) Requirement already satisfied: docopt in <https://builds.apache.org/job/beam_LoadTests_Python_GBK_Flink_Batch/ws/src/build/gradleenv/1329484227/lib/python2.7/site-packages> (from hdfs<3.0.0,>=2.1.0->apache-beam==2.15.0.dev0) (0.6.2) Requirement already satisfied: requests>=2.7.0 in <https://builds.apache.org/job/beam_LoadTests_Python_GBK_Flink_Batch/ws/src/build/gradleenv/1329484227/lib/python2.7/site-packages> (from hdfs<3.0.0,>=2.1.0->apache-beam==2.15.0.dev0) (2.22.0) Requirement already satisfied: pbr>=0.11 in <https://builds.apache.org/job/beam_LoadTests_Python_GBK_Flink_Batch/ws/src/build/gradleenv/1329484227/lib/python2.7/site-packages> (from mock<3.0.0,>=1.0.1->apache-beam==2.15.0.dev0) (5.4.1) Requirement already satisfied: funcsigs>=1; python_version < "3.3" in <https://builds.apache.org/job/beam_LoadTests_Python_GBK_Flink_Batch/ws/src/build/gradleenv/1329484227/lib/python2.7/site-packages> (from mock<3.0.0,>=1.0.1->apache-beam==2.15.0.dev0) (1.0.2) Requirement already satisfied: pyasn1>=0.1.7 in <https://builds.apache.org/job/beam_LoadTests_Python_GBK_Flink_Batch/ws/src/build/gradleenv/1329484227/lib/python2.7/site-packages> (from oauth2client<4,>=2.0.1->apache-beam==2.15.0.dev0) (0.4.5) Requirement already satisfied: pyasn1-modules>=0.0.5 in <https://builds.apache.org/job/beam_LoadTests_Python_GBK_Flink_Batch/ws/src/build/gradleenv/1329484227/lib/python2.7/site-packages> (from oauth2client<4,>=2.0.1->apache-beam==2.15.0.dev0) (0.2.5) Requirement already satisfied: rsa>=3.1.4 in <https://builds.apache.org/job/beam_LoadTests_Python_GBK_Flink_Batch/ws/src/build/gradleenv/1329484227/lib/python2.7/site-packages> (from oauth2client<4,>=2.0.1->apache-beam==2.15.0.dev0) (4.0) Requirement already satisfied: setuptools in <https://builds.apache.org/job/beam_LoadTests_Python_GBK_Flink_Batch/ws/src/build/gradleenv/1329484227/lib/python2.7/site-packages> (from protobuf<4,>=3.5.0.post1->apache-beam==2.15.0.dev0) (41.0.1) Requirement already satisfied: pyparsing>=2.1.4 in <https://builds.apache.org/job/beam_LoadTests_Python_GBK_Flink_Batch/ws/src/build/gradleenv/1329484227/lib/python2.7/site-packages> (from pydot<2,>=1.2.0->apache-beam==2.15.0.dev0) (2.4.0) Requirement already satisfied: fasteners>=0.14 in <https://builds.apache.org/job/beam_LoadTests_Python_GBK_Flink_Batch/ws/src/build/gradleenv/1329484227/lib/python2.7/site-packages> (from google-apitools<0.5.29,>=0.5.28->apache-beam==2.15.0.dev0) (0.15) Requirement already satisfied: google-api-core[grpc]<2.0.0dev,>=1.6.0 in <https://builds.apache.org/job/beam_LoadTests_Python_GBK_Flink_Batch/ws/src/build/gradleenv/1329484227/lib/python2.7/site-packages> (from google-cloud-datastore<1.8.0,>=1.7.1->apache-beam==2.15.0.dev0) (1.14.0) Requirement already satisfied: grpc-google-iam-v1<0.12dev,>=0.11.4 in <https://builds.apache.org/job/beam_LoadTests_Python_GBK_Flink_Batch/ws/src/build/gradleenv/1329484227/lib/python2.7/site-packages> (from google-cloud-pubsub<0.40.0,>=0.39.0->apache-beam==2.15.0.dev0) (0.11.4) Requirement already satisfied: google-resumable-media>=0.2.1 in <https://builds.apache.org/job/beam_LoadTests_Python_GBK_Flink_Batch/ws/src/build/gradleenv/1329484227/lib/python2.7/site-packages> (from google-cloud-bigquery<1.7.0,>=1.6.0->apache-beam==2.15.0.dev0) (0.3.2) Requirement already satisfied: googleapis-common-protos<2.0dev,>=1.5.2 in <https://builds.apache.org/job/beam_LoadTests_Python_GBK_Flink_Batch/ws/src/build/gradleenv/1329484227/lib/python2.7/site-packages> (from proto-google-cloud-datastore-v1<=0.90.4,>=0.90.0->apache-beam==2.15.0.dev0) (1.6.0) Requirement already satisfied: python-dateutil>=2.5.0 in <https://builds.apache.org/job/beam_LoadTests_Python_GBK_Flink_Batch/ws/src/build/gradleenv/1329484227/lib/python2.7/site-packages> (from pandas<0.24,>=0.23.4->apache-beam==2.15.0.dev0) (2.8.0) Requirement already satisfied: monotonic>=0.6; python_version == "2.7" in <https://builds.apache.org/job/beam_LoadTests_Python_GBK_Flink_Batch/ws/src/build/gradleenv/1329484227/lib/python2.7/site-packages> (from tenacity<6.0,>=5.0.2->apache-beam==2.15.0.dev0) (1.5) Requirement already satisfied: urllib3!=1.25.0,!=1.25.1,<1.26,>=1.21.1 in <https://builds.apache.org/job/beam_LoadTests_Python_GBK_Flink_Batch/ws/src/build/gradleenv/1329484227/lib/python2.7/site-packages> (from requests>=2.7.0->hdfs<3.0.0,>=2.1.0->apache-beam==2.15.0.dev0) (1.25.3) Requirement already satisfied: certifi>=2017.4.17 in <https://builds.apache.org/job/beam_LoadTests_Python_GBK_Flink_Batch/ws/src/build/gradleenv/1329484227/lib/python2.7/site-packages> (from requests>=2.7.0->hdfs<3.0.0,>=2.1.0->apache-beam==2.15.0.dev0) (2019.6.16) Requirement already satisfied: chardet<3.1.0,>=3.0.2 in <https://builds.apache.org/job/beam_LoadTests_Python_GBK_Flink_Batch/ws/src/build/gradleenv/1329484227/lib/python2.7/site-packages> (from requests>=2.7.0->hdfs<3.0.0,>=2.1.0->apache-beam==2.15.0.dev0) (3.0.4) Requirement already satisfied: idna<2.9,>=2.5 in <https://builds.apache.org/job/beam_LoadTests_Python_GBK_Flink_Batch/ws/src/build/gradleenv/1329484227/lib/python2.7/site-packages> (from requests>=2.7.0->hdfs<3.0.0,>=2.1.0->apache-beam==2.15.0.dev0) (2.8) Requirement already satisfied: google-auth<2.0dev,>=0.4.0 in <https://builds.apache.org/job/beam_LoadTests_Python_GBK_Flink_Batch/ws/src/build/gradleenv/1329484227/lib/python2.7/site-packages> (from google-api-core[grpc]<2.0.0dev,>=1.6.0->google-cloud-datastore<1.8.0,>=1.7.1->apache-beam==2.15.0.dev0) (1.6.3) Installing collected packages: apache-beam Found existing installation: apache-beam 2.15.0.dev0 Not uninstalling apache-beam at <https://builds.apache.org/job/beam_LoadTests_Python_GBK_Flink_Batch/ws/src/sdks/python,> outside environment <https://builds.apache.org/job/beam_LoadTests_Python_GBK_Flink_Batch/ws/src/build/gradleenv/1329484227> Can't uninstall 'apache-beam'. No files were found to uninstall. Running setup.py develop for apache-beam Successfully installed apache-beam > Task :sdks:python:apache_beam:testing:load_tests:run <https://builds.apache.org/job/beam_LoadTests_Python_GBK_Flink_Batch/ws/src/build/gradleenv/1329484227/local/lib/python2.7/site-packages/setuptools/dist.py>:472: UserWarning: Normalizing '2.15.0.dev' to '2.15.0.dev0' normalized_version, running nosetests running egg_info writing requirements to apache_beam.egg-info/requires.txt writing apache_beam.egg-info/PKG-INFO writing top-level names to apache_beam.egg-info/top_level.txt writing dependency_links to apache_beam.egg-info/dependency_links.txt writing entry points to apache_beam.egg-info/entry_points.txt reading manifest file 'apache_beam.egg-info/SOURCES.txt' reading manifest template 'MANIFEST.in' warning: no files found matching 'README.md' warning: no files found matching 'NOTICE' warning: no files found matching 'LICENSE' writing manifest file 'apache_beam.egg-info/SOURCES.txt' testGroupByKey (apache_beam.testing.load_tests.group_by_key_test.GroupByKeyTest) ... ERROR ====================================================================== ERROR: testGroupByKey (apache_beam.testing.load_tests.group_by_key_test.GroupByKeyTest) ---------------------------------------------------------------------- Traceback (most recent call last): File "<https://builds.apache.org/job/beam_LoadTests_Python_GBK_Flink_Batch/ws/src/sdks/python/apache_beam/testing/load_tests/load_test.py";,> line 70, in tearDown result = self.pipeline.run() File "<https://builds.apache.org/job/beam_LoadTests_Python_GBK_Flink_Batch/ws/src/sdks/python/apache_beam/testing/test_pipeline.py";,> line 109, in run state = result.wait_until_finish() File "<https://builds.apache.org/job/beam_LoadTests_Python_GBK_Flink_Batch/ws/src/sdks/python/apache_beam/runners/portability/portable_runner.py";,> line 446, in wait_until_finish self._job_id, self._state, self._last_error_message())) RuntimeError: Pipeline load_tests_Python_Flink_Batch_GBK_1_0724124552_42633f64-c21e-4b76-b6c0-f8b2b63d26bc failed in state FAILED: java.net.UnknownHostException: beam-loadtests-python-gbk-flink-batch-42-w-10.c.apache-beam-testing.internal -------------------- >> begin captured logging << -------------------- root: INFO: Generating grammar tables from /usr/lib/python2.7/lib2to3/Grammar.txt root: INFO: Generating grammar tables from /usr/lib/python2.7/lib2to3/PatternGrammar.txt root: INFO: Metrics will not be collected root: INFO: ==================== <function lift_combiners at 0x7fc2a8ed20c8> ==================== root: DEBUG: 12 [1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1] root: DEBUG: Stages: ['ref_AppliedPTransform_Read/Impulse_3\n Read/Impulse:beam:transform:impulse:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Read/Split_4\n Read/Split:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Read/Reshuffle/AddRandomKeys_6\n Read/Reshuffle/AddRandomKeys:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Read/Reshuffle/ReshufflePerKey/Map(reify_timestamps)_8\n Read/Reshuffle/ReshufflePerKey/Map(reify_timestamps):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Read/Reshuffle/ReshufflePerKey/GroupByKey_9\n Read/Reshuffle/ReshufflePerKey/GroupByKey:beam:transform:group_by_key:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Read/Reshuffle/ReshufflePerKey/FlatMap(restore_timestamps)_13\n Read/Reshuffle/ReshufflePerKey/FlatMap(restore_timestamps):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Read/Reshuffle/RemoveRandomKeys_14\n Read/Reshuffle/RemoveRandomKeys:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Read/ReadSplits_15\n Read/ReadSplits:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Measure time: Start_16\n Measure time: Start:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_GroupByKey 0_17\n GroupByKey 0:beam:transform:group_by_key:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Ungroup 0_21\n Ungroup 0:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Measure time: End 0_22\n Measure time: End 0:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>'] root: INFO: ==================== <function expand_sdf at 0x7fc2a8ed2140> ==================== root: DEBUG: 12 [1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1] root: DEBUG: Stages: ['ref_AppliedPTransform_Read/Impulse_3\n Read/Impulse:beam:transform:impulse:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Read/Split_4\n Read/Split:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Read/Reshuffle/AddRandomKeys_6\n Read/Reshuffle/AddRandomKeys:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Read/Reshuffle/ReshufflePerKey/Map(reify_timestamps)_8\n Read/Reshuffle/ReshufflePerKey/Map(reify_timestamps):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Read/Reshuffle/ReshufflePerKey/GroupByKey_9\n Read/Reshuffle/ReshufflePerKey/GroupByKey:beam:transform:group_by_key:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Read/Reshuffle/ReshufflePerKey/FlatMap(restore_timestamps)_13\n Read/Reshuffle/ReshufflePerKey/FlatMap(restore_timestamps):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Read/Reshuffle/RemoveRandomKeys_14\n Read/Reshuffle/RemoveRandomKeys:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Read/ReadSplits_15\n Read/ReadSplits:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Measure time: Start_16\n Measure time: Start:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_GroupByKey 0_17\n GroupByKey 0:beam:transform:group_by_key:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Ungroup 0_21\n Ungroup 0:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Measure time: End 0_22\n Measure time: End 0:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>'] root: DEBUG: Runner option 'job_name' was already added root: DEBUG: Runner option 'runner' was already added root: DEBUG: Runner option 'temp_location' was already added root: DEBUG: Runner option 'environment_cache_millis' was already added root: DEBUG: Runner option 'job_endpoint' was already added root: DEBUG: Runner option 'sdk_worker_parallelism' was already added root: DEBUG: Runner option 'streaming' was already added root: DEBUG: Runner option 'files_to_stage' was already added root: DEBUG: Runner option 'dataflow_kms_key' was already added root: DEBUG: Runner option 'enable_streaming_engine' was already added root: DEBUG: Runner option 'project' was already added root: DEBUG: Runner option 'zone' was already added root: WARNING: Discarding unparseable args: ['--publish_to_big_query=false', '--metrics_dataset=load_test', '--metrics_table=python_flink_batch_GBK_1', '--input_options={"num_records": 200000000,"key_size": 1,"value_size":9}', '--iterations=1', '--fanout=1'] root: INFO: Job state changed to RUNNING root: DEBUG: java.net.UnknownHostException: beam-loadtests-python-gbk-flink-batch-42-w-10.c.apache-beam-testing.internal at java.net.InetAddress.getAllByName0(InetAddress.java:1281) at java.net.InetAddress.getAllByName(InetAddress.java:1193) at java.net.InetAddress.getAllByName(InetAddress.java:1127) at java.net.InetAddress.getByName(InetAddress.java:1077) at org.apache.flink.runtime.rpc.akka.AkkaRpcServiceUtils.getRpcUrl(AkkaRpcServiceUtils.java:167) at org.apache.flink.runtime.rpc.akka.AkkaRpcServiceUtils.getRpcUrl(AkkaRpcServiceUtils.java:133) at org.apache.flink.runtime.highavailability.HighAvailabilityServicesUtils.createHighAvailabilityServices(HighAvailabilityServicesUtils.java:89) at org.apache.flink.client.program.ClusterClient.<init>(ClusterClient.java:159) at org.apache.flink.client.program.rest.RestClusterClient.<init>(RestClusterClient.java:185) at org.apache.flink.client.program.rest.RestClusterClient.<init>(RestClusterClient.java:158) at org.apache.flink.client.RemoteExecutor.start(RemoteExecutor.java:152) at org.apache.flink.client.RemoteExecutor.executePlanWithJars(RemoteExecutor.java:202) at org.apache.flink.client.RemoteExecutor.executePlan(RemoteExecutor.java:187) at org.apache.flink.api.java.RemoteEnvironment.execute(RemoteEnvironment.java:173) at org.apache.beam.runners.flink.FlinkBatchPortablePipelineTranslator$BatchTranslationContext.execute(FlinkBatchPortablePipelineTranslator.java:200) at org.apache.beam.runners.flink.FlinkPipelineRunner.runPipelineWithTranslator(FlinkPipelineRunner.java:87) at org.apache.beam.runners.flink.FlinkPipelineRunner.run(FlinkPipelineRunner.java:63) at org.apache.beam.runners.fnexecution.jobsubmission.JobInvocation.runPipeline(JobInvocation.java:74) at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.util.concurrent.TrustedListenableFutureTask$TrustedFutureInterruptibleTask.runInterruptibly(TrustedListenableFutureTask.java:125) at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.util.concurrent.InterruptibleTask.run(InterruptibleTask.java:57) at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.util.concurrent.TrustedListenableFutureTask.run(TrustedListenableFutureTask.java:78) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) root: ERROR: java.net.UnknownHostException: beam-loadtests-python-gbk-flink-batch-42-w-10.c.apache-beam-testing.internal root: INFO: Job state changed to FAILED --------------------- >> end captured logging << --------------------- ---------------------------------------------------------------------- XML: <https://builds.apache.org/job/beam_LoadTests_Python_GBK_Flink_Batch/ws/src/sdks/python/nosetests.xml> ---------------------------------------------------------------------- Ran 1 test in 1.104s FAILED (errors=1) > Task :sdks:python:apache_beam:testing:load_tests:run FAILED FAILURE: Build failed with an exception. * Where: Build file '<https://builds.apache.org/job/beam_LoadTests_Python_GBK_Flink_Batch/ws/src/sdks/python/apache_beam/testing/load_tests/build.gradle'> line: 49 * What went wrong: Execution failed for task ':sdks:python:apache_beam:testing:load_tests:run'. > error occurred * Try: Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights. * Get more help at https://help.gradle.org Deprecated Gradle features were used in this build, making it incompatible with Gradle 6.0. Use '--warning-mode all' to show the individual deprecation warnings. See https://docs.gradle.org/5.2.1/userguide/command_line_interface.html#sec:command_line_warnings BUILD FAILED in 5s 3 actionable tasks: 2 executed, 1 up-to-date Publishing build scan... https://gradle.com/s/brwr474ytxh6s Build step 'Invoke Gradle script' changed build result to FAILURE Build step 'Invoke Gradle script' marked build as failure --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
