See <https://builds.apache.org/job/beam_PostCommit_Python2/1046/display/redirect?page=changes>
Changes: [github] [BEAM-8659] RowJsonTest should test serialization independently ------------------------------------------ [...truncated 589.21 KB...] > Task :sdks:python:test-suites:direct:py2:hdfsIntegrationTest Stopping hdfs_it-jenkins-beam_postcommit_python2-1046_datanode_1 ... Stopping hdfs_it-jenkins-beam_postcommit_python2-1046_namenode_1 ... Stopping hdfs_it-jenkins-beam_postcommit_python2-1046_datanode_1 ... done Stopping hdfs_it-jenkins-beam_postcommit_python2-1046_namenode_1 ... done real 1m38.503s user 0m1.131s sys 0m0.157s + finally + docker-compose -p hdfs_IT-jenkins-beam_PostCommit_Python2-1046 --no-ansi down Removing hdfs_it-jenkins-beam_postcommit_python2-1046_test_1 ... Removing hdfs_it-jenkins-beam_postcommit_python2-1046_datanode_1 ... Removing hdfs_it-jenkins-beam_postcommit_python2-1046_namenode_1 ... Removing hdfs_it-jenkins-beam_postcommit_python2-1046_namenode_1 ... done Removing hdfs_it-jenkins-beam_postcommit_python2-1046_datanode_1 ... done Removing hdfs_it-jenkins-beam_postcommit_python2-1046_test_1 ... done Removing network hdfs_it-jenkins-beam_postcommit_python2-1046_test_net real 0m0.782s user 0m0.559s sys 0m0.118s > Task :sdks:python:test-suites:portable:py2:portableWordCountSparkRunnerBatch WARNING:apache_beam.io.filebasedsink:Deleting 4 existing files in target path matching: -*-of-%(num_shards)05d 19/11/21 20:56:35 INFO org.apache.beam.runners.spark.SparkPipelineRunner: Job BeamApp-jenkins-1121205611-a1de1b24_7047364c-7da6-4ef3-a5ba-b13f3c495fa3: Pipeline translated successfully. Computing outputs INFO:apache_beam.io.filebasedsink:Starting finalize_write threads with num_shards: 4 (skipped: 0), batches: 4, num_threads: 4 INFO:apache_beam.io.filebasedsink:Renamed 4 shards in 0.12 seconds. 19/11/21 20:56:53 INFO org.apache.beam.runners.spark.SparkPipelineRunner: Job BeamApp-jenkins-1121205611-a1de1b24_7047364c-7da6-4ef3-a5ba-b13f3c495fa3 finished. 19/11/21 20:56:53 WARN org.apache.beam.runners.spark.SparkPipelineResult$BatchMode: Collecting monitoring infos is not implemented yet in Spark portable runner. 19/11/21 20:56:53 INFO org.apache.beam.runners.fnexecution.artifact.AbstractArtifactRetrievalService: Manifest at /tmp/beam-temp_rfaAN/artifactsLHwVA7/job_b1469ef5-56f7-4a92-81b2-19ddd0a27c9d/MANIFEST has 1 artifact locations 19/11/21 20:56:53 INFO org.apache.beam.runners.fnexecution.artifact.BeamFileSystemArtifactStagingService: Removed dir /tmp/beam-temp_rfaAN/artifactsLHwVA7/job_b1469ef5-56f7-4a92-81b2-19ddd0a27c9d/ INFO:apache_beam.runners.portability.portable_runner:Job state changed to DONE 19/11/21 20:56:53 INFO org.apache.beam.runners.fnexecution.jobsubmission.InMemoryJobService: Getting job metrics for BeamApp-jenkins-1121205611-a1de1b24_7047364c-7da6-4ef3-a5ba-b13f3c495fa3 19/11/21 20:56:53 INFO org.apache.beam.runners.fnexecution.jobsubmission.InMemoryJobService: Finished getting job metrics for BeamApp-jenkins-1121205611-a1de1b24_7047364c-7da6-4ef3-a5ba-b13f3c495fa3 ERROR:apache_beam.runners.worker.data_plane:Failed to read inputs in the data plane. Traceback (most recent call last): File "<https://builds.apache.org/job/beam_PostCommit_Python2/ws/src/sdks/python/apache_beam/runners/worker/data_plane.py";,> line 272, in _read_inputs for elements in elements_iterator: File "<https://builds.apache.org/job/beam_PostCommit_Python2/ws/src/build/gradleenv/1866363813/local/lib/python2.7/site-packages/grpc/_channel.py";,> line 395, in next return self._next() File "<https://builds.apache.org/job/beam_PostCommit_Python2/ws/src/build/gradleenv/1866363813/local/lib/python2.7/site-packages/grpc/_channel.py";,> line 561, in _next raise self _Rendezvous: <_Rendezvous of RPC that terminated with: status = StatusCode.UNAVAILABLE details = "Socket closed" debug_error_string = "{"created":"@1574369814.320768347","description":"Error received from peer ipv4:127.0.0.1:41083","file":"src/core/lib/surface/call.cc","file_line":1055,"grpc_message":"Socket closed","grpc_status":14}" > Exception in thread run_worker_1-1: Traceback (most recent call last): File "/usr/lib/python2.7/threading.py", line 801, in __bootstrap_inner self.run() File "/usr/lib/python2.7/threading.py", line 754, in run self.__target(*self.__args, **self.__kwargs) File "<https://builds.apache.org/job/beam_PostCommit_Python2/ws/src/sdks/python/apache_beam/runners/worker/sdk_worker.py";,> line 111, in run for work_request in control_stub.Control(get_responses()): File "<https://builds.apache.org/job/beam_PostCommit_Python2/ws/src/build/gradleenv/1866363813/local/lib/python2.7/site-packages/grpc/_channel.py";,> line 395, in next return self._next() File "<https://builds.apache.org/job/beam_PostCommit_Python2/ws/src/build/gradleenv/1866363813/local/lib/python2.7/site-packages/grpc/_channel.py";,> line 561, in _next raise self _Rendezvous: <_Rendezvous of RPC that terminated with: status = StatusCode.UNAVAILABLE details = "Socket closed" debug_error_string = "{"created":"@1574369814.320824870","description":"Error received from peer ipv4:127.0.0.1:44929","file":"src/core/lib/surface/call.cc","file_line":1055,"grpc_message":"Socket closed","grpc_status":14}" > Exception in thread read_grpc_client_inputs: Traceback (most recent call last): File "/usr/lib/python2.7/threading.py", line 801, in __bootstrap_inner self.run() File "/usr/lib/python2.7/threading.py", line 754, in run self.__target(*self.__args, **self.__kwargs) File "<https://builds.apache.org/job/beam_PostCommit_Python2/ws/src/sdks/python/apache_beam/runners/worker/data_plane.py";,> line 286, in <lambda> target=lambda: self._read_inputs(elements_iterator), File "<https://builds.apache.org/job/beam_PostCommit_Python2/ws/src/sdks/python/apache_beam/runners/worker/data_plane.py";,> line 272, in _read_inputs for elements in elements_iterator: File "<https://builds.apache.org/job/beam_PostCommit_Python2/ws/src/build/gradleenv/1866363813/local/lib/python2.7/site-packages/grpc/_channel.py";,> line 395, in next return self._next() File "<https://builds.apache.org/job/beam_PostCommit_Python2/ws/src/build/gradleenv/1866363813/local/lib/python2.7/site-packages/grpc/_channel.py";,> line 561, in _next raise self _Rendezvous: <_Rendezvous of RPC that terminated with: status = StatusCode.UNAVAILABLE details = "Socket closed" debug_error_string = "{"created":"@1574369814.320768347","description":"Error received from peer ipv4:127.0.0.1:41083","file":"src/core/lib/surface/call.cc","file_line":1055,"grpc_message":"Socket closed","grpc_status":14}" > Exception in thread read_state: Traceback (most recent call last): File "/usr/lib/python2.7/threading.py", line 801, in __bootstrap_inner self.run() File "/usr/lib/python2.7/threading.py", line 754, in run self.__target(*self.__args, **self.__kwargs) File "<https://builds.apache.org/job/beam_PostCommit_Python2/ws/src/sdks/python/apache_beam/runners/worker/sdk_worker.py";,> line 532, in pull_responses for response in responses: File "<https://builds.apache.org/job/beam_PostCommit_Python2/ws/src/build/gradleenv/1866363813/local/lib/python2.7/site-packages/grpc/_channel.py";,> line 395, in next return self._next() File "<https://builds.apache.org/job/beam_PostCommit_Python2/ws/src/build/gradleenv/1866363813/local/lib/python2.7/site-packages/grpc/_channel.py";,> line 561, in _next raise self _Rendezvous: <_Rendezvous of RPC that terminated with: status = StatusCode.UNAVAILABLE details = "Socket closed" debug_error_string = "{"created":"@1574369814.320816527","description":"Error received from peer ipv4:127.0.0.1:46243","file":"src/core/lib/surface/call.cc","file_line":1055,"grpc_message":"Socket closed","grpc_status":14}" > > Task :sdks:python:test-suites:portable:py2:postCommitPy2 > Task :sdks:python:test-suites:direct:py2:mongodbioIT Using default tag: latest latest: Pulling from library/mongo 7ddbc47eeb70: Pulling fs layer c1bbdc448b72: Pulling fs layer 8c3b70e39044: Pulling fs layer 45d437916d57: Pulling fs layer e119fb0e0a55: Pulling fs layer 91f0b9bae1ea: Pulling fs layer 53e7c2967f11: Pulling fs layer 69a945568374: Pulling fs layer 93333bc225a7: Pulling fs layer 45d437916d57: Waiting e119fb0e0a55: Waiting b9c10bd6c9bd: Pulling fs layer 53e7c2967f11: Waiting 69a945568374: Waiting 7f4e3538e99c: Pulling fs layer 1164b51d180a: Pulling fs layer a715a7d71f27: Pulling fs layer b9c10bd6c9bd: Waiting 93333bc225a7: Waiting 1164b51d180a: Waiting 7f4e3538e99c: Waiting a715a7d71f27: Waiting 8c3b70e39044: Verifying Checksum 8c3b70e39044: Download complete c1bbdc448b72: Verifying Checksum c1bbdc448b72: Download complete 7ddbc47eeb70: Download complete 45d437916d57: Verifying Checksum 45d437916d57: Download complete e119fb0e0a55: Verifying Checksum e119fb0e0a55: Download complete 69a945568374: Download complete 91f0b9bae1ea: Download complete 53e7c2967f11: Verifying Checksum 53e7c2967f11: Download complete 93333bc225a7: Verifying Checksum 93333bc225a7: Download complete b9c10bd6c9bd: Verifying Checksum b9c10bd6c9bd: Download complete 1164b51d180a: Verifying Checksum 1164b51d180a: Download complete a715a7d71f27: Verifying Checksum a715a7d71f27: Download complete 7ddbc47eeb70: Pull complete c1bbdc448b72: Pull complete 8c3b70e39044: Pull complete 45d437916d57: Pull complete 7f4e3538e99c: Verifying Checksum 7f4e3538e99c: Download complete e119fb0e0a55: Pull complete 91f0b9bae1ea: Pull complete 53e7c2967f11: Pull complete 69a945568374: Pull complete 93333bc225a7: Pull complete b9c10bd6c9bd: Pull complete 7f4e3538e99c: Pull complete 1164b51d180a: Pull complete a715a7d71f27: Pull complete Digest: sha256:1a9478d8188d6be31dd2e8de076d402edf20446e54933aad7ff49f5b457d486c Status: Downloaded newer image for mongo:latest 5c708b0c85b6d3b49fa56961e0f5da294e6adf1fbb5cbcd9aeb4752506c18d49 DEPRECATION: Python 2.7 will reach the end of its life on January 1st, 2020. Please upgrade your Python as Python 2.7 won't be maintained after that date. A future version of pip will drop support for Python 2.7. More details about Python 2 support in pip, can be found at https://pip.pypa.io/en/latest/development/release-process/#python-2-support Obtaining file://<https://builds.apache.org/job/beam_PostCommit_Python2/ws/src/sdks/python> <https://builds.apache.org/job/beam_PostCommit_Python2/ws/src/sdks/python/apache_beam/__init__.py>:84: UserWarning: You are using Apache Beam with Python 2. New releases of Apache Beam will soon support Python 3 only. 'You are using Apache Beam with Python 2. ' INFO:root:Missing pipeline option (runner). Executing pipeline using the default runner: DirectRunner. INFO:__main__:Writing 100000 documents to mongodb <https://builds.apache.org/job/beam_PostCommit_Python2/ws/src/sdks/python/apache_beam/io/mongodbio_it_test.py>:70: FutureWarning: WriteToMongoDB is experimental. known_args.batch_size) INFO:apache_beam.runners.portability.fn_api_runner_transforms:==================== <function annotate_downstream_side_inputs at 0x7fda272c4cf8> ==================== INFO:apache_beam.runners.portability.fn_api_runner_transforms:==================== <function fix_side_input_pcoll_coders at 0x7fda272c4de8> ==================== INFO:apache_beam.runners.portability.fn_api_runner_transforms:==================== <function lift_combiners at 0x7fda272c4e60> ==================== INFO:apache_beam.runners.portability.fn_api_runner_transforms:==================== <function expand_sdf at 0x7fda272c4ed8> ==================== INFO:apache_beam.runners.portability.fn_api_runner_transforms:==================== <function expand_gbk at 0x7fda272c4f50> ==================== INFO:apache_beam.runners.portability.fn_api_runner_transforms:==================== <function sink_flattens at 0x7fda27b910c8> ==================== INFO:apache_beam.runners.portability.fn_api_runner_transforms:==================== <function greedily_fuse at 0x7fda27b91140> ==================== INFO:apache_beam.runners.portability.fn_api_runner_transforms:==================== <function read_to_impulse at 0x7fda27b911b8> ==================== INFO:apache_beam.runners.portability.fn_api_runner_transforms:==================== <function impulse_to_input at 0x7fda27b91230> ==================== INFO:apache_beam.runners.portability.fn_api_runner_transforms:==================== <function inject_timer_pcollections at 0x7fda27b91398> ==================== INFO:apache_beam.runners.portability.fn_api_runner_transforms:==================== <function sort_stages at 0x7fda27b91410> ==================== INFO:apache_beam.runners.portability.fn_api_runner_transforms:==================== <function window_pcollection_coders at 0x7fda27b91488> ==================== INFO:apache_beam.runners.worker.statecache:Creating state cache with size 100 INFO:apache_beam.runners.portability.fn_api_runner:Created Worker handler <apache_beam.runners.portability.fn_api_runner.EmbeddedWorkerHandler object at 0x7fda27499350> for environment urn: "beam:env:embedded_python:v1" INFO:apache_beam.runners.portability.fn_api_runner:Running (ref_AppliedPTransform_Create documents/Read_3)+((ref_AppliedPTransform_WriteToMongoDB/ParDo(_GenerateObjectIdFn)_5)+((ref_AppliedPTransform_WriteToMongoDB/Reshuffle/AddRandomKeys_7)+((ref_AppliedPTransform_WriteToMongoDB/Reshuffle/ReshufflePerKey/Map(reify_timestamps)_9)+(WriteToMongoDB/Reshuffle/ReshufflePerKey/GroupByKey/Write)))) INFO:apache_beam.runners.portability.fn_api_runner:Running ((WriteToMongoDB/Reshuffle/ReshufflePerKey/GroupByKey/Read)+(ref_AppliedPTransform_WriteToMongoDB/Reshuffle/ReshufflePerKey/FlatMap(restore_timestamps)_14))+((ref_AppliedPTransform_WriteToMongoDB/Reshuffle/RemoveRandomKeys_15)+(ref_AppliedPTransform_WriteToMongoDB/ParDo(_WriteMongoFn)_16)) INFO:__main__:Writing 100000 documents to mongodb finished in 51.667 seconds INFO:root:Missing pipeline option (runner). Executing pipeline using the default runner: DirectRunner. INFO:__main__:Reading from mongodb beam_mongodbio_it_db:integration_test_1574369816 <https://builds.apache.org/job/beam_PostCommit_Python2/ws/src/sdks/python/apache_beam/io/mongodbio_it_test.py>:85: FutureWarning: ReadFromMongoDB is experimental. | 'Map' >> beam.Map(lambda doc: doc['number']) INFO:apache_beam.runners.portability.fn_api_runner_transforms:==================== <function annotate_downstream_side_inputs at 0x7fda272c4cf8> ==================== INFO:apache_beam.runners.portability.fn_api_runner_transforms:==================== <function fix_side_input_pcoll_coders at 0x7fda272c4de8> ==================== INFO:apache_beam.runners.portability.fn_api_runner_transforms:==================== <function lift_combiners at 0x7fda272c4e60> ==================== INFO:apache_beam.runners.portability.fn_api_runner_transforms:==================== <function expand_sdf at 0x7fda272c4ed8> ==================== INFO:apache_beam.runners.portability.fn_api_runner_transforms:==================== <function expand_gbk at 0x7fda272c4f50> ==================== INFO:apache_beam.runners.portability.fn_api_runner_transforms:==================== <function sink_flattens at 0x7fda27b910c8> ==================== INFO:apache_beam.runners.portability.fn_api_runner_transforms:==================== <function greedily_fuse at 0x7fda27b91140> ==================== INFO:apache_beam.runners.portability.fn_api_runner_transforms:==================== <function read_to_impulse at 0x7fda27b911b8> ==================== INFO:apache_beam.runners.portability.fn_api_runner_transforms:==================== <function impulse_to_input at 0x7fda27b91230> ==================== INFO:apache_beam.runners.portability.fn_api_runner_transforms:==================== <function inject_timer_pcollections at 0x7fda27b91398> ==================== INFO:apache_beam.runners.portability.fn_api_runner_transforms:==================== <function sort_stages at 0x7fda27b91410> ==================== INFO:apache_beam.runners.portability.fn_api_runner_transforms:==================== <function window_pcollection_coders at 0x7fda27b91488> ==================== INFO:apache_beam.runners.worker.statecache:Creating state cache with size 100 INFO:apache_beam.runners.portability.fn_api_runner:Created Worker handler <apache_beam.runners.portability.fn_api_runner.EmbeddedWorkerHandler object at 0x7fda1d2a5710> for environment urn: "beam:env:embedded_python:v1" INFO:apache_beam.runners.portability.fn_api_runner:Running ((ref_AppliedPTransform_ReadFromMongoDB/Read_3)+((ref_AppliedPTransform_Map_4)+((ref_AppliedPTransform_assert_that/WindowInto(WindowIntoFn)_8)+((ref_AppliedPTransform_assert_that/ToVoidKey_9)+((ref_AppliedPTransform_assert_that/Group/pair_with_1_12)+(assert_that/Group/Flatten/Transcode/1))))))+(assert_that/Group/Flatten/Write/1) INFO:apache_beam.runners.portability.fn_api_runner:Running ((ref_AppliedPTransform_assert_that/Create/Read_7)+((ref_AppliedPTransform_assert_that/Group/pair_with_0_11)+(assert_that/Group/Flatten/Transcode/0)))+(assert_that/Group/Flatten/Write/0) INFO:apache_beam.runners.portability.fn_api_runner:Running (assert_that/Group/Flatten/Read)+(assert_that/Group/GroupByKey/Write) INFO:apache_beam.runners.portability.fn_api_runner:Running (assert_that/Group/GroupByKey/Read)+((ref_AppliedPTransform_assert_that/Group/Map(_merge_tagged_vals_under_key)_18)+((ref_AppliedPTransform_assert_that/Unkey_19)+(ref_AppliedPTransform_assert_that/Match_20))) INFO:__main__:Read 100000 documents from mongodb finished in 21.326 seconds FAILURE: Build failed with an exception. * Where: Build file '<https://builds.apache.org/job/beam_PostCommit_Python2/ws/src/sdks/python/build.gradle'> line: 46 * What went wrong: Execution failed for task ':sdks:python:sdist'. > Process 'command 'sh'' finished with non-zero exit value 1 * Try: Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights. * Get more help at https://help.gradle.org Deprecated Gradle features were used in this build, making it incompatible with Gradle 6.0. Use '--warning-mode all' to show the individual deprecation warnings. See https://docs.gradle.org/5.2.1/userguide/command_line_interface.html#sec:command_line_warnings BUILD FAILED in 6m 46s 114 actionable tasks: 105 executed, 6 from cache, 3 up-to-date Publishing build scan... https://gradle.com/s/cjf2owjjnp65k Build step 'Invoke Gradle script' changed build result to FAILURE Build step 'Invoke Gradle script' marked build as failure --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
