See
<https://builds.apache.org/job/beam_PostCommit_Python37/1807/display/redirect?page=changes>
Changes:
[boyuanz] Update verify_release_build script to run python tests with dev
version.
[jvilcek] [BEAM-9360] Fix equivalence check for FieldType
------------------------------------------
[...truncated 120.77 KB...]
url_read = urlopen(url)
File "/usr/lib/python3.7/urllib/request.py", line 222, in urlopen
return opener.open(url, data, timeout)
File "/usr/lib/python3.7/urllib/request.py", line 510, in open
req = Request(fullurl, data)
File "/usr/lib/python3.7/urllib/request.py", line 328, in __init__
self.full_url = url
File "/usr/lib/python3.7/urllib/request.py", line 354, in full_url
self._parse()
File "/usr/lib/python3.7/urllib/request.py", line 383, in _parse
raise ValueError("unknown url type: %r" % self.full_url)
ValueError: unknown url type:
'<https://builds.apache.org/job/beam_PostCommit_Python37/ws/src/runners/flink/1.9/job-server/build/libs/beam-runners-flink-1.9-job-server-2.21.0-SNAPSHOT.jar'>
> Task :sdks:python:test-suites:portable:py37:portableWordCountFlinkRunnerBatch
> FAILED
> Task
> :sdks:python:test-suites:portable:py37:portableWordCountFlinkRunnerStreaming
> FAILED
WARNING:apache_beam.utils.subprocess_server:Downloading job server jar from
<https://builds.apache.org/job/beam_PostCommit_Python37/ws/src/runners/flink/1.9/job-server/build/libs/beam-runners-flink-1.9-job-server-2.21.0-SNAPSHOT.jar>
Traceback (most recent call last):
File "/usr/lib/python3.7/runpy.py", line 193, in _run_module_as_main
"__main__", mod_spec)
File "/usr/lib/python3.7/runpy.py", line 85, in _run_code
exec(code, run_globals)
File
"<https://builds.apache.org/job/beam_PostCommit_Python37/ws/src/build/gradleenv/2022703441/lib/python3.7/site-packages/apache_beam/examples/wordcount.py";,>
line 142, in <module>
run()
File
"<https://builds.apache.org/job/beam_PostCommit_Python37/ws/src/build/gradleenv/2022703441/lib/python3.7/site-packages/apache_beam/examples/wordcount.py";,>
line 121, in run
result = p.run()
File
"<https://builds.apache.org/job/beam_PostCommit_Python37/ws/src/build/gradleenv/2022703441/lib/python3.7/site-packages/apache_beam/pipeline.py";,>
line 495, in run
self._options).run(False)
File
"<https://builds.apache.org/job/beam_PostCommit_Python37/ws/src/build/gradleenv/2022703441/lib/python3.7/site-packages/apache_beam/pipeline.py";,>
line 508, in run
return self.runner.run_pipeline(self, self._options)
File
"<https://builds.apache.org/job/beam_PostCommit_Python37/ws/src/build/gradleenv/2022703441/lib/python3.7/site-packages/apache_beam/runners/portability/flink_runner.py";,>
line 47, in run_pipeline
return super(FlinkRunner, self).run_pipeline(pipeline, options)
File
"<https://builds.apache.org/job/beam_PostCommit_Python37/ws/src/build/gradleenv/2022703441/lib/python3.7/site-packages/apache_beam/runners/portability/portable_runner.py";,>
line 386, in run_pipeline
job_service_handle = self.create_job_service(options)
File
"<https://builds.apache.org/job/beam_PostCommit_Python37/ws/src/build/gradleenv/2022703441/lib/python3.7/site-packages/apache_beam/runners/portability/portable_runner.py";,>
line 293, in create_job_service
return JobServiceHandle(server.start(), options)
File
"<https://builds.apache.org/job/beam_PostCommit_Python37/ws/src/build/gradleenv/2022703441/lib/python3.7/site-packages/apache_beam/runners/portability/job_server.py";,>
line 86, in start
self._endpoint = self._job_server.start()
File
"<https://builds.apache.org/job/beam_PostCommit_Python37/ws/src/build/gradleenv/2022703441/lib/python3.7/site-packages/apache_beam/runners/portability/job_server.py";,>
line 111, in start
cmd, endpoint = self.subprocess_cmd_and_endpoint()
File
"<https://builds.apache.org/job/beam_PostCommit_Python37/ws/src/build/gradleenv/2022703441/lib/python3.7/site-packages/apache_beam/runners/portability/job_server.py";,>
line 156, in subprocess_cmd_and_endpoint
jar_path = self.local_jar(self.path_to_jar())
File
"<https://builds.apache.org/job/beam_PostCommit_Python37/ws/src/build/gradleenv/2022703441/lib/python3.7/site-packages/apache_beam/runners/portability/job_server.py";,>
line 153, in local_jar
return subprocess_server.JavaJarServer.local_jar(url)
File
"<https://builds.apache.org/job/beam_PostCommit_Python37/ws/src/build/gradleenv/2022703441/lib/python3.7/site-packages/apache_beam/utils/subprocess_server.py";,>
line 204, in local_jar
url_read = urlopen(url)
File "/usr/lib/python3.7/urllib/request.py", line 222, in urlopen
return opener.open(url, data, timeout)
File "/usr/lib/python3.7/urllib/request.py", line 510, in open
req = Request(fullurl, data)
File "/usr/lib/python3.7/urllib/request.py", line 328, in __init__
self.full_url = url
File "/usr/lib/python3.7/urllib/request.py", line 354, in full_url
self._parse()
File "/usr/lib/python3.7/urllib/request.py", line 383, in _parse
raise ValueError("unknown url type: %r" % self.full_url)
ValueError: unknown url type:
'<https://builds.apache.org/job/beam_PostCommit_Python37/ws/src/runners/flink/1.9/job-server/build/libs/beam-runners-flink-1.9-job-server-2.21.0-SNAPSHOT.jar'>
> Task :sdks:python:test-suites:portable:py37:portableWordCountSparkRunnerBatch
INFO:apache_beam.runners.portability.fn_api_runner_transforms:====================
<function lift_combiners at 0x7f0e900e9400> ====================
WARNING:apache_beam.utils.subprocess_server:Starting service with ['java'
'-jar'
'<https://builds.apache.org/job/beam_PostCommit_Python37/ws/src/runners/spark/job-server/build/libs/beam-runners-spark-job-server-2.21.0-SNAPSHOT.jar'>
'--spark-master-url' 'local[4]' '--artifacts-dir'
'/tmp/beam-tempihcfh38t/artifactsr1hv8fgs' '--job-port' '45705'
'--artifact-port' '0' '--expansion-port' '0']
20/03/05 00:31:26 INFO
org.apache.beam.runners.fnexecution.jobsubmission.JobServerDriver:
ArtifactStagingService started on localhost:34053
20/03/05 00:31:26 INFO
org.apache.beam.runners.fnexecution.jobsubmission.JobServerDriver: Java
ExpansionService started on localhost:41335
20/03/05 00:31:26 INFO
org.apache.beam.runners.fnexecution.jobsubmission.JobServerDriver: JobService
started on localhost:45705
WARNING:apache_beam.options.pipeline_options:Discarding unparseable args:
['--parallelism=2', '--shutdown_sources_on_final_watermark']
20/03/05 00:31:27 INFO org.apache.beam.runners.spark.SparkJobInvoker: Invoking
job BeamApp-jenkins-0305003127-701fcef1_90bd97b4-66c5-4dcf-b4f3-0267f7054c8e
20/03/05 00:31:28 INFO
org.apache.beam.runners.fnexecution.jobsubmission.JobInvocation: Starting job
invocation
BeamApp-jenkins-0305003127-701fcef1_90bd97b4-66c5-4dcf-b4f3-0267f7054c8e
INFO:root:Waiting until the pipeline has finished because the environment
"LOOPBACK" has started a component necessary for the execution.
INFO:apache_beam.runners.portability.portable_runner:Job state changed to
STOPPED
INFO:apache_beam.runners.portability.portable_runner:Job state changed to
STARTING
INFO:apache_beam.runners.portability.portable_runner:Job state changed to
RUNNING
20/03/05 00:31:29 INFO org.apache.beam.runners.spark.SparkPipelineRunner:
PipelineOptions.filesToStage was not specified. Defaulting to files from the
classpath
20/03/05 00:31:29 INFO org.apache.beam.runners.spark.SparkPipelineRunner: Will
stage 1 files. (Enable logging at DEBUG level to see which files will be
staged.)
20/03/05 00:31:29 INFO
org.apache.beam.runners.spark.translation.SparkContextFactory: Creating a brand
new Spark Context.
20/03/05 00:31:29 WARN org.apache.hadoop.util.NativeCodeLoader: Unable to load
native-hadoop library for your platform... using builtin-java classes where
applicable
20/03/05 00:31:30 INFO org.apache.beam.runners.spark.SparkPipelineRunner:
Running job
BeamApp-jenkins-0305003127-701fcef1_90bd97b4-66c5-4dcf-b4f3-0267f7054c8e on
Spark master local[4]
20/03/05 00:31:30 INFO
org.apache.beam.runners.spark.aggregators.AggregatorsAccumulator: Instantiated
aggregators accumulator:
20/03/05 00:31:30 INFO
org.apache.beam.runners.spark.metrics.MetricsAccumulator: Instantiated metrics
accumulator: MetricQueryResults()
INFO:apache_beam.runners.worker.statecache:Creating state cache with size 0
INFO:apache_beam.runners.worker.sdk_worker:Creating insecure control channel
for localhost:36859.
INFO:apache_beam.runners.worker.sdk_worker:Control channel established.
INFO:apache_beam.runners.worker.sdk_worker:Initializing SDKHarness with
unbounded number of workers.
20/03/05 00:31:33 INFO
org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: Beam
Fn Control client connected with id 1-1
INFO:apache_beam.runners.worker.sdk_worker:Creating insecure state channel for
localhost:38587.
INFO:apache_beam.runners.worker.sdk_worker:State channel established.
INFO:apache_beam.runners.worker.data_plane:Creating client data channel for
localhost:43555
20/03/05 00:31:33 INFO
org.apache.beam.runners.fnexecution.data.GrpcDataService: Beam Fn Data client
connected.
20/03/05 00:31:34 WARN
org.apache.beam.runners.spark.translation.GroupNonMergingWindowsFunctions:
Either coder LengthPrefixCoder(ByteArrayCoder) or GlobalWindow$Coder is not
consistent with equals. That might cause issues on some runners.
WARNING:apache_beam.io.filebasedsink:Deleting 4 existing files in target path
matching: -*-of-%(num_shards)05d
20/03/05 00:54:54 INFO org.apache.beam.runners.spark.SparkPipelineRunner: Job
BeamApp-jenkins-0305003127-701fcef1_90bd97b4-66c5-4dcf-b4f3-0267f7054c8e:
Pipeline translated successfully. Computing outputs
INFO:apache_beam.io.filebasedsink:Starting finalize_write threads with
num_shards: 4 (skipped: 0), batches: 4, num_threads: 4
INFO:apache_beam.io.filebasedsink:Renamed 4 shards in 0.19 seconds.
20/03/05 00:54:59 INFO org.apache.beam.runners.spark.SparkPipelineRunner: Job
BeamApp-jenkins-0305003127-701fcef1_90bd97b4-66c5-4dcf-b4f3-0267f7054c8e
finished.
20/03/05 00:55:00 INFO
org.apache.beam.runners.fnexecution.artifact.AbstractArtifactRetrievalService:
Manifest at
/tmp/beam-tempihcfh38t/artifactsr1hv8fgs/job_3f716d01-850a-44eb-8769-5e522beb3a11/MANIFEST
has 1 artifact locations
20/03/05 00:55:00 INFO
org.apache.beam.runners.fnexecution.artifact.BeamFileSystemArtifactStagingService:
Removed dir
/tmp/beam-tempihcfh38t/artifactsr1hv8fgs/job_3f716d01-850a-44eb-8769-5e522beb3a11/
INFO:apache_beam.runners.portability.portable_runner:Job state changed to DONE
20/03/05 00:55:00 INFO
org.apache.beam.runners.fnexecution.jobsubmission.InMemoryJobService: Getting
job metrics for
BeamApp-jenkins-0305003127-701fcef1_90bd97b4-66c5-4dcf-b4f3-0267f7054c8e
20/03/05 00:55:00 INFO
org.apache.beam.runners.fnexecution.jobsubmission.InMemoryJobService: Finished
getting job metrics for
BeamApp-jenkins-0305003127-701fcef1_90bd97b4-66c5-4dcf-b4f3-0267f7054c8e
INFO:root:number of empty lines: 2
INFO:root:average word length: 3
ERROR:apache_beam.runners.worker.data_plane:Failed to read inputs in the data
plane.
Traceback (most recent call last):
File
"<https://builds.apache.org/job/beam_PostCommit_Python37/ws/src/build/gradleenv/2022703441/lib/python3.7/site-packages/apache_beam/runners/worker/data_plane.py";,>
line 428, in _read_inputs
for elements in elements_iterator:
File
"<https://builds.apache.org/job/beam_PostCommit_Python37/ws/src/build/gradleenv/2022703441/lib/python3.7/site-packages/grpc/_channel.py";,>
line 416, in __next__
return self._next()
File
"<https://builds.apache.org/job/beam_PostCommit_Python37/ws/src/build/gradleenv/2022703441/lib/python3.7/site-packages/grpc/_channel.py";,>
line 706, in _next
raise self
grpc._channel._MultiThreadedRendezvous: <_MultiThreadedRendezvous of RPC that
terminated with:
status = StatusCode.UNAVAILABLE
details = "Socket closed"
debug_error_string =
"{"created":"@1583369700.465975987","description":"Error received from peer
ipv4:127.0.0.1:43555","file":"src/core/lib/surface/call.cc","file_line":1056,"grpc_message":"Socket
closed","grpc_status":14}"
>
Exception in thread read_grpc_client_inputs:
Traceback (most recent call last):
File "/usr/lib/python3.7/threading.py", line 917, in _bootstrap_inner
self.run()
File "/usr/lib/python3.7/threading.py", line 865, in run
self._target(*self._args, **self._kwargs)
File
"<https://builds.apache.org/job/beam_PostCommit_Python37/ws/src/build/gradleenv/2022703441/lib/python3.7/site-packages/apache_beam/runners/worker/data_plane.py";,>
line 443, in <lambda>
target=lambda: self._read_inputs(elements_iterator),
File
"<https://builds.apache.org/job/beam_PostCommit_Python37/ws/src/build/gradleenv/2022703441/lib/python3.7/site-packages/apache_beam/runners/worker/data_plane.py";,>
line 428, in _read_inputs
for elements in elements_iterator:
File
"<https://builds.apache.org/job/beam_PostCommit_Python37/ws/src/build/gradleenv/2022703441/lib/python3.7/site-packages/grpc/_channel.py";,>
line 416, in __next__
return self._next()
File
"<https://builds.apache.org/job/beam_PostCommit_Python37/ws/src/build/gradleenv/2022703441/lib/python3.7/site-packages/grpc/_channel.py";,>
line 706, in _next
raise self
grpc._channel._MultiThreadedRendezvous: <_MultiThreadedRendezvous of RPC that
terminated with:
status = StatusCode.UNAVAILABLE
details = "Socket closed"
debug_error_string =
"{"created":"@1583369700.465975987","description":"Error received from peer
ipv4:127.0.0.1:43555","file":"src/core/lib/surface/call.cc","file_line":1056,"grpc_message":"Socket
closed","grpc_status":14}"
>
Exception in thread read_state:
Traceback (most recent call last):
File "/usr/lib/python3.7/threading.py", line 917, in _bootstrap_inner
self.run()
File "/usr/lib/python3.7/threading.py", line 865, in run
self._target(*self._args, **self._kwargs)
File
"<https://builds.apache.org/job/beam_PostCommit_Python37/ws/src/build/gradleenv/2022703441/lib/python3.7/site-packages/apache_beam/runners/worker/sdk_worker.py";,>
line 712, in pull_responses
for response in responses:
File
"<https://builds.apache.org/job/beam_PostCommit_Python37/ws/src/build/gradleenv/2022703441/lib/python3.7/site-packages/grpc/_channel.py";,>
line 416, in __next__
return self._next()
File
"<https://builds.apache.org/job/beam_PostCommit_Python37/ws/src/build/gradleenv/2022703441/lib/python3.7/site-packages/grpc/_channel.py";,>
line 706, in _next
raise self
grpc._channel._MultiThreadedRendezvous: <_MultiThreadedRendezvous of RPC that
terminated with:
status = StatusCode.UNAVAILABLE
details = "Socket closed"
debug_error_string =
"{"created":"@1583369700.465988997","description":"Error received from peer
ipv4:127.0.0.1:38587","file":"src/core/lib/surface/call.cc","file_line":1056,"grpc_message":"Socket
closed","grpc_status":14}"
>
Exception in thread run_worker_1-1:
Traceback (most recent call last):
File "/usr/lib/python3.7/threading.py", line 917, in _bootstrap_inner
self.run()
File "/usr/lib/python3.7/threading.py", line 865, in run
self._target(*self._args, **self._kwargs)
File
"<https://builds.apache.org/job/beam_PostCommit_Python37/ws/src/build/gradleenv/2022703441/lib/python3.7/site-packages/apache_beam/runners/worker/sdk_worker.py";,>
line 158, in run
for work_request in control_stub.Control(get_responses()):
File
"<https://builds.apache.org/job/beam_PostCommit_Python37/ws/src/build/gradleenv/2022703441/lib/python3.7/site-packages/grpc/_channel.py";,>
line 416, in __next__
return self._next()
File
"<https://builds.apache.org/job/beam_PostCommit_Python37/ws/src/build/gradleenv/2022703441/lib/python3.7/site-packages/grpc/_channel.py";,>
line 706, in _next
raise self
grpc._channel._MultiThreadedRendezvous: <_MultiThreadedRendezvous of RPC that
terminated with:
status = StatusCode.UNAVAILABLE
details = "Socket closed"
debug_error_string =
"{"created":"@1583369700.466578968","description":"Error received from peer
ipv4:127.0.0.1:36859","file":"src/core/lib/surface/call.cc","file_line":1056,"grpc_message":"Socket
closed","grpc_status":14}"
>
FAILURE: Build completed with 6 failures.
1: Task failed with an exception.
-----------
* What went wrong:
Execution failed for task ':runners:flink:1.9:jar'.
> Could not add file
> '<https://builds.apache.org/job/beam_PostCommit_Python37/ws/src/runners/flink/1.9/build/classes/java/main/org/apache/beam/runners/flink/translation/functions/FlinkExecutableStageContextFactory.class'>
> to ZIP
> '<https://builds.apache.org/job/beam_PostCommit_Python37/ws/src/runners/flink/1.9/build/libs/beam-runners-flink-1.9-2.21.0-SNAPSHOT.jar'.>
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug
option to get more log output. Run with --scan to get full insights.
==============================================================================
2: Task failed with an exception.
-----------
* What went wrong:
Execution failed for task
':sdks:python:test-suites:dataflow:py37:installGcpTest'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug
option to get more log output. Run with --scan to get full insights.
==============================================================================
3: Task failed with an exception.
-----------
* Where:
Build file
'<https://builds.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/test-suites/direct/py37/build.gradle'>
line: 60
* What went wrong:
Execution failed for task
':sdks:python:test-suites:direct:py37:hdfsIntegrationTest'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug
option to get more log output. Run with --scan to get full insights.
==============================================================================
4: Task failed with an exception.
-----------
* What went wrong:
Execution failed for task ':sdks:python:test-suites:direct:py37:installGcpTest'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug
option to get more log output. Run with --scan to get full insights.
==============================================================================
5: Task failed with an exception.
-----------
* What went wrong:
Execution failed for task
':sdks:python:test-suites:portable:py37:portableWordCountFlinkRunnerBatch'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug
option to get more log output. Run with --scan to get full insights.
==============================================================================
6: Task failed with an exception.
-----------
* What went wrong:
Execution failed for task
':sdks:python:test-suites:portable:py37:portableWordCountFlinkRunnerStreaming'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug
option to get more log output. Run with --scan to get full insights.
==============================================================================
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with
Gradle 6.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See
https://docs.gradle.org/5.2.1/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 26m 15s
83 actionable tasks: 63 executed, 20 from cache
Publishing build scan...
https://gradle.com/s/zbfvo5aazsejo
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
