Alex Amato created BEAM-11822:
---------------------------------
Summary: ValidateRunnerXlangTest.combineGloballyTest broken in
beam_PostCommit_XVR_Direct
Key: BEAM-11822
URL: https://issues.apache.org/jira/browse/BEAM-11822
Project: Beam
Issue Type: Test
Components: test-failures
Reporter: Alex Amato
[https://ci-beam.apache.org/job/beam_PostCommit_XVR_Direct/884/testReport/org.apache.beam.runners.core.construction/ValidateRunnerXlangTest/combineGloballyTest/]
======
h3. Error Message
java.lang.RuntimeException: The Runner experienced the following error during
execution: Traceback (most recent call last): File
"/home/jenkins/jenkins-slave/workspace/beam_PostCommit_XVR_Direct/src/build/gradleenv/1922375555/lib/python3.6/site-packages/apache_beam/runners/portability/local_job_service.py",
line 277, in _run_job self._pipeline_proto) File
"/home/jenkins/jenkins-slave/workspace/beam_PostCommit_XVR_Direct/src/build/gradleenv/1922375555/lib/python3.6/site-packages/apache_beam/runners/portability/fn_api_runner/fn_runner.py",
line 193, in run_via_runner_api return self.run_stages(stage_context, stages)
File
"/home/jenkins/jenkins-slave/workspace/beam_PostCommit_XVR_Direct/src/build/gradleenv/1922375555/lib/python3.6/site-packages/apache_beam/runners/portability/fn_api_runner/fn_runner.py",
line 358, in run_stages bundle_context_manager, File
"/home/jenkins/jenkins-slave/workspace/beam_PostCommit_XVR_Direct/src/build/gradleenv/1922375555/lib/python3.6/site-packages/apache_beam/runners/portability/fn_api_runner/fn_runner.py",
line 554, in _run_stage bundle_manager) File
"/home/jenkins/jenkins-slave/workspace/beam_PostCommit_XVR_Direct/src/build/gradleenv/1922375555/lib/python3.6/site-packages/apache_beam/runners/portability/fn_api_runner/fn_runner.py",
line 594, in _run_bundle data_input, data_output, input_timers,
expected_timer_output) File
"/home/jenkins/jenkins-slave/workspace/beam_PostCommit_XVR_Direct/src/build/gradleenv/1922375555/lib/python3.6/site-packages/apache_beam/runners/portability/fn_api_runner/fn_runner.py",
line 886, in process_bundle self._send_input_to_worker(process_bundle_id,
transform_id, elements) File
"/home/jenkins/jenkins-slave/workspace/beam_PostCommit_XVR_Direct/src/build/gradleenv/1922375555/lib/python3.6/site-packages/apache_beam/runners/portability/fn_api_runner/fn_runner.py",
line 762, in _send_input_to_worker for byte_stream in byte_streams: File
"/home/jenkins/jenkins-slave/workspace/beam_PostCommit_XVR_Direct/src/build/gradleenv/1922375555/lib/python3.6/site-packages/apache_beam/runners/portability/fn_api_runner/execution.py",
line 254, in __iter__ return itertools.chain(*self.partition(1)) File
"/home/jenkins/jenkins-slave/workspace/beam_PostCommit_XVR_Direct/src/build/gradleenv/1922375555/lib/python3.6/site-packages/apache_beam/runners/portability/fn_api_runner/execution.py",
line 240, in partition for wkvs in windowed_key_values(key, windowed_values):
File
"/home/jenkins/jenkins-slave/workspace/beam_PostCommit_XVR_Direct/src/build/gradleenv/1922375555/lib/python3.6/site-packages/apache_beam/transforms/trigger.py",
line 1129, in process_entire_key MIN_TIMESTAMP): File
"/home/jenkins/jenkins-slave/workspace/beam_PostCommit_XVR_Direct/src/build/gradleenv/1922375555/lib/python3.6/site-packages/apache_beam/transforms/trigger.py",
line 1308, in process_elements
self.window_fn.merge(TriggerMergeContext(all_windows)) File
"/home/jenkins/jenkins-slave/workspace/beam_PostCommit_XVR_Direct/src/build/gradleenv/1922375555/lib/python3.6/site-packages/apache_beam/runners/portability/fn_api_runner/execution.py",
line 423, in merge raise RuntimeError(result.error) RuntimeError: Traceback
(most recent call last): File
"/usr/local/lib/python3.6/site-packages/apache_beam/runners/worker/sdk_worker.py",
line 289, in _execute response = task() File
"/usr/local/lib/python3.6/site-packages/apache_beam/runners/worker/sdk_worker.py",
line 362, in <lambda> lambda: self.create_worker().do_instruction(request),
request) File
"/usr/local/lib/python3.6/site-packages/apache_beam/runners/worker/sdk_worker.py",
line 607, in do_instruction getattr(request, request_type),
request.instruction_id) File
"/usr/local/lib/python3.6/site-packages/apache_beam/runners/worker/sdk_worker.py",
line 638, in process_bundle instruction_id,
request.process_bundle_descriptor_id) File
"/usr/local/lib/python3.6/site-packages/apache_beam/runners/worker/sdk_worker.py",
line 467, in get self.data_channel_factory) File
"/usr/local/lib/python3.6/site-packages/apache_beam/runners/worker/bundle_processor.py",
line 869, in __init__ self.ops =
self.create_execution_tree(self.process_bundle_descriptor) File
"/usr/local/lib/python3.6/site-packages/apache_beam/runners/worker/bundle_processor.py",
line 926, in create_execution_tree descriptor.transforms,
key=topological_height, reverse=True)]) File
"/usr/local/lib/python3.6/site-packages/apache_beam/runners/worker/bundle_processor.py",
line 925, in <listcomp> get_operation(transform_id))) for transform_id in
sorted( File
"/usr/local/lib/python3.6/site-packages/apache_beam/runners/worker/bundle_processor.py",
line 813, in wrapper result = cache[args] = func(*args) File
"/usr/local/lib/python3.6/site-packages/apache_beam/runners/worker/bundle_processor.py",
line 907, in get_operation pcoll_id in
descriptor.transforms[transform_id].outputs.items() File
"/usr/local/lib/python3.6/site-packages/apache_beam/runners/worker/bundle_processor.py",
line 907, in <dictcomp> pcoll_id in
descriptor.transforms[transform_id].outputs.items() File
"/usr/local/lib/python3.6/site-packages/apache_beam/runners/worker/bundle_processor.py",
line 905, in <listcomp> tag: [get_operation(op) for op in
pcoll_consumers[pcoll_id]] File
"/usr/local/lib/python3.6/site-packages/apache_beam/runners/worker/bundle_processor.py",
line 813, in wrapper result = cache[args] = func(*args) File
"/usr/local/lib/python3.6/site-packages/apache_beam/runners/worker/bundle_processor.py",
line 910, in get_operation transform_id, transform_consumers) File
"/usr/local/lib/python3.6/site-packages/apache_beam/runners/worker/bundle_processor.py",
line 1199, in create_operation return creator(self, transform_id,
transform_proto, payload, consumers) File
"/usr/local/lib/python3.6/site-packages/apache_beam/runners/worker/bundle_processor.py",
line 1869, in create_merge_windows assert mapping_fn_spec.urn ==
python_urns.PICKLED_WINDOWFN AssertionError
h3. Stacktrace
java.lang.RuntimeException: The Runner experienced the following error during
execution: Traceback (most recent call last): File
"/home/jenkins/jenkins-slave/workspace/beam_PostCommit_XVR_Direct/src/build/gradleenv/1922375555/lib/python3.6/site-packages/apache_beam/runners/portability/local_job_service.py",
line 277, in _run_job self._pipeline_proto) File
"/home/jenkins/jenkins-slave/workspace/beam_PostCommit_XVR_Direct/src/build/gradleenv/1922375555/lib/python3.6/site-packages/apache_beam/runners/portability/fn_api_runner/fn_runner.py",
line 193, in run_via_runner_api return self.run_stages(stage_context, stages)
File
"/home/jenkins/jenkins-slave/workspace/beam_PostCommit_XVR_Direct/src/build/gradleenv/1922375555/lib/python3.6/site-packages/apache_beam/runners/portability/fn_api_runner/fn_runner.py",
line 358, in run_stages bundle_context_manager, File
"/home/jenkins/jenkins-slave/workspace/beam_PostCommit_XVR_Direct/src/build/gradleenv/1922375555/lib/python3.6/site-packages/apache_beam/runners/portability/fn_api_runner/fn_runner.py",
line 554, in _run_stage bundle_manager) File
"/home/jenkins/jenkins-slave/workspace/beam_PostCommit_XVR_Direct/src/build/gradleenv/1922375555/lib/python3.6/site-packages/apache_beam/runners/portability/fn_api_runner/fn_runner.py",
line 594, in _run_bundle data_input, data_output, input_timers,
expected_timer_output) File
"/home/jenkins/jenkins-slave/workspace/beam_PostCommit_XVR_Direct/src/build/gradleenv/1922375555/lib/python3.6/site-packages/apache_beam/runners/portability/fn_api_runner/fn_runner.py",
line 886, in process_bundle self._send_input_to_worker(process_bundle_id,
transform_id, elements) File
"/home/jenkins/jenkins-slave/workspace/beam_PostCommit_XVR_Direct/src/build/gradleenv/1922375555/lib/python3.6/site-packages/apache_beam/runners/portability/fn_api_runner/fn_runner.py",
line 762, in _send_input_to_worker for byte_stream in byte_streams: File
"/home/jenkins/jenkins-slave/workspace/beam_PostCommit_XVR_Direct/src/build/gradleenv/1922375555/lib/python3.6/site-packages/apache_beam/runners/portability/fn_api_runner/execution.py",
line 254, in __iter__ return itertools.chain(*self.partition(1)) File
"/home/jenkins/jenkins-slave/workspace/beam_PostCommit_XVR_Direct/src/build/gradleenv/1922375555/lib/python3.6/site-packages/apache_beam/runners/portability/fn_api_runner/execution.py",
line 240, in partition for wkvs in windowed_key_values(key, windowed_values):
File
"/home/jenkins/jenkins-slave/workspace/beam_PostCommit_XVR_Direct/src/build/gradleenv/1922375555/lib/python3.6/site-packages/apache_beam/transforms/trigger.py",
line 1129, in process_entire_key MIN_TIMESTAMP): File
"/home/jenkins/jenkins-slave/workspace/beam_PostCommit_XVR_Direct/src/build/gradleenv/1922375555/lib/python3.6/site-packages/apache_beam/transforms/trigger.py",
line 1308, in process_elements
self.window_fn.merge(TriggerMergeContext(all_windows)) File
"/home/jenkins/jenkins-slave/workspace/beam_PostCommit_XVR_Direct/src/build/gradleenv/1922375555/lib/python3.6/site-packages/apache_beam/runners/portability/fn_api_runner/execution.py",
line 423, in merge raise RuntimeError(result.error) RuntimeError: Traceback
(most recent call last): File
"/usr/local/lib/python3.6/site-packages/apache_beam/runners/worker/sdk_worker.py",
line 289, in _execute response = task() File
"/usr/local/lib/python3.6/site-packages/apache_beam/runners/worker/sdk_worker.py",
line 362, in <lambda> lambda: self.create_worker().do_instruction(request),
request) File
"/usr/local/lib/python3.6/site-packages/apache_beam/runners/worker/sdk_worker.py",
line 607, in do_instruction getattr(request, request_type),
request.instruction_id) File
"/usr/local/lib/python3.6/site-packages/apache_beam/runners/worker/sdk_worker.py",
line 638, in process_bundle instruction_id,
request.process_bundle_descriptor_id) File
"/usr/local/lib/python3.6/site-packages/apache_beam/runners/worker/sdk_worker.py",
line 467, in get self.data_channel_factory) File
"/usr/local/lib/python3.6/site-packages/apache_beam/runners/worker/bundle_processor.py",
line 869, in __init__ self.ops =
self.create_execution_tree(self.process_bundle_descriptor) File
"/usr/local/lib/python3.6/site-packages/apache_beam/runners/worker/bundle_processor.py",
line 926, in create_execution_tree descriptor.transforms,
key=topological_height, reverse=True)]) File
"/usr/local/lib/python3.6/site-packages/apache_beam/runners/worker/bundle_processor.py",
line 925, in <listcomp> get_operation(transform_id))) for transform_id in
sorted( File
"/usr/local/lib/python3.6/site-packages/apache_beam/runners/worker/bundle_processor.py",
line 813, in wrapper result = cache[args] = func(*args) File
"/usr/local/lib/python3.6/site-packages/apache_beam/runners/worker/bundle_processor.py",
line 907, in get_operation pcoll_id in
descriptor.transforms[transform_id].outputs.items() File
"/usr/local/lib/python3.6/site-packages/apache_beam/runners/worker/bundle_processor.py",
line 907, in <dictcomp> pcoll_id in
descriptor.transforms[transform_id].outputs.items() File
"/usr/local/lib/python3.6/site-packages/apache_beam/runners/worker/bundle_processor.py",
line 905, in <listcomp> tag: [get_operation(op) for op in
pcoll_consumers[pcoll_id]] File
"/usr/local/lib/python3.6/site-packages/apache_beam/runners/worker/bundle_processor.py",
line 813, in wrapper result = cache[args] = func(*args) File
"/usr/local/lib/python3.6/site-packages/apache_beam/runners/worker/bundle_processor.py",
line 910, in get_operation transform_id, transform_consumers) File
"/usr/local/lib/python3.6/site-packages/apache_beam/runners/worker/bundle_processor.py",
line 1199, in create_operation return creator(self, transform_id,
transform_proto, payload, consumers) File
"/usr/local/lib/python3.6/site-packages/apache_beam/runners/worker/bundle_processor.py",
line 1869, in create_merge_windows assert mapping_fn_spec.urn ==
python_urns.PICKLED_WINDOWFN AssertionError at
org.apache.beam.runners.portability.JobServicePipelineResult.propagateErrors(JobServicePipelineResult.java:166)
at
org.apache.beam.runners.portability.JobServicePipelineResult.waitUntilFinish(JobServicePipelineResult.java:111)
at
org.apache.beam.runners.core.construction.ValidateRunnerXlangTest.tearDown(ValidateRunnerXlangTest.java:127)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498) at
org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:59)
at
org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
at
org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:56)
at
org.junit.internal.runners.statements.RunAfters.invokeMethod(RunAfters.java:46)
at org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:33)
at org.apache.beam.sdk.testing.TestPipeline$1.evaluate(TestPipeline.java:322)
at org.junit.runners.ParentRunner$3.evaluate(ParentRunner.java:306) at
org.junit.runners.BlockJUnit4ClassRunner$1.evaluate(BlockJUnit4ClassRunner.java:100)
at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:366) at
org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:103)
at
org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:63)
at org.junit.runners.ParentRunner$4.run(ParentRunner.java:331) at
org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:79) at
org.junit.runners.ParentRunner.runChildren(ParentRunner.java:329) at
org.junit.runners.ParentRunner.access$100(ParentRunner.java:66) at
org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:293) at
org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:26)
at org.junit.runners.ParentRunner$3.evaluate(ParentRunner.java:306) at
org.junit.runners.ParentRunner.run(ParentRunner.java:413) at
org.gradle.api.internal.tasks.testing.junit.JUnitTestClassExecutor.runTestClass(JUnitTestClassExecutor.java:110)
at
org.gradle.api.internal.tasks.testing.junit.JUnitTestClassExecutor.execute(JUnitTestClassExecutor.java:58)
at
org.gradle.api.internal.tasks.testing.junit.JUnitTestClassExecutor.execute(JUnitTestClassExecutor.java:38)
at
org.gradle.api.internal.tasks.testing.junit.AbstractJUnitTestClassProcessor.processTestClass(AbstractJUnitTestClassProcessor.java:62)
at
org.gradle.api.internal.tasks.testing.SuiteTestClassProcessor.processTestClass(SuiteTestClassProcessor.java:51)
at sun.reflect.GeneratedMethodAccessor10.invoke(Unknown Source) at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498) at
org.gradle.internal.dispatch.ReflectionDispatch.dispatch(ReflectionDispatch.java:36)
at
org.gradle.internal.dispatch.ReflectionDispatch.dispatch(ReflectionDispatch.java:24)
at
org.gradle.internal.dispatch.ContextClassLoaderDispatch.dispatch(ContextClassLoaderDispatch.java:33)
at
org.gradle.internal.dispatch.ProxyDispatchAdapter$DispatchingInvocationHandler.invoke(ProxyDispatchAdapter.java:94)
at com.sun.proxy.$Proxy2.processTestClass(Unknown Source) at
org.gradle.api.internal.tasks.testing.worker.TestWorker.processTestClass(TestWorker.java:119)
at sun.reflect.GeneratedMethodAccessor9.invoke(Unknown Source) at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498) at
org.gradle.internal.dispatch.ReflectionDispatch.dispatch(ReflectionDispatch.java:36)
at
org.gradle.internal.dispatch.ReflectionDispatch.dispatch(ReflectionDispatch.java:24)
at
org.gradle.internal.remote.internal.hub.MessageHubBackedObjectConnection$DispatchWrapper.dispatch(MessageHubBackedObjectConnection.java:182)
at
org.gradle.internal.remote.internal.hub.MessageHubBackedObjectConnection$DispatchWrapper.dispatch(MessageHubBackedObjectConnection.java:164)
at
org.gradle.internal.remote.internal.hub.MessageHub$Handler.run(MessageHub.java:414)
at
org.gradle.internal.concurrent.ExecutorPolicy$CatchAndRecordFailures.onExecute(ExecutorPolicy.java:64)
at
org.gradle.internal.concurrent.ManagedExecutorImpl$1.run(ManagedExecutorImpl.java:48)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at
org.gradle.internal.concurrent.ThreadFactoryImpl$ManagedThreadRunnable.run(ThreadFactoryImpl.java:56)
at java.lang.Thread.run(Thread.java:748)
h3. Standard Error
Feb 16, 2021 12:25:10 AM org.apache.beam.runners.portability.PortableRunner run
INFO: Using job server endpoint: localhost:18091 Feb 16, 2021 12:25:10 AM
org.apache.beam.runners.portability.PortableRunner run INFO:
PrepareJobResponse: preparation_id:
"validaterunnerxlangtest0combinegloballytest-jenkins-0216002510-460cb818-a12b9848-057e-4551-b22e-c2ead482731f"
artifact_staging_endpoint \{ url: "localhost:18091" } staging_session_token:
"validaterunnerxlangtest0combinegloballytest-jenkins-0216002 ...[truncated 5165
chars]... s-0216002957-5ce43fcf-b8845aa1-4792-4171-a62d-88d0808ac838"
artifact_staging_endpoint \{ url: "localhost:18091" } staging_session_token:
"validaterunnerxlangtest0multiinputoutputwithsideinputtest-jenkins-0216002957-5ce43fcf-b8845aa1-4792-4171-a62d-88d0808ac838"
Feb 16, 2021 12:29:58 AM org.apache.beam.runners.portability.PortableRunner
run INFO: RunJobResponse: job_id:
"validaterunnerxlangtest0multiinputoutputwithsideinputtest-jenkins-0216002957-5ce43fcf-b8845aa1-4792-4171-a62d-88d0808ac838"
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
