[
https://issues.apache.org/jira/browse/BEAM-11822?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17286198#comment-17286198
]
Alex Amato commented on BEAM-11822:
-----------------------------------
Reverted a culprit commit here
[https://github.com/apache/beam/pull/14004]
Hopefully that fixes the issue
> ValidateRunnerXlangTest.combineGloballyTest broken in
> beam_PostCommit_XVR_Direct
> --------------------------------------------------------------------------------
>
> Key: BEAM-11822
> URL: https://issues.apache.org/jira/browse/BEAM-11822
> Project: Beam
> Issue Type: Test
> Components: test-failures
> Reporter: Alex Amato
> Assignee: Robert Bradshaw
> Priority: P2
> Time Spent: 1h
> Remaining Estimate: 0h
>
> [https://ci-beam.apache.org/job/beam_PostCommit_XVR_Direct/884/testReport/org.apache.beam.runners.core.construction/ValidateRunnerXlangTest/combineGloballyTest/]
> ======
>
> h3. Error Message
> java.lang.RuntimeException: The Runner experienced the following error during
> execution: Traceback (most recent call last): File
> "/home/jenkins/jenkins-slave/workspace/beam_PostCommit_XVR_Direct/src/build/gradleenv/1922375555/lib/python3.6/site-packages/apache_beam/runners/portability/local_job_service.py",
> line 277, in _run_job self._pipeline_proto) File
> "/home/jenkins/jenkins-slave/workspace/beam_PostCommit_XVR_Direct/src/build/gradleenv/1922375555/lib/python3.6/site-packages/apache_beam/runners/portability/fn_api_runner/fn_runner.py",
> line 193, in run_via_runner_api return self.run_stages(stage_context,
> stages) File
> "/home/jenkins/jenkins-slave/workspace/beam_PostCommit_XVR_Direct/src/build/gradleenv/1922375555/lib/python3.6/site-packages/apache_beam/runners/portability/fn_api_runner/fn_runner.py",
> line 358, in run_stages bundle_context_manager, File
> "/home/jenkins/jenkins-slave/workspace/beam_PostCommit_XVR_Direct/src/build/gradleenv/1922375555/lib/python3.6/site-packages/apache_beam/runners/portability/fn_api_runner/fn_runner.py",
> line 554, in _run_stage bundle_manager) File
> "/home/jenkins/jenkins-slave/workspace/beam_PostCommit_XVR_Direct/src/build/gradleenv/1922375555/lib/python3.6/site-packages/apache_beam/runners/portability/fn_api_runner/fn_runner.py",
> line 594, in _run_bundle data_input, data_output, input_timers,
> expected_timer_output) File
> "/home/jenkins/jenkins-slave/workspace/beam_PostCommit_XVR_Direct/src/build/gradleenv/1922375555/lib/python3.6/site-packages/apache_beam/runners/portability/fn_api_runner/fn_runner.py",
> line 886, in process_bundle self._send_input_to_worker(process_bundle_id,
> transform_id, elements) File
> "/home/jenkins/jenkins-slave/workspace/beam_PostCommit_XVR_Direct/src/build/gradleenv/1922375555/lib/python3.6/site-packages/apache_beam/runners/portability/fn_api_runner/fn_runner.py",
> line 762, in _send_input_to_worker for byte_stream in byte_streams: File
> "/home/jenkins/jenkins-slave/workspace/beam_PostCommit_XVR_Direct/src/build/gradleenv/1922375555/lib/python3.6/site-packages/apache_beam/runners/portability/fn_api_runner/execution.py",
> line 254, in __iter__ return itertools.chain(*self.partition(1)) File
> "/home/jenkins/jenkins-slave/workspace/beam_PostCommit_XVR_Direct/src/build/gradleenv/1922375555/lib/python3.6/site-packages/apache_beam/runners/portability/fn_api_runner/execution.py",
> line 240, in partition for wkvs in windowed_key_values(key,
> windowed_values): File
> "/home/jenkins/jenkins-slave/workspace/beam_PostCommit_XVR_Direct/src/build/gradleenv/1922375555/lib/python3.6/site-packages/apache_beam/transforms/trigger.py",
> line 1129, in process_entire_key MIN_TIMESTAMP): File
> "/home/jenkins/jenkins-slave/workspace/beam_PostCommit_XVR_Direct/src/build/gradleenv/1922375555/lib/python3.6/site-packages/apache_beam/transforms/trigger.py",
> line 1308, in process_elements
> self.window_fn.merge(TriggerMergeContext(all_windows)) File
> "/home/jenkins/jenkins-slave/workspace/beam_PostCommit_XVR_Direct/src/build/gradleenv/1922375555/lib/python3.6/site-packages/apache_beam/runners/portability/fn_api_runner/execution.py",
> line 423, in merge raise RuntimeError(result.error) RuntimeError: Traceback
> (most recent call last): File
> "/usr/local/lib/python3.6/site-packages/apache_beam/runners/worker/sdk_worker.py",
> line 289, in _execute response = task() File
> "/usr/local/lib/python3.6/site-packages/apache_beam/runners/worker/sdk_worker.py",
> line 362, in <lambda> lambda: self.create_worker().do_instruction(request),
> request) File
> "/usr/local/lib/python3.6/site-packages/apache_beam/runners/worker/sdk_worker.py",
> line 607, in do_instruction getattr(request, request_type),
> request.instruction_id) File
> "/usr/local/lib/python3.6/site-packages/apache_beam/runners/worker/sdk_worker.py",
> line 638, in process_bundle instruction_id,
> request.process_bundle_descriptor_id) File
> "/usr/local/lib/python3.6/site-packages/apache_beam/runners/worker/sdk_worker.py",
> line 467, in get self.data_channel_factory) File
> "/usr/local/lib/python3.6/site-packages/apache_beam/runners/worker/bundle_processor.py",
> line 869, in __init__ self.ops =
> self.create_execution_tree(self.process_bundle_descriptor) File
> "/usr/local/lib/python3.6/site-packages/apache_beam/runners/worker/bundle_processor.py",
> line 926, in create_execution_tree descriptor.transforms,
> key=topological_height, reverse=True)]) File
> "/usr/local/lib/python3.6/site-packages/apache_beam/runners/worker/bundle_processor.py",
> line 925, in <listcomp> get_operation(transform_id))) for transform_id in
> sorted( File
> "/usr/local/lib/python3.6/site-packages/apache_beam/runners/worker/bundle_processor.py",
> line 813, in wrapper result = cache[args] = func(*args) File
> "/usr/local/lib/python3.6/site-packages/apache_beam/runners/worker/bundle_processor.py",
> line 907, in get_operation pcoll_id in
> descriptor.transforms[transform_id].outputs.items() File
> "/usr/local/lib/python3.6/site-packages/apache_beam/runners/worker/bundle_processor.py",
> line 907, in <dictcomp> pcoll_id in
> descriptor.transforms[transform_id].outputs.items() File
> "/usr/local/lib/python3.6/site-packages/apache_beam/runners/worker/bundle_processor.py",
> line 905, in <listcomp> tag: [get_operation(op) for op in
> pcoll_consumers[pcoll_id]] File
> "/usr/local/lib/python3.6/site-packages/apache_beam/runners/worker/bundle_processor.py",
> line 813, in wrapper result = cache[args] = func(*args) File
> "/usr/local/lib/python3.6/site-packages/apache_beam/runners/worker/bundle_processor.py",
> line 910, in get_operation transform_id, transform_consumers) File
> "/usr/local/lib/python3.6/site-packages/apache_beam/runners/worker/bundle_processor.py",
> line 1199, in create_operation return creator(self, transform_id,
> transform_proto, payload, consumers) File
> "/usr/local/lib/python3.6/site-packages/apache_beam/runners/worker/bundle_processor.py",
> line 1869, in create_merge_windows assert mapping_fn_spec.urn ==
> python_urns.PICKLED_WINDOWFN AssertionError
> h3. Stacktrace
> java.lang.RuntimeException: The Runner experienced the following error during
> execution: Traceback (most recent call last): File
> "/home/jenkins/jenkins-slave/workspace/beam_PostCommit_XVR_Direct/src/build/gradleenv/1922375555/lib/python3.6/site-packages/apache_beam/runners/portability/local_job_service.py",
> line 277, in _run_job self._pipeline_proto) File
> "/home/jenkins/jenkins-slave/workspace/beam_PostCommit_XVR_Direct/src/build/gradleenv/1922375555/lib/python3.6/site-packages/apache_beam/runners/portability/fn_api_runner/fn_runner.py",
> line 193, in run_via_runner_api return self.run_stages(stage_context,
> stages) File
> "/home/jenkins/jenkins-slave/workspace/beam_PostCommit_XVR_Direct/src/build/gradleenv/1922375555/lib/python3.6/site-packages/apache_beam/runners/portability/fn_api_runner/fn_runner.py",
> line 358, in run_stages bundle_context_manager, File
> "/home/jenkins/jenkins-slave/workspace/beam_PostCommit_XVR_Direct/src/build/gradleenv/1922375555/lib/python3.6/site-packages/apache_beam/runners/portability/fn_api_runner/fn_runner.py",
> line 554, in _run_stage bundle_manager) File
> "/home/jenkins/jenkins-slave/workspace/beam_PostCommit_XVR_Direct/src/build/gradleenv/1922375555/lib/python3.6/site-packages/apache_beam/runners/portability/fn_api_runner/fn_runner.py",
> line 594, in _run_bundle data_input, data_output, input_timers,
> expected_timer_output) File
> "/home/jenkins/jenkins-slave/workspace/beam_PostCommit_XVR_Direct/src/build/gradleenv/1922375555/lib/python3.6/site-packages/apache_beam/runners/portability/fn_api_runner/fn_runner.py",
> line 886, in process_bundle self._send_input_to_worker(process_bundle_id,
> transform_id, elements) File
> "/home/jenkins/jenkins-slave/workspace/beam_PostCommit_XVR_Direct/src/build/gradleenv/1922375555/lib/python3.6/site-packages/apache_beam/runners/portability/fn_api_runner/fn_runner.py",
> line 762, in _send_input_to_worker for byte_stream in byte_streams: File
> "/home/jenkins/jenkins-slave/workspace/beam_PostCommit_XVR_Direct/src/build/gradleenv/1922375555/lib/python3.6/site-packages/apache_beam/runners/portability/fn_api_runner/execution.py",
> line 254, in __iter__ return itertools.chain(*self.partition(1)) File
> "/home/jenkins/jenkins-slave/workspace/beam_PostCommit_XVR_Direct/src/build/gradleenv/1922375555/lib/python3.6/site-packages/apache_beam/runners/portability/fn_api_runner/execution.py",
> line 240, in partition for wkvs in windowed_key_values(key,
> windowed_values): File
> "/home/jenkins/jenkins-slave/workspace/beam_PostCommit_XVR_Direct/src/build/gradleenv/1922375555/lib/python3.6/site-packages/apache_beam/transforms/trigger.py",
> line 1129, in process_entire_key MIN_TIMESTAMP): File
> "/home/jenkins/jenkins-slave/workspace/beam_PostCommit_XVR_Direct/src/build/gradleenv/1922375555/lib/python3.6/site-packages/apache_beam/transforms/trigger.py",
> line 1308, in process_elements
> self.window_fn.merge(TriggerMergeContext(all_windows)) File
> "/home/jenkins/jenkins-slave/workspace/beam_PostCommit_XVR_Direct/src/build/gradleenv/1922375555/lib/python3.6/site-packages/apache_beam/runners/portability/fn_api_runner/execution.py",
> line 423, in merge raise RuntimeError(result.error) RuntimeError: Traceback
> (most recent call last): File
> "/usr/local/lib/python3.6/site-packages/apache_beam/runners/worker/sdk_worker.py",
> line 289, in _execute response = task() File
> "/usr/local/lib/python3.6/site-packages/apache_beam/runners/worker/sdk_worker.py",
> line 362, in <lambda> lambda: self.create_worker().do_instruction(request),
> request) File
> "/usr/local/lib/python3.6/site-packages/apache_beam/runners/worker/sdk_worker.py",
> line 607, in do_instruction getattr(request, request_type),
> request.instruction_id) File
> "/usr/local/lib/python3.6/site-packages/apache_beam/runners/worker/sdk_worker.py",
> line 638, in process_bundle instruction_id,
> request.process_bundle_descriptor_id) File
> "/usr/local/lib/python3.6/site-packages/apache_beam/runners/worker/sdk_worker.py",
> line 467, in get self.data_channel_factory) File
> "/usr/local/lib/python3.6/site-packages/apache_beam/runners/worker/bundle_processor.py",
> line 869, in __init__ self.ops =
> self.create_execution_tree(self.process_bundle_descriptor) File
> "/usr/local/lib/python3.6/site-packages/apache_beam/runners/worker/bundle_processor.py",
> line 926, in create_execution_tree descriptor.transforms,
> key=topological_height, reverse=True)]) File
> "/usr/local/lib/python3.6/site-packages/apache_beam/runners/worker/bundle_processor.py",
> line 925, in <listcomp> get_operation(transform_id))) for transform_id in
> sorted( File
> "/usr/local/lib/python3.6/site-packages/apache_beam/runners/worker/bundle_processor.py",
> line 813, in wrapper result = cache[args] = func(*args) File
> "/usr/local/lib/python3.6/site-packages/apache_beam/runners/worker/bundle_processor.py",
> line 907, in get_operation pcoll_id in
> descriptor.transforms[transform_id].outputs.items() File
> "/usr/local/lib/python3.6/site-packages/apache_beam/runners/worker/bundle_processor.py",
> line 907, in <dictcomp> pcoll_id in
> descriptor.transforms[transform_id].outputs.items() File
> "/usr/local/lib/python3.6/site-packages/apache_beam/runners/worker/bundle_processor.py",
> line 905, in <listcomp> tag: [get_operation(op) for op in
> pcoll_consumers[pcoll_id]] File
> "/usr/local/lib/python3.6/site-packages/apache_beam/runners/worker/bundle_processor.py",
> line 813, in wrapper result = cache[args] = func(*args) File
> "/usr/local/lib/python3.6/site-packages/apache_beam/runners/worker/bundle_processor.py",
> line 910, in get_operation transform_id, transform_consumers) File
> "/usr/local/lib/python3.6/site-packages/apache_beam/runners/worker/bundle_processor.py",
> line 1199, in create_operation return creator(self, transform_id,
> transform_proto, payload, consumers) File
> "/usr/local/lib/python3.6/site-packages/apache_beam/runners/worker/bundle_processor.py",
> line 1869, in create_merge_windows assert mapping_fn_spec.urn ==
> python_urns.PICKLED_WINDOWFN AssertionError at
> org.apache.beam.runners.portability.JobServicePipelineResult.propagateErrors(JobServicePipelineResult.java:166)
> at
> org.apache.beam.runners.portability.JobServicePipelineResult.waitUntilFinish(JobServicePipelineResult.java:111)
> at
> org.apache.beam.runners.core.construction.ValidateRunnerXlangTest.tearDown(ValidateRunnerXlangTest.java:127)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498) at
> org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:59)
> at
> org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
> at
> org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:56)
> at
> org.junit.internal.runners.statements.RunAfters.invokeMethod(RunAfters.java:46)
> at
> org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:33)
> at org.apache.beam.sdk.testing.TestPipeline$1.evaluate(TestPipeline.java:322)
> at org.junit.runners.ParentRunner$3.evaluate(ParentRunner.java:306) at
> org.junit.runners.BlockJUnit4ClassRunner$1.evaluate(BlockJUnit4ClassRunner.java:100)
> at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:366) at
> org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:103)
> at
> org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:63)
> at org.junit.runners.ParentRunner$4.run(ParentRunner.java:331) at
> org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:79) at
> org.junit.runners.ParentRunner.runChildren(ParentRunner.java:329) at
> org.junit.runners.ParentRunner.access$100(ParentRunner.java:66) at
> org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:293) at
> org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:26)
> at org.junit.runners.ParentRunner$3.evaluate(ParentRunner.java:306) at
> org.junit.runners.ParentRunner.run(ParentRunner.java:413) at
> org.gradle.api.internal.tasks.testing.junit.JUnitTestClassExecutor.runTestClass(JUnitTestClassExecutor.java:110)
> at
> org.gradle.api.internal.tasks.testing.junit.JUnitTestClassExecutor.execute(JUnitTestClassExecutor.java:58)
> at
> org.gradle.api.internal.tasks.testing.junit.JUnitTestClassExecutor.execute(JUnitTestClassExecutor.java:38)
> at
> org.gradle.api.internal.tasks.testing.junit.AbstractJUnitTestClassProcessor.processTestClass(AbstractJUnitTestClassProcessor.java:62)
> at
> org.gradle.api.internal.tasks.testing.SuiteTestClassProcessor.processTestClass(SuiteTestClassProcessor.java:51)
> at sun.reflect.GeneratedMethodAccessor10.invoke(Unknown Source) at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498) at
> org.gradle.internal.dispatch.ReflectionDispatch.dispatch(ReflectionDispatch.java:36)
> at
> org.gradle.internal.dispatch.ReflectionDispatch.dispatch(ReflectionDispatch.java:24)
> at
> org.gradle.internal.dispatch.ContextClassLoaderDispatch.dispatch(ContextClassLoaderDispatch.java:33)
> at
> org.gradle.internal.dispatch.ProxyDispatchAdapter$DispatchingInvocationHandler.invoke(ProxyDispatchAdapter.java:94)
> at com.sun.proxy.$Proxy2.processTestClass(Unknown Source) at
> org.gradle.api.internal.tasks.testing.worker.TestWorker.processTestClass(TestWorker.java:119)
> at sun.reflect.GeneratedMethodAccessor9.invoke(Unknown Source) at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498) at
> org.gradle.internal.dispatch.ReflectionDispatch.dispatch(ReflectionDispatch.java:36)
> at
> org.gradle.internal.dispatch.ReflectionDispatch.dispatch(ReflectionDispatch.java:24)
> at
> org.gradle.internal.remote.internal.hub.MessageHubBackedObjectConnection$DispatchWrapper.dispatch(MessageHubBackedObjectConnection.java:182)
> at
> org.gradle.internal.remote.internal.hub.MessageHubBackedObjectConnection$DispatchWrapper.dispatch(MessageHubBackedObjectConnection.java:164)
> at
> org.gradle.internal.remote.internal.hub.MessageHub$Handler.run(MessageHub.java:414)
> at
> org.gradle.internal.concurrent.ExecutorPolicy$CatchAndRecordFailures.onExecute(ExecutorPolicy.java:64)
> at
> org.gradle.internal.concurrent.ManagedExecutorImpl$1.run(ManagedExecutorImpl.java:48)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
> at
> org.gradle.internal.concurrent.ThreadFactoryImpl$ManagedThreadRunnable.run(ThreadFactoryImpl.java:56)
> at java.lang.Thread.run(Thread.java:748)
> h3. Standard Error
> Feb 16, 2021 12:25:10 AM org.apache.beam.runners.portability.PortableRunner
> run INFO: Using job server endpoint: localhost:18091 Feb 16, 2021 12:25:10 AM
> org.apache.beam.runners.portability.PortableRunner run INFO:
> PrepareJobResponse: preparation_id:
> "validaterunnerxlangtest0combinegloballytest-jenkins-0216002510-460cb818-a12b9848-057e-4551-b22e-c2ead482731f"
> artifact_staging_endpoint \{ url: "localhost:18091" } staging_session_token:
> "validaterunnerxlangtest0combinegloballytest-jenkins-0216002 ...[truncated
> 5165 chars]... s-0216002957-5ce43fcf-b8845aa1-4792-4171-a62d-88d0808ac838"
> artifact_staging_endpoint \{ url: "localhost:18091" } staging_session_token:
> "validaterunnerxlangtest0multiinputoutputwithsideinputtest-jenkins-0216002957-5ce43fcf-b8845aa1-4792-4171-a62d-88d0808ac838"
> Feb 16, 2021 12:29:58 AM org.apache.beam.runners.portability.PortableRunner
> run INFO: RunJobResponse: job_id:
> "validaterunnerxlangtest0multiinputoutputwithsideinputtest-jenkins-0216002957-5ce43fcf-b8845aa1-4792-4171-a62d-88d0808ac838"
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
