[
https://issues.apache.org/jira/browse/BEAM-12847?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17478146#comment-17478146
]
Miren Esnaola commented on BEAM-12847:
--------------------------------------
The problem is still there in 2.35.0. I got the same error.
Error message from worker: generic::unknown: Traceback (most recent call
last): File
"/usr/local/lib/python3.8/site-packages/apache_beam/runners/worker/sdk_worker.py",
line 267, in _execute response = task() File
"/usr/local/lib/python3.8/site-packages/apache_beam/runners/worker/sdk_worker.py",
line 340, in <lambda> lambda: self.create_worker().do_instruction(request),
request) File
"/usr/local/lib/python3.8/site-packages/apache_beam/runners/worker/sdk_worker.py",
line 580, in do_instruction return getattr(self, request_type)( File
"/usr/local/lib/python3.8/site-packages/apache_beam/runners/worker/sdk_worker.py",
line 618, in process_bundle
bundle_processor.process_bundle(instruction_id)) File
"/usr/local/lib/python3.8/site-packages/apache_beam/runners/worker/bundle_processor.py",
line 995, in process_bundle
input_op_by_transform_id[element.transform_id].process_encoded( File
"/usr/local/lib/python3.8/site-packages/apache_beam/runners/worker/bundle_processor.py",
line 221, in process_encoded self.output(decoded_value) File
"apache_beam/runners/worker/operations.py", line 346, in
apache_beam.runners.worker.operations.Operation.output File
"apache_beam/runners/worker/operations.py", line 348, in
apache_beam.runners.worker.operations.Operation.output File
"apache_beam/runners/worker/operations.py", line 215, in
apache_beam.runners.worker.operations.SingletonConsumerSet.receive File
"apache_beam/runners/worker/operations.py", line 915, in
apache_beam.runners.worker.operations.CombineOperation.process File
"apache_beam/runners/worker/operations.py", line 919, in
apache_beam.runners.worker.operations.CombineOperation.process File
"/usr/local/lib/python3.8/site-packages/apache_beam/transforms/combiners.py",
line 879, in extract_only return
self.combine_fn.extract_output(accumulator) File
"/usr/local/lib/python3.8/site-packages/apache_beam/transforms/combiners.py",
line 516, in extract_output heap.sort(reverse=True) File
"apache_beam/transforms/cy_combiners.py", line 389, in
apache_beam.transforms.cy_combiners.ComparableValue.__lt__ AssertionError
passed through: ==> dist_proc/dax/workflow/worker/fnapi_service_impl.cc:771
On Thu, Jan 13, 2022 at 7:42 PM Valentyn Tymofieiev (Jira) <[email protected]>
--
Miren Esnaola | Google Cloud Professional Services | [email protected]
+34 917 48 66 47
> Error running Top transform in Dataflow
> ---------------------------------------
>
> Key: BEAM-12847
> URL: https://issues.apache.org/jira/browse/BEAM-12847
> Project: Beam
> Issue Type: Bug
> Components: runner-dataflow
> Affects Versions: 2.32.0
> Reporter: Miren Esnaola
> Priority: P3
>
> When running the following transform in Dataflow (the problem does not happen
> with the direct running)... I is a streaming pipeline where I am using a
> SlidingWindow.
> {code:java}
> beam.combiners.Top.Of(n=10, key=lambda item: item[1]).without_defaults()
> {code}
>
> I am getting an error
>
> {code:java}
> Error message from worker: generic::unknown: Traceback (most recent call
> last): File
> "/usr/local/lib/python3.7/site-packages/apache_beam/runners/worker/sdk_worker.py",
> line 284, in _execute response = task() File
> "/usr/local/lib/python3.7/site-packages/apache_beam/runners/worker/sdk_worker.py",
> line 357, in <lambda> lambda: self.create_worker().do_instruction(request),
> request) File
> "/usr/local/lib/python3.7/site-packages/apache_beam/runners/worker/sdk_worker.py",
> line 602, in do_instruction getattr(request, request_type),
> request.instruction_id) File
> "/usr/local/lib/python3.7/site-packages/apache_beam/runners/worker/sdk_worker.py",
> line 639, in process_bundle bundle_processor.process_bundle(instruction_id))
> File
> "/usr/local/lib/python3.7/site-packages/apache_beam/runners/worker/bundle_processor.py",
> line 994, in process_bundle element.data) File
> "/usr/local/lib/python3.7/site-packages/apache_beam/runners/worker/bundle_processor.py",
> line 222, in process_encoded self.output(decoded_value) File
> "apache_beam/runners/worker/operations.py", line 351, in
> apache_beam.runners.worker.operations.Operation.output File
> "apache_beam/runners/worker/operations.py", line 353, in
> apache_beam.runners.worker.operations.Operation.output File
> "apache_beam/runners/worker/operations.py", line 215, in
> apache_beam.runners.worker.operations.SingletonConsumerSet.receive File
> "apache_beam/runners/worker/operations.py", line 921, in
> apache_beam.runners.worker.operations.CombineOperation.process File
> "apache_beam/runners/worker/operations.py", line 925, in
> apache_beam.runners.worker.operations.CombineOperation.process File
> "/usr/local/lib/python3.7/site-packages/apache_beam/transforms/combiners.py",
> line 835, in extract_only return self.combine_fn.extract_output(accumulator)
> File
> "/usr/local/lib/python3.7/site-packages/apache_beam/transforms/combiners.py",
> line 502, in extract_output heap.sort(reverse=True) File
> "apache_beam/transforms/cy_combiners.py", line 389, in
> apache_beam.transforms.cy_combiners.ComparableValue.__lt__ AssertionError
> passed through: ==>
> dist_proc/dax/workflow/worker/fnapi_service_impl.cc:644{code}
>
> {{What might be the cause of this issue?}}
>
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
