[
https://issues.apache.org/jira/browse/BEAM-11154?focusedWorklogId=506395&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-506395
]
ASF GitHub Bot logged work on BEAM-11154:
-----------------------------------------
Author: ASF GitHub Bot
Created on: 29/Oct/20 20:41
Start Date: 29/Oct/20 20:41
Worklog Time Spent: 10m
Work Description: y1chi commented on pull request #13225:
URL: https://github.com/apache/beam/pull/13225#issuecomment-719008866
R: @kennknowles
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
Issue Time Tracking
-------------------
Worklog Id: (was: 506395)
Time Spent: 20m (was: 10m)
> Missing coder in pipeline components with dataflow runner v2
> ------------------------------------------------------------
>
> Key: BEAM-11154
> URL: https://issues.apache.org/jira/browse/BEAM-11154
> Project: Beam
> Issue Type: Bug
> Components: runner-dataflow
> Reporter: Yichi Zhang
> Assignee: Yichi Zhang
> Priority: P2
> Time Spent: 20m
> Remaining Estimate: 0h
>
> When running pipelines with Top combine function on dataflow runner v2, the
> backend complains about missing coder id for example missing
> BoundedHeapCoder1.
> After some troubleshooting this problem seems more generic:
> The step context translation phase would not recognize already registered
> Coder with incorrect hashCode() function, and will try to give it a new
> uniqified name to the pipeline_proto_coder_id,
> code pointers:
> https://github.com/apache/beam/blob/5675108933de6eb601ca2e4f21870d2ababe0ec7/runners/core-construction-java/src/main/java/org/apache/beam/runners/core/construction/SdkComponents.java#L268
> In this case, since the comparator field in BoundedHeapCoder often does not
> implement hashCode() and equals() the BoundedHeapCoder will also have a
> different hashCode() each time a new instance is created. The duplicated
> coder does not exist in already translated pipeline proto and will lead to
> the aforementioned missing coder id issue.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
