[
https://issues.apache.org/jira/browse/BEAM-14553?focusedWorklogId=777863&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-777863
]
ASF GitHub Bot logged work on BEAM-14553:
-----------------------------------------
Author: ASF GitHub Bot
Created on: 02/Jun/22 22:37
Start Date: 02/Jun/22 22:37
Worklog Time Spent: 10m
Work Description: y1chi commented on code in PR #17818:
URL: https://github.com/apache/beam/pull/17818#discussion_r888463072
##########
sdks/java/core/src/main/java/org/apache/beam/sdk/io/FileBasedSink.java:
##########
@@ -1196,7 +1196,7 @@ public static <DestinationT>
FileResultCoder<DestinationT> of(
@Override
public List<? extends Coder<?>> getCoderArguments() {
- return Arrays.asList(windowCoder);
+ return Arrays.asList(windowCoder, destinationCoder);
Review Comment:
When registering a StructuredCoder, the component coders are hashed and
compared in order to avoid duplicate. That means if we have one
FileResultCoder(IntervalWindowCoder, VoidCoder) and another
FileResultCoder(GlobalWIndowCoder, VoidCoder) they'll still cause collision if
we use Arrays.asList(destinationCoder) as components.
I didn't see the window coder is being used to encode/decode the
destinationT, the first registered FileResultCoder has serialized payload and
then deserialized correctly and was able to decode the FileResult type
correctly. But the other FileResultCoder will be deserialized from the same
payload will throw exception because the destination coder does not match.
Issue Time Tracking
-------------------
Worklog Id: (was: 777863)
Time Spent: 1h 50m (was: 1h 40m)
> Dataflow portable job submission translate FileResultCoder only with window
> coder
> ---------------------------------------------------------------------------------
>
> Key: BEAM-14553
> URL: https://issues.apache.org/jira/browse/BEAM-14553
> Project: Beam
> Issue Type: Bug
> Components: runner-dataflow
> Reporter: Yichi Zhang
> Priority: P2
> Time Spent: 1h 50m
> Remaining Estimate: 0h
>
> The destination coder is neglected, if there are multiple FileResultCoders
> with different destination coder, only first registration is successful.
--
This message was sent by Atlassian Jira
(v8.20.7#820007)
