[
https://issues.apache.org/jira/browse/BEAM-14553?focusedWorklogId=777854&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-777854
]
ASF GitHub Bot logged work on BEAM-14553:
-----------------------------------------
Author: ASF GitHub Bot
Created on: 02/Jun/22 21:55
Start Date: 02/Jun/22 21:55
Worklog Time Spent: 10m
Work Description: kennknowles commented on code in PR #17818:
URL: https://github.com/apache/beam/pull/17818#discussion_r888443505
##########
sdks/java/core/src/main/java/org/apache/beam/sdk/io/FileBasedSink.java:
##########
@@ -1196,7 +1196,7 @@ public static <DestinationT>
FileResultCoder<DestinationT> of(
@Override
public List<? extends Coder<?>> getCoderArguments() {
- return Arrays.asList(windowCoder);
+ return Arrays.asList(windowCoder, destinationCoder);
Review Comment:
You can see how it is used here:
https://github.com/apache/beam/blob/243128a8fc52798e1b58b0cf1a271d95ee7aa241/sdks/java/core/src/main/java/org/apache/beam/sdk/coders/CoderRegistry.java#L723
So if you have a `Coder<Foo<A, B, C>>` then the list of type arguments is
`A, B, C` and `getCoderArguments` needs to return a `Coder<A>, Coder<B>,
Coder<C>` that matches. So definitely before this pull request the coder
registry will try to encode `DestinationT` with the window coder.
Right now `FileResult` just holds the window without tracking the type
variable for it. I think it would be a breaking change to make it
`FileResult<WindowT, DestinationT>`. I think just changing the window coder to
the destination coder is the right thing to do.
Issue Time Tracking
-------------------
Worklog Id: (was: 777854)
Time Spent: 1.5h (was: 1h 20m)
> Dataflow portable job submission translate FileResultCoder only with window
> coder
> ---------------------------------------------------------------------------------
>
> Key: BEAM-14553
> URL: https://issues.apache.org/jira/browse/BEAM-14553
> Project: Beam
> Issue Type: Bug
> Components: runner-dataflow
> Reporter: Yichi Zhang
> Priority: P2
> Time Spent: 1.5h
> Remaining Estimate: 0h
>
> The destination coder is neglected, if there are multiple FileResultCoders
> with different destination coder, only first registration is successful.
--
This message was sent by Atlassian Jira
(v8.20.7#820007)
