[
https://issues.apache.org/jira/browse/BEAM-741?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15581093#comment-15581093
]
Kenneth Knowles edited comment on BEAM-741 at 3/21/17 2:21 AM:
---------------------------------------------------------------
Great investigation. I actually think the SDK should also always prefer the
transform's coder. But, also, for input of type {{KV<K,V>}}, the expected
behavior is for the registry to associate the type {{V}} with the value coder
and thus in this context provide exactly the same coder. So I'm going to reopen
and see about both of these.
I am struck by this conflict: the transform has some more detailed information
about its output, but also if the user sets a coder on the input PCollection,
they have even more information than a transform with a type variable, like
{{Values}}. Maybe they know something about the data distribution. If both the
registry and each transform try to adhere to the rule of propagating the user's
intent, I think they should end up largely equivalent.
was (Author: kenn):
Great investigation. I actually think the SDK should also always prefer the
transform's coder. But, also, for input of type {{KV<K,V>}}, the expected
behavior is for the registry to associate the type {{V}} with the value coder
and thus in this context provide exactly the same coder. So I'm going to reopen
and see about both of these.
I am struck by this conflict: the transform has some more detailed information
about its output, but also if the user sets a coder on the input PCollection,
they have even more information that a transform with a type variable, like
{{Values}}. Maybe they know something about the data distribution. If both the
registry and each transform try to adhere to the rule of propagating the user's
intent, I think they should end up largely equivalent.
> Values transform does not use the correct output coder when values is an
> Iterable<T>
> ------------------------------------------------------------------------------------
>
> Key: BEAM-741
> URL: https://issues.apache.org/jira/browse/BEAM-741
> Project: Beam
> Issue Type: Bug
> Components: sdk-java-core
> Reporter: Andrew Martin
> Assignee: Kenneth Knowles
> Fix For: Not applicable
>
>
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
