[
https://issues.apache.org/jira/browse/HADOOP-5979?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12717602#action_12717602
]
Devaraj Das commented on HADOOP-5979:
-------------------------------------
bq. it's simpler (the partitioner command doesn't need to know how many
partitions there are),
But still, the command needs to have an idea of how many partitions there are,
isn't it? Or maybe, you are saying that it's up to the command developer to
assume a certain partition count and implement the command... I agree that it's
simple but am not sure whether all use cases would be covered with this model..
bq. we could reuse more code that's already there (if we let the the
partitioner command output both a key and a value and pass that on to a wrapped
partitioner, like in the code sample I gave above, we even wouldn't need any
additional reading/writing logic).
What did you mean by "we wouldn't need any additional reading/writing logic" ?
There is at least that much reading/writing as your code outlined, ist it?
> Streaming partitioner should allow command, not just Java class
> ---------------------------------------------------------------
>
> Key: HADOOP-5979
> URL: https://issues.apache.org/jira/browse/HADOOP-5979
> Project: Hadoop Core
> Issue Type: Improvement
> Components: contrib/streaming
> Reporter: Klaas Bosteels
>
> Since HADOOP-4842 got committed, Streaming allows both commands and Java
> classes to be specified as mapper, reducer, and combiner, but the
> {{-partitioner}} option is still limited to Java classes only. Allowing
> commands to be specified as partitioner as well would greatly improve the
> flexibility of Streaming programs.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
