[
https://issues.apache.org/jira/browse/TEZ-1379?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14086907#comment-14086907
]
Bikas Saha commented on TEZ-1379:
---------------------------------
bq. setComparator(String comparator), setComparator(String comparator,
Configuration conf). Thoughts ?
That probably would work. The problem here seems to be that we dont have a
well-known set of configurations since the free-form Configuration was being
used earlier. So now we are struggling with that. I am afraid people will pass
the same configuration file into all these methods. Hopefully we are
maintaining a single consolidated configuration internally.
Maybe instead of a configuration per partitioner/comparator/etc we could just
accept an overall edge configuration that supplies configuration for all of the
components. That would be easier to use since such users since they probably
already have 1 consolidated conf file. So if any of your comparator/partitioner
etc require specific conf then use that one method to pass the conf for all of
them. The setFromConfiguration() method seems to suffice except that it filters
things. Maybe we can add a Tez pass through prefix that lets specific conf
through if they have that prefix. Having so many methods take a configuration
argument (advising users to filter only required confs in them) seems
burdensome to me.
bq.That's currently from TezRuntimeConfiguration
Documenting the javadoc would be useful.
> EdgeConfigurers should accept a Partitioner configuration, accept parameters
> for compression and secure shuffle
> ---------------------------------------------------------------------------------------------------------------
>
> Key: TEZ-1379
> URL: https://issues.apache.org/jira/browse/TEZ-1379
> Project: Apache Tez
> Issue Type: Sub-task
> Reporter: Siddharth Seth
> Assignee: Siddharth Seth
> Priority: Blocker
> Attachments: TEZ-1379.1.txt, TEZ-1379.2.txt
>
>
--
This message was sent by Atlassian JIRA
(v6.2#6252)
