[
https://issues.apache.org/jira/browse/HADOOP-12021?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14557135#comment-14557135
]
Joep Rottinghuis commented on HADOOP-12021:
-------------------------------------------
If you are providing a patch, you'll probably will want to make it optional to
turn on.
Configs get pushed out to the cluster for each job. Already configs can get
pretty large. Adding text will only make them larger. Each node that launches
containers for a job needs to receive these configs. Memory and network
pressure will certainly mount.
When doing work for hRaven and YARN-2928 we do find that configs can be large
and costly at times.
I suspect that changing a simple key->value map into something more complex
will be interesting.
Btw. during debugging issues on our clusters we often see unzip taking
considerable time. That usually indicated that a Configuration object is
getting created (which often reads jars from the classpath). Making that even
more expensive will probably not be great.
Perhaps you can provide a special constructor to create an extensively loaded
Configuration so that you're not burdening everybody else, but I don't know if
you need the full description to be passed around on the cluster, or if this a
client-side only use case.
> Augmenting Configuration to accomodate <description>
> ----------------------------------------------------
>
> Key: HADOOP-12021
> URL: https://issues.apache.org/jira/browse/HADOOP-12021
> Project: Hadoop Common
> Issue Type: New Feature
> Components: conf
> Reporter: Lewis John McGibbney
> Priority: Minor
> Fix For: 1.3.0, 2.8.0
>
>
> Over on the
> [common-dev|http://www.mail-archive.com/common-dev%40hadoop.apache.org/msg16099.html]
> ML I explained a use case which requires me to obtain the value of the
> Configuration <description> tags.
> [~cnauroth] advised me to raise the issue to Jira for discussion.
> I am happy to provide a patch so that the <description> values are parsed out
> of the various XML files and stored, and also that the Configuration class is
> augmented to provide accessors to accommodate the use case.
> I wanted to find out what people think about this one and whether I should
> check out Hadoop source and submit a patch. If you guys could provide some
> advice it would be appreciated.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
