[
https://issues.apache.org/jira/browse/MAPREDUCE-5894?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
jay vyas updated MAPREDUCE-5894:
--------------------------------
Description:
We recently found that when deploy hadoop 2.2 with hadoop 2.0 values
{noformat} mapreduce_shuffle {noformat} changed to {noformat}
mapreduce.shuffle {noformat} .
There are likewise many similar examples of parameters which become deprecated
over time. See
http://hadoop.apache.org/docs/r2.2.0/hadoop-project-dist/hadoop-common/DeprecatedProperties.html
I suggest we:
1) Have a list of all mandatory *current* parameters stored in the code, and
also,
2) a list of deprecated ones.
3) Then, have the build * automatically fail * if a parameter in the "madatory"
list is NOT accessed during unit tests....
this would (a) make it so that unit testing of parameters does not regress and
(b) force all updates to the code which change a parameter name, to also
include an *update to deprecated parameter list*, before build passes.
This setup could then be used to enable
4) Reporting or failing upon presence of clearly obsolete parameters which are
result of obvious human error.
was:
We recently found that when deploy hadoop 2.2 with hadoop 2.0 values
{noformat} mapreduce_shuffle {noformat} changed to {noformat}
mapreduce.shuffle {noformat} .
There are likewise many similar examples of parameters which become deprecated
over time. See
http://hadoop.apache.org/docs/r2.2.0/hadoop-project-dist/hadoop-common/DeprecatedProperties.html
I suggest we:
1) Have a list of all mandatory *current* parameters stored in the code, and
also,
2) a list of deprecated ones.
3) Then, have the build * automatically fail * a parameter in the "madatory"
list is NOT accessed. this would (a) make it so that unit testing of
parameters does not regress and (b) force all updates to the code which change
a parameter name, to also include update to deprecated parameter list, before
build passes.
> Make critical YARN properties first class citizens in the build.
> ----------------------------------------------------------------
>
> Key: MAPREDUCE-5894
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-5894
> Project: Hadoop Map/Reduce
> Issue Type: Improvement
> Reporter: jay vyas
>
> We recently found that when deploy hadoop 2.2 with hadoop 2.0 values
> {noformat} mapreduce_shuffle {noformat} changed to {noformat}
> mapreduce.shuffle {noformat} .
> There are likewise many similar examples of parameters which become
> deprecated over time. See
> http://hadoop.apache.org/docs/r2.2.0/hadoop-project-dist/hadoop-common/DeprecatedProperties.html
> I suggest we:
> 1) Have a list of all mandatory *current* parameters stored in the code, and
> also,
> 2) a list of deprecated ones.
> 3) Then, have the build * automatically fail * if a parameter in the
> "madatory" list is NOT accessed during unit tests....
> this would (a) make it so that unit testing of parameters does not regress
> and (b) force all updates to the code which change a parameter name, to also
> include an *update to deprecated parameter list*, before build passes.
> This setup could then be used to enable
> 4) Reporting or failing upon presence of clearly obsolete parameters which
> are result of obvious human error.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
