[
https://issues.apache.org/jira/browse/MAPREDUCE-7424?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
teng wang updated MAPREDUCE-7424:
---------------------------------
Description:
Suggestions to fix the document (description of mapreduce.job.sharedcache.mode
in mapred-default.xml):
There is one dependency between *mapreduce.job.sharedcache.mode* and
*yarn.sharedcache.enabled* in source code. That is,
mapreduce.job.sharedcache.mode *can work if* the shared cache
(yarn.sharedcache.enabled) is enabled. However, the document
(mapred-default.xml) does not mention it, which could affects the use of this
configuration.
The dependency code:
{code:java}
/* /apache/hadoop/mapreduce/SharedCacheConfig.java */
public void init(Configuration conf) {
if(!conf.getBoolean(YarnConfiguration.SHARED_CACHE_ENABLED,
YarnConfiguration.DEFAULT_SHARED_CACHE_ENABLED)) {
return;
}
Collection<String> configs = StringUtils.getTrimmedStringCollection(
conf.get(MRJobConfig.SHARED_CACHE_MODE,
MRJobConfig.SHARED_CACHE_MODE_DEFAULT));
if (configs.contains("files")) {
this.sharedCacheFilesEnabled = true;
}{code}
was:
Suggestions to fix the document (description of mapreduce.job.sharedcache.mode
in mapred-default.xml):
There is one dependency between mapreduce.job.sharedcache.mode and
yarn.sharedcache.enabled in source code. That is,
mapreduce.job.sharedcache.mode can work if the shared cache
(yarn.sharedcache.enabled) is enabled. However, the document
(mapred-default.xml) does not mention it, which could affects the use of this
configuration.
The dependency code:
{code:java}
/* /apache/hadoop/mapreduce/SharedCacheConfig.java */
public void init(Configuration conf) {
if(!conf.getBoolean(YarnConfiguration.SHARED_CACHE_ENABLED,
YarnConfiguration.DEFAULT_SHARED_CACHE_ENABLED)) {
return;
}
Collection<String> configs = StringUtils.getTrimmedStringCollection(
conf.get(MRJobConfig.SHARED_CACHE_MODE,
MRJobConfig.SHARED_CACHE_MODE_DEFAULT));
if (configs.contains("files")) {
this.sharedCacheFilesEnabled = true;
}{code}
> Document Fix: the dependency between mapreduce.job.sharedcache.mode and
> yarn.sharedcache.enabled
> ------------------------------------------------------------------------------------------------
>
> Key: MAPREDUCE-7424
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-7424
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: job submission
> Affects Versions: 3.3.4
> Reporter: teng wang
> Priority: Major
>
> Suggestions to fix the document (description of
> mapreduce.job.sharedcache.mode in mapred-default.xml):
> There is one dependency between *mapreduce.job.sharedcache.mode* and
> *yarn.sharedcache.enabled* in source code. That is,
> mapreduce.job.sharedcache.mode *can work if* the shared cache
> (yarn.sharedcache.enabled) is enabled. However, the document
> (mapred-default.xml) does not mention it, which could affects the use of this
> configuration.
>
> The dependency code:
> {code:java}
> /* /apache/hadoop/mapreduce/SharedCacheConfig.java */
> public void init(Configuration conf) {
> if(!conf.getBoolean(YarnConfiguration.SHARED_CACHE_ENABLED,
> YarnConfiguration.DEFAULT_SHARED_CACHE_ENABLED)) {
> return;
> }
> Collection<String> configs = StringUtils.getTrimmedStringCollection(
> conf.get(MRJobConfig.SHARED_CACHE_MODE,
> MRJobConfig.SHARED_CACHE_MODE_DEFAULT));
>
> if (configs.contains("files")) {
> this.sharedCacheFilesEnabled = true;
> }{code}
>
>
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
