LoggingResearch created MAPREDUCE-7485:
------------------------------------------
Summary: Insufficient System Outputs to Address Misconfiguration
in Shared Cache
Key: MAPREDUCE-7485
URL: https://issues.apache.org/jira/browse/MAPREDUCE-7485
Project: Hadoop Map/Reduce
Issue Type: Improvement
Components: mapreduce-client
Affects Versions: 3.3.6
Environment: Version: {{`3.3.6`}}
Location{*}:{*}
{{{}hadoop/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapreduce/SharedCacheConfig.java{}}},
in the {{init}} method, starting from Line 46.
Reporter: LoggingResearch
To enable the {{{}shared cache{}}}, users must set specific configuration
parameters: {{mapreduce.framework.name}} should be set to {{{}yarn{}}},
{{yarn.sharedcache.enable}} should be {{{}true{}}}, and
{{mapreduce.job.sharedcache.mode}} should be configured as {{enabled}} or one
of {{{}jobjar, libjars, files, archives{}}}. This requirement is implicit in
the code within {{{}org.apache.hadoop.mapreduce.SharedCacheConfig.java{}}}.
However, these relationships between the code and configuration parameters are
not documented in the configuration guides. The necessary configurations are
not mentioned in either {{mapreduce-default.xml}} or {{{}yarn-default.xml{}}}.
Consequently, users may be unable to enable the shared cache feature even if
they thoroughly review the documentation. This lack of guidance negatively
impacts user experience, system diagnosability, and observability.
*Expected Behavior:*
The system should provide clear guidance on misconfiguration related to the
shared cache. Simply relying on users to meticulously read the configuration
documentation is insufficient, especially when some configurations are
undocumented.
*How-to-Fix:*
We recommend enhancing the logging mechanism to expose these configuration
relationships. This will provide users with immediate feedback and guidance
when misconfigurations occur.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
