[
https://issues.apache.org/jira/browse/MAPREDUCE-7485?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
LoggingResearch updated MAPREDUCE-7485:
---------------------------------------
Attachment: SharedCacheConfigTest.java
> Insufficient System Outputs to Address Misconfiguration in Shared Cache
> -----------------------------------------------------------------------
>
> Key: MAPREDUCE-7485
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-7485
> Project: Hadoop Map/Reduce
> Issue Type: Improvement
> Components: mapreduce-client
> Affects Versions: 3.3.6
> Environment: Version: {{`3.3.6`}}
> Location{*}:{*}
> {{{}hadoop/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapreduce/SharedCacheConfig.java{}}},
> in the {{init}} method, starting from Line 46.
> Reporter: LoggingResearch
> Priority: Major
> Attachments: SharedCacheConfigTest.java
>
>
> To enable the {{{}shared cache{}}}, users must set specific configuration
> parameters: {{mapreduce.framework.name}} should be set to {{{}yarn{}}},
> {{yarn.sharedcache.enable}} should be {{{}true{}}}, and
> {{mapreduce.job.sharedcache.mode}} should be configured as {{enabled}} or one
> of {{{}jobjar, libjars, files, archives{}}}. This requirement is implicit in
> the code within {{{}org.apache.hadoop.mapreduce.SharedCacheConfig.java{}}}.
> However, these relationships between the code and configuration parameters
> are not documented in the configuration guides. The necessary configurations
> are not mentioned in either {{mapreduce-default.xml}} or
> {{{}yarn-default.xml{}}}. Consequently, users may be unable to enable the
> shared cache feature even if they thoroughly review the documentation. This
> lack of guidance negatively impacts user experience, system diagnosability,
> and observability.
>
> *Expected Behavior:*
> The system should provide clear guidance on misconfiguration related to the
> shared cache. Simply relying on users to meticulously read the configuration
> documentation is insufficient, especially when some configurations are
> undocumented.
>
> *How-to-Fix:*
> We recommend enhancing the logging mechanism to expose these configuration
> relationships. This will provide users with immediate feedback and guidance
> when misconfigurations occur.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
