[
https://issues.apache.org/jira/browse/YARN-6032?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15782180#comment-15782180
]
Zhaofei Meng commented on YARN-6032:
------------------------------------
We should modify use interface in ClientProtocolService to verify fs if or not
exist in hdfs.
public UseSharedCacheResourceResponse use(
UseSharedCacheResourceRequest request) throws YarnException,
IOException {
UseSharedCacheResourceResponse response =
recordFactory.newRecordInstance(UseSharedCacheResourceResponse.class);
UserGroupInformation callerUGI;
try {
callerUGI = UserGroupInformation.getCurrentUser();
} catch (IOException ie) {
LOG.info("Error getting UGI ", ie);
throw RPCUtil.getRemoteException(ie);
}
String fileName =
this.store.addResourceReference(request.getResourceKey(),
new SharedCacheResourceReference(request.getAppId(),
callerUGI.getShortUserName()));
if (fileName != null) {
if(fs.exists(new Path(fs.getHomeDirectory(),fileName))){
response
.setPath(getCacheEntryFilePath(request.getResourceKey(),
fileName));
this.metrics.incCacheHitCount();
}else{
this.store.removeResource(request.getResourceKey());
}
} else {
this.metrics.incCacheMissCount();
}
return response;
}
> scm cleaner task should rm InMemorySCMStore some cachedResources which does
> not exists in hdfs fs
> --------------------------------------------------------------------------------------------------
>
> Key: YARN-6032
> URL: https://issues.apache.org/jira/browse/YARN-6032
> Project: Hadoop YARN
> Issue Type: Bug
> Affects Versions: 2.7.1
> Reporter: Zhaofei Meng
> Fix For: 2.7.1
>
>
> If cacheresources exist in scm but not exist in hdfs,the cacheresources
> whill not rm from scm until restart scm.So we shoult add check funcion in
> cleaner task that rm the cachedResources which does not exists in hdfs fs.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
