[
https://issues.apache.org/jira/browse/HDFS-8892?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14704181#comment-14704181
]
Colin Patrick McCabe commented on HDFS-8892:
--------------------------------------------
bq. I assume slot-invalidation will happen during block-invalidation/deletes
{Primarily triggered by compaction/shard-takeover etc..}
Yes.
I guess the good thing about this patch is that it might reduce fd consumption
in some scenarios. The bad thing about this approach is that instead of only
checking the older replicas, we would have to iterate over all replicas in
order to check every slot. Also, currently the Runnable runs less often when
the replica timeout is longer, so this logic would have to be changed.
> ShortCircuitCache.CacheCleaner can add Slot.isInvalid() check too
> -----------------------------------------------------------------
>
> Key: HDFS-8892
> URL: https://issues.apache.org/jira/browse/HDFS-8892
> Project: Hadoop HDFS
> Issue Type: Improvement
> Components: hdfs-client
> Affects Versions: 2.7.1
> Reporter: Ravikumar
> Assignee: kanaka kumar avvaru
> Priority: Minor
>
> Currently CacheCleaner thread checks only for cache-expiry times. It would be
> nice if it handles an invalid-slot too in an extra-pass of evictable map…
> for(ShortCircuitReplica replica:evictable.values()) {
> if(!scr.getSlot().isValid()) {
> purge(replica);
> }
> }
> //Existing code...
> int numDemoted = demoteOldEvictableMmaped(curMs);
> int numPurged = 0;
> Long evictionTimeNs = Long.valueOf(0);
> ….
> …..
> Apps like HBase can tweak the expiry/staleness/cache-size params in
> DFS-Client, so that ShortCircuitReplica will never be closed except when Slot
> is declared invalid.
> I assume slot-invalidation will happen during block-invalidation/deletes
> {Primarily triggered by compaction/shard-takeover etc..}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
