[jira] [Updated] (HDFS-11540) PeerCache sync overhead
[ https://issues.apache.org/jira/browse/HDFS-11540?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ravikumar updated HDFS-11540: - Attachment: PeerCache.diff Please check the patch attached, based on trunk... Replaced blocking multimap with CHM along with a separate LBQ for expired sockets. These are reaped via the expiry daemon > PeerCache sync overhead > --- > > Key: HDFS-11540 > URL: https://issues.apache.org/jira/browse/HDFS-11540 > Project: Hadoop HDFS > Issue Type: Improvement > Components: hdfs-client >Affects Versions: 2.7.1 >Reporter: Ravikumar >Priority: Minor > Attachments: PeerCache.diff > > > PeerCache has a fair amount sync over-head. In addition to class level > locking, expired sockets & eldest socket are closed in critical section > during gets & puts respectively.. > A ConcurrentLinkedHashMap with appropriate fencing using DatanodeID could > speed it up? -- This message was sent by Atlassian JIRA (v6.3.15#6346) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Created] (HDFS-11540) PeerCache sync overhead
Ravikumar created HDFS-11540: Summary: PeerCache sync overhead Key: HDFS-11540 URL: https://issues.apache.org/jira/browse/HDFS-11540 Project: Hadoop HDFS Issue Type: Improvement Components: hdfs-client Affects Versions: 2.7.1 Reporter: Ravikumar Priority: Minor PeerCache has a fair amount sync over-head. In addition to class level locking, expired sockets & eldest socket are closed in critical section during gets & puts respectively.. A ConcurrentLinkedHashMap with appropriate fencing using DatanodeID could speed it up? -- This message was sent by Atlassian JIRA (v6.3.15#6346) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-3051) A zero-copy ScatterGatherRead api from FSDataInputStream
[
https://issues.apache.org/jira/browse/HDFS-3051?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15374535#comment-15374535
]
Ravikumar commented on HDFS-3051:
-
How about returning the MappedByteBuffers of all blocks for a file in local. If
there are non-local blocks, this method can simply return empty.
public List readFullyScatterGatherLocal(EnumSet options)
throws IOException {
return ((PositionedReadable)in).readFullyScatterGather(options);
}
A quick sample-impl can be like
public List readFullyScatterGatherLocal(EnumSet) throws
IOException
{
List blockRange = getBlockRange(0, getFileLength());
if(!allBlocksInLocal(blockRange))
{
return;
}
List retval = new LinkedList();
for(LocatedBlock blk:blockRange)
{
blkReader = fetchBlockReader(blk, localDNAddrPair);
ClientMmap mmap = blkReader.getClientMmap(readOptions);
mmap.setunmap(false); //Instruction to cache-eviction to avoid unmapping
this. Slots, streams & all other resources will be closed
result.add(mmap.getMappedByteBuffer());
closeBlockReader(blkReader);
}
return retval
}
Apps opening InputStreams only once (Hbase??) can call this method & use the
zero-copy buffers for reads, if file is local. If not available, they can fall
back to regular DFSInputStream. Reads can eliminate sync overheads & get same
perf as a local filesystem.
But I don't know if "leaking" MappedByteBuffers to calling code can have nasty
side-effects.
> A zero-copy ScatterGatherRead api from FSDataInputStream
>
>
> Key: HDFS-3051
> URL: https://issues.apache.org/jira/browse/HDFS-3051
> Project: Hadoop HDFS
> Issue Type: Improvement
> Components: hdfs-client, performance
>Reporter: dhruba borthakur
>Assignee: dhruba borthakur
>
> It will be nice if we can get a new API from FSDtaInputStream that allows for
> zero-copy read for hdfs readers.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-8892) ShortCircuitCache.CacheCleaner can add Slot.isInvalid() check too
[
https://issues.apache.org/jira/browse/HDFS-8892?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14720116#comment-14720116
]
Ravikumar commented on HDFS-8892:
-
Please feel free to contribute the patch [~kanaka]. I am not currently looking
to submit it,
ShortCircuitCache.CacheCleaner can add Slot.isInvalid() check too
-
Key: HDFS-8892
URL: https://issues.apache.org/jira/browse/HDFS-8892
Project: Hadoop HDFS
Issue Type: Improvement
Components: hdfs-client
Affects Versions: 2.7.1
Reporter: Ravikumar
Assignee: kanaka kumar avvaru
Priority: Minor
Currently CacheCleaner thread checks only for cache-expiry times. It would be
nice if it handles an invalid-slot too in an extra-pass of evictable map…
for(ShortCircuitReplica replica:evictable.values()) {
if(!scr.getSlot().isValid()) {
purge(replica);
}
}
//Existing code...
int numDemoted = demoteOldEvictableMmaped(curMs);
int numPurged = 0;
Long evictionTimeNs = Long.valueOf(0);
….
…..
Apps like HBase can tweak the expiry/staleness/cache-size params in
DFS-Client, so that ShortCircuitReplica will never be closed except when Slot
is declared invalid.
I assume slot-invalidation will happen during block-invalidation/deletes
{Primarily triggered by compaction/shard-takeover etc..}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HDFS-8892) ShortCircuitCache.CacheCleaner can add Slot.isInvalid() check too
[
https://issues.apache.org/jira/browse/HDFS-8892?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14710959#comment-14710959
]
Ravikumar commented on HDFS-8892:
-
I am fine if this runs in a separate thread as you said…
My app opens hadoop files only once. The block input-streams, once opened need
to stay that way forever until app instructs to close it. The close call
happens only for deleting the file or transferring the ownership of file to
another server {Like Hbase regions}
With current ShortCircuitCache, I am not able to find any knobs to tune for
such kind of behaviour.
I am planning to set all params of LRU cache high so that ShortCircuitCache
never closes any open input-stream by itself. All the stream-close calls are
handled only through invalid-slot-checker
ShortCircuitCache.CacheCleaner can add Slot.isInvalid() check too
-
Key: HDFS-8892
URL: https://issues.apache.org/jira/browse/HDFS-8892
Project: Hadoop HDFS
Issue Type: Improvement
Components: hdfs-client
Affects Versions: 2.7.1
Reporter: Ravikumar
Assignee: kanaka kumar avvaru
Priority: Minor
Currently CacheCleaner thread checks only for cache-expiry times. It would be
nice if it handles an invalid-slot too in an extra-pass of evictable map…
for(ShortCircuitReplica replica:evictable.values()) {
if(!scr.getSlot().isValid()) {
purge(replica);
}
}
//Existing code...
int numDemoted = demoteOldEvictableMmaped(curMs);
int numPurged = 0;
Long evictionTimeNs = Long.valueOf(0);
….
…..
Apps like HBase can tweak the expiry/staleness/cache-size params in
DFS-Client, so that ShortCircuitReplica will never be closed except when Slot
is declared invalid.
I assume slot-invalidation will happen during block-invalidation/deletes
{Primarily triggered by compaction/shard-takeover etc..}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HDFS-8892) ShortCircuitCache.CacheCleaner can add Slot.isInvalid() check too
[
https://issues.apache.org/jira/browse/HDFS-8892?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14706455#comment-14706455
]
Ravikumar commented on HDFS-8892:
-
Iterating over all replicas is a cause of concern. Is it possible for client to
ack a slot made invalid by DN? But my guess is that, it would needlessly
complicate this issue.
May be the patch itself is applicable for specific use-cases…
How about exposing a boolean config
dfs.client.read.shortcircuit.replica.stale.cleanup and wrapping this small
patch in a conditional block?
ShortCircuitCache.CacheCleaner can add Slot.isInvalid() check too
-
Key: HDFS-8892
URL: https://issues.apache.org/jira/browse/HDFS-8892
Project: Hadoop HDFS
Issue Type: Improvement
Components: hdfs-client
Affects Versions: 2.7.1
Reporter: Ravikumar
Assignee: kanaka kumar avvaru
Priority: Minor
Currently CacheCleaner thread checks only for cache-expiry times. It would be
nice if it handles an invalid-slot too in an extra-pass of evictable map…
for(ShortCircuitReplica replica:evictable.values()) {
if(!scr.getSlot().isValid()) {
purge(replica);
}
}
//Existing code...
int numDemoted = demoteOldEvictableMmaped(curMs);
int numPurged = 0;
Long evictionTimeNs = Long.valueOf(0);
….
…..
Apps like HBase can tweak the expiry/staleness/cache-size params in
DFS-Client, so that ShortCircuitReplica will never be closed except when Slot
is declared invalid.
I assume slot-invalidation will happen during block-invalidation/deletes
{Primarily triggered by compaction/shard-takeover etc..}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HDFS-8892) ShortCircuitCache.CacheCleaner can add Slot.isInvalid() check too
[
https://issues.apache.org/jira/browse/HDFS-8892?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14696971#comment-14696971
]
Ravikumar commented on HDFS-8892:
-
A small patch we tried. Works as expected during block-deletes
…..
if (LOG.isDebugEnabled()) {
LOG.debug(this + : cache cleaner running at + curMs);
}
//Copy the evictable, as purge removes elements from map
MapLong, ShortCircuitReplica copy = new TreeMapLong,
ShortCircuitReplica(evictable);
for(ShortCircuitReplica replica:copy.values()) {
Slot currSlot = replica.getSlot();
if(currSlot != null !currSlot.isValid()) {
if (LOG.isTraceEnabled()) {
LOG.trace(CacheCleaner: purging because slot is invalid +
replica + : + StringUtils.getStackTrace(Thread.currentThread()));
}
purge(replica);
numPurged++;
}
}
//Existing Code
int numDemoted = demoteOldEvictableMmaped(curMs);
int numPurged = 0;
….
ShortCircuitCache.CacheCleaner can add Slot.isInvalid() check too
-
Key: HDFS-8892
URL: https://issues.apache.org/jira/browse/HDFS-8892
Project: Hadoop HDFS
Issue Type: Improvement
Components: hdfs-client
Affects Versions: 2.7.1
Reporter: Ravikumar
Assignee: kanaka kumar avvaru
Priority: Minor
Currently CacheCleaner thread checks only for cache-expiry times. It would be
nice if it handles an invalid-slot too in an extra-pass of evictable map…
for(ShortCircuitReplica replica:evictable.values()) {
if(!scr.getSlot().isValid()) {
purge(replica);
}
}
//Existing code...
int numDemoted = demoteOldEvictableMmaped(curMs);
int numPurged = 0;
Long evictionTimeNs = Long.valueOf(0);
….
…..
Apps like HBase can tweak the expiry/staleness/cache-size params in
DFS-Client, so that ShortCircuitReplica will never be closed except when Slot
is declared invalid.
I assume slot-invalidation will happen during block-invalidation/deletes
{Primarily triggered by compaction/shard-takeover etc..}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Created] (HDFS-8892) ShortCircuitCache.CacheCleaner can add Slot.isInvalid() check too
Ravikumar created HDFS-8892:
---
Summary: ShortCircuitCache.CacheCleaner can add Slot.isInvalid()
check too
Key: HDFS-8892
URL: https://issues.apache.org/jira/browse/HDFS-8892
Project: Hadoop HDFS
Issue Type: Improvement
Components: hdfs-client
Affects Versions: 2.7.1
Reporter: Ravikumar
Priority: Minor
Currently CacheCleaner thread checks only for cache-expiry times. It would be
nice if it handles an invalid-slot too in an extra-pass of evictable map…
for(ShortCircuitReplica replica:evictable.values()) {
if(!scr.getSlot().isValid()) {
purge(replica);
}
}
//Existing code...
int numDemoted = demoteOldEvictableMmaped(curMs);
int numPurged = 0;
Long evictionTimeNs = Long.valueOf(0);
….
…..
Apps like HBase can tweak the expiry/staleness/cache-size params in DFS-Client,
so that ShortCircuitReplica will never be closed except when Slot is declared
invalid.
I assume slot-invalidation will happen during block-invalidation/deletes
{Primarily triggered by compaction/shard-takeover etc..}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
