Bryan Beaudreault created HDFS-16261:
----------------------------------------
Summary: Configurable grace period around deletion of invalidated
blocks
Key: HDFS-16261
URL: https://issues.apache.org/jira/browse/HDFS-16261
Project: Hadoop HDFS
Issue Type: New Feature
Reporter: Bryan Beaudreault
Assignee: Bryan Beaudreault
When a block is moved with REPLACE_BLOCK, the new location is recorded in the
NameNode and the NameNode instructs the old host to in invalidate the block
using DNA_INVALIDATE. As it stands today, this invalidation is async but tends
to happen relatively quickly.
I'm working on a feature for HBase which enables efficient healing of locality
through Balancer-style low level block moves. One issue is that HBase tends to
keep open long running DFSInputStreams and moving blocks from under them causes
lots of warns in the RegionServer and increases long tail latencies due to the
necessary retries in the DFSClient.
One way I'd like to fix this is to provide a configurable grace period on async
invalidations. This would give the DFSClient enough time to refresh block
locations before hitting any errors.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
