[
https://issues.apache.org/jira/browse/HBASE-10000?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13843956#comment-13843956
]
Ted Yu commented on HBASE-10000:
--------------------------------
I tried to add the following test to TestFSHDFSUtils by creating a subclass of
FileSystem which doesn't have isFileClosed:
{code}
/**
* Test that absence of isFileClosed doesn't make lease recovery slower.
* @throws IOException
*/
@Test (timeout = 30000)
public void testIsFileClosedAbsent() throws IOException {
// Make this time long so it is plain we broke out because of the
isFileClosed invocation.
HTU.getConfiguration().setInt("hbase.lease.recovery.dfs.timeout", 100000);
CancelableProgressable reporter =
Mockito.mock(CancelableProgressable.class);
Mockito.when(reporter.progress()).thenReturn(true);
FileSystemWithoutIsFileClosed dfs =
Mockito.mock(FileSystemWithoutIsFileClosed.class);
// Now make it so we fail the first two times -- the two fast invocations,
then we fall into
// the long loop during which we will call isFileClosed.... the next
invocation should
// therefore return true if we are to break the loop.
Mockito.when(dfs.recoverLease(FILE)).
thenReturn(false).thenReturn(false).thenReturn(true);
assertTrue(this.fsHDFSUtils.recoverDFSFileLease(dfs, FILE,
HTU.getConfiguration(), reporter,
HConstants.LEASE_RECOVERY_UNREQUESTED));
}
class FileSystemWithoutIsFileClosed extends FileSystem {
public boolean recoverLease(final Path f) throws IOException {
return true;
}
@Override
public FSDataOutputStream create(Path f,
FsPermission permission,
boolean overwrite,
int bufferSize,
short replication,
long blockSize,
Progressable progress) throws IOException {
return null;
}
@Override
public FSDataOutputStream append(Path f, int bufferSize,
Progressable progress) throws IOException {
return null;
}
@Override
public FileStatus getFileStatus(Path f) throws IOException {
return null;
}
@Override
public FSDataInputStream open(Path f, int bufferSize)
throws IOException {
return null;
}
@Override
public boolean rename(Path src, Path dst) throws IOException {
return true;
}
@Override
public boolean delete(Path f) throws IOException {
return true;
}
@Override
public boolean delete(Path f, boolean recursive) throws IOException {
return true;
}
@Override
public FileStatus[] listStatus(Path f) throws FileNotFoundException,
IOException {
return null;
}
@Override
public void setWorkingDirectory(Path new_dir) { }
@Override
public Path getWorkingDirectory() {
return null;
}
@Override
public boolean mkdirs(Path f, FsPermission permission) throws IOException {
return true;
}
@Override
public URI getUri() {
return null;
}
}
{code}
However, it won't compile - fsHDFSUtils.recoverDFSFileLease() expects a
DistributedFileSystem parameter.
I then changed DistributedFileSystem parameter to FileSystem type - FSHDFSUtils
won't compile because we call recoverLease() directly and there is no class in
between FileSystem and DistributedFileSystem which has recoverLease() method:
{code}
recovered = dfs.recoverLease(p);
{code}
Looks like using reflection on dfs to discover recoverLease() would be an
option but I don't know if it is acceptable.
> Initiate lease recovery for outstanding WAL files at the very beginning of
> recovery
> -----------------------------------------------------------------------------------
>
> Key: HBASE-10000
> URL: https://issues.apache.org/jira/browse/HBASE-10000
> Project: HBase
> Issue Type: Improvement
> Reporter: Ted Yu
> Assignee: Ted Yu
> Fix For: 0.98.1
>
> Attachments: 10000-0.96-v5.txt, 10000-0.96-v6.txt,
> 10000-recover-ts-with-pb-2.txt, 10000-recover-ts-with-pb-3.txt,
> 10000-recover-ts-with-pb-4.txt, 10000-recover-ts-with-pb-5.txt,
> 10000-recover-ts-with-pb-6.txt, 10000-v4.txt, 10000-v5.txt, 10000-v6.txt
>
>
> At the beginning of recovery, master can send lease recovery requests
> concurrently for outstanding WAL files using a thread pool.
> Each split worker would first check whether the WAL file it processes is
> closed.
> Thanks to Nicolas Liochon and Jeffery discussion with whom gave rise to this
> idea.
--
This message was sent by Atlassian JIRA
(v6.1.4#6159)
