[
https://issues.apache.org/jira/browse/HBASE-19275?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16552249#comment-16552249
]
Xu Cang commented on HBASE-19275:
---------------------------------
[~apurtell] I put some time today to debug this, below is my findings.
#createAndTestSnapshot tries to verify file existence before and after
"refreshCache", but this won't work because #getUnreferencedFiles is called,
inside this method, it calls #refreshCache here:
[https://github.com/apache/hbase/blob/c3b4f788b16ac4e0e8cfd319f495308ba4d158f5/hbase-server/src/main/java/org/apache/hadoop/hbase/master/snapshot/SnapshotFileCache.java#L190]
Also, #refreshCache will be called periodically from RefreshCacheTask.
So, I think it's good and safe to remove this check as shown below:
{quote}fs.delete(builder.getSnapshotsDir(), true);
FSUtils.logFileSystemState(fs, rootDir, LOG);
- // The files should be in cache until next refresh
- for (Path filePath: files) {
- Iterable<FileStatus> nonSnapshotFiles = getNonSnapshotFiles(cache, filePath);
- assertFalse("Cache didn't find " + filePath.getName(),
Iterables.contains(nonSnapshotFiles,
- filePath.getName()));
- }
{quote}
I am uploading a new patch for review. please let me know if I understood
something wrong.
> TestSnapshotFileCache never worked properly
> -------------------------------------------
>
> Key: HBASE-19275
> URL: https://issues.apache.org/jira/browse/HBASE-19275
> Project: HBase
> Issue Type: Sub-task
> Affects Versions: 3.0.0, 1.4.0, 1.5.0, 2.0.0
> Reporter: Andrew Purtell
> Priority: Major
> Attachments: HBASE-19275-branch-1.patch
>
>
> Error-prone noticed we were asking Iterables.contains() questions with the
> wrong type in TestSnapshotFileCache. I've attached a fixed version of the
> test. The results suggest the cache is not evicting entries properly.
> {noformat}
> java.lang.AssertionError: Cache found
> 'hdfs://localhost:52867/user/apurtell/test-data/8ce04c85-ce4b-4844-b454-5303482ade95/data/default/snapshot1/9e49edd0ab41657fb0c6ebb4d9dfad15/cf/f132e5b06f66443f8003363ed1535aac',
> but it shouldn't have.
> at org.junit.Assert.fail(Assert.java:88)
> at org.junit.Assert.assertTrue(Assert.java:41)
> at org.junit.Assert.assertFalse(Assert.java:64)
> at
> org.apache.hadoop.hbase.master.snapshot.TestSnapshotFileCache.createAndTestSnapshot(TestSnapshotFileCache.java:260)
> at
> org.apache.hadoop.hbase.master.snapshot.TestSnapshotFileCache.createAndTestSnapshotV1(TestSnapshotFileCache.java:206)
> at
> org.apache.hadoop.hbase.master.snapshot.TestSnapshotFileCache.testReloadModifiedDirectory(TestSnapshotFileCache.java:102)
> {noformat}
> {noformat}
> java.lang.AssertionError: Cache found
> 'hdfs://localhost:52867/user/apurtell/test-data/8ce04c85-ce4b-4844-b454-5303482ade95/data/default/snapshot1a/2e81adb9212c98cff970eafa006fc40b/cf/a2ec478d850e4e348359699c53b732c4',
> but it shouldn't have.
> at org.junit.Assert.fail(Assert.java:88)
> at org.junit.Assert.assertTrue(Assert.java:41)
> at org.junit.Assert.assertFalse(Assert.java:64)
> at
> org.apache.hadoop.hbase.master.snapshot.TestSnapshotFileCache.createAndTestSnapshot(TestSnapshotFileCache.java:260)
> at
> org.apache.hadoop.hbase.master.snapshot.TestSnapshotFileCache.createAndTestSnapshotV1(TestSnapshotFileCache.java:206)
> at
> org.apache.hadoop.hbase.master.snapshot.TestSnapshotFileCache.testLoadAndDelete(TestSnapshotFileCache.java:88)
> {noformat}
> These changes are part of HBASE-19239
> I've disabled the offending test cases with @Ignore in that patch, but they
> should be reenabled and fixed.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
