[
https://issues.apache.org/jira/browse/HBASE-23976?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17066235#comment-17066235
]
Wei-Chiu Chuang commented on HBASE-23976:
-----------------------------------------
I am on the master branch, reproducible on both hadoop 2 / hadoop 3 profile,
JDK 1.8.0_221, Mac.
The checksum mismatch does not fail the test after all. It was the expected
behavior. However, an assertion to verify the bucket cache persistence file can
be deleted, doesn't pass:
{noformat}
2020-03-24 14:28:35,546 ERROR [Time-limited test] bucket.BucketCache(312):
Can't restore from
file[/Users/weichiu/sandbox/hbase/hbase-server/target/test-data/7eb1e4ba-52c2-f94d-3c7b-4fdff57c8ad4/bucket.persistence]
because of
java.io.IOException: Mismatch of checksum! The persistent checksum is
N~*b��&��cȗJF, but the calculate checksum is n�.�ʼ�H�� �:��
at
org.apache.hadoop.hbase.io.hfile.bucket.PersistentIOEngine.verifyFileIntegrity(PersistentIOEngine.java:55)
at
org.apache.hadoop.hbase.io.hfile.bucket.BucketCache.parsePB(BucketCache.java:1158)
at
org.apache.hadoop.hbase.io.hfile.bucket.BucketCache.retrieveFromFile(BucketCache.java:1106)
at
org.apache.hadoop.hbase.io.hfile.bucket.BucketCache.<init>(BucketCache.java:310)
at
org.apache.hadoop.hbase.io.hfile.bucket.BucketCache.<init>(BucketCache.java:258)
at
org.apache.hadoop.hbase.io.hfile.bucket.TestVerifyBucketCacheFile.testRetrieveFromFile(TestVerifyBucketCacheFile.java:116)
...
2020-03-24 14:28:35,552 INFO [Time-limited test] bucket.BucketCache(329):
Started bucket cache;
ioengine=file:/Users/weichiu/sandbox/hbase/hbase-server/target/test-data/7eb1e4ba-52c2-f94d-3c7b-4fdff57c8ad4/bucket.cache,
capacity=32 MB, blockSize=8 KB, writerThreadNum=3, writerQLen=64,
persistencePath=/Users/weichiu/sandbox/hbase/hbase-server/target/test-data/7eb1e4ba-52c2-f94d-3c7b-4fdff57c8ad4/bucket.persistence,
bucketAllocator=org.apache.hadoop.hbase.io.hfile.bucket.BucketAllocator
2020-03-24 14:28:35,655 INFO [Time-limited test] bucket.BucketCache(1210):
Shutdown bucket cache: IO persistent=true; path to
write=/Users/weichiu/sandbox/hbase/hbase-server/target/test-data/7eb1e4ba-52c2-f94d-3c7b-4fdff57c8ad4/bucket.persistence
2020-03-24 14:28:35,655 INFO [Time-limited test-BucketCacheWriter-2]
bucket.BucketCache$WriterThread(914): Time-limited test-BucketCacheWriter-2
exiting, cacheEnabled=false
2020-03-24 14:28:35,655 INFO [Time-limited test-BucketCacheWriter-0]
bucket.BucketCache$WriterThread(914): Time-limited test-BucketCacheWriter-0
exiting, cacheEnabled=false
2020-03-24 14:28:35,655 INFO [Time-limited test-BucketCacheWriter-1]
bucket.BucketCache$WriterThread(914): Time-limited test-BucketCacheWriter-1
exiting, cacheEnabled=false
java.lang.AssertionError
at org.junit.Assert.fail(Assert.java:87)
at org.junit.Assert.assertTrue(Assert.java:42)
at org.junit.Assert.assertTrue(Assert.java:53)
at
org.apache.hadoop.hbase.io.hfile.bucket.TestVerifyBucketCacheFile.testRetrieveFromFile(TestVerifyBucketCacheFile.java:132)
...
{noformat}
Mysteriously, the persistence file is gone right after it's written. JDK bug?
{code}
private void persistToFile() throws IOException {
assert !cacheEnabled;
if (!ioEngine.isPersistent()) {
throw new IOException("Attempt to persist non-persistent cache
mappings!");
}
try (FileOutputStream fos = new FileOutputStream(persistencePath, false)) {
fos.write(ProtobufMagic.PB_MAGIC);
BucketProtoUtils.toPB(this).writeDelimitedTo(fos);
} --> persistence file is gone from my local dir at this point.
}
{code}
> [flakey test] TestVerifyBucketCacheFile
> ---------------------------------------
>
> Key: HBASE-23976
> URL: https://issues.apache.org/jira/browse/HBASE-23976
> Project: HBase
> Issue Type: Test
> Components: regionserver, test
> Affects Versions: 3.0.0
> Reporter: Nick Dimiduk
> Priority: Major
>
> I see sporadic failures in this test class. Sometimes a failure on
> {{assertTrue(file.delete())}}, an inconsistent annoyance. However, this one
> looks more sinister.
> {noformat}
> 2020-03-12 12:11:35,059 ERROR [Time-limited test] bucket.BucketCache(312):
> Can't restore from
> file[/Users/ndimiduk/repos/apache/hbase/hbase-server/target/test-data/5e5c5f5f-d5c2-94b2-8ce9-cf561f4f19f7/bucket.persistence]
> because of
> java.io.IOException: Mismatch of checksum! The persistent checksum is
> ���&#20;B&#16;k�&#3;��2�Ӏk, but the calculate checksum is
> �o���r��w��c��4
> at
> org.apache.hadoop.hbase.io.hfile.bucket.PersistentIOEngine.verifyFileIntegrity(PersistentIOEngine.java:55)
> at
> org.apache.hadoop.hbase.io.hfile.bucket.BucketCache.parsePB(BucketCache.java:1158)
> at
> org.apache.hadoop.hbase.io.hfile.bucket.BucketCache.retrieveFromFile(BucketCache.java:1106)
> at
> org.apache.hadoop.hbase.io.hfile.bucket.BucketCache.<init>(BucketCache.java:310)
> at
> org.apache.hadoop.hbase.io.hfile.bucket.BucketCache.<init>(BucketCache.java:258)
> at
> org.apache.hadoop.hbase.io.hfile.bucket.TestVerifyBucketCacheFile.testRetrieveFromFile(TestVerifyBucketCacheFile.java:116)
> {noformat}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
