[jira] [Commented] (HBASE-13301) Possible memory leak in BucketCache
[
https://issues.apache.org/jira/browse/HBASE-13301?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14494236#comment-14494236
]
Hudson commented on HBASE-13301:
SUCCESS: Integrated in HBase-0.98-on-Hadoop-1.1 #901 (See
[https://builds.apache.org/job/HBase-0.98-on-Hadoop-1.1/901/])
HBASE-13301 Possible memory leak in BucketCache (zhangduo: rev
0169cbfde8967044ef1248e8fa91dcd8abc83f14)
*
hbase-server/src/main/java/org/apache/hadoop/hbase/io/hfile/bucket/CachedEntryQueue.java
*
hbase-server/src/main/java/org/apache/hadoop/hbase/io/hfile/bucket/BucketCache.java
*
hbase-server/src/test/java/org/apache/hadoop/hbase/io/hfile/bucket/TestBucketCache.java
*
hbase-server/src/test/java/org/apache/hadoop/hbase/io/hfile/CacheTestUtils.java
* hbase-server/src/main/java/org/apache/hadoop/hbase/util/IdLock.java
> Possible memory leak in BucketCache
> ---
>
> Key: HBASE-13301
> URL: https://issues.apache.org/jira/browse/HBASE-13301
> Project: HBase
> Issue Type: Bug
> Components: BlockCache
>Reporter: zhangduo
>Assignee: zhangduo
> Fix For: 2.0.0, 1.1.0, 0.98.13, 1.0.2
>
> Attachments: HBASE-13301-0.98.patch, HBASE-13301-0.98_v1.patch,
> HBASE-13301-branch-1.0.patch, HBASE-13301-branch-1.0.patch,
> HBASE-13301-branch-1.0_v1.patch, HBASE-13301-branch-1.patch,
> HBASE-13301-branch-1_v1.patch, HBASE-13301-testcase.patch,
> HBASE-13301-testcase_v1.patch, HBASE-13301.patch, HBASE-13301_v1.patch,
> HBASE-13301_v2.patch, HBASE-13301_v3.patch
>
>
> {code:title=BucketCache.java}
> public boolean evictBlock(BlockCacheKey cacheKey) {
> ...
> if (bucketEntry.equals(backingMap.remove(cacheKey))) {
> bucketAllocator.freeBlock(bucketEntry.offset());
> realCacheSize.addAndGet(-1 * bucketEntry.getLength());
> blocksByHFile.remove(cacheKey.getHfileName(), cacheKey);
> if (removedBlock == null) {
> this.blockNumber.decrementAndGet();
> }
> } else {
> return false;
> }
> ...
> {code}
> I think the problem is here. We remove a BucketEntry that should not be
> removed by us, but we do not put it back and also do not do any clean up.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-13301) Possible memory leak in BucketCache
[
https://issues.apache.org/jira/browse/HBASE-13301?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14494152#comment-14494152
]
Hudson commented on HBASE-13301:
FAILURE: Integrated in HBase-0.98 #948 (See
[https://builds.apache.org/job/HBase-0.98/948/])
HBASE-13301 Possible memory leak in BucketCache (zhangduo: rev
0169cbfde8967044ef1248e8fa91dcd8abc83f14)
*
hbase-server/src/test/java/org/apache/hadoop/hbase/io/hfile/bucket/TestBucketCache.java
*
hbase-server/src/test/java/org/apache/hadoop/hbase/io/hfile/CacheTestUtils.java
*
hbase-server/src/main/java/org/apache/hadoop/hbase/io/hfile/bucket/BucketCache.java
* hbase-server/src/main/java/org/apache/hadoop/hbase/util/IdLock.java
*
hbase-server/src/main/java/org/apache/hadoop/hbase/io/hfile/bucket/CachedEntryQueue.java
> Possible memory leak in BucketCache
> ---
>
> Key: HBASE-13301
> URL: https://issues.apache.org/jira/browse/HBASE-13301
> Project: HBase
> Issue Type: Bug
> Components: BlockCache
>Reporter: zhangduo
>Assignee: zhangduo
> Fix For: 2.0.0, 1.1.0, 0.98.13, 1.0.2
>
> Attachments: HBASE-13301-0.98.patch, HBASE-13301-0.98_v1.patch,
> HBASE-13301-branch-1.0.patch, HBASE-13301-branch-1.0.patch,
> HBASE-13301-branch-1.0_v1.patch, HBASE-13301-branch-1.patch,
> HBASE-13301-branch-1_v1.patch, HBASE-13301-testcase.patch,
> HBASE-13301-testcase_v1.patch, HBASE-13301.patch, HBASE-13301_v1.patch,
> HBASE-13301_v2.patch, HBASE-13301_v3.patch
>
>
> {code:title=BucketCache.java}
> public boolean evictBlock(BlockCacheKey cacheKey) {
> ...
> if (bucketEntry.equals(backingMap.remove(cacheKey))) {
> bucketAllocator.freeBlock(bucketEntry.offset());
> realCacheSize.addAndGet(-1 * bucketEntry.getLength());
> blocksByHFile.remove(cacheKey.getHfileName(), cacheKey);
> if (removedBlock == null) {
> this.blockNumber.decrementAndGet();
> }
> } else {
> return false;
> }
> ...
> {code}
> I think the problem is here. We remove a BucketEntry that should not be
> removed by us, but we do not put it back and also do not do any clean up.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-13301) Possible memory leak in BucketCache
[
https://issues.apache.org/jira/browse/HBASE-13301?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14494043#comment-14494043
]
Hudson commented on HBASE-13301:
SUCCESS: Integrated in HBase-TRUNK #6375 (See
[https://builds.apache.org/job/HBase-TRUNK/6375/])
HBASE-13301 Possible memory leak in BucketCache (zhangduo: rev
4f151444b58ae85b93f76254961358932e0ffb9b)
*
hbase-server/src/main/java/org/apache/hadoop/hbase/io/hfile/bucket/BucketCache.java
* hbase-server/src/main/java/org/apache/hadoop/hbase/util/IdLock.java
*
hbase-server/src/test/java/org/apache/hadoop/hbase/io/hfile/bucket/TestBucketCache.java
*
hbase-server/src/test/java/org/apache/hadoop/hbase/io/hfile/CacheTestUtils.java
*
hbase-server/src/main/java/org/apache/hadoop/hbase/io/hfile/bucket/CachedEntryQueue.java
> Possible memory leak in BucketCache
> ---
>
> Key: HBASE-13301
> URL: https://issues.apache.org/jira/browse/HBASE-13301
> Project: HBase
> Issue Type: Bug
> Components: BlockCache
>Reporter: zhangduo
>Assignee: zhangduo
> Fix For: 2.0.0, 1.1.0, 0.98.13, 1.0.2
>
> Attachments: HBASE-13301-0.98.patch, HBASE-13301-0.98_v1.patch,
> HBASE-13301-branch-1.0.patch, HBASE-13301-branch-1.0.patch,
> HBASE-13301-branch-1.0_v1.patch, HBASE-13301-branch-1.patch,
> HBASE-13301-branch-1_v1.patch, HBASE-13301-testcase.patch,
> HBASE-13301-testcase_v1.patch, HBASE-13301.patch, HBASE-13301_v1.patch,
> HBASE-13301_v2.patch, HBASE-13301_v3.patch
>
>
> {code:title=BucketCache.java}
> public boolean evictBlock(BlockCacheKey cacheKey) {
> ...
> if (bucketEntry.equals(backingMap.remove(cacheKey))) {
> bucketAllocator.freeBlock(bucketEntry.offset());
> realCacheSize.addAndGet(-1 * bucketEntry.getLength());
> blocksByHFile.remove(cacheKey.getHfileName(), cacheKey);
> if (removedBlock == null) {
> this.blockNumber.decrementAndGet();
> }
> } else {
> return false;
> }
> ...
> {code}
> I think the problem is here. We remove a BucketEntry that should not be
> removed by us, but we do not put it back and also do not do any clean up.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-13301) Possible memory leak in BucketCache
[
https://issues.apache.org/jira/browse/HBASE-13301?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14494024#comment-14494024
]
Hudson commented on HBASE-13301:
SUCCESS: Integrated in HBase-1.1 #392 (See
[https://builds.apache.org/job/HBase-1.1/392/])
HBASE-13301 Possible memory leak in BucketCache (zhangduo: rev
bcd5c4d1371e32f8126349d671c7b82582311a84)
*
hbase-server/src/test/java/org/apache/hadoop/hbase/io/hfile/bucket/TestBucketCache.java
*
hbase-server/src/main/java/org/apache/hadoop/hbase/io/hfile/bucket/BucketCache.java
* hbase-server/src/main/java/org/apache/hadoop/hbase/util/IdLock.java
*
hbase-server/src/main/java/org/apache/hadoop/hbase/io/hfile/bucket/CachedEntryQueue.java
*
hbase-server/src/test/java/org/apache/hadoop/hbase/io/hfile/CacheTestUtils.java
> Possible memory leak in BucketCache
> ---
>
> Key: HBASE-13301
> URL: https://issues.apache.org/jira/browse/HBASE-13301
> Project: HBase
> Issue Type: Bug
> Components: BlockCache
>Reporter: zhangduo
>Assignee: zhangduo
> Fix For: 2.0.0, 1.1.0, 0.98.13, 1.0.2
>
> Attachments: HBASE-13301-0.98.patch, HBASE-13301-0.98_v1.patch,
> HBASE-13301-branch-1.0.patch, HBASE-13301-branch-1.0.patch,
> HBASE-13301-branch-1.0_v1.patch, HBASE-13301-branch-1.patch,
> HBASE-13301-branch-1_v1.patch, HBASE-13301-testcase.patch,
> HBASE-13301-testcase_v1.patch, HBASE-13301.patch, HBASE-13301_v1.patch,
> HBASE-13301_v2.patch, HBASE-13301_v3.patch
>
>
> {code:title=BucketCache.java}
> public boolean evictBlock(BlockCacheKey cacheKey) {
> ...
> if (bucketEntry.equals(backingMap.remove(cacheKey))) {
> bucketAllocator.freeBlock(bucketEntry.offset());
> realCacheSize.addAndGet(-1 * bucketEntry.getLength());
> blocksByHFile.remove(cacheKey.getHfileName(), cacheKey);
> if (removedBlock == null) {
> this.blockNumber.decrementAndGet();
> }
> } else {
> return false;
> }
> ...
> {code}
> I think the problem is here. We remove a BucketEntry that should not be
> removed by us, but we do not put it back and also do not do any clean up.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-13301) Possible memory leak in BucketCache
[
https://issues.apache.org/jira/browse/HBASE-13301?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14494015#comment-14494015
]
Hudson commented on HBASE-13301:
FAILURE: Integrated in HBase-1.0 #862 (See
[https://builds.apache.org/job/HBase-1.0/862/])
HBASE-13301 Possible memory leak in BucketCache (zhangduo: rev
b4f6ee9adeaf1d54c99615bf47d9c259eea819ac)
*
hbase-server/src/test/java/org/apache/hadoop/hbase/io/hfile/CacheTestUtils.java
* hbase-server/src/main/java/org/apache/hadoop/hbase/util/IdLock.java
*
hbase-server/src/main/java/org/apache/hadoop/hbase/io/hfile/bucket/BucketCache.java
*
hbase-server/src/test/java/org/apache/hadoop/hbase/io/hfile/bucket/TestBucketCache.java
*
hbase-server/src/main/java/org/apache/hadoop/hbase/io/hfile/bucket/CachedEntryQueue.java
> Possible memory leak in BucketCache
> ---
>
> Key: HBASE-13301
> URL: https://issues.apache.org/jira/browse/HBASE-13301
> Project: HBase
> Issue Type: Bug
> Components: BlockCache
>Reporter: zhangduo
>Assignee: zhangduo
> Fix For: 2.0.0, 1.1.0, 0.98.13, 1.0.2
>
> Attachments: HBASE-13301-0.98.patch, HBASE-13301-0.98_v1.patch,
> HBASE-13301-branch-1.0.patch, HBASE-13301-branch-1.0.patch,
> HBASE-13301-branch-1.0_v1.patch, HBASE-13301-branch-1.patch,
> HBASE-13301-branch-1_v1.patch, HBASE-13301-testcase.patch,
> HBASE-13301-testcase_v1.patch, HBASE-13301.patch, HBASE-13301_v1.patch,
> HBASE-13301_v2.patch, HBASE-13301_v3.patch
>
>
> {code:title=BucketCache.java}
> public boolean evictBlock(BlockCacheKey cacheKey) {
> ...
> if (bucketEntry.equals(backingMap.remove(cacheKey))) {
> bucketAllocator.freeBlock(bucketEntry.offset());
> realCacheSize.addAndGet(-1 * bucketEntry.getLength());
> blocksByHFile.remove(cacheKey.getHfileName(), cacheKey);
> if (removedBlock == null) {
> this.blockNumber.decrementAndGet();
> }
> } else {
> return false;
> }
> ...
> {code}
> I think the problem is here. We remove a BucketEntry that should not be
> removed by us, but we do not put it back and also do not do any clean up.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-13301) Possible memory leak in BucketCache
[
https://issues.apache.org/jira/browse/HBASE-13301?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14493851#comment-14493851
]
zhangduo commented on HBASE-13301:
--
Integrated to all branches.
Thanks all you guys who help me finish this.
> Possible memory leak in BucketCache
> ---
>
> Key: HBASE-13301
> URL: https://issues.apache.org/jira/browse/HBASE-13301
> Project: HBase
> Issue Type: Bug
> Components: BlockCache
>Reporter: zhangduo
>Assignee: zhangduo
> Fix For: 2.0.0, 1.1.0, 0.98.13, 1.0.2
>
> Attachments: HBASE-13301-0.98.patch, HBASE-13301-0.98_v1.patch,
> HBASE-13301-branch-1.0.patch, HBASE-13301-branch-1.0.patch,
> HBASE-13301-branch-1.0_v1.patch, HBASE-13301-branch-1.patch,
> HBASE-13301-branch-1_v1.patch, HBASE-13301-testcase.patch,
> HBASE-13301-testcase_v1.patch, HBASE-13301.patch, HBASE-13301_v1.patch,
> HBASE-13301_v2.patch, HBASE-13301_v3.patch
>
>
> {code:title=BucketCache.java}
> public boolean evictBlock(BlockCacheKey cacheKey) {
> ...
> if (bucketEntry.equals(backingMap.remove(cacheKey))) {
> bucketAllocator.freeBlock(bucketEntry.offset());
> realCacheSize.addAndGet(-1 * bucketEntry.getLength());
> blocksByHFile.remove(cacheKey.getHfileName(), cacheKey);
> if (removedBlock == null) {
> this.blockNumber.decrementAndGet();
> }
> } else {
> return false;
> }
> ...
> {code}
> I think the problem is here. We remove a BucketEntry that should not be
> removed by us, but we do not put it back and also do not do any clean up.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-13301) Possible memory leak in BucketCache
[
https://issues.apache.org/jira/browse/HBASE-13301?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14493843#comment-14493843
]
zhangduo commented on HBASE-13301:
--
Seems the second time is fine. Let me commit.
> Possible memory leak in BucketCache
> ---
>
> Key: HBASE-13301
> URL: https://issues.apache.org/jira/browse/HBASE-13301
> Project: HBase
> Issue Type: Bug
> Components: BlockCache
>Reporter: zhangduo
>Assignee: zhangduo
> Fix For: 2.0.0, 1.1.0, 0.98.13, 1.0.2
>
> Attachments: HBASE-13301-0.98.patch, HBASE-13301-0.98_v1.patch,
> HBASE-13301-branch-1.0.patch, HBASE-13301-branch-1.0.patch,
> HBASE-13301-branch-1.0_v1.patch, HBASE-13301-branch-1.patch,
> HBASE-13301-branch-1_v1.patch, HBASE-13301-testcase.patch,
> HBASE-13301-testcase_v1.patch, HBASE-13301.patch, HBASE-13301_v1.patch,
> HBASE-13301_v2.patch, HBASE-13301_v3.patch
>
>
> {code:title=BucketCache.java}
> public boolean evictBlock(BlockCacheKey cacheKey) {
> ...
> if (bucketEntry.equals(backingMap.remove(cacheKey))) {
> bucketAllocator.freeBlock(bucketEntry.offset());
> realCacheSize.addAndGet(-1 * bucketEntry.getLength());
> blocksByHFile.remove(cacheKey.getHfileName(), cacheKey);
> if (removedBlock == null) {
> this.blockNumber.decrementAndGet();
> }
> } else {
> return false;
> }
> ...
> {code}
> I think the problem is here. We remove a BucketEntry that should not be
> removed by us, but we do not put it back and also do not do any clean up.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-13301) Possible memory leak in BucketCache
[
https://issues.apache.org/jira/browse/HBASE-13301?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14493675#comment-14493675
]
Hadoop QA commented on HBASE-13301:
---
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12725141/HBASE-13301-0.98_v1.patch
against 0.98 branch at commit e9da064ccd8dd3c64d99e0863e36cdc1236779f1.
ATTACHMENT ID: 12725141
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 8 new
or modified tests.
{color:green}+1 hadoop versions{color}. The patch compiles with all
supported hadoop versions (2.4.1 2.5.2 2.6.0)
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 protoc{color}. The applied patch does not increase the
total number of protoc compiler warnings.
{color:red}-1 javadoc{color}. The javadoc tool appears to have generated
26 warning messages.
{color:green}+1 checkstyle{color}. The applied patch does not increase the
total number of checkstyle errors
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 2.0.3) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 lineLengths{color}. The patch does not introduce lines
longer than 100
{color:green}+1 site{color}. The mvn site goal succeeds with this patch.
{color:green}+1 core tests{color}. The patch passed unit tests in .
Test results:
https://builds.apache.org/job/PreCommit-HBASE-Build/13690//testReport/
Release Findbugs (version 2.0.3)warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/13690//artifact/patchprocess/newFindbugsWarnings.html
Checkstyle Errors:
https://builds.apache.org/job/PreCommit-HBASE-Build/13690//artifact/patchprocess/checkstyle-aggregate.html
Javadoc warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/13690//artifact/patchprocess/patchJavadocWarnings.txt
Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/13690//console
This message is automatically generated.
> Possible memory leak in BucketCache
> ---
>
> Key: HBASE-13301
> URL: https://issues.apache.org/jira/browse/HBASE-13301
> Project: HBase
> Issue Type: Bug
> Components: BlockCache
>Reporter: zhangduo
>Assignee: zhangduo
> Fix For: 2.0.0, 1.1.0, 0.98.13, 1.0.2
>
> Attachments: HBASE-13301-0.98.patch, HBASE-13301-0.98_v1.patch,
> HBASE-13301-branch-1.0.patch, HBASE-13301-branch-1.0.patch,
> HBASE-13301-branch-1.0_v1.patch, HBASE-13301-branch-1.patch,
> HBASE-13301-branch-1_v1.patch, HBASE-13301-testcase.patch,
> HBASE-13301-testcase_v1.patch, HBASE-13301.patch, HBASE-13301_v1.patch,
> HBASE-13301_v2.patch, HBASE-13301_v3.patch
>
>
> {code:title=BucketCache.java}
> public boolean evictBlock(BlockCacheKey cacheKey) {
> ...
> if (bucketEntry.equals(backingMap.remove(cacheKey))) {
> bucketAllocator.freeBlock(bucketEntry.offset());
> realCacheSize.addAndGet(-1 * bucketEntry.getLength());
> blocksByHFile.remove(cacheKey.getHfileName(), cacheKey);
> if (removedBlock == null) {
> this.blockNumber.decrementAndGet();
> }
> } else {
> return false;
> }
> ...
> {code}
> I think the problem is here. We remove a BucketEntry that should not be
> removed by us, but we do not put it back and also do not do any clean up.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-13301) Possible memory leak in BucketCache
[
https://issues.apache.org/jira/browse/HBASE-13301?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14493441#comment-14493441
]
Hadoop QA commented on HBASE-13301:
---
{color:green}+1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12725094/HBASE-13301-branch-1.0_v1.patch
against branch-1.0 branch at commit e9da064ccd8dd3c64d99e0863e36cdc1236779f1.
ATTACHMENT ID: 12725094
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 8 new
or modified tests.
{color:green}+1 hadoop versions{color}. The patch compiles with all
supported hadoop versions (2.4.1 2.5.2 2.6.0)
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 protoc{color}. The applied patch does not increase the
total number of protoc compiler warnings.
{color:green}+1 javadoc{color}. The javadoc tool did not generate any
warning messages.
{color:green}+1 checkstyle{color}. The applied patch does not increase the
total number of checkstyle errors
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 2.0.3) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 lineLengths{color}. The patch does not introduce lines
longer than 100
{color:green}+1 site{color}. The mvn site goal succeeds with this patch.
{color:green}+1 core tests{color}. The patch passed unit tests in .
Test results:
https://builds.apache.org/job/PreCommit-HBASE-Build/13687//testReport/
Release Findbugs (version 2.0.3)warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/13687//artifact/patchprocess/newFindbugsWarnings.html
Checkstyle Errors:
https://builds.apache.org/job/PreCommit-HBASE-Build/13687//artifact/patchprocess/checkstyle-aggregate.html
Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/13687//console
This message is automatically generated.
> Possible memory leak in BucketCache
> ---
>
> Key: HBASE-13301
> URL: https://issues.apache.org/jira/browse/HBASE-13301
> Project: HBase
> Issue Type: Bug
> Components: BlockCache
>Reporter: zhangduo
>Assignee: zhangduo
> Fix For: 2.0.0, 1.1.0, 0.98.13, 1.0.2
>
> Attachments: HBASE-13301-0.98.patch, HBASE-13301-0.98_v1.patch,
> HBASE-13301-branch-1.0.patch, HBASE-13301-branch-1.0.patch,
> HBASE-13301-branch-1.0_v1.patch, HBASE-13301-branch-1.patch,
> HBASE-13301-branch-1_v1.patch, HBASE-13301-testcase.patch,
> HBASE-13301-testcase_v1.patch, HBASE-13301.patch, HBASE-13301_v1.patch,
> HBASE-13301_v2.patch, HBASE-13301_v3.patch
>
>
> {code:title=BucketCache.java}
> public boolean evictBlock(BlockCacheKey cacheKey) {
> ...
> if (bucketEntry.equals(backingMap.remove(cacheKey))) {
> bucketAllocator.freeBlock(bucketEntry.offset());
> realCacheSize.addAndGet(-1 * bucketEntry.getLength());
> blocksByHFile.remove(cacheKey.getHfileName(), cacheKey);
> if (removedBlock == null) {
> this.blockNumber.decrementAndGet();
> }
> } else {
> return false;
> }
> ...
> {code}
> I think the problem is here. We remove a BucketEntry that should not be
> removed by us, but we do not put it back and also do not do any clean up.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-13301) Possible memory leak in BucketCache
[
https://issues.apache.org/jira/browse/HBASE-13301?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14493440#comment-14493440
]
Hadoop QA commented on HBASE-13301:
---
{color:green}+1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12725092/HBASE-13301-branch-1_v1.patch
against branch-1 branch at commit e9da064ccd8dd3c64d99e0863e36cdc1236779f1.
ATTACHMENT ID: 12725092
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 8 new
or modified tests.
{color:green}+1 hadoop versions{color}. The patch compiles with all
supported hadoop versions (2.4.1 2.5.2 2.6.0)
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 protoc{color}. The applied patch does not increase the
total number of protoc compiler warnings.
{color:green}+1 javadoc{color}. The javadoc tool did not generate any
warning messages.
{color:green}+1 checkstyle{color}. The applied patch does not increase the
total number of checkstyle errors
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 2.0.3) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 lineLengths{color}. The patch does not introduce lines
longer than 100
{color:green}+1 site{color}. The mvn site goal succeeds with this patch.
{color:green}+1 core tests{color}. The patch passed unit tests in .
Test results:
https://builds.apache.org/job/PreCommit-HBASE-Build/13686//testReport/
Release Findbugs (version 2.0.3)warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/13686//artifact/patchprocess/newFindbugsWarnings.html
Checkstyle Errors:
https://builds.apache.org/job/PreCommit-HBASE-Build/13686//artifact/patchprocess/checkstyle-aggregate.html
Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/13686//console
This message is automatically generated.
> Possible memory leak in BucketCache
> ---
>
> Key: HBASE-13301
> URL: https://issues.apache.org/jira/browse/HBASE-13301
> Project: HBase
> Issue Type: Bug
> Components: BlockCache
>Reporter: zhangduo
>Assignee: zhangduo
> Fix For: 2.0.0, 1.1.0, 0.98.13, 1.0.2
>
> Attachments: HBASE-13301-0.98.patch, HBASE-13301-0.98_v1.patch,
> HBASE-13301-branch-1.0.patch, HBASE-13301-branch-1.0.patch,
> HBASE-13301-branch-1.0_v1.patch, HBASE-13301-branch-1.patch,
> HBASE-13301-branch-1_v1.patch, HBASE-13301-testcase.patch,
> HBASE-13301-testcase_v1.patch, HBASE-13301.patch, HBASE-13301_v1.patch,
> HBASE-13301_v2.patch, HBASE-13301_v3.patch
>
>
> {code:title=BucketCache.java}
> public boolean evictBlock(BlockCacheKey cacheKey) {
> ...
> if (bucketEntry.equals(backingMap.remove(cacheKey))) {
> bucketAllocator.freeBlock(bucketEntry.offset());
> realCacheSize.addAndGet(-1 * bucketEntry.getLength());
> blocksByHFile.remove(cacheKey.getHfileName(), cacheKey);
> if (removedBlock == null) {
> this.blockNumber.decrementAndGet();
> }
> } else {
> return false;
> }
> ...
> {code}
> I think the problem is here. We remove a BucketEntry that should not be
> removed by us, but we do not put it back and also do not do any clean up.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-13301) Possible memory leak in BucketCache
[
https://issues.apache.org/jira/browse/HBASE-13301?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14493393#comment-14493393
]
Hadoop QA commented on HBASE-13301:
---
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12725100/HBASE-13301-0.98_v1.patch
against 0.98 branch at commit e9da064ccd8dd3c64d99e0863e36cdc1236779f1.
ATTACHMENT ID: 12725100
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 8 new
or modified tests.
{color:green}+1 hadoop versions{color}. The patch compiles with all
supported hadoop versions (2.4.1 2.5.2 2.6.0)
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 protoc{color}. The applied patch does not increase the
total number of protoc compiler warnings.
{color:red}-1 javadoc{color}. The javadoc tool appears to have generated
26 warning messages.
{color:green}+1 checkstyle{color}. The applied patch does not increase the
total number of checkstyle errors
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 2.0.3) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 lineLengths{color}. The patch does not introduce lines
longer than 100
{color:green}+1 site{color}. The mvn site goal succeeds with this patch.
{color:red}-1 core tests{color}. The patch failed these unit tests:
Test results:
https://builds.apache.org/job/PreCommit-HBASE-Build/13688//testReport/
Release Findbugs (version 2.0.3)warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/13688//artifact/patchprocess/newFindbugsWarnings.html
Checkstyle Errors:
https://builds.apache.org/job/PreCommit-HBASE-Build/13688//artifact/patchprocess/checkstyle-aggregate.html
Javadoc warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/13688//artifact/patchprocess/patchJavadocWarnings.txt
Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/13688//console
This message is automatically generated.
> Possible memory leak in BucketCache
> ---
>
> Key: HBASE-13301
> URL: https://issues.apache.org/jira/browse/HBASE-13301
> Project: HBase
> Issue Type: Bug
> Components: BlockCache
>Reporter: zhangduo
>Assignee: zhangduo
> Fix For: 2.0.0, 1.1.0, 0.98.13, 1.0.2
>
> Attachments: HBASE-13301-0.98.patch, HBASE-13301-0.98_v1.patch,
> HBASE-13301-branch-1.0.patch, HBASE-13301-branch-1.0.patch,
> HBASE-13301-branch-1.0_v1.patch, HBASE-13301-branch-1.patch,
> HBASE-13301-branch-1_v1.patch, HBASE-13301-testcase.patch,
> HBASE-13301-testcase_v1.patch, HBASE-13301.patch, HBASE-13301_v1.patch,
> HBASE-13301_v2.patch, HBASE-13301_v3.patch
>
>
> {code:title=BucketCache.java}
> public boolean evictBlock(BlockCacheKey cacheKey) {
> ...
> if (bucketEntry.equals(backingMap.remove(cacheKey))) {
> bucketAllocator.freeBlock(bucketEntry.offset());
> realCacheSize.addAndGet(-1 * bucketEntry.getLength());
> blocksByHFile.remove(cacheKey.getHfileName(), cacheKey);
> if (removedBlock == null) {
> this.blockNumber.decrementAndGet();
> }
> } else {
> return false;
> }
> ...
> {code}
> I think the problem is here. We remove a BucketEntry that should not be
> removed by us, but we do not put it back and also do not do any clean up.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-13301) Possible memory leak in BucketCache
[
https://issues.apache.org/jira/browse/HBASE-13301?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14493285#comment-14493285
]
Andrew Purtell commented on HBASE-13301:
bq. Let me port the new patch to the branches other than master.
Thanks!
> Possible memory leak in BucketCache
> ---
>
> Key: HBASE-13301
> URL: https://issues.apache.org/jira/browse/HBASE-13301
> Project: HBase
> Issue Type: Bug
> Components: BlockCache
>Reporter: zhangduo
>Assignee: zhangduo
> Fix For: 2.0.0, 1.1.0, 0.98.13, 1.0.2
>
> Attachments: HBASE-13301-0.98.patch, HBASE-13301-branch-1.0.patch,
> HBASE-13301-branch-1.0.patch, HBASE-13301-branch-1.patch,
> HBASE-13301-branch-1_v1.patch, HBASE-13301-testcase.patch,
> HBASE-13301-testcase_v1.patch, HBASE-13301.patch, HBASE-13301_v1.patch,
> HBASE-13301_v2.patch, HBASE-13301_v3.patch
>
>
> {code:title=BucketCache.java}
> public boolean evictBlock(BlockCacheKey cacheKey) {
> ...
> if (bucketEntry.equals(backingMap.remove(cacheKey))) {
> bucketAllocator.freeBlock(bucketEntry.offset());
> realCacheSize.addAndGet(-1 * bucketEntry.getLength());
> blocksByHFile.remove(cacheKey.getHfileName(), cacheKey);
> if (removedBlock == null) {
> this.blockNumber.decrementAndGet();
> }
> } else {
> return false;
> }
> ...
> {code}
> I think the problem is here. We remove a BucketEntry that should not be
> removed by us, but we do not put it back and also do not do any clean up.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-13301) Possible memory leak in BucketCache
[
https://issues.apache.org/jira/browse/HBASE-13301?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14493271#comment-14493271
]
zhangduo commented on HBASE-13301:
--
Let me port the new patch to the branches other than master.
> Possible memory leak in BucketCache
> ---
>
> Key: HBASE-13301
> URL: https://issues.apache.org/jira/browse/HBASE-13301
> Project: HBase
> Issue Type: Bug
> Components: BlockCache
>Reporter: zhangduo
>Assignee: zhangduo
> Fix For: 2.0.0, 1.1.0, 0.98.13, 1.0.2
>
> Attachments: HBASE-13301-0.98.patch, HBASE-13301-branch-1.0.patch,
> HBASE-13301-branch-1.0.patch, HBASE-13301-branch-1.patch,
> HBASE-13301-testcase.patch, HBASE-13301-testcase_v1.patch, HBASE-13301.patch,
> HBASE-13301_v1.patch, HBASE-13301_v2.patch, HBASE-13301_v3.patch
>
>
> {code:title=BucketCache.java}
> public boolean evictBlock(BlockCacheKey cacheKey) {
> ...
> if (bucketEntry.equals(backingMap.remove(cacheKey))) {
> bucketAllocator.freeBlock(bucketEntry.offset());
> realCacheSize.addAndGet(-1 * bucketEntry.getLength());
> blocksByHFile.remove(cacheKey.getHfileName(), cacheKey);
> if (removedBlock == null) {
> this.blockNumber.decrementAndGet();
> }
> } else {
> return false;
> }
> ...
> {code}
> I think the problem is here. We remove a BucketEntry that should not be
> removed by us, but we do not put it back and also do not do any clean up.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-13301) Possible memory leak in BucketCache
[
https://issues.apache.org/jira/browse/HBASE-13301?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14493260#comment-14493260
]
Andrew Purtell commented on HBASE-13301:
Yes, let's commit everywhere. I'll take care of that shortly unless objection
and watch/clean up if there's a problem.
> Possible memory leak in BucketCache
> ---
>
> Key: HBASE-13301
> URL: https://issues.apache.org/jira/browse/HBASE-13301
> Project: HBase
> Issue Type: Bug
> Components: BlockCache
>Reporter: zhangduo
>Assignee: zhangduo
> Fix For: 2.0.0, 1.1.0, 0.98.13, 1.0.2
>
> Attachments: HBASE-13301-0.98.patch, HBASE-13301-branch-1.0.patch,
> HBASE-13301-branch-1.0.patch, HBASE-13301-branch-1.patch,
> HBASE-13301-testcase.patch, HBASE-13301-testcase_v1.patch, HBASE-13301.patch,
> HBASE-13301_v1.patch, HBASE-13301_v2.patch, HBASE-13301_v3.patch
>
>
> {code:title=BucketCache.java}
> public boolean evictBlock(BlockCacheKey cacheKey) {
> ...
> if (bucketEntry.equals(backingMap.remove(cacheKey))) {
> bucketAllocator.freeBlock(bucketEntry.offset());
> realCacheSize.addAndGet(-1 * bucketEntry.getLength());
> blocksByHFile.remove(cacheKey.getHfileName(), cacheKey);
> if (removedBlock == null) {
> this.blockNumber.decrementAndGet();
> }
> } else {
> return false;
> }
> ...
> {code}
> I think the problem is here. We remove a BucketEntry that should not be
> removed by us, but we do not put it back and also do not do any clean up.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-13301) Possible memory leak in BucketCache
[
https://issues.apache.org/jira/browse/HBASE-13301?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14493037#comment-14493037
]
Enis Soztutar commented on HBASE-13301:
---
bq. Since the RC of 0.98.12 and 1.0.1 have been cut, we can push to all
branches then?
Skimmed the patch. Looks good for both, since it is an important bug fix.
> Possible memory leak in BucketCache
> ---
>
> Key: HBASE-13301
> URL: https://issues.apache.org/jira/browse/HBASE-13301
> Project: HBase
> Issue Type: Bug
> Components: BlockCache
>Reporter: zhangduo
>Assignee: zhangduo
> Fix For: 2.0.0, 1.1.0, 0.98.13, 1.0.2
>
> Attachments: HBASE-13301-0.98.patch, HBASE-13301-branch-1.0.patch,
> HBASE-13301-branch-1.0.patch, HBASE-13301-branch-1.patch,
> HBASE-13301-testcase.patch, HBASE-13301-testcase_v1.patch, HBASE-13301.patch,
> HBASE-13301_v1.patch, HBASE-13301_v2.patch, HBASE-13301_v3.patch
>
>
> {code:title=BucketCache.java}
> public boolean evictBlock(BlockCacheKey cacheKey) {
> ...
> if (bucketEntry.equals(backingMap.remove(cacheKey))) {
> bucketAllocator.freeBlock(bucketEntry.offset());
> realCacheSize.addAndGet(-1 * bucketEntry.getLength());
> blocksByHFile.remove(cacheKey.getHfileName(), cacheKey);
> if (removedBlock == null) {
> this.blockNumber.decrementAndGet();
> }
> } else {
> return false;
> }
> ...
> {code}
> I think the problem is here. We remove a BucketEntry that should not be
> removed by us, but we do not put it back and also do not do any clean up.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-13301) Possible memory leak in BucketCache
[
https://issues.apache.org/jira/browse/HBASE-13301?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14492723#comment-14492723
]
Nick Dimiduk commented on HBASE-13301:
--
Alright, sounds good. No other questions. +1.
> Possible memory leak in BucketCache
> ---
>
> Key: HBASE-13301
> URL: https://issues.apache.org/jira/browse/HBASE-13301
> Project: HBase
> Issue Type: Bug
> Components: BlockCache
>Reporter: zhangduo
>Assignee: zhangduo
> Fix For: 2.0.0, 1.1.0, 0.98.13, 1.0.2
>
> Attachments: HBASE-13301-0.98.patch, HBASE-13301-branch-1.0.patch,
> HBASE-13301-branch-1.0.patch, HBASE-13301-branch-1.patch,
> HBASE-13301-testcase.patch, HBASE-13301-testcase_v1.patch, HBASE-13301.patch,
> HBASE-13301_v1.patch, HBASE-13301_v2.patch, HBASE-13301_v3.patch
>
>
> {code:title=BucketCache.java}
> public boolean evictBlock(BlockCacheKey cacheKey) {
> ...
> if (bucketEntry.equals(backingMap.remove(cacheKey))) {
> bucketAllocator.freeBlock(bucketEntry.offset());
> realCacheSize.addAndGet(-1 * bucketEntry.getLength());
> blocksByHFile.remove(cacheKey.getHfileName(), cacheKey);
> if (removedBlock == null) {
> this.blockNumber.decrementAndGet();
> }
> } else {
> return false;
> }
> ...
> {code}
> I think the problem is here. We remove a BucketEntry that should not be
> removed by us, but we do not put it back and also do not do any clean up.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-13301) Possible memory leak in BucketCache
[
https://issues.apache.org/jira/browse/HBASE-13301?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14490819#comment-14490819
]
zhangduo commented on HBASE-13301:
--
Any other questions? [~ndimiduk]
Thanks.
> Possible memory leak in BucketCache
> ---
>
> Key: HBASE-13301
> URL: https://issues.apache.org/jira/browse/HBASE-13301
> Project: HBase
> Issue Type: Bug
> Components: BlockCache
>Reporter: zhangduo
>Assignee: zhangduo
> Fix For: 2.0.0, 1.1.0, 0.98.13, 1.0.2
>
> Attachments: HBASE-13301-0.98.patch, HBASE-13301-branch-1.0.patch,
> HBASE-13301-branch-1.0.patch, HBASE-13301-branch-1.patch,
> HBASE-13301-testcase.patch, HBASE-13301-testcase_v1.patch, HBASE-13301.patch,
> HBASE-13301_v1.patch, HBASE-13301_v2.patch, HBASE-13301_v3.patch
>
>
> {code:title=BucketCache.java}
> public boolean evictBlock(BlockCacheKey cacheKey) {
> ...
> if (bucketEntry.equals(backingMap.remove(cacheKey))) {
> bucketAllocator.freeBlock(bucketEntry.offset());
> realCacheSize.addAndGet(-1 * bucketEntry.getLength());
> blocksByHFile.remove(cacheKey.getHfileName(), cacheKey);
> if (removedBlock == null) {
> this.blockNumber.decrementAndGet();
> }
> } else {
> return false;
> }
> ...
> {code}
> I think the problem is here. We remove a BucketEntry that should not be
> removed by us, but we do not put it back and also do not do any clean up.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-13301) Possible memory leak in BucketCache
[
https://issues.apache.org/jira/browse/HBASE-13301?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14490799#comment-14490799
]
Hadoop QA commented on HBASE-13301:
---
{color:green}+1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12724728/HBASE-13301_v3.patch
against master branch at commit e994b491aca8ab2edeb60a328c690ddbc88f8b51.
ATTACHMENT ID: 12724728
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 7 new
or modified tests.
{color:green}+1 hadoop versions{color}. The patch compiles with all
supported hadoop versions (2.4.1 2.5.2 2.6.0)
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 protoc{color}. The applied patch does not increase the
total number of protoc compiler warnings.
{color:green}+1 javadoc{color}. The javadoc tool did not generate any
warning messages.
{color:green}+1 checkstyle{color}. The applied patch does not increase the
total number of checkstyle errors
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 2.0.3) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 lineLengths{color}. The patch does not introduce lines
longer than 100
{color:green}+1 site{color}. The mvn site goal succeeds with this patch.
{color:green}+1 core tests{color}. The patch passed unit tests in .
Test results:
https://builds.apache.org/job/PreCommit-HBASE-Build/13669//testReport/
Release Findbugs (version 2.0.3)warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/13669//artifact/patchprocess/newFindbugsWarnings.html
Checkstyle Errors:
https://builds.apache.org/job/PreCommit-HBASE-Build/13669//artifact/patchprocess/checkstyle-aggregate.html
Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/13669//console
This message is automatically generated.
> Possible memory leak in BucketCache
> ---
>
> Key: HBASE-13301
> URL: https://issues.apache.org/jira/browse/HBASE-13301
> Project: HBase
> Issue Type: Bug
> Components: BlockCache
>Reporter: zhangduo
>Assignee: zhangduo
> Fix For: 2.0.0, 1.1.0, 0.98.13, 1.0.2
>
> Attachments: HBASE-13301-0.98.patch, HBASE-13301-branch-1.0.patch,
> HBASE-13301-branch-1.0.patch, HBASE-13301-branch-1.patch,
> HBASE-13301-testcase.patch, HBASE-13301-testcase_v1.patch, HBASE-13301.patch,
> HBASE-13301_v1.patch, HBASE-13301_v2.patch, HBASE-13301_v3.patch
>
>
> {code:title=BucketCache.java}
> public boolean evictBlock(BlockCacheKey cacheKey) {
> ...
> if (bucketEntry.equals(backingMap.remove(cacheKey))) {
> bucketAllocator.freeBlock(bucketEntry.offset());
> realCacheSize.addAndGet(-1 * bucketEntry.getLength());
> blocksByHFile.remove(cacheKey.getHfileName(), cacheKey);
> if (removedBlock == null) {
> this.blockNumber.decrementAndGet();
> }
> } else {
> return false;
> }
> ...
> {code}
> I think the problem is here. We remove a BucketEntry that should not be
> removed by us, but we do not put it back and also do not do any clean up.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-13301) Possible memory leak in BucketCache
[
https://issues.apache.org/jira/browse/HBASE-13301?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14490582#comment-14490582
]
Nick Dimiduk commented on HBASE-13301:
--
Simple question: does your new test case fail when run on HEAD? Should be a
rare case as you say, but maybe 1/100 tests? I can't quite apply the test
without some other changes. Maybe you ran the test and saw it fail reliably?
Speaking of reliably, can you use some more dependable control structure in the
test besides sleep? Maybe a countdown latch or similar?
> Possible memory leak in BucketCache
> ---
>
> Key: HBASE-13301
> URL: https://issues.apache.org/jira/browse/HBASE-13301
> Project: HBase
> Issue Type: Bug
> Components: BlockCache
>Reporter: zhangduo
>Assignee: zhangduo
> Fix For: 2.0.0, 1.1.0, 0.98.13, 1.0.2
>
> Attachments: HBASE-13301-0.98.patch, HBASE-13301-branch-1.0.patch,
> HBASE-13301-branch-1.0.patch, HBASE-13301-branch-1.patch,
> HBASE-13301-testcase.patch, HBASE-13301-testcase_v1.patch, HBASE-13301.patch,
> HBASE-13301_v1.patch, HBASE-13301_v2.patch
>
>
> {code:title=BucketCache.java}
> public boolean evictBlock(BlockCacheKey cacheKey) {
> ...
> if (bucketEntry.equals(backingMap.remove(cacheKey))) {
> bucketAllocator.freeBlock(bucketEntry.offset());
> realCacheSize.addAndGet(-1 * bucketEntry.getLength());
> blocksByHFile.remove(cacheKey.getHfileName(), cacheKey);
> if (removedBlock == null) {
> this.blockNumber.decrementAndGet();
> }
> } else {
> return false;
> }
> ...
> {code}
> I think the problem is here. We remove a BucketEntry that should not be
> removed by us, but we do not put it back and also do not do any clean up.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-13301) Possible memory leak in BucketCache
[
https://issues.apache.org/jira/browse/HBASE-13301?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14490505#comment-14490505
]
zhangduo commented on HBASE-13301:
--
Seems pass on branch-1.0.
Since the RC of 0.98.12 and 1.0.1 have been cut, we can push to all branches
then?
[~enis] [~apurtell] Thanks.
> Possible memory leak in BucketCache
> ---
>
> Key: HBASE-13301
> URL: https://issues.apache.org/jira/browse/HBASE-13301
> Project: HBase
> Issue Type: Bug
> Components: BlockCache
>Reporter: zhangduo
>Assignee: zhangduo
> Fix For: 2.0.0, 1.1.0, 0.98.13, 1.0.2
>
> Attachments: HBASE-13301-0.98.patch, HBASE-13301-branch-1.0.patch,
> HBASE-13301-branch-1.0.patch, HBASE-13301-branch-1.patch,
> HBASE-13301-testcase.patch, HBASE-13301-testcase_v1.patch, HBASE-13301.patch,
> HBASE-13301_v1.patch, HBASE-13301_v2.patch
>
>
> {code:title=BucketCache.java}
> public boolean evictBlock(BlockCacheKey cacheKey) {
> ...
> if (bucketEntry.equals(backingMap.remove(cacheKey))) {
> bucketAllocator.freeBlock(bucketEntry.offset());
> realCacheSize.addAndGet(-1 * bucketEntry.getLength());
> blocksByHFile.remove(cacheKey.getHfileName(), cacheKey);
> if (removedBlock == null) {
> this.blockNumber.decrementAndGet();
> }
> } else {
> return false;
> }
> ...
> {code}
> I think the problem is here. We remove a BucketEntry that should not be
> removed by us, but we do not put it back and also do not do any clean up.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-13301) Possible memory leak in BucketCache
[
https://issues.apache.org/jira/browse/HBASE-13301?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14486301#comment-14486301
]
zhangduo commented on HBASE-13301:
--
Will commit later if no objections.
[~ndimiduk] [~enis] [~apurtell] OK to commit to branches other than master?
Thanks.
> Possible memory leak in BucketCache
> ---
>
> Key: HBASE-13301
> URL: https://issues.apache.org/jira/browse/HBASE-13301
> Project: HBase
> Issue Type: Bug
> Components: BlockCache
>Reporter: zhangduo
>Assignee: zhangduo
> Fix For: 2.0.0, 1.1.0, 0.98.13, 1.0.2
>
> Attachments: HBASE-13301-testcase.patch,
> HBASE-13301-testcase_v1.patch, HBASE-13301.patch, HBASE-13301_v1.patch,
> HBASE-13301_v2.patch
>
>
> {code:title=BucketCache.java}
> public boolean evictBlock(BlockCacheKey cacheKey) {
> ...
> if (bucketEntry.equals(backingMap.remove(cacheKey))) {
> bucketAllocator.freeBlock(bucketEntry.offset());
> realCacheSize.addAndGet(-1 * bucketEntry.getLength());
> blocksByHFile.remove(cacheKey.getHfileName(), cacheKey);
> if (removedBlock == null) {
> this.blockNumber.decrementAndGet();
> }
> } else {
> return false;
> }
> ...
> {code}
> I think the problem is here. We remove a BucketEntry that should not be
> removed by us, but we do not put it back and also do not do any clean up.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-13301) Possible memory leak in BucketCache
[
https://issues.apache.org/jira/browse/HBASE-13301?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14484652#comment-14484652
]
Hadoop QA commented on HBASE-13301:
---
{color:green}+1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12723804/HBASE-13301_v2.patch
against master branch at commit b6756b39c2ff5675f96a6e82dc4d68ec437f01c4.
ATTACHMENT ID: 12723804
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 7 new
or modified tests.
{color:green}+1 hadoop versions{color}. The patch compiles with all
supported hadoop versions (2.4.1 2.5.2 2.6.0)
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 protoc{color}. The applied patch does not increase the
total number of protoc compiler warnings.
{color:green}+1 javadoc{color}. The javadoc tool did not generate any
warning messages.
{color:green}+1 checkstyle{color}. The applied patch does not increase the
total number of checkstyle errors
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 2.0.3) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 lineLengths{color}. The patch does not introduce lines
longer than 100
{color:green}+1 site{color}. The mvn site goal succeeds with this patch.
{color:green}+1 core tests{color}. The patch passed unit tests in .
Test results:
https://builds.apache.org/job/PreCommit-HBASE-Build/13607//testReport/
Release Findbugs (version 2.0.3)warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/13607//artifact/patchprocess/newFindbugsWarnings.html
Checkstyle Errors:
https://builds.apache.org/job/PreCommit-HBASE-Build/13607//artifact/patchprocess/checkstyle-aggregate.html
Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/13607//console
This message is automatically generated.
> Possible memory leak in BucketCache
> ---
>
> Key: HBASE-13301
> URL: https://issues.apache.org/jira/browse/HBASE-13301
> Project: HBase
> Issue Type: Bug
> Components: BlockCache
>Reporter: zhangduo
>Assignee: zhangduo
> Fix For: 2.0.0, 1.1.0, 0.98.13, 1.0.2
>
> Attachments: HBASE-13301-testcase.patch,
> HBASE-13301-testcase_v1.patch, HBASE-13301.patch, HBASE-13301_v1.patch,
> HBASE-13301_v2.patch
>
>
> {code:title=BucketCache.java}
> public boolean evictBlock(BlockCacheKey cacheKey) {
> ...
> if (bucketEntry.equals(backingMap.remove(cacheKey))) {
> bucketAllocator.freeBlock(bucketEntry.offset());
> realCacheSize.addAndGet(-1 * bucketEntry.getLength());
> blocksByHFile.remove(cacheKey.getHfileName(), cacheKey);
> if (removedBlock == null) {
> this.blockNumber.decrementAndGet();
> }
> } else {
> return false;
> }
> ...
> {code}
> I think the problem is here. We remove a BucketEntry that should not be
> removed by us, but we do not put it back and also do not do any clean up.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-13301) Possible memory leak in BucketCache
[
https://issues.apache.org/jira/browse/HBASE-13301?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14484362#comment-14484362
]
zhangduo commented on HBASE-13301:
--
[~apurtell] A little problem about the compareTo and equals methods in
BucketEntry.
Will prepare a new patch soon.
> Possible memory leak in BucketCache
> ---
>
> Key: HBASE-13301
> URL: https://issues.apache.org/jira/browse/HBASE-13301
> Project: HBase
> Issue Type: Bug
> Components: BlockCache
>Reporter: zhangduo
>Assignee: zhangduo
> Fix For: 2.0.0, 1.1.0, 0.98.13, 1.0.2
>
> Attachments: HBASE-13301-testcase.patch,
> HBASE-13301-testcase_v1.patch, HBASE-13301.patch, HBASE-13301_v1.patch
>
>
> {code:title=BucketCache.java}
> public boolean evictBlock(BlockCacheKey cacheKey) {
> ...
> if (bucketEntry.equals(backingMap.remove(cacheKey))) {
> bucketAllocator.freeBlock(bucketEntry.offset());
> realCacheSize.addAndGet(-1 * bucketEntry.getLength());
> blocksByHFile.remove(cacheKey.getHfileName(), cacheKey);
> if (removedBlock == null) {
> this.blockNumber.decrementAndGet();
> }
> } else {
> return false;
> }
> ...
> {code}
> I think the problem is here. We remove a BucketEntry that should not be
> removed by us, but we do not put it back and also do not do any clean up.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-13301) Possible memory leak in BucketCache
[
https://issues.apache.org/jira/browse/HBASE-13301?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14483002#comment-14483002
]
Hadoop QA commented on HBASE-13301:
---
{color:green}+1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12723570/HBASE-13301_v1.patch
against master branch at commit 57f9c821287a3780384bcb37d764f13300c93a8d.
ATTACHMENT ID: 12723570
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 7 new
or modified tests.
{color:green}+1 hadoop versions{color}. The patch compiles with all
supported hadoop versions (2.4.1 2.5.2 2.6.0)
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 protoc{color}. The applied patch does not increase the
total number of protoc compiler warnings.
{color:green}+1 javadoc{color}. The javadoc tool did not generate any
warning messages.
{color:green}+1 checkstyle{color}. The applied patch does not increase the
total number of checkstyle errors
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 2.0.3) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 lineLengths{color}. The patch does not introduce lines
longer than 100
{color:green}+1 site{color}. The mvn site goal succeeds with this patch.
{color:green}+1 core tests{color}. The patch passed unit tests in .
Test results:
https://builds.apache.org/job/PreCommit-HBASE-Build/13595//testReport/
Release Findbugs (version 2.0.3)warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/13595//artifact/patchprocess/newFindbugsWarnings.html
Checkstyle Errors:
https://builds.apache.org/job/PreCommit-HBASE-Build/13595//artifact/patchprocess/checkstyle-aggregate.html
Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/13595//console
This message is automatically generated.
> Possible memory leak in BucketCache
> ---
>
> Key: HBASE-13301
> URL: https://issues.apache.org/jira/browse/HBASE-13301
> Project: HBase
> Issue Type: Bug
> Components: BlockCache
>Reporter: zhangduo
>Assignee: zhangduo
> Attachments: HBASE-13301-testcase.patch,
> HBASE-13301-testcase_v1.patch, HBASE-13301.patch, HBASE-13301_v1.patch
>
>
> {code:title=BucketCache.java}
> public boolean evictBlock(BlockCacheKey cacheKey) {
> ...
> if (bucketEntry.equals(backingMap.remove(cacheKey))) {
> bucketAllocator.freeBlock(bucketEntry.offset());
> realCacheSize.addAndGet(-1 * bucketEntry.getLength());
> blocksByHFile.remove(cacheKey.getHfileName(), cacheKey);
> if (removedBlock == null) {
> this.blockNumber.decrementAndGet();
> }
> } else {
> return false;
> }
> ...
> {code}
> I think the problem is here. We remove a BucketEntry that should not be
> removed by us, but we do not put it back and also do not do any clean up.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-13301) Possible memory leak in BucketCache
[
https://issues.apache.org/jira/browse/HBASE-13301?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14395741#comment-14395741
]
Hadoop QA commented on HBASE-13301:
---
{color:green}+1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12709418/HBASE-13301.patch
against master branch at commit fef8ae9c70f47d87baa985a66e94d7b90b861f08.
ATTACHMENT ID: 12709418
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 7 new
or modified tests.
{color:green}+1 hadoop versions{color}. The patch compiles with all
supported hadoop versions (2.4.1 2.5.2 2.6.0)
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 protoc{color}. The applied patch does not increase the
total number of protoc compiler warnings.
{color:green}+1 javadoc{color}. The javadoc tool did not generate any
warning messages.
{color:green}+1 checkstyle{color}. The applied patch does not increase the
total number of checkstyle errors
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 2.0.3) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 lineLengths{color}. The patch does not introduce lines
longer than 100
{color:green}+1 site{color}. The mvn site goal succeeds with this patch.
{color:green}+1 core tests{color}. The patch passed unit tests in .
Test results:
https://builds.apache.org/job/PreCommit-HBASE-Build/13564//testReport/
Release Findbugs (version 2.0.3)warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/13564//artifact/patchprocess/newFindbugsWarnings.html
Checkstyle Errors:
https://builds.apache.org/job/PreCommit-HBASE-Build/13564//artifact/patchprocess/checkstyle-aggregate.html
Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/13564//console
This message is automatically generated.
> Possible memory leak in BucketCache
> ---
>
> Key: HBASE-13301
> URL: https://issues.apache.org/jira/browse/HBASE-13301
> Project: HBase
> Issue Type: Bug
> Components: BlockCache
>Reporter: zhangduo
>Assignee: zhangduo
> Attachments: HBASE-13301-testcase.patch,
> HBASE-13301-testcase_v1.patch, HBASE-13301.patch
>
>
> {code:title=BucketCache.java}
> public boolean evictBlock(BlockCacheKey cacheKey) {
> ...
> if (bucketEntry.equals(backingMap.remove(cacheKey))) {
> bucketAllocator.freeBlock(bucketEntry.offset());
> realCacheSize.addAndGet(-1 * bucketEntry.getLength());
> blocksByHFile.remove(cacheKey.getHfileName(), cacheKey);
> if (removedBlock == null) {
> this.blockNumber.decrementAndGet();
> }
> } else {
> return false;
> }
> ...
> {code}
> I think the problem is here. We remove a BucketEntry that should not be
> removed by us, but we do not put it back and also do not do any clean up.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-13301) Possible memory leak in BucketCache
[
https://issues.apache.org/jira/browse/HBASE-13301?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14385682#comment-14385682
]
zhangduo commented on HBASE-13301:
--
{quote}
In btw a context switch t1 completed the caching and done evict and again
cached same block.. This seems rarest of rare case.
{quote}
Agree. But HBase is a long running service, small probability events always
occur if we keep it running long enough...
Let me revisit the whole read write path in regionserver which relates to
BlockCache and give a clear locking schema first. Then it is easier to say if
the situation in this testcase could happen.
Will come back later. Thanks.
> Possible memory leak in BucketCache
> ---
>
> Key: HBASE-13301
> URL: https://issues.apache.org/jira/browse/HBASE-13301
> Project: HBase
> Issue Type: Bug
> Components: BlockCache
>Reporter: zhangduo
>Assignee: zhangduo
> Attachments: HBASE-13301-testcase.patch
>
>
> {code:title=BucketCache.java}
> public boolean evictBlock(BlockCacheKey cacheKey) {
> ...
> if (bucketEntry.equals(backingMap.remove(cacheKey))) {
> bucketAllocator.freeBlock(bucketEntry.offset());
> realCacheSize.addAndGet(-1 * bucketEntry.getLength());
> blocksByHFile.remove(cacheKey.getHfileName(), cacheKey);
> if (removedBlock == null) {
> this.blockNumber.decrementAndGet();
> }
> } else {
> return false;
> }
> ...
> {code}
> I think the problem is here. We remove a BucketEntry that should not be
> removed by us, but we do not put it back and also do not do any clean up.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-13301) Possible memory leak in BucketCache
[
https://issues.apache.org/jira/browse/HBASE-13301?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14385138#comment-14385138
]
Anoop Sam John commented on HBASE-13301:
In btw a context switch t1 completed the caching and done evict and again
cached same block.. This seems rarest of rare case. So that is what I mean it
might not happen in a real case. (Sorry withdraw wont happen :-) )
> Possible memory leak in BucketCache
> ---
>
> Key: HBASE-13301
> URL: https://issues.apache.org/jira/browse/HBASE-13301
> Project: HBase
> Issue Type: Bug
> Components: BlockCache
>Reporter: zhangduo
>Assignee: zhangduo
> Attachments: HBASE-13301-testcase.patch
>
>
> {code:title=BucketCache.java}
> public boolean evictBlock(BlockCacheKey cacheKey) {
> ...
> if (bucketEntry.equals(backingMap.remove(cacheKey))) {
> bucketAllocator.freeBlock(bucketEntry.offset());
> realCacheSize.addAndGet(-1 * bucketEntry.getLength());
> blocksByHFile.remove(cacheKey.getHfileName(), cacheKey);
> if (removedBlock == null) {
> this.blockNumber.decrementAndGet();
> }
> } else {
> return false;
> }
> ...
> {code}
> I think the problem is here. We remove a BucketEntry that should not be
> removed by us, but we do not put it back and also do not do any clean up.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-13301) Possible memory leak in BucketCache
[
https://issues.apache.org/jira/browse/HBASE-13301?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14385133#comment-14385133
]
zhangduo commented on HBASE-13301:
--
Thanks [~anoopsamjohn], and could you explain why this won't happen?
And if this won’t happen, then maybe we just need a null check to confirm that
the block has not been evicted by others yet? A 'get and check' still make
people think that we could evict and cache again as what I did in the
testcase...
Thanks.
> Possible memory leak in BucketCache
> ---
>
> Key: HBASE-13301
> URL: https://issues.apache.org/jira/browse/HBASE-13301
> Project: HBase
> Issue Type: Bug
> Components: BlockCache
>Reporter: zhangduo
>Assignee: zhangduo
> Attachments: HBASE-13301-testcase.patch
>
>
> {code:title=BucketCache.java}
> public boolean evictBlock(BlockCacheKey cacheKey) {
> ...
> if (bucketEntry.equals(backingMap.remove(cacheKey))) {
> bucketAllocator.freeBlock(bucketEntry.offset());
> realCacheSize.addAndGet(-1 * bucketEntry.getLength());
> blocksByHFile.remove(cacheKey.getHfileName(), cacheKey);
> if (removedBlock == null) {
> this.blockNumber.decrementAndGet();
> }
> } else {
> return false;
> }
> ...
> {code}
> I think the problem is here. We remove a BucketEntry that should not be
> removed by us, but we do not put it back and also do not do any clean up.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-13301) Possible memory leak in BucketCache
[
https://issues.apache.org/jira/browse/HBASE-13301?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14383753#comment-14383753
]
Anoop Sam John commented on HBASE-13301:
{quote}
t1 cache a block
t2 evict the block but stopped before acquiring offsetLock(this is done by
holding the offsetLock with t1 in this testcase)
t1 evict the block, and cache the block again.
t2 continue evicting the block...
{quote}
Seems this won't happen in real scenarios...
But any was as you said above, it looks a buggy.. Better we can change to get
compare and remove.. The evict op is not in a hot code path. So no issue is
doing and extra op. A get for the same block will have to wait a bit more and
any way it is not going to see the block and have to fetch from DFS.
Considering that time, this extra bit of time is nothing. Good to fix as Ram
suggested.
> Possible memory leak in BucketCache
> ---
>
> Key: HBASE-13301
> URL: https://issues.apache.org/jira/browse/HBASE-13301
> Project: HBase
> Issue Type: Bug
> Components: BlockCache
>Reporter: zhangduo
>Assignee: zhangduo
> Attachments: HBASE-13301-testcase.patch
>
>
> {code:title=BucketCache.java}
> public boolean evictBlock(BlockCacheKey cacheKey) {
> ...
> if (bucketEntry.equals(backingMap.remove(cacheKey))) {
> bucketAllocator.freeBlock(bucketEntry.offset());
> realCacheSize.addAndGet(-1 * bucketEntry.getLength());
> blocksByHFile.remove(cacheKey.getHfileName(), cacheKey);
> if (removedBlock == null) {
> this.blockNumber.decrementAndGet();
> }
> } else {
> return false;
> }
> ...
> {code}
> I think the problem is here. We remove a BucketEntry that should not be
> removed by us, but we do not put it back and also do not do any clean up.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-13301) Possible memory leak in BucketCache
[
https://issues.apache.org/jira/browse/HBASE-13301?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14381237#comment-14381237
]
zhangduo commented on HBASE-13301:
--
[~ram_krish]
Yes, get and compare then remove is perfect. But it is not straight forward in
this case. We do have a IdLock, but it is only used in get and evict, and the
lock key is offset, not BlockCacheKey.
And my wonder is whether the error in this testcase could happen in real world.
Maybe the access pattern we use can avoid this error? I do not know...
Anyway, remove first then compare is not a good idea I think. If we can enter
the 'not equals' branch then no doubt, it is a bug. And if we never enter the
'not equals' branch, then why we need the compare...
Thanks.
> Possible memory leak in BucketCache
> ---
>
> Key: HBASE-13301
> URL: https://issues.apache.org/jira/browse/HBASE-13301
> Project: HBase
> Issue Type: Bug
> Components: BlockCache
>Reporter: zhangduo
>Assignee: zhangduo
> Attachments: HBASE-13301-testcase.patch
>
>
> {code:title=BucketCache.java}
> public boolean evictBlock(BlockCacheKey cacheKey) {
> ...
> if (bucketEntry.equals(backingMap.remove(cacheKey))) {
> bucketAllocator.freeBlock(bucketEntry.offset());
> realCacheSize.addAndGet(-1 * bucketEntry.getLength());
> blocksByHFile.remove(cacheKey.getHfileName(), cacheKey);
> if (removedBlock == null) {
> this.blockNumber.decrementAndGet();
> }
> } else {
> return false;
> }
> ...
> {code}
> I think the problem is here. We remove a BucketEntry that should not be
> removed by us, but we do not put it back and also do not do any clean up.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-13301) Possible memory leak in BucketCache
[
https://issues.apache.org/jira/browse/HBASE-13301?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14380145#comment-14380145
]
ramkrishna.s.vasudevan commented on HBASE-13301:
[~Apache9]
I think you have a valid point here. The soln should be to do a get() from the
map and then if it matches remove it. Great catch I would say.
> Possible memory leak in BucketCache
> ---
>
> Key: HBASE-13301
> URL: https://issues.apache.org/jira/browse/HBASE-13301
> Project: HBase
> Issue Type: Bug
> Components: BlockCache
>Reporter: zhangduo
>Assignee: zhangduo
> Attachments: HBASE-13301-testcase.patch
>
>
> {code:title=BucketCache.java}
> public boolean evictBlock(BlockCacheKey cacheKey) {
> ...
> if (bucketEntry.equals(backingMap.remove(cacheKey))) {
> bucketAllocator.freeBlock(bucketEntry.offset());
> realCacheSize.addAndGet(-1 * bucketEntry.getLength());
> blocksByHFile.remove(cacheKey.getHfileName(), cacheKey);
> if (removedBlock == null) {
> this.blockNumber.decrementAndGet();
> }
> } else {
> return false;
> }
> ...
> {code}
> I think the problem is here. We remove a BucketEntry that should not be
> removed by us, but we do not put it back and also do not do any clean up.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-13301) Possible memory leak in BucketCache
[
https://issues.apache.org/jira/browse/HBASE-13301?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14371352#comment-14371352
]
zhangduo commented on HBASE-13301:
--
Do not submit the patch since it is not a fix.
Experts needed. [~stack] (I do not know who is the right person since
[~zjushch] seems not active for a long time, so...)
> Possible memory leak in BucketCache
> ---
>
> Key: HBASE-13301
> URL: https://issues.apache.org/jira/browse/HBASE-13301
> Project: HBase
> Issue Type: Bug
> Components: BlockCache
>Reporter: zhangduo
>Assignee: zhangduo
> Attachments: HBASE-13301-testcase.patch
>
>
> {code:title=BucketCache.java}
> public boolean evictBlock(BlockCacheKey cacheKey) {
> ...
> if (bucketEntry.equals(backingMap.remove(cacheKey))) {
> bucketAllocator.freeBlock(bucketEntry.offset());
> realCacheSize.addAndGet(-1 * bucketEntry.getLength());
> blocksByHFile.remove(cacheKey.getHfileName(), cacheKey);
> if (removedBlock == null) {
> this.blockNumber.decrementAndGet();
> }
> } else {
> return false;
> }
> ...
> {code}
> I think the problem is here. We remove a BucketEntry that should not be
> removed by us, but we do not put it back and also do not do any clean up.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
