[jira] [Commented] (HBASE-17118) StoreScanner leaked in KeyValueHeap
[
https://issues.apache.org/jira/browse/HBASE-17118?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16721138#comment-16721138
]
Hudson commented on HBASE-17118:
Results for branch branch-1.3
[build #576 on
builds.a.o|https://builds.apache.org/job/HBase%20Nightly/job/branch-1.3/576/]:
(x) *{color:red}-1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-1.3/576//General_Nightly_Build_Report/]
(/) {color:green}+1 jdk7 checks{color}
-- For more information [see jdk7
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-1.3/576//JDK7_Nightly_Build_Report/]
(x) {color:red}-1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-1.3/576//JDK8_Nightly_Build_Report_(Hadoop2)/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
> StoreScanner leaked in KeyValueHeap
> ---
>
> Key: HBASE-17118
> URL: https://issues.apache.org/jira/browse/HBASE-17118
> Project: HBase
> Issue Type: Bug
>Affects Versions: 1.1.7, 1.2.4
>Reporter: binlijin
>Assignee: binlijin
>Priority: Major
> Fix For: 1.4.0, 1.2.5, 1.1.8, 1.3.3, 2.0.0
>
> Attachments: HBASE-17118-master_v1.patch,
> HBASE-17118-master_v2.patch, HBASE-17118-master_v3.patch,
> HBASE-17118-master_v4.patch, HBASE-17118-master_v5.patch,
> HBASE-17118.branch-1.1.v1.patch, HBASE-17118.branch-1.2.v1.patch,
> HBASE-17118.branch-1.addnumv1.patch, HBASE-17118.branch-1.v1.patch,
> StoreScanner.png, StoreScannerLeakHeap.png
>
>
> KeyValueHeap#generalizedSeek
> KeyValueScanner scanner = current;
> while (scanner != null) {
> Cell topKey = scanner.peek();
> ..
> boolean seekResult;
> if (isLazy && heap.size() > 0) {
> // If there is only one scanner left, we don't do lazy seek.
> seekResult = scanner.requestSeek(seekKey, forward, useBloom);
> } else {
> seekResult = NonLazyKeyValueScanner.doRealSeek(scanner, seekKey,
> forward);
> }
> ..
> scanner = heap.poll();
> }
> (1) scanner = heap.poll(); Retrieves and removes the head of this queue
> (2) scanner.requestSeek(seekKey, forward, useBloom); or
> NonLazyKeyValueScanner.doRealSeek(scanner, seekKey, forward);
> throw exception, and scanner will have no chance to close, so will cause the
> scanner leak.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-17118) StoreScanner leaked in KeyValueHeap
[
https://issues.apache.org/jira/browse/HBASE-17118?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16720751#comment-16720751
]
Hudson commented on HBASE-17118:
SUCCESS: Integrated in Jenkins build HBase-1.3-IT #509 (See
[https://builds.apache.org/job/HBase-1.3-IT/509/])
HBASE-17118 StoreScanner leaked in KeyValueHeap (binlijin) (apurtell: rev
25135dd0a13538e4dd4ffdd34dcb3120a010be3b)
* (edit)
hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/KeyValueHeap.java
> StoreScanner leaked in KeyValueHeap
> ---
>
> Key: HBASE-17118
> URL: https://issues.apache.org/jira/browse/HBASE-17118
> Project: HBase
> Issue Type: Bug
>Affects Versions: 1.1.7, 1.2.4
>Reporter: binlijin
>Assignee: binlijin
>Priority: Major
> Fix For: 1.4.0, 1.2.5, 1.1.8, 1.3.3, 2.0.0
>
> Attachments: HBASE-17118-master_v1.patch,
> HBASE-17118-master_v2.patch, HBASE-17118-master_v3.patch,
> HBASE-17118-master_v4.patch, HBASE-17118-master_v5.patch,
> HBASE-17118.branch-1.1.v1.patch, HBASE-17118.branch-1.2.v1.patch,
> HBASE-17118.branch-1.addnumv1.patch, HBASE-17118.branch-1.v1.patch,
> StoreScanner.png, StoreScannerLeakHeap.png
>
>
> KeyValueHeap#generalizedSeek
> KeyValueScanner scanner = current;
> while (scanner != null) {
> Cell topKey = scanner.peek();
> ..
> boolean seekResult;
> if (isLazy && heap.size() > 0) {
> // If there is only one scanner left, we don't do lazy seek.
> seekResult = scanner.requestSeek(seekKey, forward, useBloom);
> } else {
> seekResult = NonLazyKeyValueScanner.doRealSeek(scanner, seekKey,
> forward);
> }
> ..
> scanner = heap.poll();
> }
> (1) scanner = heap.poll(); Retrieves and removes the head of this queue
> (2) scanner.requestSeek(seekKey, forward, useBloom); or
> NonLazyKeyValueScanner.doRealSeek(scanner, seekKey, forward);
> throw exception, and scanner will have no chance to close, so will cause the
> scanner leak.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-17118) StoreScanner leaked in KeyValueHeap
[
https://issues.apache.org/jira/browse/HBASE-17118?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15720891#comment-15720891
]
binlijin commented on HBASE-17118:
--
This do not committed to branch-1.3.x, i will resolve this, and create a
subtask to back port it to branch-1.3.x
> StoreScanner leaked in KeyValueHeap
> ---
>
> Key: HBASE-17118
> URL: https://issues.apache.org/jira/browse/HBASE-17118
> Project: HBase
> Issue Type: Bug
>Affects Versions: 1.1.7, 1.2.4
>Reporter: binlijin
>Assignee: binlijin
> Fix For: 2.0.0, 1.4.0, 1.2.5, 1.1.8
>
> Attachments: HBASE-17118-master_v1.patch,
> HBASE-17118-master_v2.patch, HBASE-17118-master_v3.patch,
> HBASE-17118-master_v4.patch, HBASE-17118-master_v5.patch,
> HBASE-17118.branch-1.1.v1.patch, HBASE-17118.branch-1.2.v1.patch,
> HBASE-17118.branch-1.addnumv1.patch, HBASE-17118.branch-1.v1.patch,
> StoreScanner.png, StoreScannerLeakHeap.png
>
>
> KeyValueHeap#generalizedSeek
> KeyValueScanner scanner = current;
> while (scanner != null) {
> Cell topKey = scanner.peek();
> ..
> boolean seekResult;
> if (isLazy && heap.size() > 0) {
> // If there is only one scanner left, we don't do lazy seek.
> seekResult = scanner.requestSeek(seekKey, forward, useBloom);
> } else {
> seekResult = NonLazyKeyValueScanner.doRealSeek(scanner, seekKey,
> forward);
> }
> ..
> scanner = heap.poll();
> }
> (1) scanner = heap.poll(); Retrieves and removes the head of this queue
> (2) scanner.requestSeek(seekKey, forward, useBloom); or
> NonLazyKeyValueScanner.doRealSeek(scanner, seekKey, forward);
> throw exception, and scanner will have no chance to close, so will cause the
> scanner leak.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-17118) StoreScanner leaked in KeyValueHeap
[
https://issues.apache.org/jira/browse/HBASE-17118?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15720536#comment-15720536
]
Nick Dimiduk commented on HBASE-17118:
--
Has this this landed everywhere? Ready to close?
> StoreScanner leaked in KeyValueHeap
> ---
>
> Key: HBASE-17118
> URL: https://issues.apache.org/jira/browse/HBASE-17118
> Project: HBase
> Issue Type: Bug
>Affects Versions: 1.1.7, 1.2.4
>Reporter: binlijin
>Assignee: binlijin
> Fix For: 2.0.0, 1.4.0, 1.2.5, 1.1.8
>
> Attachments: HBASE-17118-master_v1.patch,
> HBASE-17118-master_v2.patch, HBASE-17118-master_v3.patch,
> HBASE-17118-master_v4.patch, HBASE-17118-master_v5.patch,
> HBASE-17118.branch-1.1.v1.patch, HBASE-17118.branch-1.2.v1.patch,
> HBASE-17118.branch-1.addnumv1.patch, HBASE-17118.branch-1.v1.patch,
> StoreScanner.png, StoreScannerLeakHeap.png
>
>
> KeyValueHeap#generalizedSeek
> KeyValueScanner scanner = current;
> while (scanner != null) {
> Cell topKey = scanner.peek();
> ..
> boolean seekResult;
> if (isLazy && heap.size() > 0) {
> // If there is only one scanner left, we don't do lazy seek.
> seekResult = scanner.requestSeek(seekKey, forward, useBloom);
> } else {
> seekResult = NonLazyKeyValueScanner.doRealSeek(scanner, seekKey,
> forward);
> }
> ..
> scanner = heap.poll();
> }
> (1) scanner = heap.poll(); Retrieves and removes the head of this queue
> (2) scanner.requestSeek(seekKey, forward, useBloom); or
> NonLazyKeyValueScanner.doRealSeek(scanner, seekKey, forward);
> throw exception, and scanner will have no chance to close, so will cause the
> scanner leak.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-17118) StoreScanner leaked in KeyValueHeap
[
https://issues.apache.org/jira/browse/HBASE-17118?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15697545#comment-15697545
]
Hudson commented on HBASE-17118:
SUCCESS: Integrated in Jenkins build HBase-1.1-JDK7 #1822 (See
[https://builds.apache.org/job/HBase-1.1-JDK7/1822/])
HBASE-17118 StoreScanner leaked in KeyValueHeap (binlijin: rev
b320d4909d94b416e8a995cd5545e90feecdce2a)
* (edit)
hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/KeyValueHeap.java
> StoreScanner leaked in KeyValueHeap
> ---
>
> Key: HBASE-17118
> URL: https://issues.apache.org/jira/browse/HBASE-17118
> Project: HBase
> Issue Type: Bug
>Affects Versions: 2.0.0
>Reporter: binlijin
>Assignee: binlijin
> Fix For: 2.0.0, 1.4.0
>
> Attachments: HBASE-17118-master_v1.patch,
> HBASE-17118-master_v2.patch, HBASE-17118-master_v3.patch,
> HBASE-17118-master_v4.patch, HBASE-17118-master_v5.patch,
> HBASE-17118.branch-1.1.v1.patch, HBASE-17118.branch-1.2.v1.patch,
> HBASE-17118.branch-1.addnumv1.patch, HBASE-17118.branch-1.v1.patch,
> StoreScanner.png, StoreScannerLeakHeap.png
>
>
> KeyValueHeap#generalizedSeek
> KeyValueScanner scanner = current;
> while (scanner != null) {
> Cell topKey = scanner.peek();
> ..
> boolean seekResult;
> if (isLazy && heap.size() > 0) {
> // If there is only one scanner left, we don't do lazy seek.
> seekResult = scanner.requestSeek(seekKey, forward, useBloom);
> } else {
> seekResult = NonLazyKeyValueScanner.doRealSeek(scanner, seekKey,
> forward);
> }
> ..
> scanner = heap.poll();
> }
> (1) scanner = heap.poll(); Retrieves and removes the head of this queue
> (2) scanner.requestSeek(seekKey, forward, useBloom); or
> NonLazyKeyValueScanner.doRealSeek(scanner, seekKey, forward);
> throw exception, and scanner will have no chance to close, so will cause the
> scanner leak.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-17118) StoreScanner leaked in KeyValueHeap
[
https://issues.apache.org/jira/browse/HBASE-17118?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15697487#comment-15697487
]
Hudson commented on HBASE-17118:
SUCCESS: Integrated in Jenkins build HBase-1.1-JDK8 #1906 (See
[https://builds.apache.org/job/HBase-1.1-JDK8/1906/])
HBASE-17118 StoreScanner leaked in KeyValueHeap (binlijin: rev
b320d4909d94b416e8a995cd5545e90feecdce2a)
* (edit)
hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/KeyValueHeap.java
> StoreScanner leaked in KeyValueHeap
> ---
>
> Key: HBASE-17118
> URL: https://issues.apache.org/jira/browse/HBASE-17118
> Project: HBase
> Issue Type: Bug
>Affects Versions: 2.0.0
>Reporter: binlijin
>Assignee: binlijin
> Fix For: 2.0.0, 1.4.0
>
> Attachments: HBASE-17118-master_v1.patch,
> HBASE-17118-master_v2.patch, HBASE-17118-master_v3.patch,
> HBASE-17118-master_v4.patch, HBASE-17118-master_v5.patch,
> HBASE-17118.branch-1.1.v1.patch, HBASE-17118.branch-1.2.v1.patch,
> HBASE-17118.branch-1.addnumv1.patch, HBASE-17118.branch-1.v1.patch,
> StoreScanner.png, StoreScannerLeakHeap.png
>
>
> KeyValueHeap#generalizedSeek
> KeyValueScanner scanner = current;
> while (scanner != null) {
> Cell topKey = scanner.peek();
> ..
> boolean seekResult;
> if (isLazy && heap.size() > 0) {
> // If there is only one scanner left, we don't do lazy seek.
> seekResult = scanner.requestSeek(seekKey, forward, useBloom);
> } else {
> seekResult = NonLazyKeyValueScanner.doRealSeek(scanner, seekKey,
> forward);
> }
> ..
> scanner = heap.poll();
> }
> (1) scanner = heap.poll(); Retrieves and removes the head of this queue
> (2) scanner.requestSeek(seekKey, forward, useBloom); or
> NonLazyKeyValueScanner.doRealSeek(scanner, seekKey, forward);
> throw exception, and scanner will have no chance to close, so will cause the
> scanner leak.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-17118) StoreScanner leaked in KeyValueHeap
[
https://issues.apache.org/jira/browse/HBASE-17118?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15697361#comment-15697361
]
Hudson commented on HBASE-17118:
SUCCESS: Integrated in Jenkins build HBase-1.2-JDK7 #74 (See
[https://builds.apache.org/job/HBase-1.2-JDK7/74/])
HBASE-17118 StoreScanner leaked in KeyValueHeap (binlijin: rev
f67fea4b6c2082192e939cf1efd82a9ca6423001)
* (edit)
hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/KeyValueHeap.java
> StoreScanner leaked in KeyValueHeap
> ---
>
> Key: HBASE-17118
> URL: https://issues.apache.org/jira/browse/HBASE-17118
> Project: HBase
> Issue Type: Bug
>Affects Versions: 2.0.0
>Reporter: binlijin
>Assignee: binlijin
> Fix For: 2.0.0, 1.4.0
>
> Attachments: HBASE-17118-master_v1.patch,
> HBASE-17118-master_v2.patch, HBASE-17118-master_v3.patch,
> HBASE-17118-master_v4.patch, HBASE-17118-master_v5.patch,
> HBASE-17118.branch-1.1.v1.patch, HBASE-17118.branch-1.2.v1.patch,
> HBASE-17118.branch-1.addnumv1.patch, HBASE-17118.branch-1.v1.patch,
> StoreScanner.png, StoreScannerLeakHeap.png
>
>
> KeyValueHeap#generalizedSeek
> KeyValueScanner scanner = current;
> while (scanner != null) {
> Cell topKey = scanner.peek();
> ..
> boolean seekResult;
> if (isLazy && heap.size() > 0) {
> // If there is only one scanner left, we don't do lazy seek.
> seekResult = scanner.requestSeek(seekKey, forward, useBloom);
> } else {
> seekResult = NonLazyKeyValueScanner.doRealSeek(scanner, seekKey,
> forward);
> }
> ..
> scanner = heap.poll();
> }
> (1) scanner = heap.poll(); Retrieves and removes the head of this queue
> (2) scanner.requestSeek(seekKey, forward, useBloom); or
> NonLazyKeyValueScanner.doRealSeek(scanner, seekKey, forward);
> throw exception, and scanner will have no chance to close, so will cause the
> scanner leak.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-17118) StoreScanner leaked in KeyValueHeap
[
https://issues.apache.org/jira/browse/HBASE-17118?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15697311#comment-15697311
]
Hudson commented on HBASE-17118:
SUCCESS: Integrated in Jenkins build HBase-1.2-JDK8 #68 (See
[https://builds.apache.org/job/HBase-1.2-JDK8/68/])
HBASE-17118 StoreScanner leaked in KeyValueHeap (binlijin: rev
f67fea4b6c2082192e939cf1efd82a9ca6423001)
* (edit)
hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/KeyValueHeap.java
> StoreScanner leaked in KeyValueHeap
> ---
>
> Key: HBASE-17118
> URL: https://issues.apache.org/jira/browse/HBASE-17118
> Project: HBase
> Issue Type: Bug
>Affects Versions: 2.0.0
>Reporter: binlijin
>Assignee: binlijin
> Fix For: 2.0.0, 1.4.0
>
> Attachments: HBASE-17118-master_v1.patch,
> HBASE-17118-master_v2.patch, HBASE-17118-master_v3.patch,
> HBASE-17118-master_v4.patch, HBASE-17118-master_v5.patch,
> HBASE-17118.branch-1.1.v1.patch, HBASE-17118.branch-1.2.v1.patch,
> HBASE-17118.branch-1.addnumv1.patch, HBASE-17118.branch-1.v1.patch,
> StoreScanner.png, StoreScannerLeakHeap.png
>
>
> KeyValueHeap#generalizedSeek
> KeyValueScanner scanner = current;
> while (scanner != null) {
> Cell topKey = scanner.peek();
> ..
> boolean seekResult;
> if (isLazy && heap.size() > 0) {
> // If there is only one scanner left, we don't do lazy seek.
> seekResult = scanner.requestSeek(seekKey, forward, useBloom);
> } else {
> seekResult = NonLazyKeyValueScanner.doRealSeek(scanner, seekKey,
> forward);
> }
> ..
> scanner = heap.poll();
> }
> (1) scanner = heap.poll(); Retrieves and removes the head of this queue
> (2) scanner.requestSeek(seekKey, forward, useBloom); or
> NonLazyKeyValueScanner.doRealSeek(scanner, seekKey, forward);
> throw exception, and scanner will have no chance to close, so will cause the
> scanner leak.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-17118) StoreScanner leaked in KeyValueHeap
[
https://issues.apache.org/jira/browse/HBASE-17118?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15697039#comment-15697039
]
binlijin commented on HBASE-17118:
--
Failed test not relative to this patch, run it success locally.
> StoreScanner leaked in KeyValueHeap
> ---
>
> Key: HBASE-17118
> URL: https://issues.apache.org/jira/browse/HBASE-17118
> Project: HBase
> Issue Type: Bug
>Affects Versions: 2.0.0
>Reporter: binlijin
>Assignee: binlijin
> Fix For: 2.0.0, 1.4.0
>
> Attachments: HBASE-17118-master_v1.patch,
> HBASE-17118-master_v2.patch, HBASE-17118-master_v3.patch,
> HBASE-17118-master_v4.patch, HBASE-17118-master_v5.patch,
> HBASE-17118.branch-1.1.v1.patch, HBASE-17118.branch-1.2.v1.patch,
> HBASE-17118.branch-1.addnumv1.patch, HBASE-17118.branch-1.v1.patch,
> StoreScanner.png, StoreScannerLeakHeap.png
>
>
> KeyValueHeap#generalizedSeek
> KeyValueScanner scanner = current;
> while (scanner != null) {
> Cell topKey = scanner.peek();
> ..
> boolean seekResult;
> if (isLazy && heap.size() > 0) {
> // If there is only one scanner left, we don't do lazy seek.
> seekResult = scanner.requestSeek(seekKey, forward, useBloom);
> } else {
> seekResult = NonLazyKeyValueScanner.doRealSeek(scanner, seekKey,
> forward);
> }
> ..
> scanner = heap.poll();
> }
> (1) scanner = heap.poll(); Retrieves and removes the head of this queue
> (2) scanner.requestSeek(seekKey, forward, useBloom); or
> NonLazyKeyValueScanner.doRealSeek(scanner, seekKey, forward);
> throw exception, and scanner will have no chance to close, so will cause the
> scanner leak.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-17118) StoreScanner leaked in KeyValueHeap
[
https://issues.apache.org/jira/browse/HBASE-17118?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15697036#comment-15697036
]
binlijin commented on HBASE-17118:
--
Push to branch-1.1 and branch-1.2
> StoreScanner leaked in KeyValueHeap
> ---
>
> Key: HBASE-17118
> URL: https://issues.apache.org/jira/browse/HBASE-17118
> Project: HBase
> Issue Type: Bug
>Affects Versions: 2.0.0
>Reporter: binlijin
>Assignee: binlijin
> Fix For: 2.0.0, 1.4.0
>
> Attachments: HBASE-17118-master_v1.patch,
> HBASE-17118-master_v2.patch, HBASE-17118-master_v3.patch,
> HBASE-17118-master_v4.patch, HBASE-17118-master_v5.patch,
> HBASE-17118.branch-1.1.v1.patch, HBASE-17118.branch-1.2.v1.patch,
> HBASE-17118.branch-1.addnumv1.patch, HBASE-17118.branch-1.v1.patch,
> StoreScanner.png, StoreScannerLeakHeap.png
>
>
> KeyValueHeap#generalizedSeek
> KeyValueScanner scanner = current;
> while (scanner != null) {
> Cell topKey = scanner.peek();
> ..
> boolean seekResult;
> if (isLazy && heap.size() > 0) {
> // If there is only one scanner left, we don't do lazy seek.
> seekResult = scanner.requestSeek(seekKey, forward, useBloom);
> } else {
> seekResult = NonLazyKeyValueScanner.doRealSeek(scanner, seekKey,
> forward);
> }
> ..
> scanner = heap.poll();
> }
> (1) scanner = heap.poll(); Retrieves and removes the head of this queue
> (2) scanner.requestSeek(seekKey, forward, useBloom); or
> NonLazyKeyValueScanner.doRealSeek(scanner, seekKey, forward);
> throw exception, and scanner will have no chance to close, so will cause the
> scanner leak.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-17118) StoreScanner leaked in KeyValueHeap
[
https://issues.apache.org/jira/browse/HBASE-17118?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15696203#comment-15696203
]
Hadoop QA commented on HBASE-17118:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m 17s
{color} | {color:blue} Docker mode activated. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s
{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:red}-1{color} | {color:red} test4tests {color} | {color:red} 0m 0s
{color} | {color:red} The patch doesn't appear to include any new or modified
tests. Please justify why no new tests are needed for this patch. Also please
list what manual steps were performed to verify this patch. {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
44s {color} | {color:green} branch-1.2 passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 33s
{color} | {color:green} branch-1.2 passed with JDK v1.8.0_111 {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 34s
{color} | {color:green} branch-1.2 passed with JDK v1.7.0_80 {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
57s {color} | {color:green} branch-1.2 passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
17s {color} | {color:green} branch-1.2 passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m
53s {color} | {color:green} branch-1.2 passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 25s
{color} | {color:green} branch-1.2 passed with JDK v1.8.0_111 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 33s
{color} | {color:green} branch-1.2 passed with JDK v1.7.0_80 {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
46s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 30s
{color} | {color:green} the patch passed with JDK v1.8.0_111 {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m 30s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 35s
{color} | {color:green} the patch passed with JDK v1.7.0_80 {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m 35s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
54s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
17s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s {color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
12m 59s {color} | {color:green} The patch does not cause any errors with Hadoop
2.4.0 2.4.1 2.5.0 2.5.1 2.5.2 2.6.1 2.6.2 2.6.3 2.7.1. {color} |
| {color:green}+1{color} | {color:green} hbaseprotoc {color} | {color:green} 0m
15s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 2m 5s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 23s
{color} | {color:green} the patch passed with JDK v1.8.0_111 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 32s
{color} | {color:green} the patch passed with JDK v1.7.0_80 {color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 75m 3s {color}
| {color:red} hbase-server in the patch failed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
16s {color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 102m 22s {color}
| {color:black} {color} |
\\
\\
|| Reason || Tests ||
| Failed junit tests | hadoop.hbase.mapreduce.TestMultiTableSnapshotInputFormat
|
| | hadoop.hbase.mapred.TestMultiTableSnapshotInputFormat |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=1.12.3 Server=1.12.3 Image:yetus/hbase:d6626eb |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12840518/HBASE-17118.branch-1.2.v1.patch
|
| JIRA Issue | HBASE-17118 |
| Optional Tests | asflicense javac javadoc unit findbugs hadoopcheck
hbaseanti checkstyle compile |
| uname | Linux 639c6e12a325 3.13.0-93-generic #140-Ubuntu SMP
[jira] [Commented] (HBASE-17118) StoreScanner leaked in KeyValueHeap
[
https://issues.apache.org/jira/browse/HBASE-17118?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15695978#comment-15695978
]
binlijin commented on HBASE-17118:
--
Ok, will commit it after the QA.
> StoreScanner leaked in KeyValueHeap
> ---
>
> Key: HBASE-17118
> URL: https://issues.apache.org/jira/browse/HBASE-17118
> Project: HBase
> Issue Type: Bug
>Affects Versions: 2.0.0
>Reporter: binlijin
>Assignee: binlijin
> Fix For: 2.0.0, 1.4.0
>
> Attachments: HBASE-17118-master_v1.patch,
> HBASE-17118-master_v2.patch, HBASE-17118-master_v3.patch,
> HBASE-17118-master_v4.patch, HBASE-17118-master_v5.patch,
> HBASE-17118.branch-1.1.v1.patch, HBASE-17118.branch-1.2.v1.patch,
> HBASE-17118.branch-1.addnumv1.patch, HBASE-17118.branch-1.v1.patch,
> StoreScanner.png, StoreScannerLeakHeap.png
>
>
> KeyValueHeap#generalizedSeek
> KeyValueScanner scanner = current;
> while (scanner != null) {
> Cell topKey = scanner.peek();
> ..
> boolean seekResult;
> if (isLazy && heap.size() > 0) {
> // If there is only one scanner left, we don't do lazy seek.
> seekResult = scanner.requestSeek(seekKey, forward, useBloom);
> } else {
> seekResult = NonLazyKeyValueScanner.doRealSeek(scanner, seekKey,
> forward);
> }
> ..
> scanner = heap.poll();
> }
> (1) scanner = heap.poll(); Retrieves and removes the head of this queue
> (2) scanner.requestSeek(seekKey, forward, useBloom); or
> NonLazyKeyValueScanner.doRealSeek(scanner, seekKey, forward);
> throw exception, and scanner will have no chance to close, so will cause the
> scanner leak.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-17118) StoreScanner leaked in KeyValueHeap
[
https://issues.apache.org/jira/browse/HBASE-17118?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15695852#comment-15695852
]
Yu Li commented on HBASE-17118:
---
This issue also exists in 1.1.x/1.2.x? If so, I think we should also commit
this into relative branches. [~aoxiang]
And for 1.3, maybe open another JIRA to track the backport work? Or have you
got some time to review this one [~mantonov]? Thanks.
> StoreScanner leaked in KeyValueHeap
> ---
>
> Key: HBASE-17118
> URL: https://issues.apache.org/jira/browse/HBASE-17118
> Project: HBase
> Issue Type: Bug
>Affects Versions: 2.0.0
>Reporter: binlijin
>Assignee: binlijin
> Fix For: 2.0.0, 1.4.0
>
> Attachments: HBASE-17118-master_v1.patch,
> HBASE-17118-master_v2.patch, HBASE-17118-master_v3.patch,
> HBASE-17118-master_v4.patch, HBASE-17118-master_v5.patch,
> HBASE-17118.branch-1.addnumv1.patch, HBASE-17118.branch-1.v1.patch,
> StoreScanner.png, StoreScannerLeakHeap.png
>
>
> KeyValueHeap#generalizedSeek
> KeyValueScanner scanner = current;
> while (scanner != null) {
> Cell topKey = scanner.peek();
> ..
> boolean seekResult;
> if (isLazy && heap.size() > 0) {
> // If there is only one scanner left, we don't do lazy seek.
> seekResult = scanner.requestSeek(seekKey, forward, useBloom);
> } else {
> seekResult = NonLazyKeyValueScanner.doRealSeek(scanner, seekKey,
> forward);
> }
> ..
> scanner = heap.poll();
> }
> (1) scanner = heap.poll(); Retrieves and removes the head of this queue
> (2) scanner.requestSeek(seekKey, forward, useBloom); or
> NonLazyKeyValueScanner.doRealSeek(scanner, seekKey, forward);
> throw exception, and scanner will have no chance to close, so will cause the
> scanner leak.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-17118) StoreScanner leaked in KeyValueHeap
[
https://issues.apache.org/jira/browse/HBASE-17118?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15684306#comment-15684306
]
Hudson commented on HBASE-17118:
SUCCESS: Integrated in Jenkins build HBase-1.4 #543 (See
[https://builds.apache.org/job/HBase-1.4/543/])
HBASE-17118 StoreScanner leaked in KeyValueHeap (addendum) (binlijin: rev
2da5ed098eff4b75629e014f4127901a4362bc99)
* (edit)
hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/KeyValueHeap.java
> StoreScanner leaked in KeyValueHeap
> ---
>
> Key: HBASE-17118
> URL: https://issues.apache.org/jira/browse/HBASE-17118
> Project: HBase
> Issue Type: Bug
>Affects Versions: 2.0.0
>Reporter: binlijin
>Assignee: binlijin
> Fix For: 2.0.0, 1.4.0
>
> Attachments: HBASE-17118-master_v1.patch,
> HBASE-17118-master_v2.patch, HBASE-17118-master_v3.patch,
> HBASE-17118-master_v4.patch, HBASE-17118-master_v5.patch,
> HBASE-17118.branch-1.addnumv1.patch, HBASE-17118.branch-1.v1.patch,
> StoreScanner.png, StoreScannerLeakHeap.png
>
>
> KeyValueHeap#generalizedSeek
> KeyValueScanner scanner = current;
> while (scanner != null) {
> Cell topKey = scanner.peek();
> ..
> boolean seekResult;
> if (isLazy && heap.size() > 0) {
> // If there is only one scanner left, we don't do lazy seek.
> seekResult = scanner.requestSeek(seekKey, forward, useBloom);
> } else {
> seekResult = NonLazyKeyValueScanner.doRealSeek(scanner, seekKey,
> forward);
> }
> ..
> scanner = heap.poll();
> }
> (1) scanner = heap.poll(); Retrieves and removes the head of this queue
> (2) scanner.requestSeek(seekKey, forward, useBloom); or
> NonLazyKeyValueScanner.doRealSeek(scanner, seekKey, forward);
> throw exception, and scanner will have no chance to close, so will cause the
> scanner leak.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-17118) StoreScanner leaked in KeyValueHeap
[
https://issues.apache.org/jira/browse/HBASE-17118?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15683704#comment-15683704
]
Hadoop QA commented on HBASE-17118:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m 13s
{color} | {color:blue} Docker mode activated. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s
{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:red}-1{color} | {color:red} test4tests {color} | {color:red} 0m 0s
{color} | {color:red} The patch doesn't appear to include any new or modified
tests. Please justify why no new tests are needed for this patch. Also please
list what manual steps were performed to verify this patch. {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
58s {color} | {color:green} branch-1 passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 32s
{color} | {color:green} branch-1 passed with JDK v1.8.0_111 {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 36s
{color} | {color:green} branch-1 passed with JDK v1.7.0_80 {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
56s {color} | {color:green} branch-1 passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
17s {color} | {color:green} branch-1 passed {color} |
| {color:red}-1{color} | {color:red} findbugs {color} | {color:red} 1m 57s
{color} | {color:red} hbase-server in branch-1 has 2 extant Findbugs warnings.
{color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 27s
{color} | {color:green} branch-1 passed with JDK v1.8.0_111 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 34s
{color} | {color:green} branch-1 passed with JDK v1.7.0_80 {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
42s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 30s
{color} | {color:green} the patch passed with JDK v1.8.0_111 {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m 30s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 35s
{color} | {color:green} the patch passed with JDK v1.7.0_80 {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m 35s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
56s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
16s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s {color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
15m 2s {color} | {color:green} The patch does not cause any errors with Hadoop
2.4.0 2.4.1 2.5.0 2.5.1 2.5.2 2.6.1 2.6.2 2.6.3 2.7.1. {color} |
| {color:green}+1{color} | {color:green} hbaseprotoc {color} | {color:green} 0m
15s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 2m
12s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 24s
{color} | {color:green} the patch passed with JDK v1.8.0_111 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 34s
{color} | {color:green} the patch passed with JDK v1.7.0_80 {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 82m 21s
{color} | {color:green} hbase-server in the patch passed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
22s {color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 112m 7s {color}
| {color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=1.11.2 Server=1.11.2 Image:yetus/hbase:e01ee2f |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12839785/HBASE-17118.branch-1.addnumv1.patch
|
| JIRA Issue | HBASE-17118 |
| Optional Tests | asflicense javac javadoc unit findbugs hadoopcheck
hbaseanti checkstyle compile |
| uname | Linux 87729c7ce1de 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed
Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
[jira] [Commented] (HBASE-17118) StoreScanner leaked in KeyValueHeap
[
https://issues.apache.org/jira/browse/HBASE-17118?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15683523#comment-15683523
]
binlijin commented on HBASE-17118:
--
Done, already have push to branch-1.
> StoreScanner leaked in KeyValueHeap
> ---
>
> Key: HBASE-17118
> URL: https://issues.apache.org/jira/browse/HBASE-17118
> Project: HBase
> Issue Type: Bug
>Affects Versions: 2.0.0
>Reporter: binlijin
>Assignee: binlijin
> Fix For: 2.0.0, 1.4.0
>
> Attachments: HBASE-17118-master_v1.patch,
> HBASE-17118-master_v2.patch, HBASE-17118-master_v3.patch,
> HBASE-17118-master_v4.patch, HBASE-17118-master_v5.patch,
> HBASE-17118.branch-1.addnumv1.patch, HBASE-17118.branch-1.v1.patch,
> StoreScanner.png, StoreScannerLeakHeap.png
>
>
> KeyValueHeap#generalizedSeek
> KeyValueScanner scanner = current;
> while (scanner != null) {
> Cell topKey = scanner.peek();
> ..
> boolean seekResult;
> if (isLazy && heap.size() > 0) {
> // If there is only one scanner left, we don't do lazy seek.
> seekResult = scanner.requestSeek(seekKey, forward, useBloom);
> } else {
> seekResult = NonLazyKeyValueScanner.doRealSeek(scanner, seekKey,
> forward);
> }
> ..
> scanner = heap.poll();
> }
> (1) scanner = heap.poll(); Retrieves and removes the head of this queue
> (2) scanner.requestSeek(seekKey, forward, useBloom); or
> NonLazyKeyValueScanner.doRealSeek(scanner, seekKey, forward);
> throw exception, and scanner will have no chance to close, so will cause the
> scanner leak.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-17118) StoreScanner leaked in KeyValueHeap
[
https://issues.apache.org/jira/browse/HBASE-17118?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15683475#comment-15683475
]
binlijin commented on HBASE-17118:
--
OK, thank you very much, and sorry for the mistake.
> StoreScanner leaked in KeyValueHeap
> ---
>
> Key: HBASE-17118
> URL: https://issues.apache.org/jira/browse/HBASE-17118
> Project: HBase
> Issue Type: Bug
>Affects Versions: 2.0.0
>Reporter: binlijin
>Assignee: binlijin
> Fix For: 2.0.0, 1.4.0
>
> Attachments: HBASE-17118-master_v1.patch,
> HBASE-17118-master_v2.patch, HBASE-17118-master_v3.patch,
> HBASE-17118-master_v4.patch, HBASE-17118-master_v5.patch,
> HBASE-17118.branch-1.addnumv1.patch, HBASE-17118.branch-1.v1.patch,
> StoreScanner.png, StoreScannerLeakHeap.png
>
>
> KeyValueHeap#generalizedSeek
> KeyValueScanner scanner = current;
> while (scanner != null) {
> Cell topKey = scanner.peek();
> ..
> boolean seekResult;
> if (isLazy && heap.size() > 0) {
> // If there is only one scanner left, we don't do lazy seek.
> seekResult = scanner.requestSeek(seekKey, forward, useBloom);
> } else {
> seekResult = NonLazyKeyValueScanner.doRealSeek(scanner, seekKey,
> forward);
> }
> ..
> scanner = heap.poll();
> }
> (1) scanner = heap.poll(); Retrieves and removes the head of this queue
> (2) scanner.requestSeek(seekKey, forward, useBloom); or
> NonLazyKeyValueScanner.doRealSeek(scanner, seekKey, forward);
> throw exception, and scanner will have no chance to close, so will cause the
> scanner leak.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-17118) StoreScanner leaked in KeyValueHeap
[
https://issues.apache.org/jira/browse/HBASE-17118?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15683373#comment-15683373
]
Anoop Sam John commented on HBASE-17118:
See HBASE-17137.
There is an issue with the branch-1 patch.. That missed a rethrow of the
original exception from catch block.. Master patch was ok.
> StoreScanner leaked in KeyValueHeap
> ---
>
> Key: HBASE-17118
> URL: https://issues.apache.org/jira/browse/HBASE-17118
> Project: HBase
> Issue Type: Bug
>Affects Versions: 2.0.0
>Reporter: binlijin
>Assignee: binlijin
> Fix For: 2.0.0, 1.4.0
>
> Attachments: HBASE-17118-master_v1.patch,
> HBASE-17118-master_v2.patch, HBASE-17118-master_v3.patch,
> HBASE-17118-master_v4.patch, HBASE-17118-master_v5.patch,
> HBASE-17118.branch-1.v1.patch, StoreScanner.png, StoreScannerLeakHeap.png
>
>
> KeyValueHeap#generalizedSeek
> KeyValueScanner scanner = current;
> while (scanner != null) {
> Cell topKey = scanner.peek();
> ..
> boolean seekResult;
> if (isLazy && heap.size() > 0) {
> // If there is only one scanner left, we don't do lazy seek.
> seekResult = scanner.requestSeek(seekKey, forward, useBloom);
> } else {
> seekResult = NonLazyKeyValueScanner.doRealSeek(scanner, seekKey,
> forward);
> }
> ..
> scanner = heap.poll();
> }
> (1) scanner = heap.poll(); Retrieves and removes the head of this queue
> (2) scanner.requestSeek(seekKey, forward, useBloom); or
> NonLazyKeyValueScanner.doRealSeek(scanner, seekKey, forward);
> throw exception, and scanner will have no chance to close, so will cause the
> scanner leak.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-17118) StoreScanner leaked in KeyValueHeap
[
https://issues.apache.org/jira/browse/HBASE-17118?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15682322#comment-15682322
]
binlijin commented on HBASE-17118:
--
[~yuzhih...@gmail.com] Thank you very much!
> StoreScanner leaked in KeyValueHeap
> ---
>
> Key: HBASE-17118
> URL: https://issues.apache.org/jira/browse/HBASE-17118
> Project: HBase
> Issue Type: Bug
>Affects Versions: 2.0.0
>Reporter: binlijin
>Assignee: binlijin
> Fix For: 2.0.0, 1.4.0
>
> Attachments: HBASE-17118-master_v1.patch,
> HBASE-17118-master_v2.patch, HBASE-17118-master_v3.patch,
> HBASE-17118-master_v4.patch, HBASE-17118-master_v5.patch,
> HBASE-17118.branch-1.v1.patch, StoreScanner.png, StoreScannerLeakHeap.png
>
>
> KeyValueHeap#generalizedSeek
> KeyValueScanner scanner = current;
> while (scanner != null) {
> Cell topKey = scanner.peek();
> ..
> boolean seekResult;
> if (isLazy && heap.size() > 0) {
> // If there is only one scanner left, we don't do lazy seek.
> seekResult = scanner.requestSeek(seekKey, forward, useBloom);
> } else {
> seekResult = NonLazyKeyValueScanner.doRealSeek(scanner, seekKey,
> forward);
> }
> ..
> scanner = heap.poll();
> }
> (1) scanner = heap.poll(); Retrieves and removes the head of this queue
> (2) scanner.requestSeek(seekKey, forward, useBloom); or
> NonLazyKeyValueScanner.doRealSeek(scanner, seekKey, forward);
> throw exception, and scanner will have no chance to close, so will cause the
> scanner leak.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-17118) StoreScanner leaked in KeyValueHeap
[
https://issues.apache.org/jira/browse/HBASE-17118?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15677305#comment-15677305
]
Ted Yu commented on HBASE-17118:
[~aoxiang]:
Mind bringing this to other branches now that you have the commit bit ?
For 1.3, wait for Mikhail's signal.
Thanks
> StoreScanner leaked in KeyValueHeap
> ---
>
> Key: HBASE-17118
> URL: https://issues.apache.org/jira/browse/HBASE-17118
> Project: HBase
> Issue Type: Bug
>Affects Versions: 2.0.0
>Reporter: binlijin
>Assignee: binlijin
> Fix For: 2.0.0, 1.4.0
>
> Attachments: HBASE-17118-master_v1.patch,
> HBASE-17118-master_v2.patch, HBASE-17118-master_v3.patch,
> HBASE-17118-master_v4.patch, HBASE-17118-master_v5.patch,
> HBASE-17118.branch-1.v1.patch, StoreScanner.png, StoreScannerLeakHeap.png
>
>
> KeyValueHeap#generalizedSeek
> KeyValueScanner scanner = current;
> while (scanner != null) {
> Cell topKey = scanner.peek();
> ..
> boolean seekResult;
> if (isLazy && heap.size() > 0) {
> // If there is only one scanner left, we don't do lazy seek.
> seekResult = scanner.requestSeek(seekKey, forward, useBloom);
> } else {
> seekResult = NonLazyKeyValueScanner.doRealSeek(scanner, seekKey,
> forward);
> }
> ..
> scanner = heap.poll();
> }
> (1) scanner = heap.poll(); Retrieves and removes the head of this queue
> (2) scanner.requestSeek(seekKey, forward, useBloom); or
> NonLazyKeyValueScanner.doRealSeek(scanner, seekKey, forward);
> throw exception, and scanner will have no chance to close, so will cause the
> scanner leak.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-17118) StoreScanner leaked in KeyValueHeap
[
https://issues.apache.org/jira/browse/HBASE-17118?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15674946#comment-15674946
]
Hudson commented on HBASE-17118:
SUCCESS: Integrated in Jenkins build HBase-1.4 #537 (See
[https://builds.apache.org/job/HBase-1.4/537/])
HBASE-17118 StoreScanner leaked in KeyValueHeap (binlijin) (tedyu: rev
d722b2aab72459272dc1bb05c118568c88c797ca)
* (edit)
hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/KeyValueHeap.java
> StoreScanner leaked in KeyValueHeap
> ---
>
> Key: HBASE-17118
> URL: https://issues.apache.org/jira/browse/HBASE-17118
> Project: HBase
> Issue Type: Bug
>Affects Versions: 2.0.0
>Reporter: binlijin
>Assignee: binlijin
> Fix For: 2.0.0, 1.4.0
>
> Attachments: HBASE-17118-master_v1.patch,
> HBASE-17118-master_v2.patch, HBASE-17118-master_v3.patch,
> HBASE-17118-master_v4.patch, HBASE-17118-master_v5.patch,
> HBASE-17118.branch-1.v1.patch, StoreScanner.png, StoreScannerLeakHeap.png
>
>
> KeyValueHeap#generalizedSeek
> KeyValueScanner scanner = current;
> while (scanner != null) {
> Cell topKey = scanner.peek();
> ..
> boolean seekResult;
> if (isLazy && heap.size() > 0) {
> // If there is only one scanner left, we don't do lazy seek.
> seekResult = scanner.requestSeek(seekKey, forward, useBloom);
> } else {
> seekResult = NonLazyKeyValueScanner.doRealSeek(scanner, seekKey,
> forward);
> }
> ..
> scanner = heap.poll();
> }
> (1) scanner = heap.poll(); Retrieves and removes the head of this queue
> (2) scanner.requestSeek(seekKey, forward, useBloom); or
> NonLazyKeyValueScanner.doRealSeek(scanner, seekKey, forward);
> throw exception, and scanner will have no chance to close, so will cause the
> scanner leak.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-17118) StoreScanner leaked in KeyValueHeap
[
https://issues.apache.org/jira/browse/HBASE-17118?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15674669#comment-15674669
]
Mikhail Antonov commented on HBASE-17118:
-
[~tedyu] let me have a look and come back in a bit
> StoreScanner leaked in KeyValueHeap
> ---
>
> Key: HBASE-17118
> URL: https://issues.apache.org/jira/browse/HBASE-17118
> Project: HBase
> Issue Type: Bug
>Affects Versions: 2.0.0
>Reporter: binlijin
>Assignee: binlijin
> Fix For: 2.0.0, 1.4.0
>
> Attachments: HBASE-17118-master_v1.patch,
> HBASE-17118-master_v2.patch, HBASE-17118-master_v3.patch,
> HBASE-17118-master_v4.patch, HBASE-17118-master_v5.patch,
> HBASE-17118.branch-1.v1.patch, StoreScanner.png, StoreScannerLeakHeap.png
>
>
> KeyValueHeap#generalizedSeek
> KeyValueScanner scanner = current;
> while (scanner != null) {
> Cell topKey = scanner.peek();
> ..
> boolean seekResult;
> if (isLazy && heap.size() > 0) {
> // If there is only one scanner left, we don't do lazy seek.
> seekResult = scanner.requestSeek(seekKey, forward, useBloom);
> } else {
> seekResult = NonLazyKeyValueScanner.doRealSeek(scanner, seekKey,
> forward);
> }
> ..
> scanner = heap.poll();
> }
> (1) scanner = heap.poll(); Retrieves and removes the head of this queue
> (2) scanner.requestSeek(seekKey, forward, useBloom); or
> NonLazyKeyValueScanner.doRealSeek(scanner, seekKey, forward);
> throw exception, and scanner will have no chance to close, so will cause the
> scanner leak.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-17118) StoreScanner leaked in KeyValueHeap
[
https://issues.apache.org/jira/browse/HBASE-17118?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15674651#comment-15674651
]
Hudson commented on HBASE-17118:
SUCCESS: Integrated in Jenkins build HBase-Trunk_matrix #1970 (See
[https://builds.apache.org/job/HBase-Trunk_matrix/1970/])
HBASE-17118 StoreScanner leaked in KeyValueHeap (binlijin) (tedyu: rev
f6fc94ede999766d37f38828fe36b581924fcc30)
* (edit)
hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/KeyValueHeap.java
> StoreScanner leaked in KeyValueHeap
> ---
>
> Key: HBASE-17118
> URL: https://issues.apache.org/jira/browse/HBASE-17118
> Project: HBase
> Issue Type: Bug
>Affects Versions: 2.0.0
>Reporter: binlijin
>Assignee: binlijin
> Fix For: 2.0.0, 1.4.0
>
> Attachments: HBASE-17118-master_v1.patch,
> HBASE-17118-master_v2.patch, HBASE-17118-master_v3.patch,
> HBASE-17118-master_v4.patch, HBASE-17118-master_v5.patch,
> HBASE-17118.branch-1.v1.patch, StoreScanner.png, StoreScannerLeakHeap.png
>
>
> KeyValueHeap#generalizedSeek
> KeyValueScanner scanner = current;
> while (scanner != null) {
> Cell topKey = scanner.peek();
> ..
> boolean seekResult;
> if (isLazy && heap.size() > 0) {
> // If there is only one scanner left, we don't do lazy seek.
> seekResult = scanner.requestSeek(seekKey, forward, useBloom);
> } else {
> seekResult = NonLazyKeyValueScanner.doRealSeek(scanner, seekKey,
> forward);
> }
> ..
> scanner = heap.poll();
> }
> (1) scanner = heap.poll(); Retrieves and removes the head of this queue
> (2) scanner.requestSeek(seekKey, forward, useBloom); or
> NonLazyKeyValueScanner.doRealSeek(scanner, seekKey, forward);
> throw exception, and scanner will have no chance to close, so will cause the
> scanner leak.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-17118) StoreScanner leaked in KeyValueHeap
[
https://issues.apache.org/jira/browse/HBASE-17118?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15674164#comment-15674164
]
binlijin commented on HBASE-17118:
--
Thank you very much for remind me! done.
> StoreScanner leaked in KeyValueHeap
> ---
>
> Key: HBASE-17118
> URL: https://issues.apache.org/jira/browse/HBASE-17118
> Project: HBase
> Issue Type: Bug
>Affects Versions: 2.0.0
>Reporter: binlijin
>Assignee: binlijin
> Attachments: HBASE-17118-master_v1.patch,
> HBASE-17118-master_v2.patch, HBASE-17118-master_v3.patch,
> HBASE-17118-master_v4.patch, HBASE-17118-master_v5.patch,
> HBASE-17118.branch-1.v1.patch, StoreScanner.png, StoreScannerLeakHeap.png
>
>
> KeyValueHeap#generalizedSeek
> KeyValueScanner scanner = current;
> while (scanner != null) {
> Cell topKey = scanner.peek();
> ..
> boolean seekResult;
> if (isLazy && heap.size() > 0) {
> // If there is only one scanner left, we don't do lazy seek.
> seekResult = scanner.requestSeek(seekKey, forward, useBloom);
> } else {
> seekResult = NonLazyKeyValueScanner.doRealSeek(scanner, seekKey,
> forward);
> }
> ..
> scanner = heap.poll();
> }
> (1) scanner = heap.poll(); Retrieves and removes the head of this queue
> (2) scanner.requestSeek(seekKey, forward, useBloom); or
> NonLazyKeyValueScanner.doRealSeek(scanner, seekKey, forward);
> throw exception, and scanner will have no chance to close, so will cause the
> scanner leak.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-17118) StoreScanner leaked in KeyValueHeap
[
https://issues.apache.org/jira/browse/HBASE-17118?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15674049#comment-15674049
]
Ted Yu commented on HBASE-17118:
Mind attaching patch for branch-1 ?
Thanks
> StoreScanner leaked in KeyValueHeap
> ---
>
> Key: HBASE-17118
> URL: https://issues.apache.org/jira/browse/HBASE-17118
> Project: HBase
> Issue Type: Bug
>Affects Versions: 2.0.0
>Reporter: binlijin
>Assignee: binlijin
> Attachments: HBASE-17118-master_v1.patch,
> HBASE-17118-master_v2.patch, HBASE-17118-master_v3.patch,
> HBASE-17118-master_v4.patch, HBASE-17118-master_v5.patch, StoreScanner.png,
> StoreScannerLeakHeap.png
>
>
> KeyValueHeap#generalizedSeek
> KeyValueScanner scanner = current;
> while (scanner != null) {
> Cell topKey = scanner.peek();
> ..
> boolean seekResult;
> if (isLazy && heap.size() > 0) {
> // If there is only one scanner left, we don't do lazy seek.
> seekResult = scanner.requestSeek(seekKey, forward, useBloom);
> } else {
> seekResult = NonLazyKeyValueScanner.doRealSeek(scanner, seekKey,
> forward);
> }
> ..
> scanner = heap.poll();
> }
> (1) scanner = heap.poll(); Retrieves and removes the head of this queue
> (2) scanner.requestSeek(seekKey, forward, useBloom); or
> NonLazyKeyValueScanner.doRealSeek(scanner, seekKey, forward);
> throw exception, and scanner will have no chance to close, so will cause the
> scanner leak.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-17118) StoreScanner leaked in KeyValueHeap
[
https://issues.apache.org/jira/browse/HBASE-17118?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15673525#comment-15673525
]
Hadoop QA commented on HBASE-17118:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m 15s
{color} | {color:blue} Docker mode activated. {color} |
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s {color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s
{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:red}-1{color} | {color:red} test4tests {color} | {color:red} 0m 0s
{color} | {color:red} The patch doesn't appear to include any new or modified
tests. Please justify why no new tests are needed for this patch. Also please
list what manual steps were performed to verify this patch. {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 3m
23s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 35s
{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
44s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
14s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m
41s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 27s
{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
46s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 34s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m 34s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
45s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
13s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s {color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
27m 50s {color} | {color:green} Patch does not cause any errors with Hadoop
2.6.1 2.6.2 2.6.3 2.6.4 2.6.5 2.7.1 2.7.2 2.7.3 or 3.0.0-alpha1. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m
48s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 26s
{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 89m 20s {color}
| {color:red} hbase-server in the patch failed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
16s {color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 129m 36s {color}
| {color:black} {color} |
\\
\\
|| Reason || Tests ||
| Failed junit tests | hadoop.hbase.master.TestAssignmentListener |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=1.11.2 Server=1.11.2 Image:yetus/hbase:8d52d23 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12839336/HBASE-17118-master_v5.patch
|
| JIRA Issue | HBASE-17118 |
| Optional Tests | asflicense javac javadoc unit findbugs hadoopcheck
hbaseanti checkstyle compile |
| uname | Linux 96f0efc0db76 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed
Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/component/dev-support/hbase-personality.sh
|
| git revision | master / ab55bad |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.0 |
| unit |
https://builds.apache.org/job/PreCommit-HBASE-Build/4514/artifact/patchprocess/patch-unit-hbase-server.txt
|
| unit test logs |
https://builds.apache.org/job/PreCommit-HBASE-Build/4514/artifact/patchprocess/patch-unit-hbase-server.txt
|
| Test Results |
https://builds.apache.org/job/PreCommit-HBASE-Build/4514/testReport/ |
| modules | C: hbase-server U: hbase-server |
| Console output |
https://builds.apache.org/job/PreCommit-HBASE-Build/4514/console |
| Powered by | Apache Yetus 0.3.0 http://yetus.apache.org |
This message was automatically generated.
>
[jira] [Commented] (HBASE-17118) StoreScanner leaked in KeyValueHeap
[
https://issues.apache.org/jira/browse/HBASE-17118?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15673500#comment-15673500
]
Hadoop QA commented on HBASE-17118:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m 15s
{color} | {color:blue} Docker mode activated. {color} |
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s {color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s
{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:red}-1{color} | {color:red} test4tests {color} | {color:red} 0m 0s
{color} | {color:red} The patch doesn't appear to include any new or modified
tests. Please justify why no new tests are needed for this patch. Also please
list what manual steps were performed to verify this patch. {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 3m
32s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 36s
{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
48s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
14s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m
58s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 28s
{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
47s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 38s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m 38s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
47s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
13s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s {color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
29m 47s {color} | {color:green} Patch does not cause any errors with Hadoop
2.6.1 2.6.2 2.6.3 2.6.4 2.6.5 2.7.1 2.7.2 2.7.3 or 3.0.0-alpha1. {color} |
| {color:red}-1{color} | {color:red} findbugs {color} | {color:red} 2m 10s
{color} | {color:red} hbase-server generated 1 new + 0 unchanged - 0 fixed = 1
total (was 0) {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 31s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 94m 34s
{color} | {color:green} hbase-server in the patch passed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
17s {color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 137m 55s {color}
| {color:black} {color} |
\\
\\
|| Reason || Tests ||
| FindBugs | module:hbase-server |
| | Possible null pointer dereference of scanner in
org.apache.hadoop.hbase.regionserver.KeyValueHeap.generalizedSeek(boolean,
Cell, boolean, boolean) on exception path Dereferenced at
KeyValueHeap.java:scanner in
org.apache.hadoop.hbase.regionserver.KeyValueHeap.generalizedSeek(boolean,
Cell, boolean, boolean) on exception path Dereferenced at
KeyValueHeap.java:[line 335] |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=1.11.2 Server=1.11.2 Image:yetus/hbase:8d52d23 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12839332/HBASE-17118-master_v4.patch
|
| JIRA Issue | HBASE-17118 |
| Optional Tests | asflicense javac javadoc unit findbugs hadoopcheck
hbaseanti checkstyle compile |
| uname | Linux 289f86a297d8 3.13.0-92-generic #139-Ubuntu SMP Tue Jun 28
20:42:26 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/component/dev-support/hbase-personality.sh
|
| git revision | master / ab55bad |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.0 |
| findbugs |
https://builds.apache.org/job/PreCommit-HBASE-Build/4512/artifact/patchprocess/new-findbugs-hbase-server.html
|
| Test Results |
[jira] [Commented] (HBASE-17118) StoreScanner leaked in KeyValueHeap
[
https://issues.apache.org/jira/browse/HBASE-17118?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15673308#comment-15673308
]
Duo Zhang commented on HBASE-17118:
---
+1.
> StoreScanner leaked in KeyValueHeap
> ---
>
> Key: HBASE-17118
> URL: https://issues.apache.org/jira/browse/HBASE-17118
> Project: HBase
> Issue Type: Bug
>Affects Versions: 2.0.0
>Reporter: binlijin
>Assignee: binlijin
> Attachments: HBASE-17118-master_v1.patch,
> HBASE-17118-master_v2.patch, HBASE-17118-master_v3.patch,
> HBASE-17118-master_v4.patch, HBASE-17118-master_v5.patch, StoreScanner.png,
> StoreScannerLeakHeap.png
>
>
> KeyValueHeap#generalizedSeek
> KeyValueScanner scanner = current;
> while (scanner != null) {
> Cell topKey = scanner.peek();
> ..
> boolean seekResult;
> if (isLazy && heap.size() > 0) {
> // If there is only one scanner left, we don't do lazy seek.
> seekResult = scanner.requestSeek(seekKey, forward, useBloom);
> } else {
> seekResult = NonLazyKeyValueScanner.doRealSeek(scanner, seekKey,
> forward);
> }
> ..
> scanner = heap.poll();
> }
> (1) scanner = heap.poll(); Retrieves and removes the head of this queue
> (2) scanner.requestSeek(seekKey, forward, useBloom); or
> NonLazyKeyValueScanner.doRealSeek(scanner, seekKey, forward);
> throw exception, and scanner will have no chance to close, so will cause the
> scanner leak.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-17118) StoreScanner leaked in KeyValueHeap
[
https://issues.apache.org/jira/browse/HBASE-17118?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15673303#comment-15673303
]
Yu Li commented on HBASE-17118:
---
+1 on patch v5
> StoreScanner leaked in KeyValueHeap
> ---
>
> Key: HBASE-17118
> URL: https://issues.apache.org/jira/browse/HBASE-17118
> Project: HBase
> Issue Type: Bug
>Affects Versions: 2.0.0
>Reporter: binlijin
>Assignee: binlijin
> Attachments: HBASE-17118-master_v1.patch,
> HBASE-17118-master_v2.patch, HBASE-17118-master_v3.patch,
> HBASE-17118-master_v4.patch, HBASE-17118-master_v5.patch, StoreScanner.png,
> StoreScannerLeakHeap.png
>
>
> KeyValueHeap#generalizedSeek
> KeyValueScanner scanner = current;
> while (scanner != null) {
> Cell topKey = scanner.peek();
> ..
> boolean seekResult;
> if (isLazy && heap.size() > 0) {
> // If there is only one scanner left, we don't do lazy seek.
> seekResult = scanner.requestSeek(seekKey, forward, useBloom);
> } else {
> seekResult = NonLazyKeyValueScanner.doRealSeek(scanner, seekKey,
> forward);
> }
> ..
> scanner = heap.poll();
> }
> (1) scanner = heap.poll(); Retrieves and removes the head of this queue
> (2) scanner.requestSeek(seekKey, forward, useBloom); or
> NonLazyKeyValueScanner.doRealSeek(scanner, seekKey, forward);
> throw exception, and scanner will have no chance to close, so will cause the
> scanner leak.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-17118) StoreScanner leaked in KeyValueHeap
[
https://issues.apache.org/jira/browse/HBASE-17118?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15673295#comment-15673295
]
Anoop Sam John commented on HBASE-17118:
+1
> StoreScanner leaked in KeyValueHeap
> ---
>
> Key: HBASE-17118
> URL: https://issues.apache.org/jira/browse/HBASE-17118
> Project: HBase
> Issue Type: Bug
>Affects Versions: 2.0.0
>Reporter: binlijin
>Assignee: binlijin
> Attachments: HBASE-17118-master_v1.patch,
> HBASE-17118-master_v2.patch, HBASE-17118-master_v3.patch,
> HBASE-17118-master_v4.patch, HBASE-17118-master_v5.patch, StoreScanner.png,
> StoreScannerLeakHeap.png
>
>
> KeyValueHeap#generalizedSeek
> KeyValueScanner scanner = current;
> while (scanner != null) {
> Cell topKey = scanner.peek();
> ..
> boolean seekResult;
> if (isLazy && heap.size() > 0) {
> // If there is only one scanner left, we don't do lazy seek.
> seekResult = scanner.requestSeek(seekKey, forward, useBloom);
> } else {
> seekResult = NonLazyKeyValueScanner.doRealSeek(scanner, seekKey,
> forward);
> }
> ..
> scanner = heap.poll();
> }
> (1) scanner = heap.poll(); Retrieves and removes the head of this queue
> (2) scanner.requestSeek(seekKey, forward, useBloom); or
> NonLazyKeyValueScanner.doRealSeek(scanner, seekKey, forward);
> throw exception, and scanner will have no chance to close, so will cause the
> scanner leak.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-17118) StoreScanner leaked in KeyValueHeap
[
https://issues.apache.org/jira/browse/HBASE-17118?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15673228#comment-15673228
]
binlijin commented on HBASE-17118:
--
Yes, if (scanner != null) {} is not necessary, i will remove it.
> StoreScanner leaked in KeyValueHeap
> ---
>
> Key: HBASE-17118
> URL: https://issues.apache.org/jira/browse/HBASE-17118
> Project: HBase
> Issue Type: Bug
>Affects Versions: 2.0.0
>Reporter: binlijin
>Assignee: binlijin
> Attachments: HBASE-17118-master_v1.patch,
> HBASE-17118-master_v2.patch, HBASE-17118-master_v3.patch,
> HBASE-17118-master_v4.patch, StoreScanner.png, StoreScannerLeakHeap.png
>
>
> KeyValueHeap#generalizedSeek
> KeyValueScanner scanner = current;
> while (scanner != null) {
> Cell topKey = scanner.peek();
> ..
> boolean seekResult;
> if (isLazy && heap.size() > 0) {
> // If there is only one scanner left, we don't do lazy seek.
> seekResult = scanner.requestSeek(seekKey, forward, useBloom);
> } else {
> seekResult = NonLazyKeyValueScanner.doRealSeek(scanner, seekKey,
> forward);
> }
> ..
> scanner = heap.poll();
> }
> (1) scanner = heap.poll(); Retrieves and removes the head of this queue
> (2) scanner.requestSeek(seekKey, forward, useBloom); or
> NonLazyKeyValueScanner.doRealSeek(scanner, seekKey, forward);
> throw exception, and scanner will have no chance to close, so will cause the
> scanner leak.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-17118) StoreScanner leaked in KeyValueHeap
[
https://issues.apache.org/jira/browse/HBASE-17118?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15673161#comment-15673161
]
Anoop Sam John commented on HBASE-17118:
{code}
heap.add(scanner);
310 current = pollRealKV();
{code}
Now all these under try.. pollRealKV() also throws IOE and catch may get
called. We have added back the scanner object just above.. Still scanner
point to old object. May be just in btw these 2 lines we can do scanner =
null? It is a harmless thing anyway. WDYT?
+1
> StoreScanner leaked in KeyValueHeap
> ---
>
> Key: HBASE-17118
> URL: https://issues.apache.org/jira/browse/HBASE-17118
> Project: HBase
> Issue Type: Bug
>Affects Versions: 2.0.0
>Reporter: binlijin
>Assignee: binlijin
> Attachments: HBASE-17118-master_v1.patch,
> HBASE-17118-master_v2.patch, HBASE-17118-master_v3.patch, StoreScanner.png,
> StoreScannerLeakHeap.png
>
>
> KeyValueHeap#generalizedSeek
> KeyValueScanner scanner = current;
> while (scanner != null) {
> Cell topKey = scanner.peek();
> ..
> boolean seekResult;
> if (isLazy && heap.size() > 0) {
> // If there is only one scanner left, we don't do lazy seek.
> seekResult = scanner.requestSeek(seekKey, forward, useBloom);
> } else {
> seekResult = NonLazyKeyValueScanner.doRealSeek(scanner, seekKey,
> forward);
> }
> ..
> scanner = heap.poll();
> }
> (1) scanner = heap.poll(); Retrieves and removes the head of this queue
> (2) scanner.requestSeek(seekKey, forward, useBloom); or
> NonLazyKeyValueScanner.doRealSeek(scanner, seekKey, forward);
> throw exception, and scanner will have no chance to close, so will cause the
> scanner leak.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-17118) StoreScanner leaked in KeyValueHeap
[
https://issues.apache.org/jira/browse/HBASE-17118?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15672959#comment-15672959
]
Hadoop QA commented on HBASE-17118:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m 13s
{color} | {color:blue} Docker mode activated. {color} |
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s {color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s
{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:red}-1{color} | {color:red} test4tests {color} | {color:red} 0m 0s
{color} | {color:red} The patch doesn't appear to include any new or modified
tests. Please justify why no new tests are needed for this patch. Also please
list what manual steps were performed to verify this patch. {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 3m
15s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 35s
{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
44s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
13s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m
40s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 26s
{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
45s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 34s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m 34s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
44s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
12s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s {color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
27m 10s {color} | {color:green} Patch does not cause any errors with Hadoop
2.6.1 2.6.2 2.6.3 2.6.4 2.6.5 2.7.1 2.7.2 2.7.3 or 3.0.0-alpha1. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m
48s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 24s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 91m 17s
{color} | {color:green} hbase-server in the patch passed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
16s {color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 130m 34s {color}
| {color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=1.12.3 Server=1.12.3 Image:yetus/hbase:8d52d23 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12839310/HBASE-17118-master_v3.patch
|
| JIRA Issue | HBASE-17118 |
| Optional Tests | asflicense javac javadoc unit findbugs hadoopcheck
hbaseanti checkstyle compile |
| uname | Linux 5ef5ee3d5ebf 3.13.0-95-generic #142-Ubuntu SMP Fri Aug 12
17:00:09 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/component/dev-support/hbase-personality.sh
|
| git revision | master / 48439e5 |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.0 |
| Test Results |
https://builds.apache.org/job/PreCommit-HBASE-Build/4509/testReport/ |
| modules | C: hbase-server U: hbase-server |
| Console output |
https://builds.apache.org/job/PreCommit-HBASE-Build/4509/console |
| Powered by | Apache Yetus 0.3.0 http://yetus.apache.org |
This message was automatically generated.
> StoreScanner leaked in KeyValueHeap
> ---
>
> Key: HBASE-17118
> URL: https://issues.apache.org/jira/browse/HBASE-17118
> Project: HBase
> Issue Type: Bug
>Affects Versions: 2.0.0
>Reporter: binlijin
>Assignee: binlijin
>
[jira] [Commented] (HBASE-17118) StoreScanner leaked in KeyValueHeap
[
https://issues.apache.org/jira/browse/HBASE-17118?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15672690#comment-15672690
]
binlijin commented on HBASE-17118:
--
OK
> StoreScanner leaked in KeyValueHeap
> ---
>
> Key: HBASE-17118
> URL: https://issues.apache.org/jira/browse/HBASE-17118
> Project: HBase
> Issue Type: Bug
>Reporter: binlijin
>Assignee: binlijin
> Attachments: HBASE-17118-master_v1.patch,
> HBASE-17118-master_v2.patch, HBASE-17118-master_v3.patch, StoreScanner.png,
> StoreScannerLeakHeap.png
>
>
> KeyValueHeap#generalizedSeek
> KeyValueScanner scanner = current;
> while (scanner != null) {
> Cell topKey = scanner.peek();
> ..
> boolean seekResult;
> if (isLazy && heap.size() > 0) {
> // If there is only one scanner left, we don't do lazy seek.
> seekResult = scanner.requestSeek(seekKey, forward, useBloom);
> } else {
> seekResult = NonLazyKeyValueScanner.doRealSeek(scanner, seekKey,
> forward);
> }
> ..
> scanner = heap.poll();
> }
> (1) scanner = heap.poll(); Retrieves and removes the head of this queue
> (2) scanner.requestSeek(seekKey, forward, useBloom); or
> NonLazyKeyValueScanner.doRealSeek(scanner, seekKey, forward);
> throw exception, and scanner will have no chance to close, so will cause the
> scanner leak.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-17118) StoreScanner leaked in KeyValueHeap
[
https://issues.apache.org/jira/browse/HBASE-17118?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15672684#comment-15672684
]
Ted Yu commented on HBASE-17118:
lgtm
{code}
338 LOG.error("close KeyValueScanner error", ce);
{code}
The above can be a warning.
> StoreScanner leaked in KeyValueHeap
> ---
>
> Key: HBASE-17118
> URL: https://issues.apache.org/jira/browse/HBASE-17118
> Project: HBase
> Issue Type: Bug
>Reporter: binlijin
>Assignee: binlijin
> Attachments: HBASE-17118-master_v1.patch,
> HBASE-17118-master_v2.patch, StoreScanner.png, StoreScannerLeakHeap.png
>
>
> KeyValueHeap#generalizedSeek
> KeyValueScanner scanner = current;
> while (scanner != null) {
> Cell topKey = scanner.peek();
> ..
> boolean seekResult;
> if (isLazy && heap.size() > 0) {
> // If there is only one scanner left, we don't do lazy seek.
> seekResult = scanner.requestSeek(seekKey, forward, useBloom);
> } else {
> seekResult = NonLazyKeyValueScanner.doRealSeek(scanner, seekKey,
> forward);
> }
> ..
> scanner = heap.poll();
> }
> (1) scanner = heap.poll(); Retrieves and removes the head of this queue
> (2) scanner.requestSeek(seekKey, forward, useBloom); or
> NonLazyKeyValueScanner.doRealSeek(scanner, seekKey, forward);
> throw exception, and scanner will have no chance to close, so will cause the
> scanner leak.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-17118) StoreScanner leaked in KeyValueHeap
[
https://issues.apache.org/jira/browse/HBASE-17118?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15672658#comment-15672658
]
binlijin commented on HBASE-17118:
--
Ok, done.
> StoreScanner leaked in KeyValueHeap
> ---
>
> Key: HBASE-17118
> URL: https://issues.apache.org/jira/browse/HBASE-17118
> Project: HBase
> Issue Type: Bug
>Reporter: binlijin
>Assignee: binlijin
> Attachments: HBASE-17118-master_v1.patch,
> HBASE-17118-master_v2.patch, StoreScanner.png, StoreScannerLeakHeap.png
>
>
> KeyValueHeap#generalizedSeek
> KeyValueScanner scanner = current;
> while (scanner != null) {
> Cell topKey = scanner.peek();
> ..
> boolean seekResult;
> if (isLazy && heap.size() > 0) {
> // If there is only one scanner left, we don't do lazy seek.
> seekResult = scanner.requestSeek(seekKey, forward, useBloom);
> } else {
> seekResult = NonLazyKeyValueScanner.doRealSeek(scanner, seekKey,
> forward);
> }
> ..
> scanner = heap.poll();
> }
> (1) scanner = heap.poll(); Retrieves and removes the head of this queue
> (2) scanner.requestSeek(seekKey, forward, useBloom); or
> NonLazyKeyValueScanner.doRealSeek(scanner, seekKey, forward);
> throw exception, and scanner will have no chance to close, so will cause the
> scanner leak.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-17118) StoreScanner leaked in KeyValueHeap
[
https://issues.apache.org/jira/browse/HBASE-17118?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15672657#comment-15672657
]
binlijin commented on HBASE-17118:
--
Ok, done
> StoreScanner leaked in KeyValueHeap
> ---
>
> Key: HBASE-17118
> URL: https://issues.apache.org/jira/browse/HBASE-17118
> Project: HBase
> Issue Type: Bug
>Reporter: binlijin
>Assignee: binlijin
> Attachments: HBASE-17118-master_v1.patch,
> HBASE-17118-master_v2.patch, StoreScanner.png, StoreScannerLeakHeap.png
>
>
> KeyValueHeap#generalizedSeek
> KeyValueScanner scanner = current;
> while (scanner != null) {
> Cell topKey = scanner.peek();
> ..
> boolean seekResult;
> if (isLazy && heap.size() > 0) {
> // If there is only one scanner left, we don't do lazy seek.
> seekResult = scanner.requestSeek(seekKey, forward, useBloom);
> } else {
> seekResult = NonLazyKeyValueScanner.doRealSeek(scanner, seekKey,
> forward);
> }
> ..
> scanner = heap.poll();
> }
> (1) scanner = heap.poll(); Retrieves and removes the head of this queue
> (2) scanner.requestSeek(seekKey, forward, useBloom); or
> NonLazyKeyValueScanner.doRealSeek(scanner, seekKey, forward);
> throw exception, and scanner will have no chance to close, so will cause the
> scanner leak.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-17118) StoreScanner leaked in KeyValueHeap
[
https://issues.apache.org/jira/browse/HBASE-17118?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15672525#comment-15672525
]
Duo Zhang commented on HBASE-17118:
---
Usually we should avoid catching Throwable, catching Exception is enough.
[~tedyu] For java7 or newer you can still get the original Throwable and you
can get the exception of scanner.close by calling the getSuppressed method of
the Throwable, and the suppressed exceptions will also be printed when calling
printStackTrace. But I agree that we should catch the exception of
scanner.close since it is a read operation.
Thanks.
> StoreScanner leaked in KeyValueHeap
> ---
>
> Key: HBASE-17118
> URL: https://issues.apache.org/jira/browse/HBASE-17118
> Project: HBase
> Issue Type: Bug
>Reporter: binlijin
> Attachments: HBASE-17118-master_v1.patch, StoreScanner.png,
> StoreScannerLeakHeap.png
>
>
> KeyValueHeap#generalizedSeek
> KeyValueScanner scanner = current;
> while (scanner != null) {
> Cell topKey = scanner.peek();
> ..
> boolean seekResult;
> if (isLazy && heap.size() > 0) {
> // If there is only one scanner left, we don't do lazy seek.
> seekResult = scanner.requestSeek(seekKey, forward, useBloom);
> } else {
> seekResult = NonLazyKeyValueScanner.doRealSeek(scanner, seekKey,
> forward);
> }
> ..
> scanner = heap.poll();
> }
> (1) scanner = heap.poll(); Retrieves and removes the head of this queue
> (2) scanner.requestSeek(seekKey, forward, useBloom); or
> NonLazyKeyValueScanner.doRealSeek(scanner, seekKey, forward);
> throw exception, and scanner will have no chance to close, so will cause the
> scanner leak.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-17118) StoreScanner leaked in KeyValueHeap
[
https://issues.apache.org/jira/browse/HBASE-17118?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15672497#comment-15672497
]
Ted Yu commented on HBASE-17118:
{code}
332 scanner.close();
{code}
Exception from close() call should be caught (and logged).
Otherwise this potential exception would eclipse the Throwable.
> StoreScanner leaked in KeyValueHeap
> ---
>
> Key: HBASE-17118
> URL: https://issues.apache.org/jira/browse/HBASE-17118
> Project: HBase
> Issue Type: Bug
>Reporter: binlijin
> Attachments: HBASE-17118-master_v1.patch, StoreScanner.png,
> StoreScannerLeakHeap.png
>
>
> KeyValueHeap#generalizedSeek
> KeyValueScanner scanner = current;
> while (scanner != null) {
> Cell topKey = scanner.peek();
> ..
> boolean seekResult;
> if (isLazy && heap.size() > 0) {
> // If there is only one scanner left, we don't do lazy seek.
> seekResult = scanner.requestSeek(seekKey, forward, useBloom);
> } else {
> seekResult = NonLazyKeyValueScanner.doRealSeek(scanner, seekKey,
> forward);
> }
> ..
> scanner = heap.poll();
> }
> (1) scanner = heap.poll(); Retrieves and removes the head of this queue
> (2) scanner.requestSeek(seekKey, forward, useBloom); or
> NonLazyKeyValueScanner.doRealSeek(scanner, seekKey, forward);
> throw exception, and scanner will have no chance to close, so will cause the
> scanner leak.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-17118) StoreScanner leaked in KeyValueHeap
[
https://issues.apache.org/jira/browse/HBASE-17118?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15672406#comment-15672406
]
binlijin commented on HBASE-17118:
--
We find the problem so later, and could find the HFile which cause the
IllegalArgumentException, and have no clue about the IllegalArgumentException
yet.
> StoreScanner leaked in KeyValueHeap
> ---
>
> Key: HBASE-17118
> URL: https://issues.apache.org/jira/browse/HBASE-17118
> Project: HBase
> Issue Type: Bug
>Reporter: binlijin
> Attachments: StoreScanner.png, StoreScannerLeakHeap.png
>
>
> KeyValueHeap#generalizedSeek
> KeyValueScanner scanner = current;
> while (scanner != null) {
> Cell topKey = scanner.peek();
> ..
> boolean seekResult;
> if (isLazy && heap.size() > 0) {
> // If there is only one scanner left, we don't do lazy seek.
> seekResult = scanner.requestSeek(seekKey, forward, useBloom);
> } else {
> seekResult = NonLazyKeyValueScanner.doRealSeek(scanner, seekKey,
> forward);
> }
> ..
> scanner = heap.poll();
> }
> (1) scanner = heap.poll(); Retrieves and removes the head of this queue
> (2) scanner.requestSeek(seekKey, forward, useBloom); or
> NonLazyKeyValueScanner.doRealSeek(scanner, seekKey, forward);
> throw exception, and scanner will have no change to close, so will cause the
> scanner leak.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-17118) StoreScanner leaked in KeyValueHeap
[
https://issues.apache.org/jira/browse/HBASE-17118?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15672402#comment-15672402
]
binlijin commented on HBASE-17118:
--
We encounter the problem, and we find the problem at 2016.11.10 through
unnormal gc, and dump the heap, and find the StoreScanner leak, through the
heap dump we find the StoreScanner create at 2016-11-02
{code}
2016-11-02 07:36:15,056 ERROR [B.defaultRpcServer.handler=5,queue=5,port=16020]
ipc.RpcServer: Unexpected throwable object
java.lang.IllegalArgumentException
at java.nio.Buffer.limit(Buffer.java:267)
at
org.apache.hadoop.hbase.nio.SingleByteBuff.limit(SingleByteBuff.java:91)
at
org.apache.hadoop.hbase.nio.SingleByteBuff.limit(SingleByteBuff.java:33)
at
org.apache.hadoop.hbase.io.hfile.HFileBlock.getBufferReadOnly(HFileBlock.java:393)
at
org.apache.hadoop.hbase.io.hfile.ChecksumUtil.validateBlockChecksum(ChecksumUtil.java:158)
at
org.apache.hadoop.hbase.io.hfile.HFileBlock$FSReaderImpl.validateBlockChecksum(HFileBlock.java:1737)
at
org.apache.hadoop.hbase.io.hfile.HFileBlock$FSReaderImpl.readBlockDataInternal(HFileBlock.java:1686)
at
org.apache.hadoop.hbase.io.hfile.HFileBlock$FSReaderImpl.readBlockData(HFileBlock.java:1495)
at
org.apache.hadoop.hbase.io.hfile.HFileReaderImpl.readBlock(HFileReaderImpl.java:1440)
at
org.apache.hadoop.hbase.io.hfile.HFileBlockIndex$CellBasedKeyBlockIndexReader.loadDataBlockWithScanInfo(HFileBlockIndex.java:322)
at
org.apache.hadoop.hbase.io.hfile.HFileReaderImpl$HFileScannerImpl.seekTo(HFileReaderImpl.java:817)
at
org.apache.hadoop.hbase.io.hfile.HFileReaderImpl$HFileScannerImpl.reseekTo(HFileReaderImpl.java:798)
at
org.apache.hadoop.hbase.regionserver.StoreFileScanner.reseekAtOrAfter(StoreFileScanner.java:263)
at
org.apache.hadoop.hbase.regionserver.StoreFileScanner.reseek(StoreFileScanner.java:180)
at
org.apache.hadoop.hbase.regionserver.NonLazyKeyValueScanner.doRealSeek(NonLazyKeyValueScanner.java:55)
at
org.apache.hadoop.hbase.regionserver.KeyValueHeap.generalizedSeek(KeyValueHeap.java:323)
at
org.apache.hadoop.hbase.regionserver.KeyValueHeap.reseek(KeyValueHeap.java:267)
at
org.apache.hadoop.hbase.regionserver.StoreScanner.reseek(StoreScanner.java:824)
at
org.apache.hadoop.hbase.regionserver.NonLazyKeyValueScanner.doRealSeek(NonLazyKeyValueScanner.java:55)
at
org.apache.hadoop.hbase.regionserver.NonLazyKeyValueScanner.requestSeek(NonLazyKeyValueScanner.java:39)
at
org.apache.hadoop.hbase.regionserver.KeyValueHeap.generalizedSeek(KeyValueHeap.java:321)
at
org.apache.hadoop.hbase.regionserver.KeyValueHeap.requestSeek(KeyValueHeap.java:279)
at
org.apache.hadoop.hbase.regionserver.HRegion$RegionScannerImpl.joinedHeapMayHaveData(HRegion.java:5904)
at
org.apache.hadoop.hbase.regionserver.HRegion$RegionScannerImpl.nextInternal(HRegion.java:5845)
at
org.apache.hadoop.hbase.regionserver.HRegion$RegionScannerImpl.nextRaw(HRegion.java:5589)
at
org.apache.hadoop.hbase.regionserver.RSRpcServices.scan(RSRpcServices.java:2644)
at
org.apache.hadoop.hbase.protobuf.generated.ClientProtos$ClientService$2.callBlockingMethod(ClientProtos.java:32205)
at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:839)
at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:102)
at
org.apache.hadoop.hbase.ipc.RpcExecutor.consumerLoop(RpcExecutor.java:133)
at org.apache.hadoop.hbase.ipc.RpcExecutor$1.run(RpcExecutor.java:108)
at java.lang.Thread.run(Thread.java:756)
2016-11-02 07:36:15,056 DEBUG [B.defaultRpcServer.handler=5,queue=5,port=16020]
ipc.RpcServer: B.defaultRpcServer.handler=5,queue=5,port=16020: callId: 115
service: ClientService methodName: Scan size: 29 connection: 11.180.36.86:54872
java.io.IOException
at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:894)
at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:102)
at
org.apache.hadoop.hbase.ipc.RpcExecutor.consumerLoop(RpcExecutor.java:133)
at org.apache.hadoop.hbase.ipc.RpcExecutor$1.run(RpcExecutor.java:108)
at java.lang.Thread.run(Thread.java:756)
Caused by: java.lang.IllegalArgumentException
at java.nio.Buffer.limit(Buffer.java:267)
at
org.apache.hadoop.hbase.nio.SingleByteBuff.limit(SingleByteBuff.java:91)
at
org.apache.hadoop.hbase.nio.SingleByteBuff.limit(SingleByteBuff.java:33)
at
org.apache.hadoop.hbase.io.hfile.HFileBlock.getBufferReadOnly(HFileBlock.java:393)
at
org.apache.hadoop.hbase.io.hfile.ChecksumUtil.validateBlockChecksum(ChecksumUtil.java:158)
at
org.apache.hadoop.hbase.io.hfile.HFileBlock$FSReaderImpl.validateBlockChecksum(HFileBlock.java:1737)
at
