[jira] [Updated] (HBASE-16643) Reverse scanner heap creation may not allow MSLAB closure due to inproper ref counting of segments
[
https://issues.apache.org/jira/browse/HBASE-16643?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
ramkrishna.s.vasudevan updated HBASE-16643:
---
Fix Version/s: 2.0.0
Status: Patch Available (was: Open)
> Reverse scanner heap creation may not allow MSLAB closure due to inproper ref
> counting of segments
> --
>
> Key: HBASE-16643
> URL: https://issues.apache.org/jira/browse/HBASE-16643
> Project: HBase
> Issue Type: Bug
>Reporter: ramkrishna.s.vasudevan
>Assignee: ramkrishna.s.vasudevan
>Priority: Critical
> Fix For: 2.0.0
>
> Attachments: HBASE-16643.patch
>
>
> In the reverse scanner case,
> While doing 'initBackwardHeapIfNeeded' in MemstoreScanner for setting the
> backward heap, we do a
> {code}
> if ((backwardHeap == null) && (forwardHeap != null)) {
> forwardHeap.close();
> forwardHeap = null;
> // before building the heap seek for the relevant key on the scanners,
> // for the heap to be built from the scanners correctly
> for (KeyValueScanner scan : scanners) {
> if (toLast) {
> res |= scan.seekToLastRow();
> } else {
> res |= scan.backwardSeek(cell);
> }
> }
> {code}
> forwardHeap.close(). This would internally decrement the MSLAB ref counter
> for the current active segment and snapshot segment.
> When the scan is actually closed again we do close() and that will again
> decrement the count. Here chances are there that the count would go negative
> and hence the actual MSLAB closure that checks for refCount==0 will fail.
> Apart from this, when the refCount becomes 0 after the firstClose if any
> other thread requests to close the segment, then we will end up in corrupted
> segment because the segment could be put back to the MSLAB pool.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HBASE-16643) Reverse scanner heap creation may not allow MSLAB closure due to inproper ref counting of segments
[
https://issues.apache.org/jira/browse/HBASE-16643?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
ramkrishna.s.vasudevan updated HBASE-16643:
---
Attachment: HBASE-16643.patch
Patch for 2.0. This makes MemstoreScanner to create either a KeyValueHeap or
ReversedKeyValueHeap based on the type of scan we are doing. Since
MemstoreScanner is now a Collection of SegmentScanner we need to apply the
logic of backWardSeek or seekToLastRow as we do for the actual StoreScanners
before we add on to the heap.
Also removed some unused imports in related test cases.
Made MemstoreScannerType as a first class enum. While creating the
memstoreScanner this type is passed on to decide the type of heap to be
created. Removes the unwanted code that was closing a heap and creating a
backward heap that was causing the actual problem mentioned in the JIRA title.
Ping [~anastas], [~eshcar], [~anoop.hbase] for reviews.
> Reverse scanner heap creation may not allow MSLAB closure due to inproper ref
> counting of segments
> --
>
> Key: HBASE-16643
> URL: https://issues.apache.org/jira/browse/HBASE-16643
> Project: HBase
> Issue Type: Bug
>Reporter: ramkrishna.s.vasudevan
>Assignee: ramkrishna.s.vasudevan
>Priority: Critical
> Attachments: HBASE-16643.patch
>
>
> In the reverse scanner case,
> While doing 'initBackwardHeapIfNeeded' in MemstoreScanner for setting the
> backward heap, we do a
> {code}
> if ((backwardHeap == null) && (forwardHeap != null)) {
> forwardHeap.close();
> forwardHeap = null;
> // before building the heap seek for the relevant key on the scanners,
> // for the heap to be built from the scanners correctly
> for (KeyValueScanner scan : scanners) {
> if (toLast) {
> res |= scan.seekToLastRow();
> } else {
> res |= scan.backwardSeek(cell);
> }
> }
> {code}
> forwardHeap.close(). This would internally decrement the MSLAB ref counter
> for the current active segment and snapshot segment.
> When the scan is actually closed again we do close() and that will again
> decrement the count. Here chances are there that the count would go negative
> and hence the actual MSLAB closure that checks for refCount==0 will fail.
> Apart from this, when the refCount becomes 0 after the firstClose if any
> other thread requests to close the segment, then we will end up in corrupted
> segment because the segment could be put back to the MSLAB pool.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HBASE-16643) Reverse scanner heap creation may not allow MSLAB closure due to inproper ref counting of segments
[
https://issues.apache.org/jira/browse/HBASE-16643?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
ramkrishna.s.vasudevan updated HBASE-16643:
---
Description:
In the reverse scanner case,
While doing 'initBackwardHeapIfNeeded' in MemstoreScanner for setting the
backward heap, we do a
{code}
if ((backwardHeap == null) && (forwardHeap != null)) {
forwardHeap.close();
forwardHeap = null;
// before building the heap seek for the relevant key on the scanners,
// for the heap to be built from the scanners correctly
for (KeyValueScanner scan : scanners) {
if (toLast) {
res |= scan.seekToLastRow();
} else {
res |= scan.backwardSeek(cell);
}
}
{code}
forwardHeap.close(). This would internally decrement the MSLAB ref counter for
the current active segment and snapshot segment.
When the scan is actually closed again we do close() and that will again
decrement the count. Here chances are there that the count would go negative
and hence the actual MSLAB closure that checks for refCount==0 will fail. Apart
from this, when the refCount becomes 0 after the firstClose if any other thread
requests to close the segment, then we will end up in corrupted segment because
the segment could be put back to the MSLAB pool.
was:
In the reverse scanner case,
While doing 'initBackwardHeapIfNeeded' in MemstoreScanner for setting the
backward heap, we do a
{code}
if ((backwardHeap == null) && (forwardHeap != null)) {
forwardHeap.close();
forwardHeap = null;
// before building the heap seek for the relevant key on the scanners,
// for the heap to be built from the scanners correctly
for (KeyValueScanner scan : scanners) {
if (toLast) {
res |= scan.seekToLastRow();
} else {
res |= scan.backwardSeek(cell);
}
}
{code}
forwardHeap.close(). This would internally decrement the MSLAB ref counter for
the current active segment and snapshot segment.
When the scan is actually closed again we do close() and that will again
decrement the count. Here chances are there that the count would go negative
and hence the actual MSLAB closure that checks for refCount==0 will fail. Apart
from this, when the refCount becomes 0 after the firstClose if any other thread
requests to close the segment, then we will end up in corrupted segment.
> Reverse scanner heap creation may not allow MSLAB closure due to inproper ref
> counting of segments
> --
>
> Key: HBASE-16643
> URL: https://issues.apache.org/jira/browse/HBASE-16643
> Project: HBase
> Issue Type: Bug
>Reporter: ramkrishna.s.vasudevan
>Assignee: ramkrishna.s.vasudevan
>Priority: Critical
>
> In the reverse scanner case,
> While doing 'initBackwardHeapIfNeeded' in MemstoreScanner for setting the
> backward heap, we do a
> {code}
> if ((backwardHeap == null) && (forwardHeap != null)) {
> forwardHeap.close();
> forwardHeap = null;
> // before building the heap seek for the relevant key on the scanners,
> // for the heap to be built from the scanners correctly
> for (KeyValueScanner scan : scanners) {
> if (toLast) {
> res |= scan.seekToLastRow();
> } else {
> res |= scan.backwardSeek(cell);
> }
> }
> {code}
> forwardHeap.close(). This would internally decrement the MSLAB ref counter
> for the current active segment and snapshot segment.
> When the scan is actually closed again we do close() and that will again
> decrement the count. Here chances are there that the count would go negative
> and hence the actual MSLAB closure that checks for refCount==0 will fail.
> Apart from this, when the refCount becomes 0 after the firstClose if any
> other thread requests to close the segment, then we will end up in corrupted
> segment because the segment could be put back to the MSLAB pool.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
