[jira] [Commented] (HBASE-6900) RegionScanner.reseek() creates NPE when a flush or compaction happens before the reseek.
[
https://issues.apache.org/jira/browse/HBASE-6900?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13490370#comment-13490370
]
Hudson commented on HBASE-6900:
---
Integrated in HBase-0.94-security-on-Hadoop-23 #9 (See
[https://builds.apache.org/job/HBase-0.94-security-on-Hadoop-23/9/])
HBASE-6900 RegionScanner.reseek() creates NPE when a flush or compaction
happens before the reseek. (Revision 1394378)
Result = FAILURE
larsh :
Files :
*
/hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/regionserver/StoreScanner.java
*
/hbase/branches/0.94/src/test/java/org/apache/hadoop/hbase/regionserver/TestHRegion.java
RegionScanner.reseek() creates NPE when a flush or compaction happens before
the reseek.
Key: HBASE-6900
URL: https://issues.apache.org/jira/browse/HBASE-6900
Project: HBase
Issue Type: Bug
Reporter: ramkrishna.s.vasudevan
Assignee: ramkrishna.s.vasudevan
Fix For: 0.94.2, 0.96.0
Attachments: 6900-test.txt, HBASE-6900_1.patch, HBASE-6900.patch
HBASE-5520 introduced reseek() on the RegionScanner.
Now when a scanner is created we have the StoreScanner heap. After this if a
flush or compaction happens parallely all the StoreScannerObservers are
cleared so that whenever a new next() call happens we tend to recreate the
scanner based on the latest store files.
The reseek() in StoreScanner expects the heap not to be null because always
reseek would be called from next()
{code}
public synchronized boolean reseek(KeyValue kv) throws IOException {
//Heap cannot be null, because this is only called from next() which
//guarantees that heap will never be null before this call.
if (explicitColumnQuery lazySeekEnabledGlobally) {
return heap.requestSeek(kv, true, useRowColBloom);
} else {
return heap.reseek(kv);
}
}
{code}
Now when we call RegionScanner.reseek() directly using CPs we tend to get a
NPE. In our case it happened when a major compaction was going on. I will
also attach a testcase to show the problem.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6900) RegionScanner.reseek() creates NPE when a flush or compaction happens before the reseek.
[
https://issues.apache.org/jira/browse/HBASE-6900?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13470046#comment-13470046
]
ramkrishna.s.vasudevan commented on HBASE-6900:
---
@Lars
Its better if we get it in 0.94.2. I was once again checking the lastTop
behaviour. Even if two times updateReaders happen i feel the lastTop will not
be null. Verified the same with a testcase.
{code}
// All public synchronized API calls will call 'checkReseek' which will cause
// the scanner stack to reseek if this.heap==null this.lastTop != null.
// But if two calls to updateReaders() happen without a 'next' or 'peek'
then we
// will end up calling this.peek() which would cause a reseek in the middle
of a updateReaders
// which is NOT what we want, not to mention could cause an NPE. So we
early out here.
if (this.heap == null) return;
{code}
So i feel the patch that you gave should be fine without additional checks. +1
on it.
Lars if you can commit that would be great because i cannot do the commit from
office and also i may not be able to do it today.
RegionScanner.reseek() creates NPE when a flush or compaction happens before
the reseek.
Key: HBASE-6900
URL: https://issues.apache.org/jira/browse/HBASE-6900
Project: HBase
Issue Type: Bug
Reporter: ramkrishna.s.vasudevan
Assignee: ramkrishna.s.vasudevan
Fix For: 0.94.2, 0.96.0
Attachments: 6900-test.txt, HBASE-6900_1.patch, HBASE-6900.patch
HBASE-5520 introduced reseek() on the RegionScanner.
Now when a scanner is created we have the StoreScanner heap. After this if a
flush or compaction happens parallely all the StoreScannerObservers are
cleared so that whenever a new next() call happens we tend to recreate the
scanner based on the latest store files.
The reseek() in StoreScanner expects the heap not to be null because always
reseek would be called from next()
{code}
public synchronized boolean reseek(KeyValue kv) throws IOException {
//Heap cannot be null, because this is only called from next() which
//guarantees that heap will never be null before this call.
if (explicitColumnQuery lazySeekEnabledGlobally) {
return heap.requestSeek(kv, true, useRowColBloom);
} else {
return heap.reseek(kv);
}
}
{code}
Now when we call RegionScanner.reseek() directly using CPs we tend to get a
NPE. In our case it happened when a major compaction was going on. I will
also attach a testcase to show the problem.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6900) RegionScanner.reseek() creates NPE when a flush or compaction happens before the reseek.
[
https://issues.apache.org/jira/browse/HBASE-6900?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13470102#comment-13470102
]
Hudson commented on HBASE-6900:
---
Integrated in HBase-TRUNK #3429 (See
[https://builds.apache.org/job/HBase-TRUNK/3429/])
HBASE-6900 RegionScanner.reseek() creates NPE when a flush or compaction
happens before the reseek. (Revision 1394377)
Result = FAILURE
larsh :
Files :
*
/hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/StoreScanner.java
*
/hbase/trunk/hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestHRegion.java
RegionScanner.reseek() creates NPE when a flush or compaction happens before
the reseek.
Key: HBASE-6900
URL: https://issues.apache.org/jira/browse/HBASE-6900
Project: HBase
Issue Type: Bug
Reporter: ramkrishna.s.vasudevan
Assignee: ramkrishna.s.vasudevan
Fix For: 0.94.2, 0.96.0
Attachments: 6900-test.txt, HBASE-6900_1.patch, HBASE-6900.patch
HBASE-5520 introduced reseek() on the RegionScanner.
Now when a scanner is created we have the StoreScanner heap. After this if a
flush or compaction happens parallely all the StoreScannerObservers are
cleared so that whenever a new next() call happens we tend to recreate the
scanner based on the latest store files.
The reseek() in StoreScanner expects the heap not to be null because always
reseek would be called from next()
{code}
public synchronized boolean reseek(KeyValue kv) throws IOException {
//Heap cannot be null, because this is only called from next() which
//guarantees that heap will never be null before this call.
if (explicitColumnQuery lazySeekEnabledGlobally) {
return heap.requestSeek(kv, true, useRowColBloom);
} else {
return heap.reseek(kv);
}
}
{code}
Now when we call RegionScanner.reseek() directly using CPs we tend to get a
NPE. In our case it happened when a major compaction was going on. I will
also attach a testcase to show the problem.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6900) RegionScanner.reseek() creates NPE when a flush or compaction happens before the reseek.
[
https://issues.apache.org/jira/browse/HBASE-6900?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13470101#comment-13470101
]
Hudson commented on HBASE-6900:
---
Integrated in HBase-0.94 #507 (See
[https://builds.apache.org/job/HBase-0.94/507/])
HBASE-6900 RegionScanner.reseek() creates NPE when a flush or compaction
happens before the reseek. (Revision 1394378)
Result = FAILURE
larsh :
Files :
*
/hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/regionserver/StoreScanner.java
*
/hbase/branches/0.94/src/test/java/org/apache/hadoop/hbase/regionserver/TestHRegion.java
RegionScanner.reseek() creates NPE when a flush or compaction happens before
the reseek.
Key: HBASE-6900
URL: https://issues.apache.org/jira/browse/HBASE-6900
Project: HBase
Issue Type: Bug
Reporter: ramkrishna.s.vasudevan
Assignee: ramkrishna.s.vasudevan
Fix For: 0.94.2, 0.96.0
Attachments: 6900-test.txt, HBASE-6900_1.patch, HBASE-6900.patch
HBASE-5520 introduced reseek() on the RegionScanner.
Now when a scanner is created we have the StoreScanner heap. After this if a
flush or compaction happens parallely all the StoreScannerObservers are
cleared so that whenever a new next() call happens we tend to recreate the
scanner based on the latest store files.
The reseek() in StoreScanner expects the heap not to be null because always
reseek would be called from next()
{code}
public synchronized boolean reseek(KeyValue kv) throws IOException {
//Heap cannot be null, because this is only called from next() which
//guarantees that heap will never be null before this call.
if (explicitColumnQuery lazySeekEnabledGlobally) {
return heap.requestSeek(kv, true, useRowColBloom);
} else {
return heap.reseek(kv);
}
}
{code}
Now when we call RegionScanner.reseek() directly using CPs we tend to get a
NPE. In our case it happened when a major compaction was going on. I will
also attach a testcase to show the problem.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6900) RegionScanner.reseek() creates NPE when a flush or compaction happens before the reseek.
[
https://issues.apache.org/jira/browse/HBASE-6900?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13470875#comment-13470875
]
Hudson commented on HBASE-6900:
---
Integrated in HBase-0.94-security #60 (See
[https://builds.apache.org/job/HBase-0.94-security/60/])
HBASE-6900 RegionScanner.reseek() creates NPE when a flush or compaction
happens before the reseek. (Revision 1394378)
Result = FAILURE
larsh :
Files :
*
/hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/regionserver/StoreScanner.java
*
/hbase/branches/0.94/src/test/java/org/apache/hadoop/hbase/regionserver/TestHRegion.java
RegionScanner.reseek() creates NPE when a flush or compaction happens before
the reseek.
Key: HBASE-6900
URL: https://issues.apache.org/jira/browse/HBASE-6900
Project: HBase
Issue Type: Bug
Reporter: ramkrishna.s.vasudevan
Assignee: ramkrishna.s.vasudevan
Fix For: 0.94.2, 0.96.0
Attachments: 6900-test.txt, HBASE-6900_1.patch, HBASE-6900.patch
HBASE-5520 introduced reseek() on the RegionScanner.
Now when a scanner is created we have the StoreScanner heap. After this if a
flush or compaction happens parallely all the StoreScannerObservers are
cleared so that whenever a new next() call happens we tend to recreate the
scanner based on the latest store files.
The reseek() in StoreScanner expects the heap not to be null because always
reseek would be called from next()
{code}
public synchronized boolean reseek(KeyValue kv) throws IOException {
//Heap cannot be null, because this is only called from next() which
//guarantees that heap will never be null before this call.
if (explicitColumnQuery lazySeekEnabledGlobally) {
return heap.requestSeek(kv, true, useRowColBloom);
} else {
return heap.reseek(kv);
}
}
{code}
Now when we call RegionScanner.reseek() directly using CPs we tend to get a
NPE. In our case it happened when a major compaction was going on. I will
also attach a testcase to show the problem.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6900) RegionScanner.reseek() creates NPE when a flush or compaction happens before the reseek.
[
https://issues.apache.org/jira/browse/HBASE-6900?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13469463#comment-13469463
]
Lars Hofhansl commented on HBASE-6900:
--
Thanks for explaining Ram.
Are you sure you care for the return value of checkReseek?
After you explanation it seems like you can just call checkReseek always, and
then always seek to the kv passed it. Like this:
{code}
@Override
public synchronized boolean reseek(KeyValue kv) throws IOException {
//Heap will not be null, if this is called from next() which.
//If called from RegionScanner.reseek(...) make sure the scanner
//stack is reset if needed.
checkReseek();
if (explicitColumnQuery lazySeekEnabledGlobally) {
return heap.requestSeek(kv, true, useRowColBloom);
} else {
return heap.reseek(kv);
}
}
{code}
RegionScanner.reseek() creates NPE when a flush or compaction happens before
the reseek.
Key: HBASE-6900
URL: https://issues.apache.org/jira/browse/HBASE-6900
Project: HBase
Issue Type: Bug
Reporter: ramkrishna.s.vasudevan
Assignee: ramkrishna.s.vasudevan
Fix For: 0.94.2, 0.96.0
Attachments: HBASE-6900_1.patch, HBASE-6900.patch
HBASE-5520 introduced reseek() on the RegionScanner.
Now when a scanner is created we have the StoreScanner heap. After this if a
flush or compaction happens parallely all the StoreScannerObservers are
cleared so that whenever a new next() call happens we tend to recreate the
scanner based on the latest store files.
The reseek() in StoreScanner expects the heap not to be null because always
reseek would be called from next()
{code}
public synchronized boolean reseek(KeyValue kv) throws IOException {
//Heap cannot be null, because this is only called from next() which
//guarantees that heap will never be null before this call.
if (explicitColumnQuery lazySeekEnabledGlobally) {
return heap.requestSeek(kv, true, useRowColBloom);
} else {
return heap.reseek(kv);
}
}
{code}
Now when we call RegionScanner.reseek() directly using CPs we tend to get a
NPE. In our case it happened when a major compaction was going on. I will
also attach a testcase to show the problem.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6900) RegionScanner.reseek() creates NPE when a flush or compaction happens before the reseek.
[
https://issues.apache.org/jira/browse/HBASE-6900?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13469517#comment-13469517
]
ramkrishna.s.vasudevan commented on HBASE-6900:
---
I was just seeing what if the lastTop was null so that checkReseek doesn't
reset the heap. If that will not happen then the above change should be fine
Lars. :)
Thank you.
RegionScanner.reseek() creates NPE when a flush or compaction happens before
the reseek.
Key: HBASE-6900
URL: https://issues.apache.org/jira/browse/HBASE-6900
Project: HBase
Issue Type: Bug
Reporter: ramkrishna.s.vasudevan
Assignee: ramkrishna.s.vasudevan
Fix For: 0.94.2, 0.96.0
Attachments: HBASE-6900_1.patch, HBASE-6900.patch
HBASE-5520 introduced reseek() on the RegionScanner.
Now when a scanner is created we have the StoreScanner heap. After this if a
flush or compaction happens parallely all the StoreScannerObservers are
cleared so that whenever a new next() call happens we tend to recreate the
scanner based on the latest store files.
The reseek() in StoreScanner expects the heap not to be null because always
reseek would be called from next()
{code}
public synchronized boolean reseek(KeyValue kv) throws IOException {
//Heap cannot be null, because this is only called from next() which
//guarantees that heap will never be null before this call.
if (explicitColumnQuery lazySeekEnabledGlobally) {
return heap.requestSeek(kv, true, useRowColBloom);
} else {
return heap.reseek(kv);
}
}
{code}
Now when we call RegionScanner.reseek() directly using CPs we tend to get a
NPE. In our case it happened when a major compaction was going on. I will
also attach a testcase to show the problem.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6900) RegionScanner.reseek() creates NPE when a flush or compaction happens before the reseek.
[
https://issues.apache.org/jira/browse/HBASE-6900?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13469524#comment-13469524
]
Lars Hofhansl commented on HBASE-6900:
--
Even if that happens, don't you always want to reseek to the kv that passed in?
RegionScanner.reseek() creates NPE when a flush or compaction happens before
the reseek.
Key: HBASE-6900
URL: https://issues.apache.org/jira/browse/HBASE-6900
Project: HBase
Issue Type: Bug
Reporter: ramkrishna.s.vasudevan
Assignee: ramkrishna.s.vasudevan
Fix For: 0.94.2, 0.96.0
Attachments: HBASE-6900_1.patch, HBASE-6900.patch
HBASE-5520 introduced reseek() on the RegionScanner.
Now when a scanner is created we have the StoreScanner heap. After this if a
flush or compaction happens parallely all the StoreScannerObservers are
cleared so that whenever a new next() call happens we tend to recreate the
scanner based on the latest store files.
The reseek() in StoreScanner expects the heap not to be null because always
reseek would be called from next()
{code}
public synchronized boolean reseek(KeyValue kv) throws IOException {
//Heap cannot be null, because this is only called from next() which
//guarantees that heap will never be null before this call.
if (explicitColumnQuery lazySeekEnabledGlobally) {
return heap.requestSeek(kv, true, useRowColBloom);
} else {
return heap.reseek(kv);
}
}
{code}
Now when we call RegionScanner.reseek() directly using CPs we tend to get a
NPE. In our case it happened when a major compaction was going on. I will
also attach a testcase to show the problem.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6900) RegionScanner.reseek() creates NPE when a flush or compaction happens before the reseek.
[
https://issues.apache.org/jira/browse/HBASE-6900?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13469650#comment-13469650
]
Hadoop QA commented on HBASE-6900:
--
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12547805/6900-test.txt
against trunk revision .
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 3 new
or modified tests.
{color:green}+1 hadoop2.0{color}. The patch compiles against the hadoop
2.0 profile.
{color:red}-1 javadoc{color}. The javadoc tool appears to have generated
81 warning messages.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:red}-1 findbugs{color}. The patch appears to introduce 5 new
Findbugs (version 1.3.9) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:red}-1 core tests{color}. The patch failed these unit tests:
org.apache.hadoop.hbase.io.hfile.TestForceCacheImportantBlocks
Test results:
https://builds.apache.org/job/PreCommit-HBASE-Build/3003//testReport/
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/3003//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop2-compat.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/3003//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-server.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/3003//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop1-compat.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/3003//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-common.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/3003//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html
Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/3003//console
This message is automatically generated.
RegionScanner.reseek() creates NPE when a flush or compaction happens before
the reseek.
Key: HBASE-6900
URL: https://issues.apache.org/jira/browse/HBASE-6900
Project: HBase
Issue Type: Bug
Reporter: ramkrishna.s.vasudevan
Assignee: ramkrishna.s.vasudevan
Fix For: 0.94.2, 0.96.0
Attachments: 6900-test.txt, HBASE-6900_1.patch, HBASE-6900.patch
HBASE-5520 introduced reseek() on the RegionScanner.
Now when a scanner is created we have the StoreScanner heap. After this if a
flush or compaction happens parallely all the StoreScannerObservers are
cleared so that whenever a new next() call happens we tend to recreate the
scanner based on the latest store files.
The reseek() in StoreScanner expects the heap not to be null because always
reseek would be called from next()
{code}
public synchronized boolean reseek(KeyValue kv) throws IOException {
//Heap cannot be null, because this is only called from next() which
//guarantees that heap will never be null before this call.
if (explicitColumnQuery lazySeekEnabledGlobally) {
return heap.requestSeek(kv, true, useRowColBloom);
} else {
return heap.reseek(kv);
}
}
{code}
Now when we call RegionScanner.reseek() directly using CPs we tend to get a
NPE. In our case it happened when a major compaction was going on. I will
also attach a testcase to show the problem.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6900) RegionScanner.reseek() creates NPE when a flush or compaction happens before the reseek.
[
https://issues.apache.org/jira/browse/HBASE-6900?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13469676#comment-13469676
]
Lars Hofhansl commented on HBASE-6900:
--
Failed test passed locally. Will leave it to Ram to decide whether to commit
this.
RegionScanner.reseek() creates NPE when a flush or compaction happens before
the reseek.
Key: HBASE-6900
URL: https://issues.apache.org/jira/browse/HBASE-6900
Project: HBase
Issue Type: Bug
Reporter: ramkrishna.s.vasudevan
Assignee: ramkrishna.s.vasudevan
Fix For: 0.94.2, 0.96.0
Attachments: 6900-test.txt, HBASE-6900_1.patch, HBASE-6900.patch
HBASE-5520 introduced reseek() on the RegionScanner.
Now when a scanner is created we have the StoreScanner heap. After this if a
flush or compaction happens parallely all the StoreScannerObservers are
cleared so that whenever a new next() call happens we tend to recreate the
scanner based on the latest store files.
The reseek() in StoreScanner expects the heap not to be null because always
reseek would be called from next()
{code}
public synchronized boolean reseek(KeyValue kv) throws IOException {
//Heap cannot be null, because this is only called from next() which
//guarantees that heap will never be null before this call.
if (explicitColumnQuery lazySeekEnabledGlobally) {
return heap.requestSeek(kv, true, useRowColBloom);
} else {
return heap.reseek(kv);
}
}
{code}
Now when we call RegionScanner.reseek() directly using CPs we tend to get a
NPE. In our case it happened when a major compaction was going on. I will
also attach a testcase to show the problem.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6900) RegionScanner.reseek() creates NPE when a flush or compaction happens before the reseek.
[
https://issues.apache.org/jira/browse/HBASE-6900?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13470024#comment-13470024
]
Anoop Sam John commented on HBASE-6900:
---
bq.I was just seeing what if the lastTop was null so that checkReseek doesn't
reset the heap. If that will not happen then the above change should be fine
Lars.
Store#notifyChangedReadersObservers() will set this lastTop to the old heap
peeked value. This will happen when in between of scan compaction happens or
flush.
bq.Even if that happens, don't you always want to reseek to the kv that passed
in?
Yes ideally whatever be the case when reseek() API is called from the CP expect
a reseek to happen to the passed KV
RegionScanner.reseek() creates NPE when a flush or compaction happens before
the reseek.
Key: HBASE-6900
URL: https://issues.apache.org/jira/browse/HBASE-6900
Project: HBase
Issue Type: Bug
Reporter: ramkrishna.s.vasudevan
Assignee: ramkrishna.s.vasudevan
Fix For: 0.94.2, 0.96.0
Attachments: 6900-test.txt, HBASE-6900_1.patch, HBASE-6900.patch
HBASE-5520 introduced reseek() on the RegionScanner.
Now when a scanner is created we have the StoreScanner heap. After this if a
flush or compaction happens parallely all the StoreScannerObservers are
cleared so that whenever a new next() call happens we tend to recreate the
scanner based on the latest store files.
The reseek() in StoreScanner expects the heap not to be null because always
reseek would be called from next()
{code}
public synchronized boolean reseek(KeyValue kv) throws IOException {
//Heap cannot be null, because this is only called from next() which
//guarantees that heap will never be null before this call.
if (explicitColumnQuery lazySeekEnabledGlobally) {
return heap.requestSeek(kv, true, useRowColBloom);
} else {
return heap.reseek(kv);
}
}
{code}
Now when we call RegionScanner.reseek() directly using CPs we tend to get a
NPE. In our case it happened when a major compaction was going on. I will
also attach a testcase to show the problem.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6900) RegionScanner.reseek() creates NPE when a flush or compaction happens before the reseek.
[
https://issues.apache.org/jira/browse/HBASE-6900?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13470028#comment-13470028
]
ramkrishna.s.vasudevan commented on HBASE-6900:
---
@Lars
Yes we wanted to reseek.
{code}
// this could be null.
this.lastTop = this.peek();
{code}
The above code while updating the reader says the lastTop could be null. That
is why i had to add some additional checks. Anyway will analyse once more if
it is fine i will take up your changes. Thanks once again.
RegionScanner.reseek() creates NPE when a flush or compaction happens before
the reseek.
Key: HBASE-6900
URL: https://issues.apache.org/jira/browse/HBASE-6900
Project: HBase
Issue Type: Bug
Reporter: ramkrishna.s.vasudevan
Assignee: ramkrishna.s.vasudevan
Fix For: 0.94.2, 0.96.0
Attachments: 6900-test.txt, HBASE-6900_1.patch, HBASE-6900.patch
HBASE-5520 introduced reseek() on the RegionScanner.
Now when a scanner is created we have the StoreScanner heap. After this if a
flush or compaction happens parallely all the StoreScannerObservers are
cleared so that whenever a new next() call happens we tend to recreate the
scanner based on the latest store files.
The reseek() in StoreScanner expects the heap not to be null because always
reseek would be called from next()
{code}
public synchronized boolean reseek(KeyValue kv) throws IOException {
//Heap cannot be null, because this is only called from next() which
//guarantees that heap will never be null before this call.
if (explicitColumnQuery lazySeekEnabledGlobally) {
return heap.requestSeek(kv, true, useRowColBloom);
} else {
return heap.reseek(kv);
}
}
{code}
Now when we call RegionScanner.reseek() directly using CPs we tend to get a
NPE. In our case it happened when a major compaction was going on. I will
also attach a testcase to show the problem.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6900) RegionScanner.reseek() creates NPE when a flush or compaction happens before the reseek.
[
https://issues.apache.org/jira/browse/HBASE-6900?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13470036#comment-13470036
]
Lars Hofhansl commented on HBASE-6900:
--
I'm rolling an RC as soon as HBASE-6900 is committed.
(I assume this can go into 0.94.3 if needed, which will be soon after 0.94.2)
RegionScanner.reseek() creates NPE when a flush or compaction happens before
the reseek.
Key: HBASE-6900
URL: https://issues.apache.org/jira/browse/HBASE-6900
Project: HBase
Issue Type: Bug
Reporter: ramkrishna.s.vasudevan
Assignee: ramkrishna.s.vasudevan
Fix For: 0.94.2, 0.96.0
Attachments: 6900-test.txt, HBASE-6900_1.patch, HBASE-6900.patch
HBASE-5520 introduced reseek() on the RegionScanner.
Now when a scanner is created we have the StoreScanner heap. After this if a
flush or compaction happens parallely all the StoreScannerObservers are
cleared so that whenever a new next() call happens we tend to recreate the
scanner based on the latest store files.
The reseek() in StoreScanner expects the heap not to be null because always
reseek would be called from next()
{code}
public synchronized boolean reseek(KeyValue kv) throws IOException {
//Heap cannot be null, because this is only called from next() which
//guarantees that heap will never be null before this call.
if (explicitColumnQuery lazySeekEnabledGlobally) {
return heap.requestSeek(kv, true, useRowColBloom);
} else {
return heap.reseek(kv);
}
}
{code}
Now when we call RegionScanner.reseek() directly using CPs we tend to get a
NPE. In our case it happened when a major compaction was going on. I will
also attach a testcase to show the problem.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6900) RegionScanner.reseek() creates NPE when a flush or compaction happens before the reseek.
[
https://issues.apache.org/jira/browse/HBASE-6900?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13467590#comment-13467590
]
ramkrishna.s.vasudevan commented on HBASE-6900:
---
[~lhofhansl]
Can you take a look at this? So that i can rebase the patch based on your
suggestion.
RegionScanner.reseek() creates NPE when a flush or compaction happens before
the reseek.
Key: HBASE-6900
URL: https://issues.apache.org/jira/browse/HBASE-6900
Project: HBase
Issue Type: Bug
Reporter: ramkrishna.s.vasudevan
Assignee: ramkrishna.s.vasudevan
Attachments: HBASE-6900_1.patch, HBASE-6900.patch
HBASE-5520 introduced reseek() on the RegionScanner.
Now when a scanner is created we have the StoreScanner heap. After this if a
flush or compaction happens parallely all the StoreScannerObservers are
cleared so that whenever a new next() call happens we tend to recreate the
scanner based on the latest store files.
The reseek() in StoreScanner expects the heap not to be null because always
reseek would be called from next()
{code}
public synchronized boolean reseek(KeyValue kv) throws IOException {
//Heap cannot be null, because this is only called from next() which
//guarantees that heap will never be null before this call.
if (explicitColumnQuery lazySeekEnabledGlobally) {
return heap.requestSeek(kv, true, useRowColBloom);
} else {
return heap.reseek(kv);
}
}
{code}
Now when we call RegionScanner.reseek() directly using CPs we tend to get a
NPE. In our case it happened when a major compaction was going on. I will
also attach a testcase to show the problem.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6900) RegionScanner.reseek() creates NPE when a flush or compaction happens before the reseek.
[
https://issues.apache.org/jira/browse/HBASE-6900?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13467881#comment-13467881
]
Lars Hofhansl commented on HBASE-6900:
--
[~ram_krish] Sorry for the delay.
{code}
{+if (!checkReseek() heap != null) {}
{code}
Do we need to call checkReseek here? This is a somewhat expensive operation and
we're now doing it for every reseek issued to a StoreScanner.
Could we just check for {{heap == null}} in the beginning of the method and
return {{false}} if so?
RegionScanner.reseek() creates NPE when a flush or compaction happens before
the reseek.
Key: HBASE-6900
URL: https://issues.apache.org/jira/browse/HBASE-6900
Project: HBase
Issue Type: Bug
Reporter: ramkrishna.s.vasudevan
Assignee: ramkrishna.s.vasudevan
Attachments: HBASE-6900_1.patch, HBASE-6900.patch
HBASE-5520 introduced reseek() on the RegionScanner.
Now when a scanner is created we have the StoreScanner heap. After this if a
flush or compaction happens parallely all the StoreScannerObservers are
cleared so that whenever a new next() call happens we tend to recreate the
scanner based on the latest store files.
The reseek() in StoreScanner expects the heap not to be null because always
reseek would be called from next()
{code}
public synchronized boolean reseek(KeyValue kv) throws IOException {
//Heap cannot be null, because this is only called from next() which
//guarantees that heap will never be null before this call.
if (explicitColumnQuery lazySeekEnabledGlobally) {
return heap.requestSeek(kv, true, useRowColBloom);
} else {
return heap.reseek(kv);
}
}
{code}
Now when we call RegionScanner.reseek() directly using CPs we tend to get a
NPE. In our case it happened when a major compaction was going on. I will
also attach a testcase to show the problem.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6900) RegionScanner.reseek() creates NPE when a flush or compaction happens before the reseek.
[
https://issues.apache.org/jira/browse/HBASE-6900?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13468356#comment-13468356
]
ramkrishna.s.vasudevan commented on HBASE-6900:
---
@Lars
If really the heap is not null then checkReseek will return immediately.
Attaching the checkReseek code incase you dont have eclipse with you.
{code}
private boolean checkReseek() throws IOException {
if (this.heap == null this.lastTop != null) {
resetScannerStack(this.lastTop);
if (this.heap.peek() == null
|| store.comparator.compare(this.lastTop, this.heap.peek()) != 0) {
LOG.debug(Storescanner.peek() is changed where before =
+ this.lastTop.toString() + ,and after = + this.heap.peek());
this.lastTop = null;
return true;
}
this.lastTop = null; // gone!
}
// else dont need to reseek
return false;
}
{code}
Again why i added !checkReseek() and heap!=null is because
thought heap is not null the lasttop could be null?
{code}
Could we just check for heap == null in the beginning of the method and return
false if so?
{code}
Actually we can return false on seeing heap to be null, but am not sure whether
it will be generic. Because if the CP impl is such a way that i do a reseek
and return of false implies that the given row key is not available then it
gives a different meaning. But whereas if i reset the heap and then do a
reseek then am ensuring that i am reseeking to the required kv on the newly
created heap.
Infact our internal usage of heap needs this behaviour. If you feel that is
not generic then am open to change this Lars.
Thanks a lot for your review.
RegionScanner.reseek() creates NPE when a flush or compaction happens before
the reseek.
Key: HBASE-6900
URL: https://issues.apache.org/jira/browse/HBASE-6900
Project: HBase
Issue Type: Bug
Reporter: ramkrishna.s.vasudevan
Assignee: ramkrishna.s.vasudevan
Attachments: HBASE-6900_1.patch, HBASE-6900.patch
HBASE-5520 introduced reseek() on the RegionScanner.
Now when a scanner is created we have the StoreScanner heap. After this if a
flush or compaction happens parallely all the StoreScannerObservers are
cleared so that whenever a new next() call happens we tend to recreate the
scanner based on the latest store files.
The reseek() in StoreScanner expects the heap not to be null because always
reseek would be called from next()
{code}
public synchronized boolean reseek(KeyValue kv) throws IOException {
//Heap cannot be null, because this is only called from next() which
//guarantees that heap will never be null before this call.
if (explicitColumnQuery lazySeekEnabledGlobally) {
return heap.requestSeek(kv, true, useRowColBloom);
} else {
return heap.reseek(kv);
}
}
{code}
Now when we call RegionScanner.reseek() directly using CPs we tend to get a
NPE. In our case it happened when a major compaction was going on. I will
also attach a testcase to show the problem.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6900) RegionScanner.reseek() creates NPE when a flush or compaction happens before the reseek.
[
https://issues.apache.org/jira/browse/HBASE-6900?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13466484#comment-13466484
]
ramkrishna.s.vasudevan commented on HBASE-6900:
---
@Lars/@Stack
The patch does not fix the issue. I feel that what ever next() does, by calling
checkReseek() we need to do the same thing.
RegionScanner.reseek() creates NPE when a flush or compaction happens before
the reseek.
Key: HBASE-6900
URL: https://issues.apache.org/jira/browse/HBASE-6900
Project: HBase
Issue Type: Bug
Reporter: ramkrishna.s.vasudevan
Assignee: ramkrishna.s.vasudevan
Attachments: HBASE-6900.patch
HBASE-5520 introduced reseek() on the RegionScanner.
Now when a scanner is created we have the StoreScanner heap. After this if a
flush or compaction happens parallely all the StoreScannerObservers are
cleared so that whenever a new next() call happens we tend to recreate the
scanner based on the latest store files.
The reseek() in StoreScanner expects the heap not to be null because always
reseek would be called from next()
{code}
public synchronized boolean reseek(KeyValue kv) throws IOException {
//Heap cannot be null, because this is only called from next() which
//guarantees that heap will never be null before this call.
if (explicitColumnQuery lazySeekEnabledGlobally) {
return heap.requestSeek(kv, true, useRowColBloom);
} else {
return heap.reseek(kv);
}
}
{code}
Now when we call RegionScanner.reseek() directly using CPs we tend to get a
NPE. In our case it happened when a major compaction was going on. I will
also attach a testcase to show the problem.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6900) RegionScanner.reseek() creates NPE when a flush or compaction happens before the reseek.
[
https://issues.apache.org/jira/browse/HBASE-6900?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13466532#comment-13466532
]
Hadoop QA commented on HBASE-6900:
--
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12547166/HBASE-6900_1.patch
against trunk revision .
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 3 new
or modified tests.
{color:green}+1 hadoop2.0{color}. The patch compiles against the hadoop
2.0 profile.
{color:red}-1 javadoc{color}. The javadoc tool appears to have generated
149 warning messages.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:red}-1 findbugs{color}. The patch appears to introduce 7 new
Findbugs (version 1.3.9) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:red}-1 core tests{color}. The patch failed these unit tests:
Test results:
https://builds.apache.org/job/PreCommit-HBASE-Build/2973//testReport/
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/2973//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop2-compat.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/2973//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop1-compat.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/2973//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-common.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/2973//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-server.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/2973//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html
Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/2973//console
This message is automatically generated.
RegionScanner.reseek() creates NPE when a flush or compaction happens before
the reseek.
Key: HBASE-6900
URL: https://issues.apache.org/jira/browse/HBASE-6900
Project: HBase
Issue Type: Bug
Reporter: ramkrishna.s.vasudevan
Assignee: ramkrishna.s.vasudevan
Attachments: HBASE-6900_1.patch, HBASE-6900.patch
HBASE-5520 introduced reseek() on the RegionScanner.
Now when a scanner is created we have the StoreScanner heap. After this if a
flush or compaction happens parallely all the StoreScannerObservers are
cleared so that whenever a new next() call happens we tend to recreate the
scanner based on the latest store files.
The reseek() in StoreScanner expects the heap not to be null because always
reseek would be called from next()
{code}
public synchronized boolean reseek(KeyValue kv) throws IOException {
//Heap cannot be null, because this is only called from next() which
//guarantees that heap will never be null before this call.
if (explicitColumnQuery lazySeekEnabledGlobally) {
return heap.requestSeek(kv, true, useRowColBloom);
} else {
return heap.reseek(kv);
}
}
{code}
Now when we call RegionScanner.reseek() directly using CPs we tend to get a
NPE. In our case it happened when a major compaction was going on. I will
also attach a testcase to show the problem.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6900) RegionScanner.reseek() creates NPE when a flush or compaction happens before the reseek.
[
https://issues.apache.org/jira/browse/HBASE-6900?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13466551#comment-13466551
]
Lars Hofhansl commented on HBASE-6900:
--
Hmm... Shouldn't the RegionScannerImpl.reseek perform these extra checks?
(again, I do not have a machine with Eclipse to browse the code easily ATM)
RegionScanner.reseek() creates NPE when a flush or compaction happens before
the reseek.
Key: HBASE-6900
URL: https://issues.apache.org/jira/browse/HBASE-6900
Project: HBase
Issue Type: Bug
Reporter: ramkrishna.s.vasudevan
Assignee: ramkrishna.s.vasudevan
Attachments: HBASE-6900_1.patch, HBASE-6900.patch
HBASE-5520 introduced reseek() on the RegionScanner.
Now when a scanner is created we have the StoreScanner heap. After this if a
flush or compaction happens parallely all the StoreScannerObservers are
cleared so that whenever a new next() call happens we tend to recreate the
scanner based on the latest store files.
The reseek() in StoreScanner expects the heap not to be null because always
reseek would be called from next()
{code}
public synchronized boolean reseek(KeyValue kv) throws IOException {
//Heap cannot be null, because this is only called from next() which
//guarantees that heap will never be null before this call.
if (explicitColumnQuery lazySeekEnabledGlobally) {
return heap.requestSeek(kv, true, useRowColBloom);
} else {
return heap.reseek(kv);
}
}
{code}
Now when we call RegionScanner.reseek() directly using CPs we tend to get a
NPE. In our case it happened when a major compaction was going on. I will
also attach a testcase to show the problem.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6900) RegionScanner.reseek() creates NPE when a flush or compaction happens before the reseek.
[
https://issues.apache.org/jira/browse/HBASE-6900?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13466553#comment-13466553
]
Ted Yu commented on HBASE-6900:
---
RegionScannerImpl doesn't override reseek() presently.
FYI
RegionScanner.reseek() creates NPE when a flush or compaction happens before
the reseek.
Key: HBASE-6900
URL: https://issues.apache.org/jira/browse/HBASE-6900
Project: HBase
Issue Type: Bug
Reporter: ramkrishna.s.vasudevan
Assignee: ramkrishna.s.vasudevan
Attachments: HBASE-6900_1.patch, HBASE-6900.patch
HBASE-5520 introduced reseek() on the RegionScanner.
Now when a scanner is created we have the StoreScanner heap. After this if a
flush or compaction happens parallely all the StoreScannerObservers are
cleared so that whenever a new next() call happens we tend to recreate the
scanner based on the latest store files.
The reseek() in StoreScanner expects the heap not to be null because always
reseek would be called from next()
{code}
public synchronized boolean reseek(KeyValue kv) throws IOException {
//Heap cannot be null, because this is only called from next() which
//guarantees that heap will never be null before this call.
if (explicitColumnQuery lazySeekEnabledGlobally) {
return heap.requestSeek(kv, true, useRowColBloom);
} else {
return heap.reseek(kv);
}
}
{code}
Now when we call RegionScanner.reseek() directly using CPs we tend to get a
NPE. In our case it happened when a major compaction was going on. I will
also attach a testcase to show the problem.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6900) RegionScanner.reseek() creates NPE when a flush or compaction happens before the reseek.
[
https://issues.apache.org/jira/browse/HBASE-6900?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13466555#comment-13466555
]
Ted Yu commented on HBASE-6900:
---
I was looking at 0.92 code base.
In trunk, RegionScannerImpl has reseek().
RegionScanner.reseek() creates NPE when a flush or compaction happens before
the reseek.
Key: HBASE-6900
URL: https://issues.apache.org/jira/browse/HBASE-6900
Project: HBase
Issue Type: Bug
Reporter: ramkrishna.s.vasudevan
Assignee: ramkrishna.s.vasudevan
Attachments: HBASE-6900_1.patch, HBASE-6900.patch
HBASE-5520 introduced reseek() on the RegionScanner.
Now when a scanner is created we have the StoreScanner heap. After this if a
flush or compaction happens parallely all the StoreScannerObservers are
cleared so that whenever a new next() call happens we tend to recreate the
scanner based on the latest store files.
The reseek() in StoreScanner expects the heap not to be null because always
reseek would be called from next()
{code}
public synchronized boolean reseek(KeyValue kv) throws IOException {
//Heap cannot be null, because this is only called from next() which
//guarantees that heap will never be null before this call.
if (explicitColumnQuery lazySeekEnabledGlobally) {
return heap.requestSeek(kv, true, useRowColBloom);
} else {
return heap.reseek(kv);
}
}
{code}
Now when we call RegionScanner.reseek() directly using CPs we tend to get a
NPE. In our case it happened when a major compaction was going on. I will
also attach a testcase to show the problem.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6900) RegionScanner.reseek() creates NPE when a flush or compaction happens before the reseek.
[
https://issues.apache.org/jira/browse/HBASE-6900?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13466561#comment-13466561
]
Ted Yu commented on HBASE-6900:
---
RegionScannerImpl.reseek() would call this.storeHeap.requestSeek(), ending up
in StoreScanner.reseek().
So I think Ram's change should be fine.
RegionScanner.reseek() creates NPE when a flush or compaction happens before
the reseek.
Key: HBASE-6900
URL: https://issues.apache.org/jira/browse/HBASE-6900
Project: HBase
Issue Type: Bug
Reporter: ramkrishna.s.vasudevan
Assignee: ramkrishna.s.vasudevan
Attachments: HBASE-6900_1.patch, HBASE-6900.patch
HBASE-5520 introduced reseek() on the RegionScanner.
Now when a scanner is created we have the StoreScanner heap. After this if a
flush or compaction happens parallely all the StoreScannerObservers are
cleared so that whenever a new next() call happens we tend to recreate the
scanner based on the latest store files.
The reseek() in StoreScanner expects the heap not to be null because always
reseek would be called from next()
{code}
public synchronized boolean reseek(KeyValue kv) throws IOException {
//Heap cannot be null, because this is only called from next() which
//guarantees that heap will never be null before this call.
if (explicitColumnQuery lazySeekEnabledGlobally) {
return heap.requestSeek(kv, true, useRowColBloom);
} else {
return heap.reseek(kv);
}
}
{code}
Now when we call RegionScanner.reseek() directly using CPs we tend to get a
NPE. In our case it happened when a major compaction was going on. I will
also attach a testcase to show the problem.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6900) RegionScanner.reseek() creates NPE when a flush or compaction happens before the reseek.
[
https://issues.apache.org/jira/browse/HBASE-6900?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13466562#comment-13466562
]
Lars Hofhansl commented on HBASE-6900:
--
yes, but StoreScanner should not need to be changed for this
RegionScanner.reseek() creates NPE when a flush or compaction happens before
the reseek.
Key: HBASE-6900
URL: https://issues.apache.org/jira/browse/HBASE-6900
Project: HBase
Issue Type: Bug
Reporter: ramkrishna.s.vasudevan
Assignee: ramkrishna.s.vasudevan
Attachments: HBASE-6900_1.patch, HBASE-6900.patch
HBASE-5520 introduced reseek() on the RegionScanner.
Now when a scanner is created we have the StoreScanner heap. After this if a
flush or compaction happens parallely all the StoreScannerObservers are
cleared so that whenever a new next() call happens we tend to recreate the
scanner based on the latest store files.
The reseek() in StoreScanner expects the heap not to be null because always
reseek would be called from next()
{code}
public synchronized boolean reseek(KeyValue kv) throws IOException {
//Heap cannot be null, because this is only called from next() which
//guarantees that heap will never be null before this call.
if (explicitColumnQuery lazySeekEnabledGlobally) {
return heap.requestSeek(kv, true, useRowColBloom);
} else {
return heap.reseek(kv);
}
}
{code}
Now when we call RegionScanner.reseek() directly using CPs we tend to get a
NPE. In our case it happened when a major compaction was going on. I will
also attach a testcase to show the problem.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6900) RegionScanner.reseek() creates NPE when a flush or compaction happens before the reseek.
[
https://issues.apache.org/jira/browse/HBASE-6900?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13466640#comment-13466640
]
ramkrishna.s.vasudevan commented on HBASE-6900:
---
@Lars
I too first thought of performing these checks in the RegionScannerImpl level
but
- The StoreScanner heap is not exposed outside as the storeHeap internally has
the Storescanner heap(which is on the memstore and the storefiles). The
KeyValueScanner is the one that invokes the reseek() api.
- RegionScanner can have customised implementations also.
What do you feel Lars?
RegionScanner.reseek() creates NPE when a flush or compaction happens before
the reseek.
Key: HBASE-6900
URL: https://issues.apache.org/jira/browse/HBASE-6900
Project: HBase
Issue Type: Bug
Reporter: ramkrishna.s.vasudevan
Assignee: ramkrishna.s.vasudevan
Attachments: HBASE-6900_1.patch, HBASE-6900.patch
HBASE-5520 introduced reseek() on the RegionScanner.
Now when a scanner is created we have the StoreScanner heap. After this if a
flush or compaction happens parallely all the StoreScannerObservers are
cleared so that whenever a new next() call happens we tend to recreate the
scanner based on the latest store files.
The reseek() in StoreScanner expects the heap not to be null because always
reseek would be called from next()
{code}
public synchronized boolean reseek(KeyValue kv) throws IOException {
//Heap cannot be null, because this is only called from next() which
//guarantees that heap will never be null before this call.
if (explicitColumnQuery lazySeekEnabledGlobally) {
return heap.requestSeek(kv, true, useRowColBloom);
} else {
return heap.reseek(kv);
}
}
{code}
Now when we call RegionScanner.reseek() directly using CPs we tend to get a
NPE. In our case it happened when a major compaction was going on. I will
also attach a testcase to show the problem.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6900) RegionScanner.reseek() creates NPE when a flush or compaction happens before the reseek.
[
https://issues.apache.org/jira/browse/HBASE-6900?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13466187#comment-13466187
]
ramkrishna.s.vasudevan commented on HBASE-6900:
---
{code}
2012-09-29 17:02:10,765 INFO [main] regionserver.HStore(1045): Completed major
compaction of 1 file(s) in family of
testToShowNPEOnRegionScannerReseek,,1348918329648.73944279562b7d512a9f318cc90098d2.
into 279f1073e9ba4e6db93d93eab86258cf, size=782.0; total size for store is
782.0
java.lang.NullPointerException
at
org.apache.hadoop.hbase.regionserver.StoreScanner.reseek(StoreScanner.java:569)
at
org.apache.hadoop.hbase.regionserver.NonLazyKeyValueScanner.doRealSeek(NonLazyKeyValueScanner.java:55)
at
org.apache.hadoop.hbase.regionserver.NonLazyKeyValueScanner.requestSeek(NonLazyKeyValueScanner.java:39)
at
org.apache.hadoop.hbase.regionserver.KeyValueHeap.generalizedSeek(KeyValueHeap.java:319)
at
org.apache.hadoop.hbase.regionserver.KeyValueHeap.requestSeek(KeyValueHeap.java:278)
at
org.apache.hadoop.hbase.regionserver.HRegion$RegionScannerImpl.reseek(HRegion.java:3588)
at
org.apache.hadoop.hbase.regionserver.TestHRegion.testToShowNPEOnRegionScannerReseek(TestHRegion.java:232)
{code}
RegionScanner.reseek() creates NPE when a flush or compaction happens before
the reseek.
Key: HBASE-6900
URL: https://issues.apache.org/jira/browse/HBASE-6900
Project: HBase
Issue Type: Bug
Reporter: ramkrishna.s.vasudevan
Assignee: ramkrishna.s.vasudevan
Attachments: HBASE-6900.patch
HBASE-5520 introduced reseek() on the RegionScanner.
Now when a scanner is created we have the StoreScanner heap. After this if a
flush or compaction happens parallely all the StoreScannerObservers are
cleared so that whenever a new next() call happens we tend to recreate the
scanner based on the latest store files.
The reseek() in StoreScanner expects the heap not to be null because always
reseek would be called from next()
{code}
public synchronized boolean reseek(KeyValue kv) throws IOException {
//Heap cannot be null, because this is only called from next() which
//guarantees that heap will never be null before this call.
if (explicitColumnQuery lazySeekEnabledGlobally) {
return heap.requestSeek(kv, true, useRowColBloom);
} else {
return heap.reseek(kv);
}
}
{code}
Now when we call RegionScanner.reseek() directly using CPs we tend to get a
NPE. In our case it happened when a major compaction was going on. I will
also attach a testcase to show the problem.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6900) RegionScanner.reseek() creates NPE when a flush or compaction happens before the reseek.
[
https://issues.apache.org/jira/browse/HBASE-6900?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13466302#comment-13466302
]
stack commented on HBASE-6900:
--
Patch looks good Ram. Doesn't fix the issue though? What would you suggest?
RegionScanner.reseek() creates NPE when a flush or compaction happens before
the reseek.
Key: HBASE-6900
URL: https://issues.apache.org/jira/browse/HBASE-6900
Project: HBase
Issue Type: Bug
Reporter: ramkrishna.s.vasudevan
Assignee: ramkrishna.s.vasudevan
Attachments: HBASE-6900.patch
HBASE-5520 introduced reseek() on the RegionScanner.
Now when a scanner is created we have the StoreScanner heap. After this if a
flush or compaction happens parallely all the StoreScannerObservers are
cleared so that whenever a new next() call happens we tend to recreate the
scanner based on the latest store files.
The reseek() in StoreScanner expects the heap not to be null because always
reseek would be called from next()
{code}
public synchronized boolean reseek(KeyValue kv) throws IOException {
//Heap cannot be null, because this is only called from next() which
//guarantees that heap will never be null before this call.
if (explicitColumnQuery lazySeekEnabledGlobally) {
return heap.requestSeek(kv, true, useRowColBloom);
} else {
return heap.reseek(kv);
}
}
{code}
Now when we call RegionScanner.reseek() directly using CPs we tend to get a
NPE. In our case it happened when a major compaction was going on. I will
also attach a testcase to show the problem.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6900) RegionScanner.reseek() creates NPE when a flush or compaction happens before the reseek.
[
https://issues.apache.org/jira/browse/HBASE-6900?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13466322#comment-13466322
]
Lars Hofhansl commented on HBASE-6900:
--
The patch is just to confirm the issue?
If this is to deal with something that is possibly null we should do that (i.e.
add a null check and not catch throwable).
My dev machine is dead currently so I cannot look at code easily. From your
description reseek should do whatever check and setup next() performs, right?
RegionScanner.reseek() creates NPE when a flush or compaction happens before
the reseek.
Key: HBASE-6900
URL: https://issues.apache.org/jira/browse/HBASE-6900
Project: HBase
Issue Type: Bug
Reporter: ramkrishna.s.vasudevan
Assignee: ramkrishna.s.vasudevan
Attachments: HBASE-6900.patch
HBASE-5520 introduced reseek() on the RegionScanner.
Now when a scanner is created we have the StoreScanner heap. After this if a
flush or compaction happens parallely all the StoreScannerObservers are
cleared so that whenever a new next() call happens we tend to recreate the
scanner based on the latest store files.
The reseek() in StoreScanner expects the heap not to be null because always
reseek would be called from next()
{code}
public synchronized boolean reseek(KeyValue kv) throws IOException {
//Heap cannot be null, because this is only called from next() which
//guarantees that heap will never be null before this call.
if (explicitColumnQuery lazySeekEnabledGlobally) {
return heap.requestSeek(kv, true, useRowColBloom);
} else {
return heap.reseek(kv);
}
}
{code}
Now when we call RegionScanner.reseek() directly using CPs we tend to get a
NPE. In our case it happened when a major compaction was going on. I will
also attach a testcase to show the problem.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
