[
https://issues.apache.org/jira/browse/HBASE-14895?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15052479#comment-15052479
]
Anoop Sam John commented on HBASE-14895:
----------------------------------------
one issue found wrt maintain the scanners in a new List in StoreScanner (These
are memstore scanner and store file scanners at the begin of the scan)
Over the scan time, some of the scanners would have reached its end and we
would have eventually closed them off. And the PriorityQ in KVHeap would have
removed them as well. But we are still holding those refs in this new List in
Store Scanner.
Now after a flush we will use this new List to recreate new Heap. We will
remove old MemstoreScaner and add new MemstoreScanner as well as scanner to the
new flushed file. And use all of them to create new KVHeap. (This might include
already closed StoreFileScanners as well).
In KVHeap constructor
{code}
KeyValueHeap(List<? extends KeyValueScanner> scanners,
KVScannerComparator comparator) throws IOException {
this.comparator = comparator;
if (!scanners.isEmpty()) {
this.heap = new PriorityQueue<KeyValueScanner>(scanners.size(),
this.comparator);
for (KeyValueScanner scanner : scanners) {
if (scanner.peek() != null) {
this.heap.add(scanner);
} else {
this.scannersForDelayedClose.add(scanner);
}
}
this.current = pollRealKV();
}
}
{code}
So the closed scanners wil have peek Cell as null we wont add them to heap.
Good. But we will add them to delayed close and after one shipped call we will
try close them again
StoreFileScanner#close()
{code}
public void close() {
cur = null;
this.hfs.close();
if (this.reader != null) {
this.reader.decrementRefCount();
}
}
{code}
No impact because of hfs close. But the decr read ref count will be a problem
and can cause issue.
In a simple way we can we can have closed flag in StoreFileScanner which can be
used to check when close call on second time or close call nullifying the
reader and hfs on StoreFileScanner.
Or else if we dont need to maintain a new List of scanners in StoreScanner and
make use of the current scanners in the KVHeap for recreating the new heap,
that would have looked better. No need to maintain same scanners in 2 data
structures.
> Seek only to the newly flushed file on scanner reset on flush
> -------------------------------------------------------------
>
> Key: HBASE-14895
> URL: https://issues.apache.org/jira/browse/HBASE-14895
> Project: HBase
> Issue Type: Sub-task
> Reporter: ramkrishna.s.vasudevan
> Assignee: ramkrishna.s.vasudevan
> Fix For: 2.0.0
>
> Attachments: HBASE-14895.patch, HBASE-14895_1.patch,
> HBASE-14895_1.patch, HBASE-14895_2.patch
>
>
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
