[
https://issues.apache.org/jira/browse/HBASE-12295?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14564140#comment-14564140
]
ramkrishna.s.vasudevan commented on HBASE-12295:
------------------------------------------------
bq.You can't figure whether L1 or L2 looking at the key?
You mean add some different type of keys for the L1 or L2 case? Can check this.
bq.We'll have to dig in on why. You'd think w/ less intermediaries that it
would be faster.
Yes that was the reason. When we were writing every cell to the socket
directly things were really slow.
bq.But pulling the rpc controller back into HRegion is a little perverse.
Okie, we could see if we can make the REsult carry this cellblock. Will check
on this once again.
bq.(maybe this is not possible....). That said, for now at least, the creation
of CellBlock is a good place for triggering decrement of backing block
reference.
Yes. Right.
bq. If that is not enough, can we mimic scan context in Get?
Will see on this if it is really possible.
bq.So, as is, we'll make a copy per registered CP?
Yes but only if the scanner returned from the CP hook is not implementing the
new interface. But if we allow the REgionScanner itself to use that new
interface and the CP tries to use that new API diligently like how close()
would be used, then we are safe we don't need to do that copy. Anyway for
non-java case we need to do that copy.
bq.You want a delayed close, one that completes only after all outstanding
scanners are done? Can we have scanner close set up one of these?
Ya, I tried out something yesterday. Seems to work. But checking on all the
corner cases, the next patch may have this change. However I doubt on how a
scan next() can be handled. It needs some explicit way of decrementing the ref
count alone.
But generally the call to close() would be an logical end to the ongoing read
process including decrementing the ref count on the current list of blocks.
bq.So the boolean does what for compacted files? It says, don't evict files
that are being read though they have been compacted? (Is this like the
finalizeScan case at all? Where you want to do a delayed close until no more
references?)
A kind of. But this would happen purely internally. Nothing like explicitly
calling from the compaciton scanners to set this boolean. Just the block block
in the Bucket cache itself would know this and do it.
bq.One other thought is how you folks thinking of testing this stuff?
Ya as a first measure we have written unit tests to test out these scenarios
and ensure we are doing the ref counting correctly. We are adding more tests to
it so that we cover the basic scenarios.
For the cluster testing we are planning to reduce the bucket cache size and
ensure we have frequent eviction scenarios and verify whether any of the
results are getting corrupted. Any other suggestion you have here?
> Prevent block eviction under us if reads are in progress from the BBs
> ---------------------------------------------------------------------
>
> Key: HBASE-12295
> URL: https://issues.apache.org/jira/browse/HBASE-12295
> Project: HBase
> Issue Type: Sub-task
> Components: regionserver, Scanners
> Reporter: ramkrishna.s.vasudevan
> Assignee: ramkrishna.s.vasudevan
> Fix For: 2.0.0
>
> Attachments: HBASE-12295.pdf, HBASE-12295_trunk.patch
>
>
> While we try to serve the reads from the BBs directly from the block cache,
> we need to ensure that the blocks does not get evicted under us while
> reading. This JIRA is to discuss and implement a strategy for the same.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
