[
https://issues.apache.org/jira/browse/HBASE-11425?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14961753#comment-14961753
]
Anoop Sam John commented on HBASE-11425:
----------------------------------------
One doubt after seeing the logs... The OOME comes during a time when some
compaction happens?
There was one open item we discussed after the shipped() call during the
compaction.. This API gets called during the scan flow after we ship a set
of rows back to client.. So all the blocks other than the cur block we came
across during this scan, can get released. (Ref count decrements).
But during compaction this call is not at all happening and only at the end one
close happens. We can correct this.. I have a quick patch for that. U will
be interested to see it and test it with once boss?
> Cell/DBB end-to-end on the read-path
> ------------------------------------
>
> Key: HBASE-11425
> URL: https://issues.apache.org/jira/browse/HBASE-11425
> Project: HBase
> Issue Type: Umbrella
> Components: regionserver, Scanners
> Affects Versions: 0.99.0
> Reporter: Anoop Sam John
> Assignee: Anoop Sam John
> Attachments: BenchmarkTestCode.zip, Benchmarks_Tests.docx, GC pics
> with evictions_4G heap.png, HBASE-11425-E2E-NotComplete.patch,
> HBASE-11425.patch, Offheap reads in HBase using BBs_V2.pdf, Offheap reads in
> HBase using BBs_final.pdf, Screen Shot 2015-10-16 at 5.13.22 PM.png, gc.png,
> gets.png, heap.png, load.png, median.png, ram.log
>
>
> Umbrella jira to make sure we can have blocks cached in offheap backed cache.
> In the entire read path, we can refer to this offheap buffer and avoid onheap
> copying.
> The high level items I can identify as of now are
> 1. Avoid the array() call on BB in read path.. (This is there in many
> classes. We can handle class by class)
> 2. Support Buffer based getter APIs in cell. In read path we will create a
> new Cell with backed by BB. Will need in CellComparator, Filter (like SCVF),
> CPs etc.
> 3. Avoid KeyValue.ensureKeyValue() calls in read path - This make byte copy.
> 4. Remove all CP hooks (which are already deprecated) which deal with KVs.
> (In read path)
> Will add subtasks under this.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
