[
https://issues.apache.org/jira/browse/HBASE-11425?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14523448#comment-14523448
]
stack commented on HBASE-11425:
-------------------------------
[~anoop.hbase] The write up is excellent -- especially the bit where you dumb
it down listing out conclusion at end of each test and do up the table with all
results. Suggest you do an edit, add a conclusion, and post the jmh files on
this issue (I could not open them from the doc) rather than in the doc, and
then post a note to dev list pointing at these findings since we are going to
build on top of them going forward (we should put it on hbase blog?). Very nice.
Here are some comments on the doc that might help w/ the edit.
Suggest you add sentence after first on why we want to go offheap (this will
make the doc 'standalone')
Change: "When we support E2E off heap support and in turn support Cell also
backed by off heap memory, we have to make sure to select the best performing
data structure/framework for this off heap storage."
to
"When we implement E2E off heap support, we have to make sure to select the
best performing data structure/framework."
s/below test is/below tests are/
s/pros like, our PRC layer/pros such as our RPC layer already/
s/HDFS/an HDFS/
Change "But the NIO Buffer APIs have complaints over its performance and
methods not inlineable"
to
But NIO ByteBuffers can be slow (boundary checks and/or some methods may not
inline).
Change: "This make us to think for netty also as it seems better performing.
(Really? We have test results below)"
to
"This makes us look to Netty ByteBuf as a possibly better performing
alternative."
On first test, add sentence comparing difference between onheap and offheap
runs (onheap looks 30% slower compared to onheap?) Do same comparing jdk8 to
jdk7? (Hmm... yeah, would like to see the code... smile)
s/Similar way/Similarly/
s/The next test cmopare/The next test compares/
s/come almost/come out almost/
s/test what I/test that I/
> Cell/DBB end-to-end on the read-path
> ------------------------------------
>
> Key: HBASE-11425
> URL: https://issues.apache.org/jira/browse/HBASE-11425
> Project: HBase
> Issue Type: Umbrella
> Components: regionserver, Scanners
> Affects Versions: 0.99.0
> Reporter: Anoop Sam John
> Assignee: Anoop Sam John
> Attachments: Benchmarks_Tests.docx,
> HBASE-11425-E2E-NotComplete.patch, HBASE-11425.patch, Offheap reads in HBase
> using BBs_V2.pdf, Offheap reads in HBase using BBs_final.pdf
>
>
> Umbrella jira to make sure we can have blocks cached in offheap backed cache.
> In the entire read path, we can refer to this offheap buffer and avoid onheap
> copying.
> The high level items I can identify as of now are
> 1. Avoid the array() call on BB in read path.. (This is there in many
> classes. We can handle class by class)
> 2. Support Buffer based getter APIs in cell. In read path we will create a
> new Cell with backed by BB. Will need in CellComparator, Filter (like SCVF),
> CPs etc.
> 3. Avoid KeyValue.ensureKeyValue() calls in read path - This make byte copy.
> 4. Remove all CP hooks (which are already deprecated) which deal with KVs.
> (In read path)
> Will add subtasks under this.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
