[
https://issues.apache.org/jira/browse/HBASE-1024?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12678622#action_12678622
]
stack commented on HBASE-1024:
------------------------------
Bunch of us discussed the zero-copy above this evening at HUG6. In particular
remove HStoreKey and just do byte array all the ways into the HFile and then on
the way out, carry the HFile key and value all the ways out to IPC. Will make
an issue once I have better handle on it.
> Rearchitect regionserver I/O
> ----------------------------
>
> Key: HBASE-1024
> URL: https://issues.apache.org/jira/browse/HBASE-1024
> Project: Hadoop HBase
> Issue Type: Improvement
> Components: io, regionserver
> Reporter: Andrew Purtell
>
> Consider supporting:
> * 2GB store files
> * 1TB per node (500 store files)
> * Cell values up to ~100MB
> * Typical use case of RS running with 1GB of heap only
> Some ideas:
> * Drop MapFile and make a custom store file format with (competing) design
> goals:
> ** heap efficiency
> ** fast lookups
> ** minimize I/O operations
> ** optimize for typical DFS blocksizes (8MB, 64MB)
> * MRU cache for filehandles and store file indexes
> * Memory mapped store file indexes -- don't hold the indexes in heap; rely on
> the OS blockcache for performance
> * "Zero copy" I/O from IPC to store file and vice versa, like NIO buffers
>
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
