[
https://issues.apache.org/jira/browse/HBASE-8109?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13704909#comment-13704909
]
Sergey Shelukhin commented on HBASE-8109:
-----------------------------------------
btw, given that there is probably no advantage in terms on contiguity when we
are reading across a block boundary in the same file, compared to reading
separate files, we can prototype by having some sort of over-files that contain
multiple one-block files. As far as I know, Accumulo is able to make use of
bazillion files on each server, so it should be doable.
> HBase can manage blocks instead of (or inside) files in HDFS
> ------------------------------------------------------------
>
> Key: HBASE-8109
> URL: https://issues.apache.org/jira/browse/HBASE-8109
> Project: HBase
> Issue Type: Brainstorming
> Reporter: Sergey Shelukhin
>
> Prompted by previous non-Hadoop experience and some dev list discussions, and
> after talking to some HDFS people about blocks.
> HBase could improve a lot by managing HDFS blocks instead of files, and
> reusing the blocks among other things. Some areas that could improve are
> splits, compactions, management of large blobs, locality enforcement.
> I was told that block APIs in Hadoop 2 are well-isolated, but not exposed
> yet. They can easily be exposed, and as one of the first potential users we
> could get to help shape them. Two areas that from my limited understanding is
> currently fuzzy are namespaces for blocks, and ref-counting.
> We should come up with list of initial scenarios to figure out what we need
> from block API (locality, detecting/enforcing block boundary/variable size
> blocks, reusing one block, ...).
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
