[
https://issues.apache.org/jira/browse/HBASE-13991?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14607544#comment-14607544
]
stack commented on HBASE-13991:
-------------------------------
[~benlau] Thank you for writing up design and for the helpful experiments.
"Simply put, HDFS was not designed to handle the existence of millions of
files/directories in a single directory. "
Yes.
How about just making a four level bucket of all HFiles. From
http://static.googleusercontent.com/media/research.google.com/en/us/archive/bigtable-osdi06.pdf,
"Each tablet’s SSTables are registered in the METADATA table." (Our
[~mbertozzi] has been going on about going this route for ever -- see his
design attached to https://issues.apache.org/jira/browse/HBASE-7806... It has
some in common w/ yours).
Thanks [~benlau]
> Hierarchical Layout for Humongous Tables
> ----------------------------------------
>
> Key: HBASE-13991
> URL: https://issues.apache.org/jira/browse/HBASE-13991
> Project: HBase
> Issue Type: Sub-task
> Reporter: Ben Lau
> Assignee: Ben Lau
> Attachments: HBASE-13991-master.patch, HumongousTableDoc.pdf
>
>
> Add support for humongous tables via a hierarchical layout for regions on
> filesystem.
> Credit for most of this code goes to Huaiyu Zhu.
> Latest version of the patch is available on the review board:
> https://reviews.apache.org/r/36029/
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
