[ 
https://issues.apache.org/jira/browse/HBASE-14439?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Work on HBASE-14439 started by Sean Busbey.
-------------------------------------------
> New/Improved Filesystem Abstractions
> ------------------------------------
>
>                 Key: HBASE-14439
>                 URL: https://issues.apache.org/jira/browse/HBASE-14439
>             Project: HBase
>          Issue Type: Sub-task
>            Reporter: Ben Lau
>            Assignee: Sean Busbey
>         Attachments: abstraction.patch
>
>
> Ticket for work in progress on new FileSystem abstractions.  Previously, we 
> (Yahoo) submitted a ticket that would add support for humongous (1 million 
> region+) tables via a hierarchical layout (HBASE-13991).  However open source 
> is moving in a similar but not identical direction in the future and so the 
> patch will not be merged into open source.
> We will be working on a different patch now with folks from open source.  It 
> will create/add to 2 layers-- a path abstraction layer and a use-oriented 
> abstraction layer.  The path abstraction layer is epitomized by classes like 
> FsUtils (and in the patch new classes like AFsLayout).  The use oriented 
> abstraction layer is epitomized by existing classes like 
> MasterFileSystem/HRegionFileSystem (and possibly new classes later) that 
> build on the path abstraction layer and focus on 'doing things' (eg creating 
> regions) and less on the gritty details like the paths.
> This work on abstracting and isolating the paths from the use cases will help 
> Yahoo not diverge too much from open source with its internal 'Humongous' 
> table hierarchical layout, while also helping open source move further 
> towards the eventual goal of redoing the FS layout in a similar (but 
> different) hierarchical layout later that focuses on data directory 
> uniformity (unlike the humongous patch) and storing hierarchy in the meta 
> table instead which enables new optimizations (see HBASE-14090.)
> Attached to this ticket is some work we've done at Yahoo so far that will be 
> put into an open source HBase branch for further collaboration.  The patch is 
> not meant to be complete yet and is a work in progress.  (Please wait on 
> patch comments/reviews.)  It also includes some Yahoo-specific 'humongous' 
> layout code that will be removed before submission in open source.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Reply via email to