Sean Busbey updated HBASE-14439:
    Assignee: Sean Busbey  (was: Matteo Bertozzi)

Making another go at this effort. I've moved the experimental implementation 
from last year to the branch {{hbase-14439-experimental}} and posted a WIP on 
{{hbase-14439}} so that I can work with [~uagashe] in the public repo before 
getting things to a level of done-ness needed to patch master.

the WIP has a bunch of gaps. I've got some notes on them in {{TODO}} lines. 
Most notably, it currently will pass {{mvn -DskipTests compile}} but won't pass 
{{mvn -DskipTests test-compile}}.

> New/Improved Filesystem Abstractions
> ------------------------------------
>                 Key: HBASE-14439
>                 URL: https://issues.apache.org/jira/browse/HBASE-14439
>             Project: HBase
>          Issue Type: Sub-task
>            Reporter: Ben Lau
>            Assignee: Sean Busbey
>         Attachments: abstraction.patch
> Ticket for work in progress on new FileSystem abstractions.  Previously, we 
> (Yahoo) submitted a ticket that would add support for humongous (1 million 
> region+) tables via a hierarchical layout (HBASE-13991).  However open source 
> is moving in a similar but not identical direction in the future and so the 
> patch will not be merged into open source.
> We will be working on a different patch now with folks from open source.  It 
> will create/add to 2 layers-- a path abstraction layer and a use-oriented 
> abstraction layer.  The path abstraction layer is epitomized by classes like 
> FsUtils (and in the patch new classes like AFsLayout).  The use oriented 
> abstraction layer is epitomized by existing classes like 
> MasterFileSystem/HRegionFileSystem (and possibly new classes later) that 
> build on the path abstraction layer and focus on 'doing things' (eg creating 
> regions) and less on the gritty details like the paths.
> This work on abstracting and isolating the paths from the use cases will help 
> Yahoo not diverge too much from open source with its internal 'Humongous' 
> table hierarchical layout, while also helping open source move further 
> towards the eventual goal of redoing the FS layout in a similar (but 
> different) hierarchical layout later that focuses on data directory 
> uniformity (unlike the humongous patch) and storing hierarchy in the meta 
> table instead which enables new optimizations (see HBASE-14090.)
> Attached to this ticket is some work we've done at Yahoo so far that will be 
> put into an open source HBase branch for further collaboration.  The patch is 
> not meant to be complete yet and is a work in progress.  (Please wait on 
> patch comments/reviews.)  It also includes some Yahoo-specific 'humongous' 
> layout code that will be removed before submission in open source.

This message was sent by Atlassian JIRA

Reply via email to