[
https://issues.apache.org/jira/browse/HBASE-14439?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sean Busbey updated HBASE-14439:
--------------------------------
Assignee: Sean Busbey (was: Matteo Bertozzi)
Making another go at this effort. I've moved the experimental implementation
from last year to the branch {{hbase-14439-experimental}} and posted a WIP on
{{hbase-14439}} so that I can work with [~uagashe] in the public repo before
getting things to a level of done-ness needed to patch master.
the WIP has a bunch of gaps. I've got some notes on them in {{TODO}} lines.
Most notably, it currently will pass {{mvn -DskipTests compile}} but won't pass
{{mvn -DskipTests test-compile}}.
> New/Improved Filesystem Abstractions
> ------------------------------------
>
> Key: HBASE-14439
> URL: https://issues.apache.org/jira/browse/HBASE-14439
> Project: HBase
> Issue Type: Sub-task
> Reporter: Ben Lau
> Assignee: Sean Busbey
> Attachments: abstraction.patch
>
>
> Ticket for work in progress on new FileSystem abstractions. Previously, we
> (Yahoo) submitted a ticket that would add support for humongous (1 million
> region+) tables via a hierarchical layout (HBASE-13991). However open source
> is moving in a similar but not identical direction in the future and so the
> patch will not be merged into open source.
> We will be working on a different patch now with folks from open source. It
> will create/add to 2 layers-- a path abstraction layer and a use-oriented
> abstraction layer. The path abstraction layer is epitomized by classes like
> FsUtils (and in the patch new classes like AFsLayout). The use oriented
> abstraction layer is epitomized by existing classes like
> MasterFileSystem/HRegionFileSystem (and possibly new classes later) that
> build on the path abstraction layer and focus on 'doing things' (eg creating
> regions) and less on the gritty details like the paths.
> This work on abstracting and isolating the paths from the use cases will help
> Yahoo not diverge too much from open source with its internal 'Humongous'
> table hierarchical layout, while also helping open source move further
> towards the eventual goal of redoing the FS layout in a similar (but
> different) hierarchical layout later that focuses on data directory
> uniformity (unlike the humongous patch) and storing hierarchy in the meta
> table instead which enables new optimizations (see HBASE-14090.)
> Attached to this ticket is some work we've done at Yahoo so far that will be
> put into an open source HBase branch for further collaboration. The patch is
> not meant to be complete yet and is a work in progress. (Please wait on
> patch comments/reviews.) It also includes some Yahoo-specific 'humongous'
> layout code that will be removed before submission in open source.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
