Zach York commented on HBASE-20429:

[~apurtell] I'll look into the specifics in a little bit, but I feel like 
relying less on the FS (atomic renames for example) might be the right way to 
go here. A while back there was some work done (or proposed) to have HBase 
handle the file metadata somewhere to avoid the necessity of renames (HBase 
would update the path/location in this table so in effect, the rename would be 
atomic). I didn't spend a ton of time looking for the old issues, but I think 
this one was related: HBASE-14090. 

[~stack] and [~uagashe] did some initial planning on it and I planned to help 
out, but got sidelined by other stuff. They might be able to chime in here as 

> Support for mixed or write-heavy workloads on non-HDFS filesystems
> ------------------------------------------------------------------
>                 Key: HBASE-20429
>                 URL: https://issues.apache.org/jira/browse/HBASE-20429
>             Project: HBase
>          Issue Type: Umbrella
>            Reporter: Andrew Purtell
>            Priority: Major
> We can support reasonably well use cases on non-HDFS filesystems, like S3, 
> where an external writer has loaded (and continues to load) HFiles via the 
> bulk load mechanism, and then we serve out a read only workload at the HBase 
> API.
> Mixed workloads or write-heavy workloads won't fare as well. In fact, data 
> loss seems certain. It will depend in the specific filesystem, but all of the 
> S3 backed Hadoop filesystems suffer from a couple of obvious problems, 
> notably a lack of atomic rename. 
> This umbrella will serve to collect some related ideas for consideration.

This message was sent by Atlassian JIRA

Reply via email to