[
https://issues.apache.org/jira/browse/HBASE-20429?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16596423#comment-16596423
]
Steve Loughran commented on HBASE-20429:
----------------------------------------
BTW, HADOOP-15691 is my latest iteration of having each FS declare its
capabilities. As I've noted at the end, as well as through a new interface, we
could expose this as new config options you can look for in
fsInstance.getCon().get("option"), provided the FS instances clone their
supplied configs and then patch them. This would let you check to see what an
FS offered.
w.r.t s3guard, need to know what semantics you get. With S3Guard you get
consistent listings, but rename is still sub-atomic
thanks for promising to invite me to any discussions, as long as it not via
Amazon Chime or Skype for Business I'm up for it.
> Support for mixed or write-heavy workloads on non-HDFS filesystems
> ------------------------------------------------------------------
>
> Key: HBASE-20429
> URL: https://issues.apache.org/jira/browse/HBASE-20429
> Project: HBase
> Issue Type: Umbrella
> Reporter: Andrew Purtell
> Priority: Major
>
> We can support reasonably well use cases on non-HDFS filesystems, like S3,
> where an external writer has loaded (and continues to load) HFiles via the
> bulk load mechanism, and then we serve out a read only workload at the HBase
> API.
> Mixed workloads or write-heavy workloads won't fare as well. In fact, data
> loss seems certain. It will depend in the specific filesystem, but all of the
> S3 backed Hadoop filesystems suffer from a couple of obvious problems,
> notably a lack of atomic rename.
> This umbrella will serve to collect some related ideas for consideration.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
