[
https://issues.apache.org/jira/browse/HBASE-28216?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17914976#comment-17914976
]
Ray Mattingly commented on HBASE-28216:
---------------------------------------
We're in the process of upgrading our production systems from 2.5 to 2.6.1, so
I can't speak to EC+2.6.x specifically. But we're successfully using erasure
coding on several of our largest clusters, most of which are running a branch
of 2.5 (with several 2.6+ changes backported). Here's a rundown of our
experience so far:
# If you're constrained by data size, then EC will likely save you
significantly on the bottomline. Assuming that data size continues to be your
only bottleneck, it could likely save you as much as 50%.
# All reads will have bad locality on an EC cluster. This obviously hurts
performance. Less obviously, this also hurts isolation — if one DataNode has a
subtly impaired disk, for example, then it could affect the performance of
otherwise unrelated RegionServer processes that are running on other nodes.
We're accustomed to running at ~100% locality, so this has been a bit of a
headache.
# Because locality is no longer a thing, you can also use the HDFS balancer
without worrying about which paths it's touching. Anecdotally, I would
encourage you to use it because DataNode disk utilization will be pretty
important in balancing the load of an EC cluster.
# DataNode decommissions take a lot longer and are a lot more expensive. We're
still tweaking the various levers available to us to find the correct balance
that will decom a DN as quickly as possible, while not saturating the disk
throughput of the DN (which causes its own problems, since a decom'ing DN is
still in the hot path of read requests).
> HDFS erasure coding support for table data dirs
> -----------------------------------------------
>
> Key: HBASE-28216
> URL: https://issues.apache.org/jira/browse/HBASE-28216
> Project: HBase
> Issue Type: New Feature
> Reporter: Bryan Beaudreault
> Assignee: Bryan Beaudreault
> Priority: Major
> Labels: patch-available, pull-request-available
> Fix For: 2.6.0, 3.0.0-beta-2
>
>
> [Erasure
> coding|https://hadoop.apache.org/docs/stable/hadoop-project-dist/hadoop-hdfs/HDFSErasureCoding.html]
> (EC) is a hadoop-3 feature which can drastically reduce storage
> requirements, at the expense of locality. At my company we have a few hbase
> clusters which are extremely data dense and take mostly write traffic, fewer
> reads (cold data). We'd like to reduce the cost of these clusters, and EC is
> a great way to do that since it can reduce replication related storage costs
> by 50%.
> It's possible to enable EC policies on sub directories of HDFS. One can
> manually set this with {{{}hdfs ec -setPolicy -path
> /hbase/data/default/usertable -policy xxxx{}}}. This can work without any
> hbase support.
> One problem with that is a lack of visibility by operators into which tables
> might have EC enabled. I think this is where HBase can help. Here's my
> proposal:
> * Add a new TableDescriptor and ColumnDescriptor field ERASURE_CODING_POLICY
> * In ModifyTableProcedure preflightChecks, if ERASURE_CODING_POLICY is set,
> verify that the requested policy is available and enabled via
> DistributedFileSystem.
> getErasureCodingPolicies().
> * During ModifyTableProcedure, add a new state for
> MODIFY_TABLE_SYNC_ERASURE_CODING_POLICY.
> ** When adding or changing a policy, use DistributedFileSystem.
> setErasureCodingPolicy to sync it for the data and archive dir of that table
> (or column in table)
> ** When removing the property or setting it to empty, use
> DistributedFileSystem.
> unsetErasureCodingPolicy to remove it from the data and archive dir.
> Since this new API is in hadoop-3 only, we'll need to add a reflection
> wrapper class for managing the calls and verifying that the API is available.
> We'll similarly do that API check in preflightChecks.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
