[
https://issues.apache.org/jira/browse/HBASE-28216?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17810604#comment-17810604
]
ASF GitHub Bot commented on HBASE-28216:
----------------------------------------
Apache-HBase commented on PR #5591:
URL: https://github.com/apache/hbase/pull/5591#issuecomment-1909148092
:broken_heart: **-1 overall**
| Vote | Subsystem | Runtime | Comment |
|:----:|----------:|--------:|:--------|
| +0 :ok: | reexec | 0m 55s | Docker mode activated. |
| -0 :warning: | yetus | 0m 5s | Unprocessed flag(s):
--brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list
--whitespace-tabs-ignore-list --quick-hadoopcheck |
||| _ Prechecks _ |
||| _ branch-2 Compile Tests _ |
| +0 :ok: | mvndep | 0m 16s | Maven dependency ordering for branch |
| +1 :green_heart: | mvninstall | 2m 58s | branch-2 passed |
| +1 :green_heart: | compile | 1m 57s | branch-2 passed |
| +1 :green_heart: | shadedjars | 5m 5s | branch has no errors when
building our shaded downstream artifacts. |
| +1 :green_heart: | javadoc | 1m 6s | branch-2 passed |
||| _ Patch Compile Tests _ |
| +0 :ok: | mvndep | 0m 13s | Maven dependency ordering for patch |
| +1 :green_heart: | mvninstall | 2m 53s | the patch passed |
| +1 :green_heart: | compile | 2m 9s | the patch passed |
| +1 :green_heart: | javac | 2m 9s | the patch passed |
| +1 :green_heart: | shadedjars | 5m 20s | patch has no errors when
building our shaded downstream artifacts. |
| +1 :green_heart: | javadoc | 1m 8s | the patch passed |
||| _ Other Tests _ |
| +1 :green_heart: | unit | 0m 31s | hbase-protocol-shaded in the patch
passed. |
| +1 :green_heart: | unit | 8m 25s | hbase-client in the patch passed.
|
| -1 :x: | unit | 241m 33s | hbase-server in the patch failed. |
| +1 :green_heart: | unit | 8m 10s | hbase-shell in the patch passed. |
| | | 288m 12s | |
| Subsystem | Report/Notes |
|----------:|:-------------|
| Docker | ClientAPI=1.44 ServerAPI=1.44 base:
https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5591/4/artifact/yetus-jdk8-hadoop2-check/output/Dockerfile
|
| GITHUB PR | https://github.com/apache/hbase/pull/5591 |
| Optional Tests | javac javadoc unit shadedjars compile |
| uname | Linux 6fd72ae33876 5.4.0-169-generic #187-Ubuntu SMP Thu Nov 23
14:52:28 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/hbase-personality.sh |
| git revision | branch-2 / 97974024a8 |
| Default Java | Temurin-1.8.0_352-b08 |
| unit |
https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5591/4/artifact/yetus-jdk8-hadoop2-check/output/patch-unit-hbase-server.txt
|
| Test Results |
https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5591/4/testReport/
|
| Max. process+thread count | 4118 (vs. ulimit of 30000) |
| modules | C: hbase-protocol-shaded hbase-client hbase-server hbase-shell
U: . |
| Console output |
https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5591/4/console
|
| versions | git=2.34.1 maven=3.8.6 |
| Powered by | Apache Yetus 0.12.0 https://yetus.apache.org |
This message was automatically generated.
> HDFS erasure coding support for table data dirs
> -----------------------------------------------
>
> Key: HBASE-28216
> URL: https://issues.apache.org/jira/browse/HBASE-28216
> Project: HBase
> Issue Type: New Feature
> Reporter: Bryan Beaudreault
> Assignee: Bryan Beaudreault
> Priority: Major
> Labels: patch-available
>
> [Erasure
> coding|https://hadoop.apache.org/docs/stable/hadoop-project-dist/hadoop-hdfs/HDFSErasureCoding.html]
> (EC) is a hadoop-3 feature which can drastically reduce storage
> requirements, at the expense of locality. At my company we have a few hbase
> clusters which are extremely data dense and take mostly write traffic, fewer
> reads (cold data). We'd like to reduce the cost of these clusters, and EC is
> a great way to do that since it can reduce replication related storage costs
> by 50%.
> It's possible to enable EC policies on sub directories of HDFS. One can
> manually set this with {{{}hdfs ec -setPolicy -path
> /hbase/data/default/usertable -policy xxxx{}}}. This can work without any
> hbase support.
> One problem with that is a lack of visibility by operators into which tables
> might have EC enabled. I think this is where HBase can help. Here's my
> proposal:
> * Add a new TableDescriptor and ColumnDescriptor field ERASURE_CODING_POLICY
> * In ModifyTableProcedure preflightChecks, if ERASURE_CODING_POLICY is set,
> verify that the requested policy is available and enabled via
> DistributedFileSystem.
> getErasureCodingPolicies().
> * During ModifyTableProcedure, add a new state for
> MODIFY_TABLE_SYNC_ERASURE_CODING_POLICY.
> ** When adding or changing a policy, use DistributedFileSystem.
> setErasureCodingPolicy to sync it for the data and archive dir of that table
> (or column in table)
> ** When removing the property or setting it to empty, use
> DistributedFileSystem.
> unsetErasureCodingPolicy to remove it from the data and archive dir.
> Since this new API is in hadoop-3 only, we'll need to add a reflection
> wrapper class for managing the calls and verifying that the API is available.
> We'll similarly do that API check in preflightChecks.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
