[
https://issues.apache.org/jira/browse/HBASE-28216?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17810145#comment-17810145
]
ASF GitHub Bot commented on HBASE-28216:
----------------------------------------
Apache-HBase commented on PR #5591:
URL: https://github.com/apache/hbase/pull/5591#issuecomment-1907152047
:confetti_ball: **+1 overall**
| Vote | Subsystem | Runtime | Comment |
|:----:|----------:|--------:|:--------|
| +0 :ok: | reexec | 0m 37s | Docker mode activated. |
||| _ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | No case conflicting files
found. |
| +0 :ok: | prototool | 0m 0s | prototool was not available. |
| +1 :green_heart: | hbaseanti | 0m 0s | Patch does not have any
anti-patterns. |
| +1 :green_heart: | @author | 0m 0s | The patch does not contain any
@author tags. |
||| _ branch-2 Compile Tests _ |
| +0 :ok: | mvndep | 0m 12s | Maven dependency ordering for branch |
| +1 :green_heart: | mvninstall | 3m 18s | branch-2 passed |
| +1 :green_heart: | compile | 4m 11s | branch-2 passed |
| +1 :green_heart: | checkstyle | 1m 4s | branch-2 passed |
| +1 :green_heart: | spotless | 0m 44s | branch has no errors when
running spotless:check. |
| +1 :green_heart: | spotbugs | 4m 16s | branch-2 passed |
||| _ Patch Compile Tests _ |
| +0 :ok: | mvndep | 0m 18s | Maven dependency ordering for patch |
| +1 :green_heart: | mvninstall | 2m 52s | the patch passed |
| +1 :green_heart: | compile | 4m 4s | the patch passed |
| +1 :green_heart: | cc | 4m 4s | the patch passed |
| +1 :green_heart: | javac | 4m 4s | the patch passed |
| -0 :warning: | checkstyle | 0m 35s | hbase-server: The patch
generated 1 new + 7 unchanged - 0 fixed = 8 total (was 7) |
| -0 :warning: | rubocop | 0m 8s | The patch generated 8 new + 701
unchanged - 3 fixed = 709 total (was 704) |
| +1 :green_heart: | whitespace | 0m 0s | The patch has no whitespace
issues. |
| +1 :green_heart: | hadoopcheck | 9m 46s | Patch does not cause any
errors with Hadoop 2.10.2 or 3.3.6. |
| +1 :green_heart: | hbaseprotoc | 1m 28s | the patch passed |
| +1 :green_heart: | spotless | 0m 42s | patch has no errors when
running spotless:check. |
| +1 :green_heart: | spotbugs | 4m 39s | the patch passed |
||| _ Other Tests _ |
| +1 :green_heart: | asflicense | 0m 30s | The patch does not generate
ASF License warnings. |
| | | 41m 57s | |
| Subsystem | Report/Notes |
|----------:|:-------------|
| Docker | ClientAPI=1.43 ServerAPI=1.43 base:
https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5591/3/artifact/yetus-general-check/output/Dockerfile
|
| GITHUB PR | https://github.com/apache/hbase/pull/5591 |
| Optional Tests | dupname asflicense javac spotbugs hadoopcheck hbaseanti
spotless checkstyle compile cc hbaseprotoc prototool rubocop |
| uname | Linux f3b39c633c58 5.4.0-1103-aws #111~18.04.1-Ubuntu SMP Tue May
23 20:04:10 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/hbase-personality.sh |
| git revision | branch-2 / 5488acfeaa |
| Default Java | Eclipse Adoptium-11.0.17+8 |
| checkstyle |
https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5591/3/artifact/yetus-general-check/output/diff-checkstyle-hbase-server.txt
|
| rubocop |
https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5591/3/artifact/yetus-general-check/output/diff-patch-rubocop.txt
|
| Max. process+thread count | 80 (vs. ulimit of 30000) |
| modules | C: hbase-protocol-shaded hbase-client hbase-server hbase-shell
U: . |
| Console output |
https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5591/3/console
|
| versions | git=2.34.1 maven=3.8.6 spotbugs=4.7.3 rubocop=1.37.1 |
| Powered by | Apache Yetus 0.12.0 https://yetus.apache.org |
This message was automatically generated.
> HDFS erasure coding support for table data dirs
> -----------------------------------------------
>
> Key: HBASE-28216
> URL: https://issues.apache.org/jira/browse/HBASE-28216
> Project: HBase
> Issue Type: New Feature
> Reporter: Bryan Beaudreault
> Assignee: Bryan Beaudreault
> Priority: Major
> Labels: patch-available
>
> [Erasure
> coding|https://hadoop.apache.org/docs/stable/hadoop-project-dist/hadoop-hdfs/HDFSErasureCoding.html]
> (EC) is a hadoop-3 feature which can drastically reduce storage
> requirements, at the expense of locality. At my company we have a few hbase
> clusters which are extremely data dense and take mostly write traffic, fewer
> reads (cold data). We'd like to reduce the cost of these clusters, and EC is
> a great way to do that since it can reduce replication related storage costs
> by 50%.
> It's possible to enable EC policies on sub directories of HDFS. One can
> manually set this with {{{}hdfs ec -setPolicy -path
> /hbase/data/default/usertable -policy xxxx{}}}. This can work without any
> hbase support.
> One problem with that is a lack of visibility by operators into which tables
> might have EC enabled. I think this is where HBase can help. Here's my
> proposal:
> * Add a new TableDescriptor and ColumnDescriptor field ERASURE_CODING_POLICY
> * In ModifyTableProcedure preflightChecks, if ERASURE_CODING_POLICY is set,
> verify that the requested policy is available and enabled via
> DistributedFileSystem.
> getErasureCodingPolicies().
> * During ModifyTableProcedure, add a new state for
> MODIFY_TABLE_SYNC_ERASURE_CODING_POLICY.
> ** When adding or changing a policy, use DistributedFileSystem.
> setErasureCodingPolicy to sync it for the data and archive dir of that table
> (or column in table)
> ** When removing the property or setting it to empty, use
> DistributedFileSystem.
> unsetErasureCodingPolicy to remove it from the data and archive dir.
> Since this new API is in hadoop-3 only, we'll need to add a reflection
> wrapper class for managing the calls and verifying that the API is available.
> We'll similarly do that API check in preflightChecks.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
