[jira] [Commented] (HBASE-27632) Refactor WAL.Reader implementation so we can better support WAL splitting and replication
[
https://issues.apache.org/jira/browse/HBASE-27632?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17699166#comment-17699166
]
Hudson commented on HBASE-27632:
Results for branch branch-2
[build #764 on
builds.a.o|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-2/764/]:
(/) *{color:green}+1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-2/764/General_20Nightly_20Build_20Report/]
(/) {color:green}+1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-2/764/JDK8_20Nightly_20Build_20Report_20_28Hadoop2_29/]
(/) {color:green}+1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-2/764/JDK8_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(/) {color:green}+1 jdk11 hadoop3 checks{color}
-- For more information [see jdk11
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-2/764/JDK11_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
(/) {color:green}+1 client integration test{color}
> Refactor WAL.Reader implementation so we can better support WAL splitting and
> replication
> -
>
> Key: HBASE-27632
> URL: https://issues.apache.org/jira/browse/HBASE-27632
> Project: HBase
> Issue Type: Improvement
> Components: Replication, wal
>Reporter: Duo Zhang
>Assignee: Duo Zhang
>Priority: Major
> Fix For: 2.6.0, 3.0.0-alpha-4
>
>
> WAL splitting and replication has completely different requirements on WAL
> reader, the latter one is much comlicated as it wants to tail a WAL file
> which is currently being written.
> I think split them into two classes is better here.
> But we exposes two configurations out
> 'hbase.regionserver.hlog.reader.impl'
> 'hbase.regionserver.hlog.writer.impl'
> which indicates that we only have one implementation for wal reader.
> These two configurations are inrtoduced in 0.96, for reading the pre 0.94
> sequence file format WAL file. And starting from 2.0.0, we have already
> dropped the support of reading sequence file format WAL file, so I think the
> configurations are useless now.
> Now we stil require users to specify SecureProtobufLogReader and
> SecureProtobufLogWriter when enable WAL encryption, but actually they are
> both the only choice, so just use 'hbase.regionserver.wal.encryption' to
> determine whether we need to use SecureProtobufLogWriter to write WAL. And
> for reading, I do not think we should rely on the configuration, we should
> detect whether the WAL file itself is encrypted(I believe we have already
> done this in SecureProtobufLogReader.readHeader.
> So in general, I prefer we just mark these two configurations deprecated and
> do not use them in our code base, so we can have different Reader
> implementations for WAL splitting and replication.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Commented] (HBASE-27688) HFile splitting occurs during bulkload, the CREATE_TIME_TS of hfileinfo is 0
[
https://issues.apache.org/jira/browse/HBASE-27688?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17699164#comment-17699164
]
Duo Zhang commented on HBASE-27688:
---
Based on branch-2 please~
> HFile splitting occurs during bulkload, the CREATE_TIME_TS of hfileinfo is 0
>
>
> Key: HBASE-27688
> URL: https://issues.apache.org/jira/browse/HBASE-27688
> Project: HBase
> Issue Type: Bug
>Reporter: alan.zhao
>Assignee: alan.zhao
>Priority: Major
>
> If HFile splitting occurs during bulkload, the CREATE_TIME_TS of hfileinfo
> =0,When HFile is copied after splitting, CREATE_TIME_TS of the original file
> is not copied。
> {code:java}
> ##BulkLoadHFilesTool.class
> /**
> * Copy half of an HFile into a new HFile.
> */
> private static void copyHFileHalf(Configuration conf, Path inFile, Path
> outFile,
> Reference reference, ColumnFamilyDescriptor familyDescriptor) throws
> IOException {
> FileSystem fs = inFile.getFileSystem(conf);
> CacheConfig cacheConf = CacheConfig.DISABLED;
> HalfStoreFileReader halfReader = null;
> StoreFileWriter halfWriter = null;
> try {
> 。。。
> HFileContext hFileContext = new
> HFileContextBuilder().withCompression(compression)
> .withChecksumType(StoreUtils.getChecksumType(conf))
>
> .withBytesPerCheckSum(StoreUtils.getBytesPerChecksum(conf)).withBlockSize(blocksize)
>
> .withDataBlockEncoding(familyDescriptor.getDataBlockEncoding()).withIncludesTags(true)
> .build();
> // TODO .withCreateTime(EnvironmentEdgeManager.currentTime())
> halfWriter = new StoreFileWriter.Builder(conf, cacheConf,
> fs).withFilePath(outFile)
> .withBloomType(bloomFilterType).withFileContext(hFileContext).build();
> HFileScanner scanner = halfReader.getScanner(false, false, false);
> scanner.seekTo();
> do {
> halfWriter.append(scanner.getCell());
> } while (scanner.next());
> for (Map.Entry entry : fileInfo.entrySet()) {
> if (shouldCopyHFileMetaKey(entry.getKey())) {
> halfWriter.appendFileInfo(entry.getKey(), entry.getValue());
> }
> }
> } finally {
> 。。。
> }
> }
> ##get lastMajorCompactionTs metric
> lastMajorCompactionTs = this.region.getOldestHfileTs(true);
> ...
> long now = EnvironmentEdgeManager.currentTime();
> return now - lastMajorCompactionTs;
> ...
> ##
> public long getOldestHfileTs(boolean majorCompactionOnly) throws IOException {
> long result = Long.MAX_VALUE;
> for (HStore store : stores.values()) {
> Collection storeFiles = store.getStorefiles();
>...
> for (HStoreFile file : storeFiles) {
> StoreFileReader sfReader = file.getReader();
> ...
> result = Math.min(result, reader.getFileContext().getFileCreateTime());
> }
> }
> return result == Long.MAX_VALUE ? 0 : result;
> }{code}
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Commented] (HBASE-27688) HFile splitting occurs during bulkload, the CREATE_TIME_TS of hfileinfo is 0
[
https://issues.apache.org/jira/browse/HBASE-27688?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17699163#comment-17699163
]
alan.zhao commented on HBASE-27688:
---
ok,thanks, I will submit a pr again. Do I need to specify the branch based on
which?
> HFile splitting occurs during bulkload, the CREATE_TIME_TS of hfileinfo is 0
>
>
> Key: HBASE-27688
> URL: https://issues.apache.org/jira/browse/HBASE-27688
> Project: HBase
> Issue Type: Bug
>Reporter: alan.zhao
>Assignee: alan.zhao
>Priority: Major
>
> If HFile splitting occurs during bulkload, the CREATE_TIME_TS of hfileinfo
> =0,When HFile is copied after splitting, CREATE_TIME_TS of the original file
> is not copied。
> {code:java}
> ##BulkLoadHFilesTool.class
> /**
> * Copy half of an HFile into a new HFile.
> */
> private static void copyHFileHalf(Configuration conf, Path inFile, Path
> outFile,
> Reference reference, ColumnFamilyDescriptor familyDescriptor) throws
> IOException {
> FileSystem fs = inFile.getFileSystem(conf);
> CacheConfig cacheConf = CacheConfig.DISABLED;
> HalfStoreFileReader halfReader = null;
> StoreFileWriter halfWriter = null;
> try {
> 。。。
> HFileContext hFileContext = new
> HFileContextBuilder().withCompression(compression)
> .withChecksumType(StoreUtils.getChecksumType(conf))
>
> .withBytesPerCheckSum(StoreUtils.getBytesPerChecksum(conf)).withBlockSize(blocksize)
>
> .withDataBlockEncoding(familyDescriptor.getDataBlockEncoding()).withIncludesTags(true)
> .build();
> // TODO .withCreateTime(EnvironmentEdgeManager.currentTime())
> halfWriter = new StoreFileWriter.Builder(conf, cacheConf,
> fs).withFilePath(outFile)
> .withBloomType(bloomFilterType).withFileContext(hFileContext).build();
> HFileScanner scanner = halfReader.getScanner(false, false, false);
> scanner.seekTo();
> do {
> halfWriter.append(scanner.getCell());
> } while (scanner.next());
> for (Map.Entry entry : fileInfo.entrySet()) {
> if (shouldCopyHFileMetaKey(entry.getKey())) {
> halfWriter.appendFileInfo(entry.getKey(), entry.getValue());
> }
> }
> } finally {
> 。。。
> }
> }
> ##get lastMajorCompactionTs metric
> lastMajorCompactionTs = this.region.getOldestHfileTs(true);
> ...
> long now = EnvironmentEdgeManager.currentTime();
> return now - lastMajorCompactionTs;
> ...
> ##
> public long getOldestHfileTs(boolean majorCompactionOnly) throws IOException {
> long result = Long.MAX_VALUE;
> for (HStore store : stores.values()) {
> Collection storeFiles = store.getStorefiles();
>...
> for (HStoreFile file : storeFiles) {
> StoreFileReader sfReader = file.getReader();
> ...
> result = Math.min(result, reader.getFileContext().getFileCreateTime());
> }
> }
> return result == Long.MAX_VALUE ? 0 : result;
> }{code}
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[GitHub] [hbase] Apache-HBase commented on pull request #4545: HBASE-26034 Add support to take parallel backups
Apache-HBase commented on PR #4545: URL: https://github.com/apache/hbase/pull/4545#issuecomment-1464829102 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 0m 59s | Docker mode activated. | ||| _ Prechecks _ | | +1 :green_heart: | dupname | 0m 0s | No case conflicting files found. | | +1 :green_heart: | hbaseanti | 0m 0s | Patch does not have any anti-patterns. | | +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. | ||| _ master Compile Tests _ | | +1 :green_heart: | mvninstall | 3m 38s | master passed | | +1 :green_heart: | compile | 0m 22s | master passed | | +1 :green_heart: | checkstyle | 0m 8s | master passed | | +1 :green_heart: | spotless | 0m 42s | branch has no errors when running spotless:check. | | +1 :green_heart: | spotbugs | 0m 28s | master passed | ||| _ Patch Compile Tests _ | | +1 :green_heart: | mvninstall | 3m 36s | the patch passed | | +1 :green_heart: | compile | 0m 24s | the patch passed | | +1 :green_heart: | javac | 0m 24s | the patch passed | | +1 :green_heart: | checkstyle | 0m 8s | the patch passed | | +1 :green_heart: | whitespace | 0m 0s | The patch has no whitespace issues. | | +1 :green_heart: | hadoopcheck | 12m 49s | Patch does not cause any errors with Hadoop 3.2.4 3.3.4. | | +1 :green_heart: | spotless | 0m 41s | patch has no errors when running spotless:check. | | +1 :green_heart: | spotbugs | 0m 38s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | asflicense | 0m 7s | The patch does not generate ASF License warnings. | | | | 32m 23s | | | Subsystem | Report/Notes | |--:|:-| | Docker | ClientAPI=1.42 ServerAPI=1.42 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4545/4/artifact/yetus-general-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/4545 | | Optional Tests | dupname asflicense spotless javac spotbugs hadoopcheck hbaseanti checkstyle compile | | uname | Linux 83b10f1cdbc3 5.4.0-135-generic #152-Ubuntu SMP Wed Nov 23 20:19:22 UTC 2022 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / 72d5a46899 | | Default Java | Eclipse Adoptium-11.0.17+8 | | Max. process+thread count | 86 (vs. ulimit of 3) | | modules | C: hbase-backup U: hbase-backup | | Console output | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4545/4/console | | versions | git=2.34.1 maven=3.8.6 spotbugs=4.7.3 | | Powered by | Apache Yetus 0.12.0 https://yetus.apache.org | This message was automatically generated. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #4545: HBASE-26034 Add support to take parallel backups
Apache-HBase commented on PR #4545: URL: https://github.com/apache/hbase/pull/4545#issuecomment-1464828500 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 0m 26s | Docker mode activated. | | -0 :warning: | yetus | 0m 3s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck | ||| _ Prechecks _ | ||| _ master Compile Tests _ | | +1 :green_heart: | mvninstall | 3m 55s | master passed | | +1 :green_heart: | compile | 0m 15s | master passed | | +1 :green_heart: | shadedjars | 4m 51s | branch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 0m 13s | master passed | ||| _ Patch Compile Tests _ | | +1 :green_heart: | mvninstall | 3m 29s | the patch passed | | +1 :green_heart: | compile | 0m 16s | the patch passed | | +1 :green_heart: | javac | 0m 16s | the patch passed | | +1 :green_heart: | shadedjars | 4m 48s | patch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 0m 12s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | unit | 8m 20s | hbase-backup in the patch passed. | | | | 27m 55s | | | Subsystem | Report/Notes | |--:|:-| | Docker | ClientAPI=1.42 ServerAPI=1.42 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4545/4/artifact/yetus-jdk11-hadoop3-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/4545 | | Optional Tests | javac javadoc unit shadedjars compile | | uname | Linux 2bcc01dcdb9d 5.4.0-1092-aws #100~18.04.2-Ubuntu SMP Tue Nov 29 08:39:52 UTC 2022 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / 72d5a46899 | | Default Java | Eclipse Adoptium-11.0.17+8 | | Test Results | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4545/4/testReport/ | | Max. process+thread count | 3580 (vs. ulimit of 3) | | modules | C: hbase-backup U: hbase-backup | | Console output | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4545/4/console | | versions | git=2.34.1 maven=3.8.6 | | Powered by | Apache Yetus 0.12.0 https://yetus.apache.org | This message was automatically generated. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #4545: HBASE-26034 Add support to take parallel backups
Apache-HBase commented on PR #4545: URL: https://github.com/apache/hbase/pull/4545#issuecomment-1464828340 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 0m 56s | Docker mode activated. | | -0 :warning: | yetus | 0m 2s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck | ||| _ Prechecks _ | ||| _ master Compile Tests _ | | +1 :green_heart: | mvninstall | 2m 47s | master passed | | +1 :green_heart: | compile | 0m 17s | master passed | | +1 :green_heart: | shadedjars | 4m 21s | branch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 0m 10s | master passed | ||| _ Patch Compile Tests _ | | +1 :green_heart: | mvninstall | 2m 54s | the patch passed | | +1 :green_heart: | compile | 0m 16s | the patch passed | | +1 :green_heart: | javac | 0m 16s | the patch passed | | +1 :green_heart: | shadedjars | 4m 19s | patch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 0m 13s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | unit | 9m 22s | hbase-backup in the patch passed. | | | | 26m 42s | | | Subsystem | Report/Notes | |--:|:-| | Docker | ClientAPI=1.42 ServerAPI=1.42 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4545/4/artifact/yetus-jdk8-hadoop3-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/4545 | | Optional Tests | javac javadoc unit shadedjars compile | | uname | Linux 627d371adcc5 5.4.0-137-generic #154-Ubuntu SMP Thu Jan 5 17:03:22 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / 72d5a46899 | | Default Java | Temurin-1.8.0_352-b08 | | Test Results | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4545/4/testReport/ | | Max. process+thread count | 3098 (vs. ulimit of 3) | | modules | C: hbase-backup U: hbase-backup | | Console output | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4545/4/console | | versions | git=2.34.1 maven=3.8.6 | | Powered by | Apache Yetus 0.12.0 https://yetus.apache.org | This message was automatically generated. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #4545: HBASE-26034 Add support to take parallel backups
Apache-HBase commented on PR #4545: URL: https://github.com/apache/hbase/pull/4545#issuecomment-1464822587 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 1m 1s | Docker mode activated. | ||| _ Prechecks _ | | +1 :green_heart: | dupname | 0m 0s | No case conflicting files found. | | +1 :green_heart: | hbaseanti | 0m 0s | Patch does not have any anti-patterns. | | +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. | ||| _ master Compile Tests _ | | +1 :green_heart: | mvninstall | 3m 30s | master passed | | +1 :green_heart: | compile | 0m 30s | master passed | | +1 :green_heart: | checkstyle | 0m 16s | master passed | | +1 :green_heart: | spotless | 1m 2s | branch has no errors when running spotless:check. | | +1 :green_heart: | spotbugs | 0m 38s | master passed | ||| _ Patch Compile Tests _ | | +1 :green_heart: | mvninstall | 3m 51s | the patch passed | | +1 :green_heart: | compile | 0m 25s | the patch passed | | +1 :green_heart: | javac | 0m 25s | the patch passed | | +1 :green_heart: | checkstyle | 0m 10s | the patch passed | | +1 :green_heart: | whitespace | 0m 0s | The patch has no whitespace issues. | | +1 :green_heart: | hadoopcheck | 13m 39s | Patch does not cause any errors with Hadoop 3.2.4 3.3.4. | | +1 :green_heart: | spotless | 0m 42s | patch has no errors when running spotless:check. | | +1 :green_heart: | spotbugs | 0m 38s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | asflicense | 0m 10s | The patch does not generate ASF License warnings. | | | | 34m 22s | | | Subsystem | Report/Notes | |--:|:-| | Docker | ClientAPI=1.42 ServerAPI=1.42 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4545/3/artifact/yetus-general-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/4545 | | Optional Tests | dupname asflicense spotless javac spotbugs hadoopcheck hbaseanti checkstyle compile | | uname | Linux 78e017ab3a13 5.4.0-135-generic #152-Ubuntu SMP Wed Nov 23 20:19:22 UTC 2022 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / 72d5a46899 | | Default Java | Eclipse Adoptium-11.0.17+8 | | Max. process+thread count | 85 (vs. ulimit of 3) | | modules | C: hbase-backup U: hbase-backup | | Console output | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4545/3/console | | versions | git=2.34.1 maven=3.8.6 spotbugs=4.7.3 | | Powered by | Apache Yetus 0.12.0 https://yetus.apache.org | This message was automatically generated. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #4545: HBASE-26034 Add support to take parallel backups
Apache-HBase commented on PR #4545: URL: https://github.com/apache/hbase/pull/4545#issuecomment-1464821805 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 0m 25s | Docker mode activated. | | -0 :warning: | yetus | 0m 3s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck | ||| _ Prechecks _ | ||| _ master Compile Tests _ | | +1 :green_heart: | mvninstall | 3m 56s | master passed | | +1 :green_heart: | compile | 0m 15s | master passed | | +1 :green_heart: | shadedjars | 4m 59s | branch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 0m 14s | master passed | ||| _ Patch Compile Tests _ | | +1 :green_heart: | mvninstall | 3m 36s | the patch passed | | +1 :green_heart: | compile | 0m 18s | the patch passed | | +1 :green_heart: | javac | 0m 18s | the patch passed | | +1 :green_heart: | shadedjars | 4m 57s | patch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 0m 12s | the patch passed | ||| _ Other Tests _ | | -1 :x: | unit | 8m 37s | hbase-backup in the patch failed. | | | | 28m 38s | | | Subsystem | Report/Notes | |--:|:-| | Docker | ClientAPI=1.42 ServerAPI=1.42 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4545/3/artifact/yetus-jdk11-hadoop3-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/4545 | | Optional Tests | javac javadoc unit shadedjars compile | | uname | Linux 8ef5067e0518 5.4.0-1092-aws #100~18.04.2-Ubuntu SMP Tue Nov 29 08:39:52 UTC 2022 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / 72d5a46899 | | Default Java | Eclipse Adoptium-11.0.17+8 | | unit | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4545/3/artifact/yetus-jdk11-hadoop3-check/output/patch-unit-hbase-backup.txt | | Test Results | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4545/3/testReport/ | | Max. process+thread count | 3619 (vs. ulimit of 3) | | modules | C: hbase-backup U: hbase-backup | | Console output | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4545/3/console | | versions | git=2.34.1 maven=3.8.6 | | Powered by | Apache Yetus 0.12.0 https://yetus.apache.org | This message was automatically generated. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #4545: HBASE-26034 Add support to take parallel backups
Apache-HBase commented on PR #4545: URL: https://github.com/apache/hbase/pull/4545#issuecomment-1464821536 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 0m 54s | Docker mode activated. | | -0 :warning: | yetus | 0m 3s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck | ||| _ Prechecks _ | ||| _ master Compile Tests _ | | +1 :green_heart: | mvninstall | 3m 5s | master passed | | +1 :green_heart: | compile | 0m 16s | master passed | | +1 :green_heart: | shadedjars | 4m 22s | branch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 0m 14s | master passed | ||| _ Patch Compile Tests _ | | +1 :green_heart: | mvninstall | 2m 52s | the patch passed | | +1 :green_heart: | compile | 0m 16s | the patch passed | | +1 :green_heart: | javac | 0m 16s | the patch passed | | +1 :green_heart: | shadedjars | 4m 20s | patch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 0m 13s | the patch passed | ||| _ Other Tests _ | | -1 :x: | unit | 9m 20s | hbase-backup in the patch failed. | | | | 27m 2s | | | Subsystem | Report/Notes | |--:|:-| | Docker | ClientAPI=1.42 ServerAPI=1.42 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4545/3/artifact/yetus-jdk8-hadoop3-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/4545 | | Optional Tests | javac javadoc unit shadedjars compile | | uname | Linux c52d732fe4be 5.4.0-137-generic #154-Ubuntu SMP Thu Jan 5 17:03:22 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / 72d5a46899 | | Default Java | Temurin-1.8.0_352-b08 | | unit | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4545/3/artifact/yetus-jdk8-hadoop3-check/output/patch-unit-hbase-backup.txt | | Test Results | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4545/3/testReport/ | | Max. process+thread count | 3162 (vs. ulimit of 3) | | modules | C: hbase-backup U: hbase-backup | | Console output | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4545/3/console | | versions | git=2.34.1 maven=3.8.6 | | Powered by | Apache Yetus 0.12.0 https://yetus.apache.org | This message was automatically generated. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Commented] (HBASE-27704) Quotas can drastically overflow configured limit
[
https://issues.apache.org/jira/browse/HBASE-27704?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17699152#comment-17699152
]
Bryan Beaudreault commented on HBASE-27704:
---
Thanks Duo. Due to how quotas work in hbase, I think we need to handle
overconsumption. This is because we have a model like this:
{code:java}
// consume an estimate of resources (100 bytes for gets, 1000 for scans)
OperationQuota quota = quotaManager.checkQuota(region, operation);
try {
// do stuff and count bytes of response
quota.addScanResult(result);
} finally {
// consume actual amount from limiter
quota.close();
}{code}
The actual amount consumed in quota.close() can be drastically larger than the
estimate. Even for scans where we limit them using ScannerContext and
quota.getReadAvailable(), with enough concurrency the value of getReadAvailable
is not sufficient. Gets are not limited at all, so can be even worse here when
hitting large rows.
Say you have a limit of 1MB/s. A large scan comes in, checkQuota consumes
estimate of 1000 bytes. getReadAvailable says there's 0.99MB available. Scan
wants to return 10MB but is limited to 0.99MB by ScannerContext. This consumes
the entire quota, which seems fine. Next request would get an exception, until
quota available builds up to at least 1000 bytes.
But lets say 15 of these scans came in at once. There's no synchronization on
getReadAvailable, so they all get about 0.99MB back after consuming estimate,
and all want to read 10MB. They are each limited to 0.99MB, but in total that
equals up to almost 15MB. When the scans finish, the 15MB is subtracted from
the quota available. We should really be at -14MB, but we are at 0MB due to
this bug.
That large burst should cause the client to have to back off for 14s, as
they've consumed way too much quota. But since we stop at 0MB, in 1 second we
are back at full quota. The pattern can repeat at this point.
So this pattern can lead to a situation where even though we are limiting to
1MB/sec, the client is actually able to achieve almost 15MB/sec. It could be
higher with more concurrency.
I think there are two options to solve this:
# Use better estimates in checkQuota. If we could achieve a better estimate,
getReadAvailable would be more accurate. But this would be hard to do cheaply,
and would not eliminate the problem just minimize it a bit.
# Properly handle over consumption by allowing the quota available to go below
0, as originally intended. This is the fairest way to ensure the quota is
actually honored. It does mean clients might see "black out periods" where they
can't submit anything if they way exceed the quota. We could potentially work
around that by limiting how far negative we can go, or allowing it to fill up
to 0 faster than usual. These don't seem necessary in my testing though.
> Quotas can drastically overflow configured limit
>
>
> Key: HBASE-27704
> URL: https://issues.apache.org/jira/browse/HBASE-27704
> Project: HBase
> Issue Type: Bug
>Reporter: Bryan Beaudreault
>Priority: Major
> Attachments: Screenshot 2023-03-10 at 5.17.51 PM.png
>
>
> The original implementation did not allow exceeding quota. For example, you
> specify a limit of 10 resource/sec and consume 20 resources, it takes 1.1
> seconds to be able submit another request. This was covered by the
> [testOverconsumption in
> TestRateLimiter|https://github.com/apache/hbase/blame/587b0b4f20bdc0415b6541023e611b69c87dba15/hbase-server/src/test/java/org/apache/hadoop/hbase/quotas/TestRateLimiter.java#L97].
> As an incidental part of HBASE-13686, that logic was changed. There is no
> mention of the reasoning behind the change in the issue comments or review
> board, I think it was missed. The goal of that issue was to add different
> refill strategies, but it also modified the over consumption. The
> testOverconsumption was [split out for both refill
> strategies|https://github.com/apache/hbase/blame/master/hbase-server/src/test/java/org/apache/hadoop/hbase/quotas/TestRateLimiter.java#L104-L159],
> but the core reasoning was lost. The comment says:
> {code:java}
> // 10 resources are available, but we need to consume 20 resources109
> // Verify that we have to wait at least 1.1sec to have 1 resource available
> {code}
> But the actual test was updated to only require a new resource after 100ms.
> This is incorrect.
> The problem is, when consuming if you go negative it sets to 0
> [here|https://github.com/apache/hbase/blame/master/hbase-server/src/main/java/org/apache/hadoop/hbase/quotas/RateLimiter.java#L187-L191].
> Additionally, when refilling the new logic does a Math.max(0, available +
> refillAmount):
>
[GitHub] [hbase] Apache-HBase commented on pull request #4544: HBASE-26322 add rsgroup support for backup
Apache-HBase commented on PR #4544: URL: https://github.com/apache/hbase/pull/4544#issuecomment-1464803117 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 1m 5s | Docker mode activated. | | -0 :warning: | yetus | 0m 2s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck | ||| _ Prechecks _ | ||| _ master Compile Tests _ | | +0 :ok: | mvndep | 0m 11s | Maven dependency ordering for branch | | +1 :green_heart: | mvninstall | 4m 15s | master passed | | +1 :green_heart: | compile | 1m 13s | master passed | | +1 :green_heart: | shadedjars | 5m 31s | branch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 0m 43s | master passed | ||| _ Patch Compile Tests _ | | +0 :ok: | mvndep | 0m 13s | Maven dependency ordering for patch | | +1 :green_heart: | mvninstall | 3m 34s | the patch passed | | +1 :green_heart: | compile | 1m 4s | the patch passed | | +1 :green_heart: | javac | 1m 4s | the patch passed | | +1 :green_heart: | shadedjars | 4m 59s | patch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 0m 44s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | unit | 2m 58s | hbase-common in the patch passed. | | +1 :green_heart: | unit | 11m 51s | hbase-backup in the patch passed. | | +1 :green_heart: | unit | 0m 44s | hbase-it in the patch passed. | | | | 40m 50s | | | Subsystem | Report/Notes | |--:|:-| | Docker | ClientAPI=1.42 ServerAPI=1.42 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4544/2/artifact/yetus-jdk11-hadoop3-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/4544 | | Optional Tests | javac javadoc unit shadedjars compile | | uname | Linux de911ba1d2d9 5.4.0-135-generic #152-Ubuntu SMP Wed Nov 23 20:19:22 UTC 2022 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / 72d5a46899 | | Default Java | Eclipse Adoptium-11.0.17+8 | | Test Results | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4544/2/testReport/ | | Max. process+thread count | 3346 (vs. ulimit of 3) | | modules | C: hbase-common hbase-backup hbase-it U: . | | Console output | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4544/2/console | | versions | git=2.34.1 maven=3.8.6 | | Powered by | Apache Yetus 0.12.0 https://yetus.apache.org | This message was automatically generated. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #4544: HBASE-26322 add rsgroup support for backup
Apache-HBase commented on PR #4544: URL: https://github.com/apache/hbase/pull/4544#issuecomment-1464802545 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 1m 0s | Docker mode activated. | ||| _ Prechecks _ | | +1 :green_heart: | dupname | 0m 0s | No case conflicting files found. | | +1 :green_heart: | hbaseanti | 0m 0s | Patch does not have any anti-patterns. | | +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. | ||| _ master Compile Tests _ | | +0 :ok: | mvndep | 0m 12s | Maven dependency ordering for branch | | +1 :green_heart: | mvninstall | 3m 34s | master passed | | +1 :green_heart: | compile | 1m 25s | master passed | | +1 :green_heart: | checkstyle | 0m 37s | master passed | | +1 :green_heart: | spotless | 0m 43s | branch has no errors when running spotless:check. | | +1 :green_heart: | spotbugs | 1m 24s | master passed | ||| _ Patch Compile Tests _ | | +0 :ok: | mvndep | 0m 11s | Maven dependency ordering for patch | | +1 :green_heart: | mvninstall | 3m 22s | the patch passed | | +1 :green_heart: | compile | 1m 21s | the patch passed | | +1 :green_heart: | javac | 1m 21s | the patch passed | | +1 :green_heart: | checkstyle | 0m 37s | the patch passed | | +1 :green_heart: | whitespace | 0m 0s | The patch has no whitespace issues. | | +1 :green_heart: | hadoopcheck | 12m 28s | Patch does not cause any errors with Hadoop 3.2.4 3.3.4. | | +1 :green_heart: | spotless | 0m 40s | patch has no errors when running spotless:check. | | +1 :green_heart: | spotbugs | 1m 46s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | asflicense | 0m 29s | The patch does not generate ASF License warnings. | | | | 37m 37s | | | Subsystem | Report/Notes | |--:|:-| | Docker | ClientAPI=1.42 ServerAPI=1.42 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4544/2/artifact/yetus-general-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/4544 | | Optional Tests | dupname asflicense javac spotbugs hadoopcheck hbaseanti spotless checkstyle compile | | uname | Linux 4070304c53f8 5.4.0-135-generic #152-Ubuntu SMP Wed Nov 23 20:19:22 UTC 2022 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / 72d5a46899 | | Default Java | Eclipse Adoptium-11.0.17+8 | | Max. process+thread count | 81 (vs. ulimit of 3) | | modules | C: hbase-common hbase-backup hbase-it U: . | | Console output | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4544/2/console | | versions | git=2.34.1 maven=3.8.6 spotbugs=4.7.3 | | Powered by | Apache Yetus 0.12.0 https://yetus.apache.org | This message was automatically generated. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #4544: HBASE-26322 add rsgroup support for backup
Apache-HBase commented on PR #4544: URL: https://github.com/apache/hbase/pull/4544#issuecomment-1464801596 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 0m 24s | Docker mode activated. | | -0 :warning: | yetus | 0m 3s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck | ||| _ Prechecks _ | ||| _ master Compile Tests _ | | +0 :ok: | mvndep | 0m 13s | Maven dependency ordering for branch | | +1 :green_heart: | mvninstall | 2m 56s | master passed | | +1 :green_heart: | compile | 0m 40s | master passed | | +1 :green_heart: | shadedjars | 4m 31s | branch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 0m 32s | master passed | ||| _ Patch Compile Tests _ | | +0 :ok: | mvndep | 0m 11s | Maven dependency ordering for patch | | +1 :green_heart: | mvninstall | 2m 42s | the patch passed | | +1 :green_heart: | compile | 0m 41s | the patch passed | | +1 :green_heart: | javac | 0m 41s | the patch passed | | +1 :green_heart: | shadedjars | 4m 29s | patch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 0m 31s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | unit | 1m 40s | hbase-common in the patch passed. | | +1 :green_heart: | unit | 10m 56s | hbase-backup in the patch passed. | | +1 :green_heart: | unit | 0m 35s | hbase-it in the patch passed. | | | | 32m 39s | | | Subsystem | Report/Notes | |--:|:-| | Docker | ClientAPI=1.42 ServerAPI=1.42 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4544/2/artifact/yetus-jdk8-hadoop3-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/4544 | | Optional Tests | javac javadoc unit shadedjars compile | | uname | Linux 7d27f6243c25 5.4.0-1094-aws #102~18.04.1-Ubuntu SMP Tue Jan 10 21:07:03 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / 72d5a46899 | | Default Java | Temurin-1.8.0_352-b08 | | Test Results | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4544/2/testReport/ | | Max. process+thread count | 3535 (vs. ulimit of 3) | | modules | C: hbase-common hbase-backup hbase-it U: . | | Console output | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4544/2/console | | versions | git=2.34.1 maven=3.8.6 | | Powered by | Apache Yetus 0.12.0 https://yetus.apache.org | This message was automatically generated. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Commented] (HBASE-27704) Quotas can drastically overflow configured limit
[
https://issues.apache.org/jira/browse/HBASE-27704?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17699146#comment-17699146
]
Duo Zhang commented on HBASE-27704:
---
[~zghao]
> Quotas can drastically overflow configured limit
>
>
> Key: HBASE-27704
> URL: https://issues.apache.org/jira/browse/HBASE-27704
> Project: HBase
> Issue Type: Bug
>Reporter: Bryan Beaudreault
>Priority: Major
> Attachments: Screenshot 2023-03-10 at 5.17.51 PM.png
>
>
> The original implementation did not allow exceeding quota. For example, you
> specify a limit of 10 resource/sec and consume 20 resources, it takes 1.1
> seconds to be able submit another request. This was covered by the
> [testOverconsumption in
> TestRateLimiter|https://github.com/apache/hbase/blame/587b0b4f20bdc0415b6541023e611b69c87dba15/hbase-server/src/test/java/org/apache/hadoop/hbase/quotas/TestRateLimiter.java#L97].
> As an incidental part of HBASE-13686, that logic was changed. There is no
> mention of the reasoning behind the change in the issue comments or review
> board, I think it was missed. The goal of that issue was to add different
> refill strategies, but it also modified the over consumption. The
> testOverconsumption was [split out for both refill
> strategies|https://github.com/apache/hbase/blame/master/hbase-server/src/test/java/org/apache/hadoop/hbase/quotas/TestRateLimiter.java#L104-L159],
> but the core reasoning was lost. The comment says:
> {code:java}
> // 10 resources are available, but we need to consume 20 resources109
> // Verify that we have to wait at least 1.1sec to have 1 resource available
> {code}
> But the actual test was updated to only require a new resource after 100ms.
> This is incorrect.
> The problem is, when consuming if you go negative it sets to 0
> [here|https://github.com/apache/hbase/blame/master/hbase-server/src/main/java/org/apache/hadoop/hbase/quotas/RateLimiter.java#L187-L191].
> Additionally, when refilling the new logic does a Math.max(0, available +
> refillAmount):
> [here|https://github.com/apache/hbase/blame/master/hbase-server/src/main/java/org/apache/hadoop/hbase/quotas/RateLimiter.java#L159-L163].
> So it's really impossible to get below 0, which is impractical for a rate
> limiter.
> With this setup it's very easy to drastically overconsume the rate limiter.
> See attached screenshot, which shows two humps. The first one has the current
> logic, the second hump has my fix which removes both of those problems. The
> rate limit was set to 500mb/s, but I was easily able to go over 700 mb/s
> without the fix.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Assigned] (HBASE-27702) Remove 'hbase.regionserver.hlog.writer.impl' config
[ https://issues.apache.org/jira/browse/HBASE-27702?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Duo Zhang reassigned HBASE-27702: - Assignee: Duo Zhang > Remove 'hbase.regionserver.hlog.writer.impl' config > --- > > Key: HBASE-27702 > URL: https://issues.apache.org/jira/browse/HBASE-27702 > Project: HBase > Issue Type: Task > Components: wal >Reporter: Duo Zhang >Assignee: Duo Zhang >Priority: Major > > We do not support sequence file based WAL file any more, and we should also > automatically choose SecureProtobufWALWriter if WAL entryption is enabled. > This is a follow up issue for HBASE-27632. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Work started] (HBASE-27702) Remove 'hbase.regionserver.hlog.writer.impl' config
[ https://issues.apache.org/jira/browse/HBASE-27702?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Work on HBASE-27702 started by Duo Zhang. - > Remove 'hbase.regionserver.hlog.writer.impl' config > --- > > Key: HBASE-27702 > URL: https://issues.apache.org/jira/browse/HBASE-27702 > Project: HBase > Issue Type: Task > Components: wal >Reporter: Duo Zhang >Assignee: Duo Zhang >Priority: Major > > We do not support sequence file based WAL file any more, and we should also > automatically choose SecureProtobufWALWriter if WAL entryption is enabled. > This is a follow up issue for HBASE-27632. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (HBASE-27704) Quotas can drastically overflow configured limit
[
https://issues.apache.org/jira/browse/HBASE-27704?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Bryan Beaudreault updated HBASE-27704:
--
Description:
The original implementation did not allow exceeding quota. For example, you
specify a limit of 10 resource/sec and consume 20 resources, it takes 1.1
seconds to be able submit another request. This was covered by the
[testOverconsumption in
TestRateLimiter|https://github.com/apache/hbase/blame/587b0b4f20bdc0415b6541023e611b69c87dba15/hbase-server/src/test/java/org/apache/hadoop/hbase/quotas/TestRateLimiter.java#L97].
As an incidental part of HBASE-13686, that logic was changed. There is no
mention of the reasoning behind the change in the issue comments or review
board, I think it was missed. The goal of that issue was to add different
refill strategies, but it also modified the over consumption. The
testOverconsumption was [split out for both refill
strategies|https://github.com/apache/hbase/blame/master/hbase-server/src/test/java/org/apache/hadoop/hbase/quotas/TestRateLimiter.java#L104-L159],
but the core reasoning was lost. The comment says:
{code:java}
// 10 resources are available, but we need to consume 20 resources109
// Verify that we have to wait at least 1.1sec to have 1 resource available
{code}
But the actual test was updated to only require a new resource after 100ms.
This is incorrect.
The problem is, when consuming if you go negative it sets to 0
[here|https://github.com/apache/hbase/blame/master/hbase-server/src/main/java/org/apache/hadoop/hbase/quotas/RateLimiter.java#L187-L191].
Additionally, when refilling the new logic does a Math.max(0, available +
refillAmount):
[here|https://github.com/apache/hbase/blame/master/hbase-server/src/main/java/org/apache/hadoop/hbase/quotas/RateLimiter.java#L159-L163].
So it's really impossible to get below 0, which is impractical for a rate
limiter.
With this setup it's very easy to drastically overconsume the rate limiter. See
attached screenshot, which shows two humps. The first one has the current
logic, the second hump has my fix which removes both of those problems. The
rate limit was set to 500mb/s, but I was easily able to go over 700 mb/s
without the fix.
was:
The original implementation did not allow exceeding quota. For example, you
specify a limit of 10 resource/sec and submit 20 resources, it takes 1.1
seconds to be able submit another request. This was covered by the
[testOverconsumption in
TestRateLimiter|https://github.com/apache/hbase/blame/587b0b4f20bdc0415b6541023e611b69c87dba15/hbase-server/src/test/java/org/apache/hadoop/hbase/quotas/TestRateLimiter.java#L97].
As an incidental part of HBASE-13686, that logic was changed. There is no
mention of the reasoning behind the change in the issue comments or review
board, I think it was missed. The goal of that issue was to add different
refill strategies, but it also modified the over consumption. The
testOverconsumption was [split out for both refill
strategies|https://github.com/apache/hbase/blame/master/hbase-server/src/test/java/org/apache/hadoop/hbase/quotas/TestRateLimiter.java#L104-L159],
but the core reasoning was lost. The comment says:
{code:java}
// 10 resources are available, but we need to consume 20 resources109
// Verify that we have to wait at least 1.1sec to have 1 resource available
{code}
But the actual test was updated to only require a new resource after 100ms.
This is incorrect.
The problem is, when consuming if you go negative it sets to 0
[here|https://github.com/apache/hbase/blame/master/hbase-server/src/main/java/org/apache/hadoop/hbase/quotas/RateLimiter.java#L187-L191].
Additionally, when refilling the new logic does a Math.max(0, refillAmount):
[here|https://github.com/apache/hbase/blame/master/hbase-server/src/main/java/org/apache/hadoop/hbase/quotas/RateLimiter.java#L159-L163].
So it's really impossible to get below 0, which is impractical for a rate
limiter.
With this setup it's very easy to drastically overconsume the rate limiter. See
attached screenshot, which shows two humps. The first one has the current
logic, the second hump has my fix which removes both of those problems. The
rate limit was set to 500mb/s, but I was easily able to go over 700 mb/s
without the fix.
> Quotas can drastically overflow configured limit
>
>
> Key: HBASE-27704
> URL: https://issues.apache.org/jira/browse/HBASE-27704
> Project: HBase
> Issue Type: Bug
>Reporter: Bryan Beaudreault
>Priority: Major
> Attachments: Screenshot 2023-03-10 at 5.17.51 PM.png
>
>
> The original implementation did not allow exceeding quota. For example, you
> specify a limit of 10 resource/sec and consume 20 resources, it takes 1.1
> seconds to be able submit another request. This was
[jira] [Updated] (HBASE-27704) Quotas can drastically overflow configured limit
[
https://issues.apache.org/jira/browse/HBASE-27704?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Bryan Beaudreault updated HBASE-27704:
--
Attachment: Screenshot 2023-03-10 at 5.17.51 PM.png
> Quotas can drastically overflow configured limit
>
>
> Key: HBASE-27704
> URL: https://issues.apache.org/jira/browse/HBASE-27704
> Project: HBase
> Issue Type: Bug
>Reporter: Bryan Beaudreault
>Priority: Major
> Attachments: Screenshot 2023-03-10 at 5.17.51 PM.png
>
>
> The original implementation did not allow exceeding quota. For example, you
> specify a limit of 10 resource/sec and submit 20 resources, it takes 1.1
> seconds to be able submit another request. This was covered by the
> [testOverconsumption in
> TestRateLimiter|https://github.com/apache/hbase/blame/587b0b4f20bdc0415b6541023e611b69c87dba15/hbase-server/src/test/java/org/apache/hadoop/hbase/quotas/TestRateLimiter.java#L97].
> As an incidental part of HBASE-13686, that logic was changed. There is no
> mention of the reasoning behind the change in the issue comments or review
> board, I think it was missed. The goal of that issue was to add different
> refill strategies, but it also modified the over consumption. The
> testOverconsumption was [split out for both refill
> strategies|https://github.com/apache/hbase/blame/master/hbase-server/src/test/java/org/apache/hadoop/hbase/quotas/TestRateLimiter.java#L104-L159],
> but the core reasoning was lost. The comment says:
> {code:java}
> // 10 resources are available, but we need to consume 20 resources109
> // Verify that we have to wait at least 1.1sec to have 1 resource available
> {code}
> But the actual test was updated to only require a new resource after 100ms.
> This is incorrect.
> The problem is, when consuming if you go negative it sets to 0
> [here|https://github.com/apache/hbase/blame/master/hbase-server/src/main/java/org/apache/hadoop/hbase/quotas/RateLimiter.java#L187-L191].
> Additionally, when refilling the new logic does a Math.max(0, refillAmount):
> [here|https://github.com/apache/hbase/blame/master/hbase-server/src/main/java/org/apache/hadoop/hbase/quotas/RateLimiter.java#L159-L163].
> So it's really impossible to get below 0, which is impractical for a rate
> limiter.
> With this setup it's very easy to drastically overconsume the rate limiter.
> See attached screenshot, which shows two humps. The first one has the current
> logic, the second hump has my fix which removes both of those problems. The
> rate limit was set to 500mb/s, but I was easily able to go over 700 mb/s
> without the fix.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Created] (HBASE-27704) Quotas can drastically overflow configured limit
Bryan Beaudreault created HBASE-27704:
-
Summary: Quotas can drastically overflow configured limit
Key: HBASE-27704
URL: https://issues.apache.org/jira/browse/HBASE-27704
Project: HBase
Issue Type: Bug
Reporter: Bryan Beaudreault
Attachments: Screenshot 2023-03-10 at 5.17.51 PM.png
The original implementation did not allow exceeding quota. For example, you
specify a limit of 10 resource/sec and submit 20 resources, it takes 1.1
seconds to be able submit another request. This was covered by the
[testOverconsumption in
TestRateLimiter|https://github.com/apache/hbase/blame/587b0b4f20bdc0415b6541023e611b69c87dba15/hbase-server/src/test/java/org/apache/hadoop/hbase/quotas/TestRateLimiter.java#L97].
As an incidental part of HBASE-13686, that logic was changed. There is no
mention of the reasoning behind the change in the issue comments or review
board, I think it was missed. The goal of that issue was to add different
refill strategies, but it also modified the over consumption. The
testOverconsumption was [split out for both refill
strategies|https://github.com/apache/hbase/blame/master/hbase-server/src/test/java/org/apache/hadoop/hbase/quotas/TestRateLimiter.java#L104-L159],
but the core reasoning was lost. The comment says:
{code:java}
// 10 resources are available, but we need to consume 20 resources109
// Verify that we have to wait at least 1.1sec to have 1 resource available
{code}
But the actual test was updated to only require a new resource after 100ms.
This is incorrect.
The problem is, when consuming if you go negative it sets to 0
[here|https://github.com/apache/hbase/blame/master/hbase-server/src/main/java/org/apache/hadoop/hbase/quotas/RateLimiter.java#L187-L191].
Additionally, when refilling the new logic does a Math.max(0, refillAmount):
[here|https://github.com/apache/hbase/blame/master/hbase-server/src/main/java/org/apache/hadoop/hbase/quotas/RateLimiter.java#L159-L163].
So it's really impossible to get below 0, which is impractical for a rate
limiter.
With this setup it's very easy to drastically overconsume the rate limiter. See
attached screenshot, which shows two humps. The first one has the current
logic, the second hump has my fix which removes both of those problems. The
rate limit was set to 500mb/s, but I was easily able to go over 700 mb/s
without the fix.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Resolved] (HBASE-27685) Enable code coverage reporting to SonarQube in HBase
[ https://issues.apache.org/jira/browse/HBASE-27685?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Peter Somogyi resolved HBASE-27685. --- Fix Version/s: 2.6.0 3.0.0-alpha-4 2.4.17 2.5.4 Resolution: Fixed Merged to branch-2.4+. Thanks [~dora.horvath]! > Enable code coverage reporting to SonarQube in HBase > > > Key: HBASE-27685 > URL: https://issues.apache.org/jira/browse/HBASE-27685 > Project: HBase > Issue Type: Task >Reporter: Dóra Horváth >Assignee: Dóra Horváth >Priority: Minor > Fix For: 2.6.0, 3.0.0-alpha-4, 2.4.17, 2.5.4 > > -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (HBASE-27615) Add group of regionServer on Master webUI
[
https://issues.apache.org/jira/browse/HBASE-27615?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17699084#comment-17699084
]
Hudson commented on HBASE-27615:
Results for branch master
[build #792 on
builds.a.o|https://ci-hbase.apache.org/job/HBase%20Nightly/job/master/792/]:
(/) *{color:green}+1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/master/792/General_20Nightly_20Build_20Report/]
(/) {color:green}+1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/master/792/JDK8_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(/) {color:green}+1 jdk11 hadoop3 checks{color}
-- For more information [see jdk11
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/master/792/JDK11_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
(/) {color:green}+1 client integration test{color}
> Add group of regionServer on Master webUI
> -
>
> Key: HBASE-27615
> URL: https://issues.apache.org/jira/browse/HBASE-27615
> Project: HBase
> Issue Type: Improvement
>Reporter: tianhang tang
>Assignee: tianhang tang
>Priority: Major
> Attachments: image-2023-02-06-12-04-03-503.png
>
>
> We do have a RSGroupList on webUI now, but it is still a little inconvenient
> if I just want to know which group a specific regionServer belongs to.
> So add this info on webUI:
> !image-2023-02-06-12-04-03-503.png|width=889,height=174!
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[GitHub] [hbase] petersomogyi merged pull request #5076: HBASE-27685 Enable code coverage reporting to SonarQube in HBase
petersomogyi merged PR #5076: URL: https://github.com/apache/hbase/pull/5076 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #5076: HBASE-27685 Enable code coverage reporting to SonarQube in HBase
Apache-HBase commented on PR #5076: URL: https://github.com/apache/hbase/pull/5076#issuecomment-1464194911 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 1m 7s | Docker mode activated. | | -0 :warning: | yetus | 0m 3s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck | ||| _ Prechecks _ | ||| _ master Compile Tests _ | | +0 :ok: | mvndep | 0m 19s | Maven dependency ordering for branch | | +1 :green_heart: | mvninstall | 2m 50s | master passed | | +1 :green_heart: | compile | 1m 34s | master passed | | +1 :green_heart: | shadedjars | 4m 16s | branch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 1m 34s | master passed | ||| _ Patch Compile Tests _ | | +0 :ok: | mvndep | 0m 14s | Maven dependency ordering for patch | | +1 :green_heart: | mvninstall | 2m 48s | the patch passed | | +1 :green_heart: | compile | 1m 36s | the patch passed | | +1 :green_heart: | javac | 1m 36s | the patch passed | | +1 :green_heart: | shadedjars | 4m 17s | patch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 1m 33s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | unit | 354m 25s | root in the patch passed. | | | | 383m 26s | | | Subsystem | Report/Notes | |--:|:-| | Docker | ClientAPI=1.42 ServerAPI=1.42 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5076/5/artifact/yetus-jdk8-hadoop3-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/5076 | | Optional Tests | javac javadoc unit shadedjars compile | | uname | Linux 78329962a0d4 5.4.0-135-generic #152-Ubuntu SMP Wed Nov 23 20:19:22 UTC 2022 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / 0491524eb1 | | Default Java | Temurin-1.8.0_352-b08 | | Test Results | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5076/5/testReport/ | | Max. process+thread count | 4336 (vs. ulimit of 3) | | modules | C: hbase-it . U: . | | Console output | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5076/5/console | | versions | git=2.34.1 maven=3.8.6 | | Powered by | Apache Yetus 0.12.0 https://yetus.apache.org | This message was automatically generated. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #5076: HBASE-27685 Enable code coverage reporting to SonarQube in HBase
Apache-HBase commented on PR #5076: URL: https://github.com/apache/hbase/pull/5076#issuecomment-1464100756 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 0m 52s | Docker mode activated. | | -0 :warning: | yetus | 0m 3s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck | ||| _ Prechecks _ | ||| _ master Compile Tests _ | | +0 :ok: | mvndep | 0m 18s | Maven dependency ordering for branch | | +1 :green_heart: | mvninstall | 3m 22s | master passed | | +1 :green_heart: | compile | 1m 54s | master passed | | +1 :green_heart: | shadedjars | 4m 26s | branch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 2m 4s | master passed | ||| _ Patch Compile Tests _ | | +0 :ok: | mvndep | 0m 15s | Maven dependency ordering for patch | | +1 :green_heart: | mvninstall | 3m 19s | the patch passed | | +1 :green_heart: | compile | 1m 53s | the patch passed | | +1 :green_heart: | javac | 1m 53s | the patch passed | | +1 :green_heart: | shadedjars | 4m 24s | patch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 2m 6s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | unit | 277m 27s | root in the patch passed. | | | | 308m 42s | | | Subsystem | Report/Notes | |--:|:-| | Docker | ClientAPI=1.42 ServerAPI=1.42 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5076/5/artifact/yetus-jdk11-hadoop3-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/5076 | | Optional Tests | javac javadoc unit shadedjars compile | | uname | Linux edfa78a6b0ee 5.4.0-137-generic #154-Ubuntu SMP Thu Jan 5 17:03:22 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / 0491524eb1 | | Default Java | Eclipse Adoptium-11.0.17+8 | | Test Results | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5076/5/testReport/ | | Max. process+thread count | 4899 (vs. ulimit of 3) | | modules | C: hbase-it . U: . | | Console output | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5076/5/console | | versions | git=2.34.1 maven=3.8.6 | | Powered by | Apache Yetus 0.12.0 https://yetus.apache.org | This message was automatically generated. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #5095: HBASE-27701 Fix a misspelling of the ZstdCodec
Apache-HBase commented on PR #5095: URL: https://github.com/apache/hbase/pull/5095#issuecomment-1463973692 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 0m 58s | Docker mode activated. | ||| _ Prechecks _ | | +1 :green_heart: | dupname | 0m 0s | No case conflicting files found. | | +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. | ||| _ master Compile Tests _ | | +1 :green_heart: | mvninstall | 3m 47s | master passed | | +0 :ok: | refguide | 2m 54s | branch has no errors when building the reference guide. See footer for rendered docs, which you should manually inspect. | | +1 :green_heart: | spotless | 0m 45s | branch has no errors when running spotless:check. | ||| _ Patch Compile Tests _ | | +1 :green_heart: | mvninstall | 3m 20s | the patch passed | | +1 :green_heart: | whitespace | 0m 0s | The patch has no whitespace issues. | | +0 :ok: | refguide | 2m 12s | patch has no errors when building the reference guide. See footer for rendered docs, which you should manually inspect. | | +1 :green_heart: | spotless | 0m 41s | patch has no errors when running spotless:check. | ||| _ Other Tests _ | | +1 :green_heart: | asflicense | 0m 13s | The patch does not generate ASF License warnings. | | | | 16m 21s | | | Subsystem | Report/Notes | |--:|:-| | Docker | ClientAPI=1.42 ServerAPI=1.42 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5095/1/artifact/yetus-general-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/5095 | | Optional Tests | dupname asflicense refguide spotless | | uname | Linux 2101c920f7c3 5.4.0-135-generic #152-Ubuntu SMP Wed Nov 23 20:19:22 UTC 2022 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / e48c4485db | | refguide | https://nightlies.apache.org/hbase/HBase-PreCommit-GitHub-PR/PR-5095/1/yetus-general-check/output/branch-site/book.html | | refguide | https://nightlies.apache.org/hbase/HBase-PreCommit-GitHub-PR/PR-5095/1/yetus-general-check/output/patch-site/book.html | | Max. process+thread count | 86 (vs. ulimit of 3) | | modules | C: . U: . | | Console output | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5095/1/console | | versions | git=2.34.1 maven=3.8.6 | | Powered by | Apache Yetus 0.12.0 https://yetus.apache.org | This message was automatically generated. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Updated] (HBASE-27702) Remove 'hbase.regionserver.hlog.writer.impl' config
[ https://issues.apache.org/jira/browse/HBASE-27702?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Duo Zhang updated HBASE-27702: -- Component/s: wal > Remove 'hbase.regionserver.hlog.writer.impl' config > --- > > Key: HBASE-27702 > URL: https://issues.apache.org/jira/browse/HBASE-27702 > Project: HBase > Issue Type: Task > Components: wal >Reporter: Duo Zhang >Priority: Major > > We do not support sequence file based WAL file any more, and we should also > automatically choose SecureProtobufWALWriter if WAL entryption is enabled. > This is a follow up issue for HBASE-27632. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (HBASE-27703) Update ref guide about how to enable WAL encryption
Duo Zhang created HBASE-27703: - Summary: Update ref guide about how to enable WAL encryption Key: HBASE-27703 URL: https://issues.apache.org/jira/browse/HBASE-27703 Project: HBase Issue Type: Task Components: documentation, security, wal Reporter: Duo Zhang AFter HBASE-27632 and HBASE-27702, we do not need to specify the WAL implementation any more. Update ref guide about this change. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[GitHub] [hbase] Apache-HBase commented on pull request #5095: HBASE-27701 Fix a misspelling of the ZstdCodec
Apache-HBase commented on PR #5095: URL: https://github.com/apache/hbase/pull/5095#issuecomment-1463952379 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 0m 23s | Docker mode activated. | | -0 :warning: | yetus | 0m 3s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck | ||| _ Prechecks _ | ||| _ master Compile Tests _ | ||| _ Patch Compile Tests _ | ||| _ Other Tests _ | | | | 1m 36s | | | Subsystem | Report/Notes | |--:|:-| | Docker | ClientAPI=1.42 ServerAPI=1.42 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5095/1/artifact/yetus-jdk8-hadoop3-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/5095 | | Optional Tests | | | uname | Linux af3f39a77655 5.4.0-1094-aws #102~18.04.1-Ubuntu SMP Tue Jan 10 21:07:03 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / e48c4485db | | Max. process+thread count | 33 (vs. ulimit of 3) | | modules | C: . U: . | | Console output | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5095/1/console | | versions | git=2.34.1 maven=3.8.6 | | Powered by | Apache Yetus 0.12.0 https://yetus.apache.org | This message was automatically generated. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #5095: HBASE-27701 Fix a misspelling of the ZstdCodec
Apache-HBase commented on PR #5095: URL: https://github.com/apache/hbase/pull/5095#issuecomment-1463952188 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 0m 26s | Docker mode activated. | | -0 :warning: | yetus | 0m 3s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck | ||| _ Prechecks _ | ||| _ master Compile Tests _ | ||| _ Patch Compile Tests _ | ||| _ Other Tests _ | | | | 1m 35s | | | Subsystem | Report/Notes | |--:|:-| | Docker | ClientAPI=1.42 ServerAPI=1.42 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5095/1/artifact/yetus-jdk11-hadoop3-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/5095 | | Optional Tests | | | uname | Linux 444291939ccd 5.4.0-1092-aws #100~18.04.2-Ubuntu SMP Tue Nov 29 08:39:52 UTC 2022 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / e48c4485db | | Max. process+thread count | 41 (vs. ulimit of 3) | | modules | C: . U: . | | Console output | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5095/1/console | | versions | git=2.34.1 maven=3.8.6 | | Powered by | Apache Yetus 0.12.0 https://yetus.apache.org | This message was automatically generated. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Created] (HBASE-27702) Remove 'hbase.regionserver.hlog.writer.impl' config
Duo Zhang created HBASE-27702: - Summary: Remove 'hbase.regionserver.hlog.writer.impl' config Key: HBASE-27702 URL: https://issues.apache.org/jira/browse/HBASE-27702 Project: HBase Issue Type: Task Reporter: Duo Zhang We do not support sequence file based WAL file any more, and we should also automatically choose SecureProtobufWALWriter if WAL entryption is enabled. This is a follow up issue for HBASE-27632. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[GitHub] [hbase] frensjan opened a new pull request, #5095: HBASE-27701 Fix a misspelling of the ZstdCodec
frensjan opened a new pull request, #5095: URL: https://github.com/apache/hbase/pull/5095 Fix a misspelling of the ZstdCodec in compression documentation. See also https://issues.apache.org/jira/browse/HBASE-27701. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Resolved] (HBASE-27632) Refactor WAL.Reader implementation so we can better support WAL splitting and replication
[ https://issues.apache.org/jira/browse/HBASE-27632?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Duo Zhang resolved HBASE-27632. --- Fix Version/s: 2.6.0 3.0.0-alpha-4 Hadoop Flags: Reviewed Resolution: Fixed Pushed to master and branch-2. Thanks all for helping and thanks [~GeorryHuang] for reviewing! > Refactor WAL.Reader implementation so we can better support WAL splitting and > replication > - > > Key: HBASE-27632 > URL: https://issues.apache.org/jira/browse/HBASE-27632 > Project: HBase > Issue Type: Improvement > Components: Replication, wal >Reporter: Duo Zhang >Assignee: Duo Zhang >Priority: Major > Fix For: 2.6.0, 3.0.0-alpha-4 > > > WAL splitting and replication has completely different requirements on WAL > reader, the latter one is much comlicated as it wants to tail a WAL file > which is currently being written. > I think split them into two classes is better here. > But we exposes two configurations out > 'hbase.regionserver.hlog.reader.impl' > 'hbase.regionserver.hlog.writer.impl' > which indicates that we only have one implementation for wal reader. > These two configurations are inrtoduced in 0.96, for reading the pre 0.94 > sequence file format WAL file. And starting from 2.0.0, we have already > dropped the support of reading sequence file format WAL file, so I think the > configurations are useless now. > Now we stil require users to specify SecureProtobufLogReader and > SecureProtobufLogWriter when enable WAL encryption, but actually they are > both the only choice, so just use 'hbase.regionserver.wal.encryption' to > determine whether we need to use SecureProtobufLogWriter to write WAL. And > for reading, I do not think we should rely on the configuration, we should > detect whether the WAL file itself is encrypted(I believe we have already > done this in SecureProtobufLogReader.readHeader. > So in general, I prefer we just mark these two configurations deprecated and > do not use them in our code base, so we can have different Reader > implementations for WAL splitting and replication. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (HBASE-27701) ZStdCodec codec implementation class documentation typo
Frens Jan Rumph created HBASE-27701:
---
Summary: ZStdCodec codec implementation class documentation typo
Key: HBASE-27701
URL: https://issues.apache.org/jira/browse/HBASE-27701
Project: HBase
Issue Type: Bug
Components: documentation
Reporter: Frens Jan Rumph
As mentioned in the [u...@hbase.apache.org|mailto:u...@hbase.apache.org]
mailing list I noticed a small typo in the documentation on compression for
Zstd. The codec implementation class in the documentation is listed as
{{org.apache.hadoop.hbase.io.compress.zstd.ZStdCodec}} while the actual class
is written with a lower case S: {{ZStdCodec.}}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Updated] (HBASE-27632) Refactor WAL.Reader implementation so we can better support WAL splitting and replication
[ https://issues.apache.org/jira/browse/HBASE-27632?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Duo Zhang updated HBASE-27632: -- Release Note: Introduced two types of WAL reader, WALStreamReader and WALTailingReader. The former one is for most use cases where we only need to read closed WAL files, such as splitting and replaying, and the second one is for replication, where we need to tail a WAL file which is being written currently. The config 'hbase.regionserver.hlog.reader.impl' is removed, as now we have two types of WAL reader. And starting from at least 0.98, we do not support sequence file format WAL file any more, the only accepted format is protobuf. And for encrypted WAL files, now you do not need to specify SecureProtobufLogReader either, as the new implementation will choose to use SecureWALCellCodec based on the properties in WALHeader. There is still a config called 'hbase.regionserver.wal.stream.reader.impl', for specifying the implementation for WALStreamReader. This is only designed to be used in HBase itself, for injecting some error when writing UTs, downstream users should not use it as it is unstable. > Refactor WAL.Reader implementation so we can better support WAL splitting and > replication > - > > Key: HBASE-27632 > URL: https://issues.apache.org/jira/browse/HBASE-27632 > Project: HBase > Issue Type: Improvement > Components: Replication, wal >Reporter: Duo Zhang >Assignee: Duo Zhang >Priority: Major > > WAL splitting and replication has completely different requirements on WAL > reader, the latter one is much comlicated as it wants to tail a WAL file > which is currently being written. > I think split them into two classes is better here. > But we exposes two configurations out > 'hbase.regionserver.hlog.reader.impl' > 'hbase.regionserver.hlog.writer.impl' > which indicates that we only have one implementation for wal reader. > These two configurations are inrtoduced in 0.96, for reading the pre 0.94 > sequence file format WAL file. And starting from 2.0.0, we have already > dropped the support of reading sequence file format WAL file, so I think the > configurations are useless now. > Now we stil require users to specify SecureProtobufLogReader and > SecureProtobufLogWriter when enable WAL encryption, but actually they are > both the only choice, so just use 'hbase.regionserver.wal.encryption' to > determine whether we need to use SecureProtobufLogWriter to write WAL. And > for reading, I do not think we should rely on the configuration, we should > detect whether the WAL file itself is encrypted(I believe we have already > done this in SecureProtobufLogReader.readHeader. > So in general, I prefer we just mark these two configurations deprecated and > do not use them in our code base, so we can have different Reader > implementations for WAL splitting and replication. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[GitHub] [hbase] Apache9 merged pull request #5055: HBASE-27632 Refactor WAL.Reader implementation so we can better suppo…
Apache9 merged PR #5055: URL: https://github.com/apache/hbase/pull/5055 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Commented] (HBASE-27688) HFile splitting occurs during bulkload, the CREATE_TIME_TS of hfileinfo is 0
[
https://issues.apache.org/jira/browse/HBASE-27688?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17698945#comment-17698945
]
Duo Zhang commented on HBASE-27688:
---
The code for branch-2.x is a bit different so the patch can not be applied to
branch-2.x cleanly.
Please provide a PR for branch-2? [~alanlemma]
Thanks.
> HFile splitting occurs during bulkload, the CREATE_TIME_TS of hfileinfo is 0
>
>
> Key: HBASE-27688
> URL: https://issues.apache.org/jira/browse/HBASE-27688
> Project: HBase
> Issue Type: Bug
>Reporter: alan.zhao
>Priority: Major
>
> If HFile splitting occurs during bulkload, the CREATE_TIME_TS of hfileinfo
> =0,When HFile is copied after splitting, CREATE_TIME_TS of the original file
> is not copied。
> {code:java}
> ##BulkLoadHFilesTool.class
> /**
> * Copy half of an HFile into a new HFile.
> */
> private static void copyHFileHalf(Configuration conf, Path inFile, Path
> outFile,
> Reference reference, ColumnFamilyDescriptor familyDescriptor) throws
> IOException {
> FileSystem fs = inFile.getFileSystem(conf);
> CacheConfig cacheConf = CacheConfig.DISABLED;
> HalfStoreFileReader halfReader = null;
> StoreFileWriter halfWriter = null;
> try {
> 。。。
> HFileContext hFileContext = new
> HFileContextBuilder().withCompression(compression)
> .withChecksumType(StoreUtils.getChecksumType(conf))
>
> .withBytesPerCheckSum(StoreUtils.getBytesPerChecksum(conf)).withBlockSize(blocksize)
>
> .withDataBlockEncoding(familyDescriptor.getDataBlockEncoding()).withIncludesTags(true)
> .build();
> // TODO .withCreateTime(EnvironmentEdgeManager.currentTime())
> halfWriter = new StoreFileWriter.Builder(conf, cacheConf,
> fs).withFilePath(outFile)
> .withBloomType(bloomFilterType).withFileContext(hFileContext).build();
> HFileScanner scanner = halfReader.getScanner(false, false, false);
> scanner.seekTo();
> do {
> halfWriter.append(scanner.getCell());
> } while (scanner.next());
> for (Map.Entry entry : fileInfo.entrySet()) {
> if (shouldCopyHFileMetaKey(entry.getKey())) {
> halfWriter.appendFileInfo(entry.getKey(), entry.getValue());
> }
> }
> } finally {
> 。。。
> }
> }
> ##get lastMajorCompactionTs metric
> lastMajorCompactionTs = this.region.getOldestHfileTs(true);
> ...
> long now = EnvironmentEdgeManager.currentTime();
> return now - lastMajorCompactionTs;
> ...
> ##
> public long getOldestHfileTs(boolean majorCompactionOnly) throws IOException {
> long result = Long.MAX_VALUE;
> for (HStore store : stores.values()) {
> Collection storeFiles = store.getStorefiles();
>...
> for (HStoreFile file : storeFiles) {
> StoreFileReader sfReader = file.getReader();
> ...
> result = Math.min(result, reader.getFileContext().getFileCreateTime());
> }
> }
> return result == Long.MAX_VALUE ? 0 : result;
> }{code}
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Assigned] (HBASE-27688) HFile splitting occurs during bulkload, the CREATE_TIME_TS of hfileinfo is 0
[
https://issues.apache.org/jira/browse/HBASE-27688?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Duo Zhang reassigned HBASE-27688:
-
Assignee: alan.zhao
> HFile splitting occurs during bulkload, the CREATE_TIME_TS of hfileinfo is 0
>
>
> Key: HBASE-27688
> URL: https://issues.apache.org/jira/browse/HBASE-27688
> Project: HBase
> Issue Type: Bug
>Reporter: alan.zhao
>Assignee: alan.zhao
>Priority: Major
>
> If HFile splitting occurs during bulkload, the CREATE_TIME_TS of hfileinfo
> =0,When HFile is copied after splitting, CREATE_TIME_TS of the original file
> is not copied。
> {code:java}
> ##BulkLoadHFilesTool.class
> /**
> * Copy half of an HFile into a new HFile.
> */
> private static void copyHFileHalf(Configuration conf, Path inFile, Path
> outFile,
> Reference reference, ColumnFamilyDescriptor familyDescriptor) throws
> IOException {
> FileSystem fs = inFile.getFileSystem(conf);
> CacheConfig cacheConf = CacheConfig.DISABLED;
> HalfStoreFileReader halfReader = null;
> StoreFileWriter halfWriter = null;
> try {
> 。。。
> HFileContext hFileContext = new
> HFileContextBuilder().withCompression(compression)
> .withChecksumType(StoreUtils.getChecksumType(conf))
>
> .withBytesPerCheckSum(StoreUtils.getBytesPerChecksum(conf)).withBlockSize(blocksize)
>
> .withDataBlockEncoding(familyDescriptor.getDataBlockEncoding()).withIncludesTags(true)
> .build();
> // TODO .withCreateTime(EnvironmentEdgeManager.currentTime())
> halfWriter = new StoreFileWriter.Builder(conf, cacheConf,
> fs).withFilePath(outFile)
> .withBloomType(bloomFilterType).withFileContext(hFileContext).build();
> HFileScanner scanner = halfReader.getScanner(false, false, false);
> scanner.seekTo();
> do {
> halfWriter.append(scanner.getCell());
> } while (scanner.next());
> for (Map.Entry entry : fileInfo.entrySet()) {
> if (shouldCopyHFileMetaKey(entry.getKey())) {
> halfWriter.appendFileInfo(entry.getKey(), entry.getValue());
> }
> }
> } finally {
> 。。。
> }
> }
> ##get lastMajorCompactionTs metric
> lastMajorCompactionTs = this.region.getOldestHfileTs(true);
> ...
> long now = EnvironmentEdgeManager.currentTime();
> return now - lastMajorCompactionTs;
> ...
> ##
> public long getOldestHfileTs(boolean majorCompactionOnly) throws IOException {
> long result = Long.MAX_VALUE;
> for (HStore store : stores.values()) {
> Collection storeFiles = store.getStorefiles();
>...
> for (HStoreFile file : storeFiles) {
> StoreFileReader sfReader = file.getReader();
> ...
> result = Math.min(result, reader.getFileContext().getFileCreateTime());
> }
> }
> return result == Long.MAX_VALUE ? 0 : result;
> }{code}
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[GitHub] [hbase] Apache9 merged pull request #5082: HBASE-27688 HFile splitting occurs during bulkload, the CREATE_TIME_TS of hfileinfo is 0
Apache9 merged PR #5082: URL: https://github.com/apache/hbase/pull/5082 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] Apache9 commented on pull request #5082: HBASE-27688 HFile splitting occurs during bulkload, the CREATE_TIME_TS of hfileinfo is 0
Apache9 commented on PR #5082: URL: https://github.com/apache/hbase/pull/5082#issuecomment-1463824561 Should not be related. Let me merge. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #5076: HBASE-27685 Enable code coverage reporting to SonarQube in HBase
Apache-HBase commented on PR #5076: URL: https://github.com/apache/hbase/pull/5076#issuecomment-1463744459 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 0m 37s | Docker mode activated. | ||| _ Prechecks _ | | +1 :green_heart: | dupname | 0m 0s | No case conflicting files found. | | +0 :ok: | markdownlint | 0m 0s | markdownlint was not available. | | +0 :ok: | shelldocs | 0m 0s | Shelldocs was not available. | | +1 :green_heart: | @author | 0m 1s | The patch does not contain any @author tags. | ||| _ master Compile Tests _ | | +0 :ok: | mvndep | 0m 36s | Maven dependency ordering for branch | | +1 :green_heart: | mvninstall | 3m 57s | master passed | | +1 :green_heart: | compile | 5m 1s | master passed | | +1 :green_heart: | spotless | 0m 41s | branch has no errors when running spotless:check. | ||| _ Patch Compile Tests _ | | +0 :ok: | mvndep | 0m 14s | Maven dependency ordering for patch | | +1 :green_heart: | mvninstall | 3m 34s | the patch passed | | +1 :green_heart: | compile | 5m 5s | the patch passed | | +1 :green_heart: | javac | 5m 5s | the patch passed | | +1 :green_heart: | shellcheck | 0m 1s | There were no new shellcheck issues. | | +1 :green_heart: | whitespace | 0m 0s | The patch has no whitespace issues. | | +1 :green_heart: | xml | 0m 2s | The patch has no ill-formed XML file. | | +1 :green_heart: | hadoopcheck | 13m 58s | Patch does not cause any errors with Hadoop 3.2.4 3.3.4. | | +1 :green_heart: | spotless | 0m 43s | patch has no errors when running spotless:check. | ||| _ Other Tests _ | | +1 :green_heart: | asflicense | 0m 17s | The patch does not generate ASF License warnings. | | | | 43m 14s | | | Subsystem | Report/Notes | |--:|:-| | Docker | ClientAPI=1.42 ServerAPI=1.42 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5076/5/artifact/yetus-general-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/5076 | | Optional Tests | dupname asflicense spotless markdownlint shellcheck shelldocs javac hadoopcheck xml compile | | uname | Linux dce8c9b4dc4b 5.4.0-1092-aws #100~18.04.2-Ubuntu SMP Tue Nov 29 08:39:52 UTC 2022 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / 0491524eb1 | | Default Java | Eclipse Adoptium-11.0.17+8 | | Max. process+thread count | 82 (vs. ulimit of 3) | | modules | C: hbase-it . U: . | | Console output | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5076/5/console | | versions | git=2.34.1 maven=3.8.6 shellcheck=0.8.0 | | Powered by | Apache Yetus 0.12.0 https://yetus.apache.org | This message was automatically generated. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Commented] (HBASE-26352) Provide HBase upgrade guidelines from 1.6 to 2.4+ versions
[ https://issues.apache.org/jira/browse/HBASE-26352?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17698835#comment-17698835 ] Jack Yang commented on HBASE-26352: --- OK thanks [~vjasani] for your advice. I'll try then. > Provide HBase upgrade guidelines from 1.6 to 2.4+ versions > -- > > Key: HBASE-26352 > URL: https://issues.apache.org/jira/browse/HBASE-26352 > Project: HBase > Issue Type: Bug >Reporter: Viraj Jasani >Assignee: Viraj Jasani >Priority: Major > Fix For: 3.0.0-alpha-4 > > > Provide some ref guide under section: > [https://hbase.apache.org/book.html#upgrade2.0.rolling.upgrades] > This should include experience of performing in-place rolling upgrade > (without downtime) from 1.6/1.7 to 2.4+ release versions. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (HBASE-27109) Move replication queue storage from zookeeper to a separated HBase table
[
https://issues.apache.org/jira/browse/HBASE-27109?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17698821#comment-17698821
]
Hudson commented on HBASE-27109:
Results for branch HBASE-27109/table_based_rqs
[build #52 on
builds.a.o|https://ci-hbase.apache.org/job/HBase%20Nightly/job/HBASE-27109%252Ftable_based_rqs/52/]:
(x) *{color:red}-1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/HBASE-27109%252Ftable_based_rqs/52/General_20Nightly_20Build_20Report/]
(/) {color:green}+1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/HBASE-27109%252Ftable_based_rqs/52/JDK8_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(x) {color:red}-1 jdk11 hadoop3 checks{color}
-- For more information [see jdk11
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/HBASE-27109%252Ftable_based_rqs/52/JDK11_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
(/) {color:green}+1 client integration test{color}
> Move replication queue storage from zookeeper to a separated HBase table
>
>
> Key: HBASE-27109
> URL: https://issues.apache.org/jira/browse/HBASE-27109
> Project: HBase
> Issue Type: New Feature
> Components: Replication
>Reporter: Duo Zhang
>Assignee: Duo Zhang
>Priority: Major
>
> This is a more specific issue based on the works which are already done in
> HBASE-15867.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[GitHub] [hbase] petersomogyi commented on pull request #5076: HBASE-27685 Enable code coverage reporting to SonarQube in HBase
petersomogyi commented on PR #5076: URL: https://github.com/apache/hbase/pull/5076#issuecomment-1463467550 Please fix the spotless and shellcheck issues. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
