[GitHub] [hbase-connectors] busbey commented on pull request #94: HBASE-26841 Replace log4j with reload4j for hbase-connector
busbey commented on pull request #94: URL: https://github.com/apache/hbase-connectors/pull/94#issuecomment-1068742586 I wonder if we should be more aggressive here in the connectors repo. For the spark connector we don't need to include any specific logging backend, because we should get that from spark. For the kafka proxy by default we get a classpath from hbase that probably includes a logging backend. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #4216: HBASE-26720 ExportSnapshot should validate the source snapshot before copying files
Apache-HBase commented on pull request #4216: URL: https://github.com/apache/hbase/pull/4216#issuecomment-1068698785 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 1m 11s | Docker mode activated. | | -0 :warning: | yetus | 0m 3s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck --skip-errorprone --exclude-tests-url --build-thread --surefire-first-part-fork-count --surefire-second-part-fork-count | ||| _ Prechecks _ | ||| _ master Compile Tests _ | | +1 :green_heart: | mvninstall | 6m 25s | master passed | | +1 :green_heart: | compile | 0m 30s | master passed | | +1 :green_heart: | javadoc | 0m 20s | master passed | ||| _ Patch Compile Tests _ | | +1 :green_heart: | mvninstall | 0m 32s | the patch passed | | +1 :green_heart: | compile | 0m 30s | the patch passed | | +1 :green_heart: | javac | 0m 30s | the patch passed | | +1 :green_heart: | javadoc | 0m 20s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | unit | 8m 58s | hbase-mapreduce in the patch passed. | | | | 19m 31s | | | Subsystem | Report/Notes | |--:|:-| | Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4216/5/artifact/yetus-jdk11-hadoop3-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/4216 | | Optional Tests | javac javadoc unit compile | | uname | Linux 525b1916d66c 5.4.0-90-generic #101-Ubuntu SMP Fri Oct 15 20:00:55 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / c4ec5230f2 | | Default Java | AdoptOpenJDK-11.0.10+9 | | Test Results | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4216/5/testReport/ | | Max. process+thread count | 849 (vs. ulimit of 3) | | modules | C: hbase-mapreduce U: hbase-mapreduce | | Console output | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4216/5/console | | versions | git=2.17.1 maven=3.6.3 | | Powered by | Apache Yetus 0.12.0 https://yetus.apache.org | This message was automatically generated. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #4216: HBASE-26720 ExportSnapshot should validate the source snapshot before copying files
Apache-HBase commented on pull request #4216: URL: https://github.com/apache/hbase/pull/4216#issuecomment-1068697865 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 1m 10s | Docker mode activated. | | -0 :warning: | yetus | 0m 2s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck --skip-errorprone --exclude-tests-url --build-thread --surefire-first-part-fork-count --surefire-second-part-fork-count | ||| _ Prechecks _ | ||| _ master Compile Tests _ | | +1 :green_heart: | mvninstall | 5m 23s | master passed | | +1 :green_heart: | compile | 0m 23s | master passed | | +1 :green_heart: | javadoc | 0m 16s | master passed | ||| _ Patch Compile Tests _ | | +1 :green_heart: | mvninstall | 0m 28s | the patch passed | | +1 :green_heart: | compile | 0m 26s | the patch passed | | +1 :green_heart: | javac | 0m 26s | the patch passed | | +1 :green_heart: | javadoc | 0m 17s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | unit | 7m 54s | hbase-mapreduce in the patch passed. | | | | 17m 4s | | | Subsystem | Report/Notes | |--:|:-| | Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4216/5/artifact/yetus-jdk8-hadoop3-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/4216 | | Optional Tests | javac javadoc unit compile | | uname | Linux 79789a6ef02a 5.4.0-90-generic #101-Ubuntu SMP Fri Oct 15 20:00:55 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / c4ec5230f2 | | Default Java | AdoptOpenJDK-1.8.0_282-b08 | | Test Results | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4216/5/testReport/ | | Max. process+thread count | 856 (vs. ulimit of 3) | | modules | C: hbase-mapreduce U: hbase-mapreduce | | Console output | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4216/5/console | | versions | git=2.17.1 maven=3.6.3 | | Powered by | Apache Yetus 0.12.0 https://yetus.apache.org | This message was automatically generated. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] sunhelly commented on a change in pull request #4205: Hbase 26275, ROWPREFIXFILTER doesn't work when execute deleteall with table hbase:meta
sunhelly commented on a change in pull request #4205:
URL: https://github.com/apache/hbase/pull/4205#discussion_r827594422
##
File path: hbase-shell/src/main/ruby/hbase/table.rb
##
@@ -243,7 +243,7 @@ def _deleteall_internal(row, column = nil,
args = {}, all_version = true)
# delete operation doesn't need read permission. Retaining the read
check for
# meta table as a part of HBASE-5837.
- if is_meta_table?
+ if is_meta_table? and !(row.is_a?(Hash) and row.key?('ROWPREFIXFILTER'))
Review comment:
Of course, you can try to make the descriptions more clearly.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [hbase] xicm commented on a change in pull request #4205: Hbase 26275, ROWPREFIXFILTER doesn't work when execute deleteall with table hbase:meta
xicm commented on a change in pull request #4205:
URL: https://github.com/apache/hbase/pull/4205#discussion_r827592988
##
File path: hbase-shell/src/main/ruby/hbase/table.rb
##
@@ -243,7 +243,7 @@ def _deleteall_internal(row, column = nil,
args = {}, all_version = true)
# delete operation doesn't need read permission. Retaining the read
check for
# meta table as a part of HBASE-5837.
- if is_meta_table?
+ if is_meta_table? and !(row.is_a?(Hash) and row.key?('ROWPREFIXFILTER'))
Review comment:
Yes, I have this concen too. Let me fix the error message?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #4216: HBASE-26720 ExportSnapshot should validate the source snapshot before copying files
Apache-HBase commented on pull request #4216: URL: https://github.com/apache/hbase/pull/4216#issuecomment-1068690480 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 0m 0s | Docker mode activated. | | -1 :x: | docker | 0m 0s | Docker command '/usr/bin/docker' is too old (1.41 < API v 1.27.0). | | Subsystem | Report/Notes | |--:|:-| | Console output | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4216/5/console | | versions | git=2.17.1 | | Powered by | Apache Yetus 0.12.0 https://yetus.apache.org | This message was automatically generated. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] sunhelly commented on a change in pull request #4205: Hbase 26275, ROWPREFIXFILTER doesn't work when execute deleteall with table hbase:meta
sunhelly commented on a change in pull request #4205:
URL: https://github.com/apache/hbase/pull/4205#discussion_r827591672
##
File path: hbase-shell/src/main/ruby/hbase/table.rb
##
@@ -243,7 +243,7 @@ def _deleteall_internal(row, column = nil,
args = {}, all_version = true)
# delete operation doesn't need read permission. Retaining the read
check for
# meta table as a part of HBASE-5837.
- if is_meta_table?
+ if is_meta_table? and !(row.is_a?(Hash) and row.key?('ROWPREFIXFILTER'))
Review comment:
It's dangerous to delete rows in meta table by 'ROWPREFIXFILTER', right?
If the rowPrefix matches some rows unexpected, the meta data will be deleted
unexpected. I think maybe this is why the origin design seperated meta table
here.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #4216: HBASE-26720 ExportSnapshot should validate the source snapshot before copying files
Apache-HBase commented on pull request #4216: URL: https://github.com/apache/hbase/pull/4216#issuecomment-1068678615 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 1m 12s | Docker mode activated. | | -0 :warning: | yetus | 0m 3s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck --skip-errorprone --exclude-tests-url --build-thread --surefire-first-part-fork-count --surefire-second-part-fork-count | ||| _ Prechecks _ | ||| _ master Compile Tests _ | | +1 :green_heart: | mvninstall | 7m 4s | master passed | | +1 :green_heart: | compile | 0m 29s | master passed | | +1 :green_heart: | javadoc | 0m 22s | master passed | ||| _ Patch Compile Tests _ | | +1 :green_heart: | mvninstall | 0m 31s | the patch passed | | +1 :green_heart: | compile | 0m 27s | the patch passed | | +1 :green_heart: | javac | 0m 27s | the patch passed | | +1 :green_heart: | javadoc | 0m 19s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | unit | 8m 51s | hbase-mapreduce in the patch passed. | | | | 20m 3s | | | Subsystem | Report/Notes | |--:|:-| | Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4216/4/artifact/yetus-jdk11-hadoop3-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/4216 | | Optional Tests | javac javadoc unit compile | | uname | Linux e0888d71c596 5.4.0-90-generic #101-Ubuntu SMP Fri Oct 15 20:00:55 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / c4ec5230f2 | | Default Java | AdoptOpenJDK-11.0.10+9 | | Test Results | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4216/4/testReport/ | | Max. process+thread count | 898 (vs. ulimit of 3) | | modules | C: hbase-mapreduce U: hbase-mapreduce | | Console output | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4216/4/console | | versions | git=2.17.1 maven=3.6.3 | | Powered by | Apache Yetus 0.12.0 https://yetus.apache.org | This message was automatically generated. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #4216: HBASE-26720 ExportSnapshot should validate the source snapshot before copying files
Apache-HBase commented on pull request #4216: URL: https://github.com/apache/hbase/pull/4216#issuecomment-1068677801 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 1m 8s | Docker mode activated. | | -0 :warning: | yetus | 0m 2s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck --skip-errorprone --exclude-tests-url --build-thread --surefire-first-part-fork-count --surefire-second-part-fork-count | ||| _ Prechecks _ | ||| _ master Compile Tests _ | | +1 :green_heart: | mvninstall | 5m 57s | master passed | | +1 :green_heart: | compile | 0m 25s | master passed | | +1 :green_heart: | javadoc | 0m 19s | master passed | ||| _ Patch Compile Tests _ | | +1 :green_heart: | mvninstall | 0m 27s | the patch passed | | +1 :green_heart: | compile | 0m 25s | the patch passed | | +1 :green_heart: | javac | 0m 25s | the patch passed | | +1 :green_heart: | javadoc | 0m 16s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | unit | 8m 14s | hbase-mapreduce in the patch passed. | | | | 17m 55s | | | Subsystem | Report/Notes | |--:|:-| | Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4216/4/artifact/yetus-jdk8-hadoop3-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/4216 | | Optional Tests | javac javadoc unit compile | | uname | Linux c31bf5e42d83 5.4.0-90-generic #101-Ubuntu SMP Fri Oct 15 20:00:55 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / c4ec5230f2 | | Default Java | AdoptOpenJDK-1.8.0_282-b08 | | Test Results | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4216/4/testReport/ | | Max. process+thread count | 866 (vs. ulimit of 3) | | modules | C: hbase-mapreduce U: hbase-mapreduce | | Console output | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4216/4/console | | versions | git=2.17.1 maven=3.6.3 | | Powered by | Apache Yetus 0.12.0 https://yetus.apache.org | This message was automatically generated. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #4216: HBASE-26720 ExportSnapshot should validate the source snapshot before copying files
Apache-HBase commented on pull request #4216: URL: https://github.com/apache/hbase/pull/4216#issuecomment-1068671304 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 0m 0s | Docker mode activated. | | -1 :x: | docker | 0m 1s | Docker command '/usr/bin/docker' is too old (1.41 < API v 1.27.0). | | Subsystem | Report/Notes | |--:|:-| | Console output | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4216/4/console | | versions | git=2.17.1 | | Powered by | Apache Yetus 0.12.0 https://yetus.apache.org | This message was automatically generated. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] xicm commented on a change in pull request #4216: HBASE-26720 ExportSnapshot should validate the source snapshot before copying files
xicm commented on a change in pull request #4216:
URL: https://github.com/apache/hbase/pull/4216#discussion_r827574748
##
File path:
hbase-mapreduce/src/main/java/org/apache/hadoop/hbase/snapshot/ExportSnapshot.java
##
@@ -996,6 +1000,12 @@ public int doWork() throws IOException {
LOG.debug("outputFs={}, outputRoot={}, skipTmp={},
initialOutputSnapshotDir={}",
outputFs, outputRoot.toString(), skipTmp, initialOutputSnapshotDir);
+// Verify snapshot source before copying files
+if (verifySource) {
+ LOG.info("Verify snapshot source.");
Review comment:
I added inputFs, inputRoot, snapshotDir values in the log, srcConf
contains too many things that I didn't add.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[jira] [Commented] (HBASE-26611) Changing SFT implementation on disabled table is dangerous
[
https://issues.apache.org/jira/browse/HBASE-26611?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17507290#comment-17507290
]
Hudson commented on HBASE-26611:
Results for branch branch-2
[build #485 on
builds.a.o|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-2/485/]:
(x) *{color:red}-1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-2/485/General_20Nightly_20Build_20Report/]
(x) {color:red}-1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-2/485/JDK8_20Nightly_20Build_20Report_20_28Hadoop2_29/]
(x) {color:red}-1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-2/485/JDK8_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(x) {color:red}-1 jdk11 hadoop3 checks{color}
-- For more information [see jdk11
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-2/485/JDK11_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
(x) {color:red}-1 client integration test{color}
--Failed when running client tests on top of Hadoop 2. [see log for
details|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-2/485//artifact/output-integration/hadoop-2.log].
(note that this means we didn't run on Hadoop 3)
> Changing SFT implementation on disabled table is dangerous
> --
>
> Key: HBASE-26611
> URL: https://issues.apache.org/jira/browse/HBASE-26611
> Project: HBase
> Issue Type: Sub-task
> Components: HFile, proc-v2
>Reporter: Duo Zhang
>Assignee: Duo Zhang
>Priority: Critical
> Fix For: 2.6.0, 3.0.0-alpha-3
>
>
> As our current design is to have a MIGRATION SFT so when region opens, we
> will mirror the store files to the dst SFT.
> But for a disabled table, the region will not be reopened so we have no
> chance to mirror the store files list, so after user enable the table, it may
> find out that all the store files are gone...
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
[jira] [Commented] (HBASE-26840) Fix NPE in the retry of logroller
[
https://issues.apache.org/jira/browse/HBASE-26840?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17507291#comment-17507291
]
Hudson commented on HBASE-26840:
Results for branch branch-2
[build #485 on
builds.a.o|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-2/485/]:
(x) *{color:red}-1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-2/485/General_20Nightly_20Build_20Report/]
(x) {color:red}-1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-2/485/JDK8_20Nightly_20Build_20Report_20_28Hadoop2_29/]
(x) {color:red}-1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-2/485/JDK8_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(x) {color:red}-1 jdk11 hadoop3 checks{color}
-- For more information [see jdk11
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-2/485/JDK11_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
(x) {color:red}-1 client integration test{color}
--Failed when running client tests on top of Hadoop 2. [see log for
details|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-2/485//artifact/output-integration/hadoop-2.log].
(note that this means we didn't run on Hadoop 3)
> Fix NPE in the retry of logroller
> -
>
> Key: HBASE-26840
> URL: https://issues.apache.org/jira/browse/HBASE-26840
> Project: HBase
> Issue Type: Bug
> Components: wal
>Affects Versions: 2.4.10
>Reporter: Xiaolin Ha
>Assignee: Xiaolin Ha
>Priority: Minor
> Fix For: 2.5.0, 2.6.0, 3.0.0-alpha-3, 2.4.11
>
>
> {code:java}
> LOG.warn("Retry to roll log, nAttempts={}, waiting time={}ms, sleeping 1s to
> retry,"
> + " last excepiton= {}", nAttempts, waitingTime,
> ioe.getCause().getClass().getSimpleName()); {code}
> ioe.getCause() maybe null.
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
[jira] [Commented] (HBASE-26833) Avoid waiting to clear buffer usage of ReplicationSourceShipper when aborting the RS
[
https://issues.apache.org/jira/browse/HBASE-26833?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17507292#comment-17507292
]
Hudson commented on HBASE-26833:
Results for branch branch-2
[build #485 on
builds.a.o|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-2/485/]:
(x) *{color:red}-1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-2/485/General_20Nightly_20Build_20Report/]
(x) {color:red}-1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-2/485/JDK8_20Nightly_20Build_20Report_20_28Hadoop2_29/]
(x) {color:red}-1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-2/485/JDK8_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(x) {color:red}-1 jdk11 hadoop3 checks{color}
-- For more information [see jdk11
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-2/485/JDK11_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
(x) {color:red}-1 client integration test{color}
--Failed when running client tests on top of Hadoop 2. [see log for
details|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-2/485//artifact/output-integration/hadoop-2.log].
(note that this means we didn't run on Hadoop 3)
> Avoid waiting to clear buffer usage of ReplicationSourceShipper when aborting
> the RS
>
>
> Key: HBASE-26833
> URL: https://issues.apache.org/jira/browse/HBASE-26833
> Project: HBase
> Issue Type: Improvement
> Components: regionserver, Replication
>Affects Versions: 2.4.10
>Reporter: Xiaolin Ha
>Assignee: Xiaolin Ha
>Priority: Major
> Fix For: 2.5.0, 2.6.0, 3.0.0-alpha-3, 2.4.11
>
>
> HBASE-24813 introduced the clear of buffer used in replication source
> shipper, but there is sleep in the method, if the variable sleepForRetries
> has a large value, and there are many wal groups, the aborting of RS may last
> a long time, but we should only do some necessary things in the aborting
> progress.
> {code:java}
> void clearWALEntryBatch() {
> long timeout = System.currentTimeMillis() + this.shipEditsTimeout;
> while(this.isAlive() || this.entryReader.isAlive()){
> try {
> if (System.currentTimeMillis() >= timeout) {
> LOG.warn("Shipper clearWALEntryBatch method timed out whilst waiting
> reader/shipper "
> + "thread to stop. Not cleaning buffer usage. Shipper alive: {};
> Reader alive: {}",
> this.source.getPeerId(), this.isAlive(),
> this.entryReader.isAlive());
> return;
> } else {
> // Wait both shipper and reader threads to stop
> Thread.sleep(this.sleepForRetries);
> }
> } catch (InterruptedException e) {
> LOG.warn("{} Interrupted while waiting {} to stop on
> clearWALEntryBatch. "
> + "Not cleaning buffer usage: {}", this.source.getPeerId(),
> this.getName(), e);
> return;
> }
> }
> .. {code}
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
[jira] [Commented] (HBASE-26835) Rewrite TestLruAdaptiveBlockCache to make it more stable
[
https://issues.apache.org/jira/browse/HBASE-26835?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17507289#comment-17507289
]
Hudson commented on HBASE-26835:
Results for branch branch-2
[build #485 on
builds.a.o|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-2/485/]:
(x) *{color:red}-1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-2/485/General_20Nightly_20Build_20Report/]
(x) {color:red}-1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-2/485/JDK8_20Nightly_20Build_20Report_20_28Hadoop2_29/]
(x) {color:red}-1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-2/485/JDK8_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(x) {color:red}-1 jdk11 hadoop3 checks{color}
-- For more information [see jdk11
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-2/485/JDK11_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
(x) {color:red}-1 client integration test{color}
--Failed when running client tests on top of Hadoop 2. [see log for
details|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-2/485//artifact/output-integration/hadoop-2.log].
(note that this means we didn't run on Hadoop 3)
> Rewrite TestLruAdaptiveBlockCache to make it more stable
>
>
> Key: HBASE-26835
> URL: https://issues.apache.org/jira/browse/HBASE-26835
> Project: HBase
> Issue Type: Improvement
> Components: test
>Reporter: Duo Zhang
>Assignee: Duo Zhang
>Priority: Major
> Fix For: 2.5.0, 3.0.0-alpha-3, 2.4.11
>
>
> Similiar to HBASE-26830.
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
[GitHub] [hbase] Apache-HBase commented on pull request #4231: HBASE-26848 Set java.io.tmpdir on mvn command when running jenkins job
Apache-HBase commented on pull request #4231: URL: https://github.com/apache/hbase/pull/4231#issuecomment-1068619438 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Commented] (HBASE-26833) Avoid waiting to clear buffer usage of ReplicationSourceShipper when aborting the RS
[
https://issues.apache.org/jira/browse/HBASE-26833?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17507288#comment-17507288
]
Hudson commented on HBASE-26833:
Results for branch branch-2.4
[build #307 on
builds.a.o|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-2.4/307/]:
(x) *{color:red}-1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-2.4/307/General_20Nightly_20Build_20Report/]
(x) {color:red}-1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-2.4/307/JDK8_20Nightly_20Build_20Report_20_28Hadoop2_29/]
(x) {color:red}-1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-2.4/307/JDK8_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(x) {color:red}-1 jdk11 hadoop3 checks{color}
-- For more information [see jdk11
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-2.4/307/JDK11_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
(/) {color:green}+1 client integration test{color}
> Avoid waiting to clear buffer usage of ReplicationSourceShipper when aborting
> the RS
>
>
> Key: HBASE-26833
> URL: https://issues.apache.org/jira/browse/HBASE-26833
> Project: HBase
> Issue Type: Improvement
> Components: regionserver, Replication
>Affects Versions: 2.4.10
>Reporter: Xiaolin Ha
>Assignee: Xiaolin Ha
>Priority: Major
> Fix For: 2.5.0, 2.6.0, 3.0.0-alpha-3, 2.4.11
>
>
> HBASE-24813 introduced the clear of buffer used in replication source
> shipper, but there is sleep in the method, if the variable sleepForRetries
> has a large value, and there are many wal groups, the aborting of RS may last
> a long time, but we should only do some necessary things in the aborting
> progress.
> {code:java}
> void clearWALEntryBatch() {
> long timeout = System.currentTimeMillis() + this.shipEditsTimeout;
> while(this.isAlive() || this.entryReader.isAlive()){
> try {
> if (System.currentTimeMillis() >= timeout) {
> LOG.warn("Shipper clearWALEntryBatch method timed out whilst waiting
> reader/shipper "
> + "thread to stop. Not cleaning buffer usage. Shipper alive: {};
> Reader alive: {}",
> this.source.getPeerId(), this.isAlive(),
> this.entryReader.isAlive());
> return;
> } else {
> // Wait both shipper and reader threads to stop
> Thread.sleep(this.sleepForRetries);
> }
> } catch (InterruptedException e) {
> LOG.warn("{} Interrupted while waiting {} to stop on
> clearWALEntryBatch. "
> + "Not cleaning buffer usage: {}", this.source.getPeerId(),
> this.getName(), e);
> return;
> }
> }
> .. {code}
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
[GitHub] [hbase] Apache-HBase commented on pull request #4231: HBASE-26848 Set java.io.tmpdir on mvn command when running jenkins job
Apache-HBase commented on pull request #4231: URL: https://github.com/apache/hbase/pull/4231#issuecomment-1068618771 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #4231: HBASE-26848 Set java.io.tmpdir on mvn command when running jenkins job
Apache-HBase commented on pull request #4231: URL: https://github.com/apache/hbase/pull/4231#issuecomment-1068618545 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Commented] (HBASE-26537) FuzzyRowFilter backwards compatibility
[
https://issues.apache.org/jira/browse/HBASE-26537?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17507284#comment-17507284
]
Hudson commented on HBASE-26537:
Results for branch branch-2.4
[build #307 on
builds.a.o|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-2.4/307/]:
(x) *{color:red}-1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-2.4/307/General_20Nightly_20Build_20Report/]
(x) {color:red}-1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-2.4/307/JDK8_20Nightly_20Build_20Report_20_28Hadoop2_29/]
(x) {color:red}-1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-2.4/307/JDK8_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(x) {color:red}-1 jdk11 hadoop3 checks{color}
-- For more information [see jdk11
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-2.4/307/JDK11_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
(/) {color:green}+1 client integration test{color}
> FuzzyRowFilter backwards compatibility
> --
>
> Key: HBASE-26537
> URL: https://issues.apache.org/jira/browse/HBASE-26537
> Project: HBase
> Issue Type: Bug
>Affects Versions: 1.1.13, 1.2.12, 1.3.6
>Reporter: Bryan Beaudreault
>Assignee: Bryan Beaudreault
>Priority: Major
> Fix For: 2.5.0, 1.8.0, 2.6.0
>
>
> HBASE-15676 introduced a backwards incompatible change which makes it
> impossible to upgrade in our designated upgraded order (server first, then
> client) without potential bad results. Worse, the failure mode is silent – a
> pre-HBASE-15676 client would incorrectly receive 0 results from a
> post-HBASE-15676 server.
> I solved this internally as part of our upgrade from 1.2.0 to 2.4.6 by adding
> a new proto field to switch between the two implementations. I'm submitting
> this Jira to capture and potentially backport that fix for anyone else who
> encounters it.
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
[jira] [Commented] (HBASE-26835) Rewrite TestLruAdaptiveBlockCache to make it more stable
[
https://issues.apache.org/jira/browse/HBASE-26835?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17507286#comment-17507286
]
Hudson commented on HBASE-26835:
Results for branch branch-2.4
[build #307 on
builds.a.o|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-2.4/307/]:
(x) *{color:red}-1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-2.4/307/General_20Nightly_20Build_20Report/]
(x) {color:red}-1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-2.4/307/JDK8_20Nightly_20Build_20Report_20_28Hadoop2_29/]
(x) {color:red}-1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-2.4/307/JDK8_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(x) {color:red}-1 jdk11 hadoop3 checks{color}
-- For more information [see jdk11
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-2.4/307/JDK11_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
(/) {color:green}+1 client integration test{color}
> Rewrite TestLruAdaptiveBlockCache to make it more stable
>
>
> Key: HBASE-26835
> URL: https://issues.apache.org/jira/browse/HBASE-26835
> Project: HBase
> Issue Type: Improvement
> Components: test
>Reporter: Duo Zhang
>Assignee: Duo Zhang
>Priority: Major
> Fix For: 2.5.0, 3.0.0-alpha-3, 2.4.11
>
>
> Similiar to HBASE-26830.
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
[jira] [Commented] (HBASE-26839) Fix compatibility issues in 2.4.11RC0
[
https://issues.apache.org/jira/browse/HBASE-26839?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17507285#comment-17507285
]
Hudson commented on HBASE-26839:
Results for branch branch-2.4
[build #307 on
builds.a.o|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-2.4/307/]:
(x) *{color:red}-1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-2.4/307/General_20Nightly_20Build_20Report/]
(x) {color:red}-1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-2.4/307/JDK8_20Nightly_20Build_20Report_20_28Hadoop2_29/]
(x) {color:red}-1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-2.4/307/JDK8_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(x) {color:red}-1 jdk11 hadoop3 checks{color}
-- For more information [see jdk11
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-2.4/307/JDK11_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
(/) {color:green}+1 client integration test{color}
> Fix compatibility issues in 2.4.11RC0
> -
>
> Key: HBASE-26839
> URL: https://issues.apache.org/jira/browse/HBASE-26839
> Project: HBase
> Issue Type: Task
>Reporter: Andrew Kyle Purtell
>Assignee: Andrew Kyle Purtell
>Priority: Minor
> Fix For: 2.4.11
>
>
> package org.apache.hadoop.hbase.http
> SecurityHeadersFilter
> LimitedPrivate(CONFIG)
> Removed method: SecurityHeadersFilter.getDefaultParameters ( Configuration
> conf ) [static] : Map
> package org.apache.hadoop.hbase.filter
> RandomRowFilter
> Public
> Field random of type java.util.Random has been removed from this class. A
> client program may be interrupted by NoSuchFieldError exception.
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
[jira] [Commented] (HBASE-26603) Cherry pick HBASE-26537 to branch-2.4
[
https://issues.apache.org/jira/browse/HBASE-26603?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17507283#comment-17507283
]
Hudson commented on HBASE-26603:
Results for branch branch-2.4
[build #307 on
builds.a.o|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-2.4/307/]:
(x) *{color:red}-1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-2.4/307/General_20Nightly_20Build_20Report/]
(x) {color:red}-1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-2.4/307/JDK8_20Nightly_20Build_20Report_20_28Hadoop2_29/]
(x) {color:red}-1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-2.4/307/JDK8_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(x) {color:red}-1 jdk11 hadoop3 checks{color}
-- For more information [see jdk11
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-2.4/307/JDK11_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
(/) {color:green}+1 client integration test{color}
> Cherry pick HBASE-26537 to branch-2.4
> -
>
> Key: HBASE-26603
> URL: https://issues.apache.org/jira/browse/HBASE-26603
> Project: HBase
> Issue Type: Sub-task
>Reporter: Andrew Kyle Purtell
>Assignee: Andrew Kyle Purtell
>Priority: Major
> Fix For: 2.4.11
>
>
> This may need more discussion for going into a patch release (or not).
> Consider for 2.4.10.
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
[jira] [Commented] (HBASE-26840) Fix NPE in the retry of logroller
[
https://issues.apache.org/jira/browse/HBASE-26840?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17507287#comment-17507287
]
Hudson commented on HBASE-26840:
Results for branch branch-2.4
[build #307 on
builds.a.o|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-2.4/307/]:
(x) *{color:red}-1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-2.4/307/General_20Nightly_20Build_20Report/]
(x) {color:red}-1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-2.4/307/JDK8_20Nightly_20Build_20Report_20_28Hadoop2_29/]
(x) {color:red}-1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-2.4/307/JDK8_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(x) {color:red}-1 jdk11 hadoop3 checks{color}
-- For more information [see jdk11
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-2.4/307/JDK11_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
(/) {color:green}+1 client integration test{color}
> Fix NPE in the retry of logroller
> -
>
> Key: HBASE-26840
> URL: https://issues.apache.org/jira/browse/HBASE-26840
> Project: HBase
> Issue Type: Bug
> Components: wal
>Affects Versions: 2.4.10
>Reporter: Xiaolin Ha
>Assignee: Xiaolin Ha
>Priority: Minor
> Fix For: 2.5.0, 2.6.0, 3.0.0-alpha-3, 2.4.11
>
>
> {code:java}
> LOG.warn("Retry to roll log, nAttempts={}, waiting time={}ms, sleeping 1s to
> retry,"
> + " last excepiton= {}", nAttempts, waitingTime,
> ioe.getCause().getClass().getSimpleName()); {code}
> ioe.getCause() maybe null.
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
[GitHub] [hbase] Apache9 commented on a change in pull request #4221: Forward-port "HBASE-26834 Adapt ConnectionRule for both sync and async connections" to master
Apache9 commented on a change in pull request #4221:
URL: https://github.com/apache/hbase/pull/4221#discussion_r827526115
##
File path:
hbase-endpoint/src/test/java/org/apache/hadoop/hbase/coprocessor/TestCoprocessorEndpointTracing.java
##
@@ -228,7 +228,7 @@ public void onError(Throwable error) {
@Test
public void traceSyncTableEndpointCall() throws Exception {
-final Connection connection =
connectionRule.getConnection().toConnection();
+final Connection connection = connectionRule.getSyncConnection();
Review comment:
I also do not have very strong opinion on which one is better but if
want to add the 'Sync' prefix, then we need to change a bunch of classes, like
SyncTable, SyncAdmin, etc, and most of them are marked as IA.Public. So in
practical, I suggest we just keep it as is, at least one style is better then
two styles in our code base...
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[jira] [Updated] (HBASE-26848) Set java.io.tmpdir on mvn command when running jenkins job
[ https://issues.apache.org/jira/browse/HBASE-26848?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Duo Zhang updated HBASE-26848: -- Summary: Set java.io.tmpdir on mvn command when running jenkins job (was: Set java.io.tmpdir on mvn command when running UTs in jenkins job) > Set java.io.tmpdir on mvn command when running jenkins job > -- > > Key: HBASE-26848 > URL: https://issues.apache.org/jira/browse/HBASE-26848 > Project: HBase > Issue Type: Improvement > Components: jenkins, test >Reporter: Duo Zhang >Assignee: Duo Zhang >Priority: Major > > Recently on hbase1-10, it is easy to eat all the space of /tmp. I've checked > several times, it is always the stderr*deferred file, which is the temp > output of our UTs. > I think it means we have some problem for some of our UTs which could > generate very big output, but if the tmp is full, it is not easy for us to > find out the root cause as we need to delete the file to recover first... > Looking at the code of surefire > https://github.com/apache/maven-surefire/blob/68bca29b23d27dba3e06cf5e73a3fdc1a22598db/maven-surefire-common/src/main/java/org/apache/maven/plugin/surefire/report/Utf8RecodingDeferredFileOutputStream.java#L69 > It just creates the file with jdk's Files.createTempFile, the location can be > controlled by java.io.tmpdir, so let's point it to the target directory under > the jenkins workspace to avoid wat all the spaces of /tmp first. -- This message was sent by Atlassian Jira (v8.20.1#820001)
[jira] [Updated] (HBASE-26848) Set java.io.tmpdir on mvn command when running UTs in jenkins job
[ https://issues.apache.org/jira/browse/HBASE-26848?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Duo Zhang updated HBASE-26848: -- Summary: Set java.io.tmpdir on mvn command when running UTs in jenkins job (was: Refactor the surefire plugin defination in our poms) > Set java.io.tmpdir on mvn command when running UTs in jenkins job > - > > Key: HBASE-26848 > URL: https://issues.apache.org/jira/browse/HBASE-26848 > Project: HBase > Issue Type: Improvement > Components: jenkins, test >Reporter: Duo Zhang >Assignee: Duo Zhang >Priority: Major > > Recently on hbase1-10, it is easy to eat all the space of /tmp. I've checked > several times, it is always the stderr*deferred file, which is the temp > output of our UTs. > I think it means we have some problem for some of our UTs which could > generate very big output, but if the tmp is full, it is not easy for us to > find out the root cause as we need to delete the file to recover first... > Looking at the code of surefire > https://github.com/apache/maven-surefire/blob/68bca29b23d27dba3e06cf5e73a3fdc1a22598db/maven-surefire-common/src/main/java/org/apache/maven/plugin/surefire/report/Utf8RecodingDeferredFileOutputStream.java#L69 > It just creates the file with jdk's Files.createTempFile, the location can be > controlled by java.io.tmpdir, so let's point it to the target directory under > the jenkins workspace to avoid wat all the spaces of /tmp first. -- This message was sent by Atlassian Jira (v8.20.1#820001)
[jira] [Commented] (HBASE-26848) Refactor the surefire plugin defination in our poms
[ https://issues.apache.org/jira/browse/HBASE-26848?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17507274#comment-17507274 ] Duo Zhang commented on HBASE-26848: --- OK, I think I found the problem... The systemPropertyVariables is used to set system properties when starting new processes for running UTs, but the surefire plugin itself is running in the process of mvn, so you need to add -Djava.io.tmpdir in the mvn command line if you want to relocate the stdout and stderr output... Let me prepare a PR. > Refactor the surefire plugin defination in our poms > --- > > Key: HBASE-26848 > URL: https://issues.apache.org/jira/browse/HBASE-26848 > Project: HBase > Issue Type: Improvement > Components: jenkins, test >Reporter: Duo Zhang >Assignee: Duo Zhang >Priority: Major > > Recently on hbase1-10, it is easy to eat all the space of /tmp. I've checked > several times, it is always the stderr*deferred file, which is the temp > output of our UTs. > I think it means we have some problem for some of our UTs which could > generate very big output, but if the tmp is full, it is not easy for us to > find out the root cause as we need to delete the file to recover first... > Looking at the code of surefire > https://github.com/apache/maven-surefire/blob/68bca29b23d27dba3e06cf5e73a3fdc1a22598db/maven-surefire-common/src/main/java/org/apache/maven/plugin/surefire/report/Utf8RecodingDeferredFileOutputStream.java#L69 > It just creates the file with jdk's Files.createTempFile, the location can be > controlled by java.io.tmpdir, so let's point it to the target directory under > the jenkins workspace to avoid wat all the spaces of /tmp first. -- This message was sent by Atlassian Jira (v8.20.1#820001)
[GitHub] [hbase] Apache-HBase commented on pull request #4181: HBASE-26810 Add dynamic configuration support for system coprocessors
Apache-HBase commented on pull request #4181: URL: https://github.com/apache/hbase/pull/4181#issuecomment-1068582142 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 0m 25s | Docker mode activated. | | -0 :warning: | yetus | 0m 2s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck --skip-errorprone --exclude-tests-url --build-thread --surefire-first-part-fork-count --surefire-second-part-fork-count | ||| _ Prechecks _ | ||| _ master Compile Tests _ | | +0 :ok: | mvndep | 0m 17s | Maven dependency ordering for branch | | +1 :green_heart: | mvninstall | 3m 42s | master passed | | +1 :green_heart: | compile | 2m 53s | master passed | | +1 :green_heart: | javadoc | 2m 6s | master passed | ||| _ Patch Compile Tests _ | | +0 :ok: | mvndep | 0m 10s | Maven dependency ordering for patch | | +1 :green_heart: | mvninstall | 4m 16s | the patch passed | | +1 :green_heart: | compile | 2m 58s | the patch passed | | +1 :green_heart: | javac | 2m 58s | the patch passed | | +1 :green_heart: | javadoc | 2m 10s | the patch passed | ||| _ Other Tests _ | | -1 :x: | unit | 132m 30s | hbase-server in the patch failed. | | -1 :x: | unit | 225m 39s | root in the patch failed. | | | | 378m 26s | | | Subsystem | Report/Notes | |--:|:-| | Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4181/9/artifact/yetus-jdk8-hadoop3-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/4181 | | Optional Tests | javac javadoc unit compile | | uname | Linux 1f7854b8a5f6 5.4.0-90-generic #101-Ubuntu SMP Fri Oct 15 20:00:55 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / d050217a3d | | Default Java | AdoptOpenJDK-1.8.0_282-b08 | | unit | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4181/9/artifact/yetus-jdk8-hadoop3-check/output/patch-unit-hbase-server.txt | | unit | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4181/9/artifact/yetus-jdk8-hadoop3-check/output/patch-unit-root.txt | | Test Results | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4181/9/testReport/ | | Max. process+thread count | 3199 (vs. ulimit of 3) | | modules | C: hbase-server . U: . | | Console output | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4181/9/console | | versions | git=2.17.1 maven=3.6.3 | | Powered by | Apache Yetus 0.12.0 https://yetus.apache.org | This message was automatically generated. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #4181: HBASE-26810 Add dynamic configuration support for system coprocessors
Apache-HBase commented on pull request #4181: URL: https://github.com/apache/hbase/pull/4181#issuecomment-1068566766 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 0m 40s | Docker mode activated. | | -0 :warning: | yetus | 0m 2s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck --skip-errorprone --exclude-tests-url --build-thread --surefire-first-part-fork-count --surefire-second-part-fork-count | ||| _ Prechecks _ | ||| _ master Compile Tests _ | | +0 :ok: | mvndep | 0m 50s | Maven dependency ordering for branch | | +1 :green_heart: | mvninstall | 5m 11s | master passed | | +1 :green_heart: | compile | 3m 46s | master passed | | +1 :green_heart: | javadoc | 3m 0s | master passed | ||| _ Patch Compile Tests _ | | +0 :ok: | mvndep | 0m 10s | Maven dependency ordering for patch | | +1 :green_heart: | mvninstall | 5m 20s | the patch passed | | +1 :green_heart: | compile | 3m 39s | the patch passed | | +1 :green_heart: | javac | 3m 39s | the patch passed | | +1 :green_heart: | javadoc | 2m 58s | the patch passed | ||| _ Other Tests _ | | -1 :x: | unit | 138m 12s | hbase-server in the patch failed. | | -1 :x: | unit | 190m 58s | root in the patch failed. | | | | 356m 40s | | | Subsystem | Report/Notes | |--:|:-| | Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4181/9/artifact/yetus-jdk11-hadoop3-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/4181 | | Optional Tests | javac javadoc unit compile | | uname | Linux 7a9293542a35 5.4.0-1025-aws #25~18.04.1-Ubuntu SMP Fri Sep 11 12:03:04 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / d050217a3d | | Default Java | AdoptOpenJDK-11.0.10+9 | | unit | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4181/9/artifact/yetus-jdk11-hadoop3-check/output/patch-unit-hbase-server.txt | | unit | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4181/9/artifact/yetus-jdk11-hadoop3-check/output/patch-unit-root.txt | | Test Results | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4181/9/testReport/ | | Max. process+thread count | 2193 (vs. ulimit of 3) | | modules | C: hbase-server . U: . | | Console output | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4181/9/console | | versions | git=2.17.1 maven=3.6.3 | | Powered by | Apache Yetus 0.12.0 https://yetus.apache.org | This message was automatically generated. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Commented] (HBASE-25958) Region Visualizer: Implement a Region Size histogram
[ https://issues.apache.org/jira/browse/HBASE-25958?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17507217#comment-17507217 ] Nick Dimiduk commented on HBASE-25958: -- I have the visualization implemented. I have not gotten the rest of the JavaScript to play along. https://github.com/ndimiduk/hbase/commit/bdebffeca9b1ce8bb3c7fb5cda81e45a2b2cc782 > Region Visualizer: Implement a Region Size histogram > > > Key: HBASE-25958 > URL: https://issues.apache.org/jira/browse/HBASE-25958 > Project: HBase > Issue Type: Improvement > Components: master, UI >Reporter: Nick Dimiduk >Assignee: Nick Dimiduk >Priority: Major > > Add a new view to the Region Visualizer that displays a histogram of region > size. Make it so that the visualization can by selected, rather than showing > multiple charts at once. -- This message was sent by Atlassian Jira (v8.20.1#820001)
[jira] [Resolved] (HBASE-25865) Visualize current state of region assignment
[
https://issues.apache.org/jira/browse/HBASE-25865?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Nick Dimiduk resolved HBASE-25865.
--
Resolution: Fixed
> Visualize current state of region assignment
>
>
> Key: HBASE-25865
> URL: https://issues.apache.org/jira/browse/HBASE-25865
> Project: HBase
> Issue Type: New Feature
> Components: master, Operability, Usability
>Reporter: Nick Dimiduk
>Assignee: Nick Dimiduk
>Priority: Blocker
> Fix For: 2.5.0, 2.6.0, 3.0.0-alpha-3
>
> Attachments: image1.png, image2.png
>
>
> After several months of debugging and tuning the balancer and normalizer on a
> large production cluster, we found that working from visualizations of the
> current region state was very useful for understanding behaviors and
> quantifying improvements we made along the way. Specifically, we found that a
> chart of total assigned region count and total assigned region store files
> size per table per host was immensely useful for tuning the balancer.
> Histograms of store file size made understanding normalizer activity much
> more intuitive.
> Our scripts would parse the output of the shell's {{status 'detailed'}}
> command, extract the desired metric, and produce charts. I'd like to build
> into the master UI the equivalent functionality, with data coming directly
> from the {{ClusterMetrics object}}, and data rendered into an interactive
> chart rendered in the browser.
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
[jira] [Commented] (HBASE-25865) Visualize current state of region assignment
[
https://issues.apache.org/jira/browse/HBASE-25865?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17507211#comment-17507211
]
Nick Dimiduk commented on HBASE-25865:
--
I have converted HBASE-25958 to an independent JIRA because I don't think it's
critical for blocking 2.5.0.
> Visualize current state of region assignment
>
>
> Key: HBASE-25865
> URL: https://issues.apache.org/jira/browse/HBASE-25865
> Project: HBase
> Issue Type: New Feature
> Components: master, Operability, Usability
>Reporter: Nick Dimiduk
>Assignee: Nick Dimiduk
>Priority: Blocker
> Fix For: 2.5.0, 2.6.0, 3.0.0-alpha-3
>
> Attachments: image1.png, image2.png
>
>
> After several months of debugging and tuning the balancer and normalizer on a
> large production cluster, we found that working from visualizations of the
> current region state was very useful for understanding behaviors and
> quantifying improvements we made along the way. Specifically, we found that a
> chart of total assigned region count and total assigned region store files
> size per table per host was immensely useful for tuning the balancer.
> Histograms of store file size made understanding normalizer activity much
> more intuitive.
> Our scripts would parse the output of the shell's {{status 'detailed'}}
> command, extract the desired metric, and produce charts. I'd like to build
> into the master UI the equivalent functionality, with data coming directly
> from the {{ClusterMetrics object}}, and data rendered into an interactive
> chart rendered in the browser.
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
[jira] [Updated] (HBASE-25865) Visualize current state of region assignment
[
https://issues.apache.org/jira/browse/HBASE-25865?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Nick Dimiduk updated HBASE-25865:
-
Fix Version/s: 2.5.0
> Visualize current state of region assignment
>
>
> Key: HBASE-25865
> URL: https://issues.apache.org/jira/browse/HBASE-25865
> Project: HBase
> Issue Type: New Feature
> Components: master, Operability, Usability
>Reporter: Nick Dimiduk
>Assignee: Nick Dimiduk
>Priority: Blocker
> Fix For: 2.5.0, 2.6.0, 3.0.0-alpha-3
>
> Attachments: image1.png, image2.png
>
>
> After several months of debugging and tuning the balancer and normalizer on a
> large production cluster, we found that working from visualizations of the
> current region state was very useful for understanding behaviors and
> quantifying improvements we made along the way. Specifically, we found that a
> chart of total assigned region count and total assigned region store files
> size per table per host was immensely useful for tuning the balancer.
> Histograms of store file size made understanding normalizer activity much
> more intuitive.
> Our scripts would parse the output of the shell's {{status 'detailed'}}
> command, extract the desired metric, and produce charts. I'd like to build
> into the master UI the equivalent functionality, with data coming directly
> from the {{ClusterMetrics object}}, and data rendered into an interactive
> chart rendered in the browser.
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
[jira] [Commented] (HBASE-25958) Region Visualizer: Implement a Region Size histogram
[ https://issues.apache.org/jira/browse/HBASE-25958?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17507209#comment-17507209 ] Nick Dimiduk commented on HBASE-25958: -- FYI [~apurtell], I have converted this issue to an independent JIRA because I don't think it's critical for blocking 2.5.0. > Region Visualizer: Implement a Region Size histogram > > > Key: HBASE-25958 > URL: https://issues.apache.org/jira/browse/HBASE-25958 > Project: HBase > Issue Type: Improvement > Components: master, UI >Reporter: Nick Dimiduk >Assignee: Nick Dimiduk >Priority: Major > > Add a new view to the Region Visualizer that displays a histogram of region > size. Make it so that the visualization can by selected, rather than showing > multiple charts at once. -- This message was sent by Atlassian Jira (v8.20.1#820001)
[jira] [Updated] (HBASE-25958) Region Visualizer: Implement a Region Size histogram
[ https://issues.apache.org/jira/browse/HBASE-25958?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nick Dimiduk updated HBASE-25958: - Summary: Region Visualizer: Implement a Region Size histogram (was: Implement a Region Size histogram) > Region Visualizer: Implement a Region Size histogram > > > Key: HBASE-25958 > URL: https://issues.apache.org/jira/browse/HBASE-25958 > Project: HBase > Issue Type: Improvement > Components: master, UI >Reporter: Nick Dimiduk >Assignee: Nick Dimiduk >Priority: Major > > Add a new view to the Region Visualizer that displays a histogram of region > size. Make it so that the visualization can by selected, rather than showing > multiple charts at once. -- This message was sent by Atlassian Jira (v8.20.1#820001)
[jira] [Updated] (HBASE-25958) Implement a Region Size histogram
[ https://issues.apache.org/jira/browse/HBASE-25958?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nick Dimiduk updated HBASE-25958: - Description: Add a new view to the Region Visualizer that displays a histogram of region size. Make it so that the visualization can by selected, rather than showing multiple charts at once. (was: Add a new view that displays a histogram of region size. Make it so that the visualization can by selected, rather than showing multiple charts at once.) > Implement a Region Size histogram > - > > Key: HBASE-25958 > URL: https://issues.apache.org/jira/browse/HBASE-25958 > Project: HBase > Issue Type: Improvement > Components: master, UI >Reporter: Nick Dimiduk >Assignee: Nick Dimiduk >Priority: Major > > Add a new view to the Region Visualizer that displays a histogram of region > size. Make it so that the visualization can by selected, rather than showing > multiple charts at once. -- This message was sent by Atlassian Jira (v8.20.1#820001)
[jira] [Updated] (HBASE-25958) Implement a Region Size histogram
[ https://issues.apache.org/jira/browse/HBASE-25958?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nick Dimiduk updated HBASE-25958: - Parent: (was: HBASE-25865) Issue Type: Improvement (was: Sub-task) > Implement a Region Size histogram > - > > Key: HBASE-25958 > URL: https://issues.apache.org/jira/browse/HBASE-25958 > Project: HBase > Issue Type: Improvement > Components: master, UI >Reporter: Nick Dimiduk >Assignee: Nick Dimiduk >Priority: Major > > Add a new view that displays a histogram of region size. Make it so that the > visualization can by selected, rather than showing multiple charts at once. -- This message was sent by Atlassian Jira (v8.20.1#820001)
[jira] [Updated] (HBASE-25896) Implement a Region Visualization on Master WebUI
[ https://issues.apache.org/jira/browse/HBASE-25896?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nick Dimiduk updated HBASE-25896: - Fix Version/s: 2.5.0 > Implement a Region Visualization on Master WebUI > > > Key: HBASE-25896 > URL: https://issues.apache.org/jira/browse/HBASE-25896 > Project: HBase > Issue Type: Sub-task >Reporter: Nick Dimiduk >Assignee: Nick Dimiduk >Priority: Major > Fix For: 2.5.0, 2.6.0, 3.0.0-alpha-3 > > > Demonstrate end-to-end delivery of an interesting visualization. -- This message was sent by Atlassian Jira (v8.20.1#820001)
[jira] [Updated] (HBASE-25896) Implement a Region Visualization on Master WebUI
[ https://issues.apache.org/jira/browse/HBASE-25896?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nick Dimiduk updated HBASE-25896: - Resolution: Fixed Status: Resolved (was: Patch Available) > Implement a Region Visualization on Master WebUI > > > Key: HBASE-25896 > URL: https://issues.apache.org/jira/browse/HBASE-25896 > Project: HBase > Issue Type: Sub-task >Reporter: Nick Dimiduk >Assignee: Nick Dimiduk >Priority: Major > Fix For: 2.5.0, 2.6.0, 3.0.0-alpha-3 > > > Demonstrate end-to-end delivery of an interesting visualization. -- This message was sent by Atlassian Jira (v8.20.1#820001)
[GitHub] [hbase] ndimiduk merged pull request #4230: Backport "HBASE-25896 Implement a Region Visualization on Master WebUI" to branch-2.5
ndimiduk merged pull request #4230: URL: https://github.com/apache/hbase/pull/4230 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Updated] (HBASE-25896) Implement a Region Visualization on Master WebUI
[ https://issues.apache.org/jira/browse/HBASE-25896?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nick Dimiduk updated HBASE-25896: - Fix Version/s: 2.6.0 > Implement a Region Visualization on Master WebUI > > > Key: HBASE-25896 > URL: https://issues.apache.org/jira/browse/HBASE-25896 > Project: HBase > Issue Type: Sub-task >Reporter: Nick Dimiduk >Assignee: Nick Dimiduk >Priority: Major > Fix For: 2.6.0, 3.0.0-alpha-3 > > > Demonstrate end-to-end delivery of an interesting visualization. -- This message was sent by Atlassian Jira (v8.20.1#820001)
[GitHub] [hbase] ndimiduk opened a new pull request #4230: Backport "HBASE-25896 Implement a Region Visualization on Master WebUI" to branch-2.5
ndimiduk opened a new pull request #4230: URL: https://github.com/apache/hbase/pull/4230 This is a demonstration of visualization of regions on the cluster. The visualization is a stacked bar chart showing total storefile size per table per region server, with the x-axis being server names, the y-axis being storfile size, and the bars stacked per table. The visualization is generated entirely on the fly from within the browser, implemented using Vega Lite. So far, Vega appears to handle rendering this visualization for a cluster of over 700 region servers with approximately 300,000 regions. Per [0], include an update to the top-level LICENSE.txt. Also update LICENSE files in all binary distributions (i.e., jars), by way of LICENSE.vm. Vega uses a BSD 3-clause variant without advertising clause, and as such is a "Category A" license, per [1]. No changes are made to the NOTICE files, as per the existing example of bundling the minified JQuery, which is also a Category A license. [0]: https://infra.apache.org/licensing-howto.html [1]: https://www.apache.org/legal/resolved.html#category-a Signed-off-by: Andrew Purtell -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Commented] (HBASE-26812) ShortCircuitingClusterConnection fails to close RegionScanners when making short-circuited calls
[
https://issues.apache.org/jira/browse/HBASE-26812?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17507205#comment-17507205
]
Lars Hofhansl commented on HBASE-26812:
---
Thanks [~comnetwork]. That would indeed work for this instance.
There is still a time-bomb in HBase, though. Each time you cal Get - or
anything that creates a RegionScanner - via the short circuited client you will
leave StoreScanner and all its references around forever.
So IMHO this should be fixed in HBase.
After reflecting a bit, maybe the best is to remove the short-circuit
optimization from HBase, since it is not currently working correctly.
> ShortCircuitingClusterConnection fails to close RegionScanners when making
> short-circuited calls
>
>
> Key: HBASE-26812
> URL: https://issues.apache.org/jira/browse/HBASE-26812
> Project: HBase
> Issue Type: Bug
>Affects Versions: 2.4.9
>Reporter: Lars Hofhansl
>Priority: Critical
>
> Just ran into this on the Phoenix side.
> We retrieve a Connection via
> {{{}RegionCoprocessorEnvironment.createConnection... getTable(...){}}}. And
> then call get on that table. The Get's key happens to be local. Now each call
> to table.get() leaves an open StoreScanner around forever. (verified with a
> memory profiler).
> There references are held via
> RegionScannerImpl.storeHeap.scannersForDelayedClose. Eventially the
> RegionServer goes into a GC of death and can only ended with kill -9.
> The reason appears to be that in this case there is no currentCall context.
> Some time in 2.x the Rpc handler/call was made responsible for closing open
> region scanners, but we forgot to handle {{ShortCircuitingClusterConnection}}
> It's not immediately clear how to fix this. But it does make
> ShortCircuitingClusterConnection useless and dangerous. If you use it, you
> *will* create a giant memory leak.
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
[GitHub] [hbase] ndimiduk merged pull request #4229: Backport "HBASE-25896 Implement a Region Visualization on Master WebUI" to branch-2
ndimiduk merged pull request #4229: URL: https://github.com/apache/hbase/pull/4229 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] ndimiduk opened a new pull request #4229: Backport "HBASE-25896 Implement a Region Visualization on Master WebUI" to branch-2
ndimiduk opened a new pull request #4229: URL: https://github.com/apache/hbase/pull/4229 This is a demonstration of visualization of regions on the cluster. The visualization is a stacked bar chart showing total storefile size per table per region server, with the x-axis being server names, the y-axis being storfile size, and the bars stacked per table. The visualization is generated entirely on the fly from within the browser, implemented using Vega Lite. So far, Vega appears to handle rendering this visualization for a cluster of over 700 region servers with approximately 300,000 regions. Per [0], include an update to the top-level LICENSE.txt. Also update LICENSE files in all binary distributions (i.e., jars), by way of LICENSE.vm. Vega uses a BSD 3-clause variant without advertising clause, and as such is a "Category A" license, per [1]. No changes are made to the NOTICE files, as per the existing example of bundling the minified JQuery, which is also a Category A license. [0]: https://infra.apache.org/licensing-howto.html [1]: https://www.apache.org/legal/resolved.html#category-a Signed-off-by: Andrew Purtell -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] ndimiduk merged pull request #4228: HBASE-25896 Implement a Region Visualization on Master WebUI
ndimiduk merged pull request #4228: URL: https://github.com/apache/hbase/pull/4228 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] ndimiduk opened a new pull request #4228: HBASE-25896 Implement a Region Visualization on Master WebUI
ndimiduk opened a new pull request #4228: URL: https://github.com/apache/hbase/pull/4228 Carrying forward +1 from #4178 . I missed a test change that PR didn't catch for me. I needed an addendum but did a revert. So this is everything. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] ndimiduk merged pull request #4227: Revert "HBASE-25896 Implement a Region Visualization on Master WebUI"
ndimiduk merged pull request #4227: URL: https://github.com/apache/hbase/pull/4227 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] ndimiduk opened a new pull request #4227: Revert "HBASE-25896 Implement a Region Visualization on Master WebUI"
ndimiduk opened a new pull request #4227: URL: https://github.com/apache/hbase/pull/4227 Reverts apache/hbase#4178 This change needs to also update a test. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] ndimiduk commented on pull request #4220: HBASE-26834 Adapt ConnectionRule for both sync and async connections (branch-2)
ndimiduk commented on pull request #4220: URL: https://github.com/apache/hbase/pull/4220#issuecomment-1068395256 @busbey my earlier diagnosis was not entirely accurate -- this is not PR's fault. I reproduce the same behavior when I run a single test out of a single module (with JDK11, Hadoop3, branch-2). i.e., ``` $ env JAVA_HOME=/Library/Java/JavaVirtualMachines/adoptopenjdk-11.jdk/Contents/Home mvn clean install -Dhadoop.profile=3.0 $ env JAVA_HOME=/Library/Java/JavaVirtualMachines/adoptopenjdk-11.jdk/Contents/Home mvn -pl hbase-server test -Dhadoop.profile=3.0 -Dtest=TestApiV1ClusterMetricsResource ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Updated] (HBASE-25896) Implement a Region Visualization on Master WebUI
[ https://issues.apache.org/jira/browse/HBASE-25896?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nick Dimiduk updated HBASE-25896: - Fix Version/s: 3.0.0-alpha-3 > Implement a Region Visualization on Master WebUI > > > Key: HBASE-25896 > URL: https://issues.apache.org/jira/browse/HBASE-25896 > Project: HBase > Issue Type: Sub-task >Reporter: Nick Dimiduk >Assignee: Nick Dimiduk >Priority: Major > Fix For: 3.0.0-alpha-3 > > > Demonstrate end-to-end delivery of an interesting visualization. -- This message was sent by Atlassian Jira (v8.20.1#820001)
[GitHub] [hbase] ndimiduk commented on pull request #4178: HBASE-25896 Implement a Region Visualization on Master WebUI
ndimiduk commented on pull request #4178: URL: https://github.com/apache/hbase/pull/4178#issuecomment-1068383609 Rebased and addressed checkstyle and whitespace issues. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] ndimiduk merged pull request #4178: HBASE-25896 Implement a Region Visualization on Master WebUI
ndimiduk merged pull request #4178: URL: https://github.com/apache/hbase/pull/4178 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] ndimiduk commented on a change in pull request #4221: Forward-port "HBASE-26834 Adapt ConnectionRule for both sync and async connections" to master
ndimiduk commented on a change in pull request #4221:
URL: https://github.com/apache/hbase/pull/4221#discussion_r827333658
##
File path:
hbase-endpoint/src/test/java/org/apache/hadoop/hbase/coprocessor/TestCoprocessorEndpointTracing.java
##
@@ -228,7 +228,7 @@ public void onError(Throwable error) {
@Test
public void traceSyncTableEndpointCall() throws Exception {
-final Connection connection =
connectionRule.getConnection().toConnection();
+final Connection connection = connectionRule.getSyncConnection();
Review comment:
Hmm, you like that better? I think we use those words in our public API
because we had to maintain backward compatibility, hence keeping
`getConnection` and not renaming it to `getSyncConnection`. For this test code,
I thought explicit names would be better. I don't feel strongly, was just my
option as I wrote the code.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [hbase] ndimiduk commented on pull request #4220: HBASE-26834 Adapt ConnectionRule for both sync and async connections (branch-2)
ndimiduk commented on pull request #4220: URL: https://github.com/apache/hbase/pull/4220#issuecomment-1068368751 Okay, well. JDK11 aside, the failures in JDK8 build look like worker resource exhaustion and unrelated to this change. None of those test classes use this Rule, so... -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] frostruan commented on pull request #4225: HBASE-26842 fix TestSnapshotProcedure for branch-2
frostruan commented on pull request #4225: URL: https://github.com/apache/hbase/pull/4225#issuecomment-1068335723 There are five tests failing. The two tests in TestSnapshotFromAdmin fail because that they are designed for the old synchronous implementation, which is polling result from server periodically, now we use a new implementation, which is blocking on the future until timeout. So I think we can move these two tests directly. The two tests for testing AsyncFlushSnapshot fail because that we always choose to using SnapshotProcedure to take snapshot. If SnapshotProcedure is enabled, master will run a SnapshotProcedure and return a procid to the client, then the client will pull result by the procid. If SnapshotProcedure is disabled, then master will take a zk-coordinated snapshot and the client will keep asking master if the snapshot is finished by calling isSnapshotDone method. So The isSnapshotDone method is only used for zk-coordinated snapshots but in these two tests, we actually run a SnapshotProcedure, which will trigger an UnknownSnapshotException. I'll try to fix these problems as soon as possible. The last failing test org.apache.hadoop.hbase.master.cleaner.TestSnapshotFromMaster.testSnapshotHFileArchiving is not very stable. I haven't find the reason why it failed yet. I'll keep trying. Thanks. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase-connectors] taklwu edited a comment on pull request #94: HBASE-26841 Replace log4j with reload4j for hbase-connector
taklwu edited a comment on pull request #94: URL: https://github.com/apache/hbase-connectors/pull/94#issuecomment-1068331863 yeah, other than the hbase-connector itself, as you figured out that it could be version upgrade in hbase , hadoop , and spark-core. then my concern is, if we should postpone the hbase-connector 1.1.0 release till then ? or should we have a release without fixing log4j issue? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase-connectors] taklwu edited a comment on pull request #94: HBASE-26841 Replace log4j with reload4j for hbase-connector
taklwu edited a comment on pull request #94: URL: https://github.com/apache/hbase-connectors/pull/94#issuecomment-1068331863 yeah, as you figured out that it could be version upgrade in hbase , hadoop , and spark-core. then my concern is, if we should postpone the hbase-connector 1.1.0 release till then ? or should we have a release without fixing log4j issue? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase-connectors] taklwu commented on pull request #94: HBASE-26841 Replace log4j with reload4j for hbase-connector
taklwu commented on pull request #94: URL: https://github.com/apache/hbase-connectors/pull/94#issuecomment-1068331863 yeah, as you figured out that it could be an version upgrade in hbase , hadoop , and spark-core. then my concern is, if we should postpone the hbase-connector 1.1.0 release till then ? or should we have a release without fixing log4j issue? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #4181: HBASE-26810 Add dynamic configuration support for system coprocessors
Apache-HBase commented on pull request #4181: URL: https://github.com/apache/hbase/pull/4181#issuecomment-1068320821 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 0m 42s | Docker mode activated. | | -0 :warning: | yetus | 0m 3s | Unprocessed flag(s): --quick-hadoopcheck --exclude-tests-url --asf-nightlies-general-check-base --build-thread --surefire-first-part-fork-count --surefire-second-part-fork-count | ||| _ Prechecks _ | | +1 :green_heart: | dupname | 0m 0s | No case conflicting files found. | | +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. | ||| _ master Compile Tests _ | | +0 :ok: | mvndep | 0m 23s | Maven dependency ordering for branch | | +1 :green_heart: | mvninstall | 3m 57s | master passed | | +1 :green_heart: | compile | 3m 4s | master passed | | +1 :green_heart: | checkstyle | 1m 55s | master passed | | +1 :green_heart: | spotbugs | 11m 23s | master passed | ||| _ Patch Compile Tests _ | | +0 :ok: | mvndep | 0m 10s | Maven dependency ordering for patch | | +1 :green_heart: | mvninstall | 4m 37s | the patch passed | | +1 :green_heart: | compile | 3m 2s | the patch passed | | +1 :green_heart: | javac | 3m 2s | the patch passed | | +1 :green_heart: | checkstyle | 1m 53s | the patch passed | | +1 :green_heart: | whitespace | 0m 0s | The patch has no whitespace issues. | | +1 :green_heart: | spotbugs | 11m 30s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | asflicense | 0m 18s | The patch does not generate ASF License warnings. | | | | 44m 15s | | | Subsystem | Report/Notes | |--:|:-| | Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4181/9/artifact/yetus-general-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/4181 | | Optional Tests | dupname asflicense javac spotbugs checkstyle compile | | uname | Linux b27e6719aca0 5.4.0-1025-aws #25~18.04.1-Ubuntu SMP Fri Sep 11 12:03:04 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / d050217a3d | | Default Java | AdoptOpenJDK-1.8.0_282-b08 | | Max. process+thread count | 57 (vs. ulimit of 3) | | modules | C: hbase-server . U: . | | Console output | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4181/9/console | | versions | git=2.17.1 maven=3.6.3 spotbugs=4.2.2 | | Powered by | Apache Yetus 0.12.0 https://yetus.apache.org | This message was automatically generated. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Updated] (HBASE-26772) Shell suspended in background
[
https://issues.apache.org/jira/browse/HBASE-26772?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Andrew Kyle Purtell updated HBASE-26772:
Fix Version/s: 2.6.0
> Shell suspended in background
> -
>
> Key: HBASE-26772
> URL: https://issues.apache.org/jira/browse/HBASE-26772
> Project: HBase
> Issue Type: Bug
> Components: shell
>Reporter: Peter Somogyi
>Assignee: Peter Somogyi
>Priority: Minor
> Fix For: 2.5.0, 2.6.0, 3.0.0-alpha-3, 2.4.11
>
>
> The hbase shell process hangs when running in the background with piped input.
> {noformat}
> $ cat command.sh
> #!/bin/bash
> echo "list" | bin/hbase shell > shell.out 2>&1
> $ ./command.sh &
> [1] 62907
> $
> [1] + 62907 suspended (tty output) ./command.sh{noformat}
> This regression was introduced with JRuby 9.1 -> 9.2 upgrade and the process
> hangs on this line:
> [https://github.com/apache/hbase/blob/master/hbase-shell/src/main/ruby/jar-bootstrap.rb#L41]
>
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
[jira] [Updated] (HBASE-23303) Add security headers to REST server/info page
[
https://issues.apache.org/jira/browse/HBASE-23303?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Andrew Kyle Purtell updated HBASE-23303:
Fix Version/s: 2.5.0
2.6.0
3.0.0-alpha-3
2.4.11
(was: 3.0.0-alpha-1)
(was: 2.3.0)
(was: 2.2.3)
(was: 2.1.9)
> Add security headers to REST server/info page
> -
>
> Key: HBASE-23303
> URL: https://issues.apache.org/jira/browse/HBASE-23303
> Project: HBase
> Issue Type: Improvement
> Components: REST
>Affects Versions: 3.0.0-alpha-1, 2.0.6, 2.1.7, 2.2.2
>Reporter: Andor Molnar
>Assignee: Andor Molnar
>Priority: Major
> Fix For: 2.5.0, 2.6.0, 3.0.0-alpha-3, 2.4.11
>
>
> Vulnerability scanners suggest that the following extra headers should be
> added to both Info/Rest server endpoints which are exposed by {{hbase-rest}}
> project.
> * X-Frame-Options: SAMEORIGIN
> * X-Xss-Protection: 1; mode=block
> * X-Content-Type-Options: nosniff
> * Strict-Transport-Security: “max-age=63072000;includeSubDomains;preload”
> * Content-Security-Policy: default-src https: data: 'unsafe-inline'
> 'unsafe-eval'
> Info server already has "X-Frame-Options: DENY" which is more restrictive
> than "SAMEORIGIN", so it's probably fine. All of three headers are missing
> from REST responses.
> I'll put together a patch to resolve this.
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
[GitHub] [hbase] busbey commented on pull request #4220: HBASE-26834 Adapt ConnectionRule for both sync and async connections (branch-2)
busbey commented on pull request #4220: URL: https://github.com/apache/hbase/pull/4220#issuecomment-1068227412 > Oh. That mvn invocation is missing the -Dhadoop.profile=3.0 that is required for building under JDK11. that flag is supposed to get set over here: https://github.com/apache/hbase/blob/master/dev-support/hbase-personality.sh#L170-L175 so either our branch pattern matching is broken, something else is overriding the setting, or this execution flow somehow skips `personality_modules` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Resolved] (HBASE-26828) Increase the concurrency when running UTs in pre commit job
[ https://issues.apache.org/jira/browse/HBASE-26828?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Kyle Purtell resolved HBASE-26828. - Fix Version/s: 2.5.0 2.6.0 3.0.0-alpha-3 2.4.11 Resolution: Fixed > Increase the concurrency when running UTs in pre commit job > --- > > Key: HBASE-26828 > URL: https://issues.apache.org/jira/browse/HBASE-26828 > Project: HBase > Issue Type: Improvement > Components: jenkins, test >Reporter: Duo Zhang >Assignee: Duo Zhang >Priority: Major > Fix For: 2.5.0, 2.6.0, 3.0.0-alpha-3, 2.4.11 > > > The default forkCount is 0.25C. We used to run on a 16 cores machine so the > concurrency is 4, but now our build nodes are all 8 cores, so the concurrency > is 2. The build time is way more slower than before. > Since we only have 2 slots on the build node, I think it is OK to increase > the forkCount to 0.5C. But I'm not sure whether we should change the default > value in pom, or we just set it in the pre commit job. -- This message was sent by Atlassian Jira (v8.20.1#820001)
[jira] [Comment Edited] (HBASE-26828) Increase the concurrency when running UTs in pre commit job
[ https://issues.apache.org/jira/browse/HBASE-26828?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17507057#comment-17507057 ] Andrew Kyle Purtell edited comment on HBASE-26828 at 3/15/22, 4:59 PM: --- This has been committed to branch-2.4. In order for the change log to be correct for 2.4.11RC1 I have to resolve this JIRA right now. was (Author: apurtell): This has been committed to branch-2.4. In order for the change log to be correct I have to resolve this JIRA now. > Increase the concurrency when running UTs in pre commit job > --- > > Key: HBASE-26828 > URL: https://issues.apache.org/jira/browse/HBASE-26828 > Project: HBase > Issue Type: Improvement > Components: jenkins, test >Reporter: Duo Zhang >Assignee: Duo Zhang >Priority: Major > Fix For: 2.5.0, 2.6.0, 3.0.0-alpha-3, 2.4.11 > > > The default forkCount is 0.25C. We used to run on a 16 cores machine so the > concurrency is 4, but now our build nodes are all 8 cores, so the concurrency > is 2. The build time is way more slower than before. > Since we only have 2 slots on the build node, I think it is OK to increase > the forkCount to 0.5C. But I'm not sure whether we should change the default > value in pom, or we just set it in the pre commit job. -- This message was sent by Atlassian Jira (v8.20.1#820001)
[jira] [Commented] (HBASE-26828) Increase the concurrency when running UTs in pre commit job
[ https://issues.apache.org/jira/browse/HBASE-26828?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17507057#comment-17507057 ] Andrew Kyle Purtell commented on HBASE-26828: - This has been committed to branch-2.4. In order for the change log to be correct I have to resolve this JIRA now. > Increase the concurrency when running UTs in pre commit job > --- > > Key: HBASE-26828 > URL: https://issues.apache.org/jira/browse/HBASE-26828 > Project: HBase > Issue Type: Improvement > Components: jenkins, test >Reporter: Duo Zhang >Assignee: Duo Zhang >Priority: Major > > The default forkCount is 0.25C. We used to run on a 16 cores machine so the > concurrency is 4, but now our build nodes are all 8 cores, so the concurrency > is 2. The build time is way more slower than before. > Since we only have 2 slots on the build node, I think it is OK to increase > the forkCount to 0.5C. But I'm not sure whether we should change the default > value in pom, or we just set it in the pre commit job. -- This message was sent by Atlassian Jira (v8.20.1#820001)
[jira] [Updated] (HBASE-26833) Avoid waiting to clear buffer usage of ReplicationSourceShipper when aborting the RS
[
https://issues.apache.org/jira/browse/HBASE-26833?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Andrew Kyle Purtell updated HBASE-26833:
Fix Version/s: 2.4.11
(was: 2.4.12)
> Avoid waiting to clear buffer usage of ReplicationSourceShipper when aborting
> the RS
>
>
> Key: HBASE-26833
> URL: https://issues.apache.org/jira/browse/HBASE-26833
> Project: HBase
> Issue Type: Improvement
> Components: regionserver, Replication
>Affects Versions: 2.4.10
>Reporter: Xiaolin Ha
>Assignee: Xiaolin Ha
>Priority: Major
> Fix For: 2.5.0, 2.6.0, 3.0.0-alpha-3, 2.4.11
>
>
> HBASE-24813 introduced the clear of buffer used in replication source
> shipper, but there is sleep in the method, if the variable sleepForRetries
> has a large value, and there are many wal groups, the aborting of RS may last
> a long time, but we should only do some necessary things in the aborting
> progress.
> {code:java}
> void clearWALEntryBatch() {
> long timeout = System.currentTimeMillis() + this.shipEditsTimeout;
> while(this.isAlive() || this.entryReader.isAlive()){
> try {
> if (System.currentTimeMillis() >= timeout) {
> LOG.warn("Shipper clearWALEntryBatch method timed out whilst waiting
> reader/shipper "
> + "thread to stop. Not cleaning buffer usage. Shipper alive: {};
> Reader alive: {}",
> this.source.getPeerId(), this.isAlive(),
> this.entryReader.isAlive());
> return;
> } else {
> // Wait both shipper and reader threads to stop
> Thread.sleep(this.sleepForRetries);
> }
> } catch (InterruptedException e) {
> LOG.warn("{} Interrupted while waiting {} to stop on
> clearWALEntryBatch. "
> + "Not cleaning buffer usage: {}", this.source.getPeerId(),
> this.getName(), e);
> return;
> }
> }
> .. {code}
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
[jira] [Updated] (HBASE-26835) Rewrite TestLruAdaptiveBlockCache to make it more stable
[ https://issues.apache.org/jira/browse/HBASE-26835?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Kyle Purtell updated HBASE-26835: Fix Version/s: 2.4.11 (was: 2.4.12) > Rewrite TestLruAdaptiveBlockCache to make it more stable > > > Key: HBASE-26835 > URL: https://issues.apache.org/jira/browse/HBASE-26835 > Project: HBase > Issue Type: Improvement > Components: test >Reporter: Duo Zhang >Assignee: Duo Zhang >Priority: Major > Fix For: 2.5.0, 3.0.0-alpha-3, 2.4.11 > > > Similiar to HBASE-26830. -- This message was sent by Atlassian Jira (v8.20.1#820001)
[GitHub] [hbase] wchevreuil commented on a change in pull request #4223: HBASE-26838 Junit jar is not included in the hbase tar ball, causing …
wchevreuil commented on a change in pull request #4223: URL: https://github.com/apache/hbase/pull/4223#discussion_r827207342 ## File path: hbase-assembly/src/main/assembly/hadoop-three-compat.xml ## @@ -261,6 +262,14 @@ io.opentelemetry.javaagent:* + + + + lib/test Review comment: You mean add to hbase classpath by default? Junit used to be shipped until it was scoped for test only on all modules, at least since release 2.4 (In 2.2 it was still shipped). In general, HBase processes seem to work pretty fine without it on the CP. The only setback is for the test tooling, such as `IntegrationTestIngest`, which will now fail without junit lib in the classpath. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase-connectors] busbey commented on pull request #94: HBASE-26841 Replace log4j with reload4j for hbase-connector
busbey commented on pull request #94: URL: https://github.com/apache/hbase-connectors/pull/94#issuecomment-1068217000 the expectation is this would work with the upcoming hbase 2.4.11 release with reload4j right? we'll need to revisit what to do once we get a hbase 2.y with log4j 2 instead of reload4j, but I think that can wait until the time comes. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Commented] (HBASE-26552) Introduce retry to logroller to avoid abort
[
https://issues.apache.org/jira/browse/HBASE-26552?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17507051#comment-17507051
]
Andrew Kyle Purtell commented on HBASE-26552:
-
Thanks [~Xiaolin Ha], I will make a RC1.
> Introduce retry to logroller to avoid abort
> ---
>
> Key: HBASE-26552
> URL: https://issues.apache.org/jira/browse/HBASE-26552
> Project: HBase
> Issue Type: Improvement
> Components: wal
>Affects Versions: 3.0.0-alpha-2, 2.4.10
>Reporter: Xiaolin Ha
>Assignee: Xiaolin Ha
>Priority: Major
> Fix For: 2.5.0, 2.6.0, 3.0.0-alpha-3, 2.4.11
>
>
> When calling RollController#rollWal in AbstractWALRoller, the regionserver
> may abort when encounters exception,
> {code:java}
> ...
> } catch (FailedLogCloseException | ConnectException e) {
> abort("Failed log close in log roller", e);
> } catch (IOException ex) {
> // Abort if we get here. We probably won't recover an IOE. HBASE-1132
> abort("IOE in log roller",
> ex instanceof RemoteException ? ((RemoteException)
> ex).unwrapRemoteException() : ex);
> } catch (Exception ex) {
> LOG.error("Log rolling failed", ex);
> abort("Log rolling failed", ex);
> } {code}
> I think we should support retry of rollWal here to avoid recovering the
> service by killing regionserver. The restart of regionserver is costly and
> very not friendly to the availability.
> I find that when creating new writer for the WAL in
> FanOutOneBlockAsyncDFSOutputHelper#createOutput, it supports retry to
> addBlock by setting this config "hbase.fs.async.create.retries". The idea of
> retry to roll WAL is similar to it, they both try best to make roll WAL
> succeed.
> But the initialization of new WAL writer also includes flushing the write
> buffer flush and waiting until it is completed by
> AsyncProtobufLogWriter#writeMagicAndWALHeader, which can also fail by some
> hardware reasons. The regionserver connected to the datanodes after addBlock,
> but that not means the magic and header can be flushed successfully.
> {code:java}
> protected long writeMagicAndWALHeader(byte[] magic, WALHeader header) throws
> IOException {
> return write(future -> {
> output.write(magic);
> try {
> header.writeDelimitedTo(asyncOutputWrapper);
> } catch (IOException e) {
> // should not happen
> throw new AssertionError(e);
> }
> addListener(output.flush(false), (len, error) -> {
> if (error != null) {
> future.completeExceptionally(error);
> } else {
> future.complete(len);
> }
> });
> });
> }{code}
> We have found that in our production clusters, there exists aborting of
> regionservers that caused by "IOE in log roller". And the practice in our
> clusters is that just one more retry of rollWal can make the WAL roll
> complete and continue serving.
>
>
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
[GitHub] [hbase] Apache-HBase commented on pull request #4226: HBASE-26837 Set SFT config when creating TableDescriptor in TestClone…
Apache-HBase commented on pull request #4226: URL: https://github.com/apache/hbase/pull/4226#issuecomment-1068132113 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 1m 23s | Docker mode activated. | | -0 :warning: | yetus | 0m 3s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck --skip-errorprone --exclude-tests-url --build-thread --surefire-first-part-fork-count --surefire-second-part-fork-count | ||| _ Prechecks _ | ||| _ master Compile Tests _ | | +1 :green_heart: | mvninstall | 6m 4s | master passed | | +1 :green_heart: | compile | 1m 2s | master passed | | +1 :green_heart: | javadoc | 0m 39s | master passed | ||| _ Patch Compile Tests _ | | +1 :green_heart: | mvninstall | 1m 5s | the patch passed | | +1 :green_heart: | compile | 0m 59s | the patch passed | | +1 :green_heart: | javac | 0m 59s | the patch passed | | +1 :green_heart: | javadoc | 0m 35s | the patch passed | ||| _ Other Tests _ | | -1 :x: | unit | 189m 41s | hbase-server in the patch failed. | | | | 202m 56s | | | Subsystem | Report/Notes | |--:|:-| | Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4226/1/artifact/yetus-jdk8-hadoop3-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/4226 | | Optional Tests | javac javadoc unit compile | | uname | Linux 73c237e38dc3 5.4.0-90-generic #101-Ubuntu SMP Fri Oct 15 20:00:55 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / a816204793 | | Default Java | AdoptOpenJDK-1.8.0_282-b08 | | unit | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4226/1/artifact/yetus-jdk8-hadoop3-check/output/patch-unit-hbase-server.txt | | Test Results | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4226/1/testReport/ | | Max. process+thread count | 1840 (vs. ulimit of 3) | | modules | C: hbase-server U: hbase-server | | Console output | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4226/1/console | | versions | git=2.17.1 maven=3.6.3 | | Powered by | Apache Yetus 0.12.0 https://yetus.apache.org | This message was automatically generated. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Commented] (HBASE-25895) Implement a Cluster Metrics JSON endpoint
[
https://issues.apache.org/jira/browse/HBASE-25895?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17506984#comment-17506984
]
Hudson commented on HBASE-25895:
Results for branch branch-2.5
[build #64 on
builds.a.o|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-2.5/64/]:
(x) *{color:red}-1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-2.5/64/General_20Nightly_20Build_20Report/]
(/) {color:green}+1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-2.5/64/JDK8_20Nightly_20Build_20Report_20_28Hadoop2_29/]
(/) {color:green}+1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-2.5/64/JDK8_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(x) {color:red}-1 jdk11 hadoop3 checks{color}
-- For more information [see jdk11
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-2.5/64/JDK11_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(x) {color:red}-1 source release artifact{color}
-- See build output for details.
(x) {color:red}-1 client integration test{color}
-- Something went wrong with this stage, [check relevant console
output|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-2.5/64//console].
> Implement a Cluster Metrics JSON endpoint
> -
>
> Key: HBASE-25895
> URL: https://issues.apache.org/jira/browse/HBASE-25895
> Project: HBase
> Issue Type: Sub-task
>Reporter: Nick Dimiduk
>Assignee: Nick Dimiduk
>Priority: Major
> Fix For: 2.5.0, 2.6.0, 3.0.0-alpha-3
>
>
> In order to build a dynamic visualization functionality, we need access to
> the underlying dataset. I think we can do a lot by exposing the
> ClusterMetrics already available via the {{o.a.h.h.client.Admin}} interface.
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
[jira] [Commented] (HBASE-26846) Move AsyncProcess configs to HConstants
[
https://issues.apache.org/jira/browse/HBASE-26846?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17506982#comment-17506982
]
Bryan Beaudreault commented on HBASE-26846:
---
Thanks for your feedback Duo. I submitted a discussion thread to the dev list.
One option might be to annotate these classes themselves as
IA.LimitedPrivate(CONFIG) instead so that users can reference the configs at
their own discretion knowing they may change.
> Move AsyncProcess configs to HConstants
> ---
>
> Key: HBASE-26846
> URL: https://issues.apache.org/jira/browse/HBASE-26846
> Project: HBase
> Issue Type: Improvement
>Reporter: Bryan Beaudreault
>Assignee: Bryan Beaudreault
>Priority: Minor
>
> AsyncProcess has a few public static config constants, but the class itself
> is package-private so users cannot access them. Moving them to HConstants
> will make it easier for users to change these settings:
> {code:java}
> hbase.client.primaryCallTimeout.multiget
> hbase.client.start.log.errors.counter
> hbase.client.log.batcherrors.details
> hbase.client.log.detail.period.ms
> {code}
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
[jira] [Commented] (HBASE-26848) Refactor the surefire plugin defination in our poms
[ https://issues.apache.org/jira/browse/HBASE-26848?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17506971#comment-17506971 ] Duo Zhang commented on HBASE-26848: --- Maybe the problem is that we define surefire plugin in some of the sub modules where we have a separated systemPropertyVariables config section. Let me have a try. > Refactor the surefire plugin defination in our poms > --- > > Key: HBASE-26848 > URL: https://issues.apache.org/jira/browse/HBASE-26848 > Project: HBase > Issue Type: Improvement > Components: jenkins, test >Reporter: Duo Zhang >Assignee: Duo Zhang >Priority: Major > > Recently on hbase1-10, it is easy to eat all the space of /tmp. I've checked > several times, it is always the stderr*deferred file, which is the temp > output of our UTs. > I think it means we have some problem for some of our UTs which could > generate very big output, but if the tmp is full, it is not easy for us to > find out the root cause as we need to delete the file to recover first... > Looking at the code of surefire > https://github.com/apache/maven-surefire/blob/68bca29b23d27dba3e06cf5e73a3fdc1a22598db/maven-surefire-common/src/main/java/org/apache/maven/plugin/surefire/report/Utf8RecodingDeferredFileOutputStream.java#L69 > It just creates the file with jdk's Files.createTempFile, the location can be > controlled by java.io.tmpdir, so let's point it to the target directory under > the jenkins workspace to avoid wat all the spaces of /tmp first. -- This message was sent by Atlassian Jira (v8.20.1#820001)
[jira] [Updated] (HBASE-26848) Refactor the surefire plugin defination in our poms
[ https://issues.apache.org/jira/browse/HBASE-26848?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Duo Zhang updated HBASE-26848: -- Summary: Refactor the surefire plugin defination in our poms (was: Set java.io.tmpdir when executing maven surefire plugin) > Refactor the surefire plugin defination in our poms > --- > > Key: HBASE-26848 > URL: https://issues.apache.org/jira/browse/HBASE-26848 > Project: HBase > Issue Type: Improvement > Components: jenkins, test >Reporter: Duo Zhang >Assignee: Duo Zhang >Priority: Major > > Recently on hbase1-10, it is easy to eat all the space of /tmp. I've checked > several times, it is always the stderr*deferred file, which is the temp > output of our UTs. > I think it means we have some problem for some of our UTs which could > generate very big output, but if the tmp is full, it is not easy for us to > find out the root cause as we need to delete the file to recover first... > Looking at the code of surefire > https://github.com/apache/maven-surefire/blob/68bca29b23d27dba3e06cf5e73a3fdc1a22598db/maven-surefire-common/src/main/java/org/apache/maven/plugin/surefire/report/Utf8RecodingDeferredFileOutputStream.java#L69 > It just creates the file with jdk's Files.createTempFile, the location can be > controlled by java.io.tmpdir, so let's point it to the target directory under > the jenkins workspace to avoid wat all the spaces of /tmp first. -- This message was sent by Atlassian Jira (v8.20.1#820001)
[GitHub] [hbase] Apache-HBase commented on pull request #4225: HBASE-26842 fix TestSnapshotProcedure for branch-2
Apache-HBase commented on pull request #4225: URL: https://github.com/apache/hbase/pull/4225#issuecomment-1068102823 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 1m 9s | Docker mode activated. | | -0 :warning: | yetus | 0m 6s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck --skip-errorprone --exclude-tests-url --build-thread --surefire-first-part-fork-count --surefire-second-part-fork-count | ||| _ Prechecks _ | ||| _ branch-2 Compile Tests _ | | +0 :ok: | mvndep | 0m 16s | Maven dependency ordering for branch | | +1 :green_heart: | mvninstall | 5m 3s | branch-2 passed | | +1 :green_heart: | compile | 1m 24s | branch-2 passed | | +1 :green_heart: | javadoc | 0m 54s | branch-2 passed | ||| _ Patch Compile Tests _ | | +0 :ok: | mvndep | 0m 15s | Maven dependency ordering for patch | | +1 :green_heart: | mvninstall | 1m 28s | the patch passed | | +1 :green_heart: | compile | 1m 19s | the patch passed | | +1 :green_heart: | javac | 1m 19s | the patch passed | | +1 :green_heart: | javadoc | 0m 54s | the patch passed | ||| _ Other Tests _ | | -1 :x: | unit | 2m 26s | hbase-client in the patch failed. | | -1 :x: | unit | 161m 33s | hbase-server in the patch failed. | | | | 178m 10s | | | Subsystem | Report/Notes | |--:|:-| | Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4225/1/artifact/yetus-jdk8-hadoop2-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/4225 | | Optional Tests | javac javadoc unit compile | | uname | Linux f36185558175 5.4.0-1025-aws #25~18.04.1-Ubuntu SMP Fri Sep 11 12:03:04 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | branch-2 / e6879b8fd7 | | Default Java | AdoptOpenJDK-1.8.0_282-b08 | | unit | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4225/1/artifact/yetus-jdk8-hadoop2-check/output/patch-unit-hbase-client.txt | | unit | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4225/1/artifact/yetus-jdk8-hadoop2-check/output/patch-unit-hbase-server.txt | | Test Results | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4225/1/testReport/ | | Max. process+thread count | 1884 (vs. ulimit of 12500) | | modules | C: hbase-client hbase-server U: . | | Console output | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4225/1/console | | versions | git=2.17.1 maven=3.6.3 | | Powered by | Apache Yetus 0.12.0 https://yetus.apache.org | This message was automatically generated. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Commented] (HBASE-26848) Set java.io.tmpdir when executing maven surefire plugin
[ https://issues.apache.org/jira/browse/HBASE-26848?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17506966#comment-17506966 ] Duo Zhang commented on HBASE-26848: --- OK, checked the pom.xml, we should have already set java.io.tmpdir to project.build.directory. Need to find out why it does not work sometimes. > Set java.io.tmpdir when executing maven surefire plugin > --- > > Key: HBASE-26848 > URL: https://issues.apache.org/jira/browse/HBASE-26848 > Project: HBase > Issue Type: Improvement > Components: jenkins, test >Reporter: Duo Zhang >Assignee: Duo Zhang >Priority: Major > > Recently on hbase1-10, it is easy to eat all the space of /tmp. I've checked > several times, it is always the stderr*deferred file, which is the temp > output of our UTs. > I think it means we have some problem for some of our UTs which could > generate very big output, but if the tmp is full, it is not easy for us to > find out the root cause as we need to delete the file to recover first... > Looking at the code of surefire > https://github.com/apache/maven-surefire/blob/68bca29b23d27dba3e06cf5e73a3fdc1a22598db/maven-surefire-common/src/main/java/org/apache/maven/plugin/surefire/report/Utf8RecodingDeferredFileOutputStream.java#L69 > It just creates the file with jdk's Files.createTempFile, the location can be > controlled by java.io.tmpdir, so let's point it to the target directory under > the jenkins workspace to avoid wat all the spaces of /tmp first. -- This message was sent by Atlassian Jira (v8.20.1#820001)
[jira] [Updated] (HBASE-26848) Set java.io.tmpdir when executing maven surefire plugin
[ https://issues.apache.org/jira/browse/HBASE-26848?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Duo Zhang updated HBASE-26848: -- Description: Recently on hbase1-10, it is easy to eat all the space of /tmp. I've checked several times, it is always the stderr*deferred file, which is the temp output of our UTs. I think it means we have some problem for some of our UTs which could generate very big output, but if the tmp is full, it is not easy for us to find out the root cause as we need to delete the file to recover first... Looking at the code of surefire https://github.com/apache/maven-surefire/blob/68bca29b23d27dba3e06cf5e73a3fdc1a22598db/maven-surefire-common/src/main/java/org/apache/maven/plugin/surefire/report/Utf8RecodingDeferredFileOutputStream.java#L69 It just creates the file with jdk's Files.createTempFile, the location can be controlled by java.io.tmpdir, so let's point it to the target directory under the jenkins workspace to avoid wat all the spaces of /tmp first. was: Recently on hbase1-10, it is easy to eat all the space of /tmp. I've checked several times, it is always the stderr*deferred file, which is the temp output of our UTs. I think it means we have some problem for some of our UTs which could generate very big output, but if the tmp is full, it is not easy for us to find out the root cause as we need to delete the file to recover first... Looking at the code of surefire https://github.com/apache/maven-surefire/blob/68bca29b23d27dba3e06cf5e73a3fdc1a22598db/maven-surefire-common/src/main/java/org/apache/maven/plugin/surefire/report/Utf8RecodingDeferredFileOutputStream.java#L69 It just creates the file with jdk's Files.createTempFile, the location can be controlled by java.io.tmpDir, so let's point it to the target directory under the jenkins workspace to avoid wat all the spaces of /tmp first. > Set java.io.tmpdir when executing maven surefire plugin > --- > > Key: HBASE-26848 > URL: https://issues.apache.org/jira/browse/HBASE-26848 > Project: HBase > Issue Type: Improvement > Components: jenkins, test >Reporter: Duo Zhang >Assignee: Duo Zhang >Priority: Major > > Recently on hbase1-10, it is easy to eat all the space of /tmp. I've checked > several times, it is always the stderr*deferred file, which is the temp > output of our UTs. > I think it means we have some problem for some of our UTs which could > generate very big output, but if the tmp is full, it is not easy for us to > find out the root cause as we need to delete the file to recover first... > Looking at the code of surefire > https://github.com/apache/maven-surefire/blob/68bca29b23d27dba3e06cf5e73a3fdc1a22598db/maven-surefire-common/src/main/java/org/apache/maven/plugin/surefire/report/Utf8RecodingDeferredFileOutputStream.java#L69 > It just creates the file with jdk's Files.createTempFile, the location can be > controlled by java.io.tmpdir, so let's point it to the target directory under > the jenkins workspace to avoid wat all the spaces of /tmp first. -- This message was sent by Atlassian Jira (v8.20.1#820001)
[jira] [Updated] (HBASE-26848) Set java.io.tmpdir when executing maven surefire plugin
[ https://issues.apache.org/jira/browse/HBASE-26848?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Duo Zhang updated HBASE-26848: -- Summary: Set java.io.tmpdir when executing maven surefire plugin (was: Set java.io.tmpDir when executing maven surefire plugin) > Set java.io.tmpdir when executing maven surefire plugin > --- > > Key: HBASE-26848 > URL: https://issues.apache.org/jira/browse/HBASE-26848 > Project: HBase > Issue Type: Improvement > Components: jenkins, test >Reporter: Duo Zhang >Assignee: Duo Zhang >Priority: Major > > Recently on hbase1-10, it is easy to eat all the space of /tmp. I've checked > several times, it is always the stderr*deferred file, which is the temp > output of our UTs. > I think it means we have some problem for some of our UTs which could > generate very big output, but if the tmp is full, it is not easy for us to > find out the root cause as we need to delete the file to recover first... > Looking at the code of surefire > https://github.com/apache/maven-surefire/blob/68bca29b23d27dba3e06cf5e73a3fdc1a22598db/maven-surefire-common/src/main/java/org/apache/maven/plugin/surefire/report/Utf8RecodingDeferredFileOutputStream.java#L69 > It just creates the file with jdk's Files.createTempFile, the location can be > controlled by java.io.tmpDir, so let's point it to the target directory under > the jenkins workspace to avoid wat all the spaces of /tmp first. -- This message was sent by Atlassian Jira (v8.20.1#820001)
[jira] [Assigned] (HBASE-26848) Set java.io.tmpDir when executing maven surefire plugin
[ https://issues.apache.org/jira/browse/HBASE-26848?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Duo Zhang reassigned HBASE-26848: - Assignee: Duo Zhang > Set java.io.tmpDir when executing maven surefire plugin > --- > > Key: HBASE-26848 > URL: https://issues.apache.org/jira/browse/HBASE-26848 > Project: HBase > Issue Type: Improvement > Components: jenkins, test >Reporter: Duo Zhang >Assignee: Duo Zhang >Priority: Major > > Recently on hbase1-10, it is easy to eat all the space of /tmp. I've checked > several times, it is always the stderr*deferred file, which is the temp > output of our UTs. > I think it means we have some problem for some of our UTs which could > generate very big output, but if the tmp is full, it is not easy for us to > find out the root cause as we need to delete the file to recover first... > Looking at the code of surefire > https://github.com/apache/maven-surefire/blob/68bca29b23d27dba3e06cf5e73a3fdc1a22598db/maven-surefire-common/src/main/java/org/apache/maven/plugin/surefire/report/Utf8RecodingDeferredFileOutputStream.java#L69 > It just creates the file with jdk's Files.createTempFile, the location can be > controlled by java.io.tmpDir, so let's point it to the target directory under > the jenkins workspace to avoid wat all the spaces of /tmp first. -- This message was sent by Atlassian Jira (v8.20.1#820001)
[jira] [Created] (HBASE-26848) Set java.io.tmpDir when executing maven surefire plugin
Duo Zhang created HBASE-26848: - Summary: Set java.io.tmpDir when executing maven surefire plugin Key: HBASE-26848 URL: https://issues.apache.org/jira/browse/HBASE-26848 Project: HBase Issue Type: Improvement Components: jenkins, test Reporter: Duo Zhang Recently on hbase1-10, it is easy to eat all the space of /tmp. I've checked several times, it is always the stderr*deferred file, which is the temp output of our UTs. I think it means we have some problem for some of our UTs which could generate very big output, but if the tmp is full, it is not easy for us to find out the root cause as we need to delete the file to recover first... Looking at the code of surefire https://github.com/apache/maven-surefire/blob/68bca29b23d27dba3e06cf5e73a3fdc1a22598db/maven-surefire-common/src/main/java/org/apache/maven/plugin/surefire/report/Utf8RecodingDeferredFileOutputStream.java#L69 It just creates the file with jdk's Files.createTempFile, the location can be controlled by java.io.tmpDir, so let's point it to the target directory under the jenkins workspace to avoid wat all the spaces of /tmp first. -- This message was sent by Atlassian Jira (v8.20.1#820001)
[GitHub] [hbase] Apache9 commented on a change in pull request #4106: HBASE-26545 Implement tracing of scan
Apache9 commented on a change in pull request #4106:
URL: https://github.com/apache/hbase/pull/4106#discussion_r827056634
##
File path:
hbase-server/src/test/java/org/apache/hadoop/hbase/client/AbstractTestAsyncTableScan.java
##
@@ -17,29 +17,95 @@
*/
package org.apache.hadoop.hbase.client;
+import static org.hamcrest.MatcherAssert.assertThat;
+import static org.hamcrest.Matchers.allOf;
+import static org.hamcrest.Matchers.endsWith;
+import static org.hamcrest.Matchers.hasItem;
+import static org.hamcrest.Matchers.hasProperty;
+import static org.hamcrest.Matchers.isA;
import static org.junit.Assert.assertEquals;
-import static org.junit.Assert.assertTrue;
-
+import static org.junit.Assert.assertThrows;
+import static org.junit.Assert.fail;
+import io.opentelemetry.sdk.trace.data.SpanData;
import java.io.IOException;
import java.io.UncheckedIOException;
import java.util.Arrays;
import java.util.List;
+import java.util.concurrent.ExecutionException;
import java.util.concurrent.ForkJoinPool;
+import java.util.concurrent.TimeUnit;
import java.util.function.Supplier;
import java.util.stream.Collectors;
import java.util.stream.IntStream;
+import org.apache.hadoop.conf.Configuration;
+import org.apache.hadoop.hbase.ConnectionRule;
import org.apache.hadoop.hbase.HBaseTestingUtil;
+import org.apache.hadoop.hbase.MatcherPredicate;
+import org.apache.hadoop.hbase.MiniClusterRule;
+import org.apache.hadoop.hbase.StartTestingClusterOption;
import org.apache.hadoop.hbase.TableName;
+import org.apache.hadoop.hbase.Waiter;
import org.apache.hadoop.hbase.regionserver.NoSuchColumnFamilyException;
+import org.apache.hadoop.hbase.trace.OpenTelemetryClassRule;
+import org.apache.hadoop.hbase.trace.OpenTelemetryTestRule;
+import org.apache.hadoop.hbase.trace.TraceUtil;
import org.apache.hadoop.hbase.util.Bytes;
+import org.apache.hadoop.hbase.util.JVMClusterUtil;
import org.apache.hadoop.hbase.util.Pair;
-import org.junit.AfterClass;
-import org.junit.BeforeClass;
+import org.hamcrest.Matcher;
+import org.junit.ClassRule;
+import org.junit.Rule;
import org.junit.Test;
+import org.junit.rules.ExternalResource;
+import org.junit.rules.RuleChain;
+import org.junit.rules.TestName;
+import org.junit.rules.TestRule;
public abstract class AbstractTestAsyncTableScan {
- protected static final HBaseTestingUtil TEST_UTIL = new HBaseTestingUtil();
+ protected static final OpenTelemetryClassRule otelClassRule =
OpenTelemetryClassRule.create();
+ protected static final MiniClusterRule miniClusterRule =
MiniClusterRule.newBuilder()
+.setMiniClusterOption(StartTestingClusterOption.builder()
+ .numWorkers(3)
+ .build())
+.build();
+
+ protected static final ConnectionRule connectionRule =
+new ConnectionRule(miniClusterRule::createConnection);
+
+ private static final class Setup extends ExternalResource {
Review comment:
Can upgrading to junit5 solve this problem?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #4226: HBASE-26837 Set SFT config when creating TableDescriptor in TestClone…
Apache-HBase commented on pull request #4226: URL: https://github.com/apache/hbase/pull/4226#issuecomment-1068057530 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 1m 43s | Docker mode activated. | | -0 :warning: | yetus | 0m 3s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck --skip-errorprone --exclude-tests-url --build-thread --surefire-first-part-fork-count --surefire-second-part-fork-count | ||| _ Prechecks _ | ||| _ master Compile Tests _ | | +1 :green_heart: | mvninstall | 4m 43s | master passed | | +1 :green_heart: | compile | 0m 41s | master passed | | +1 :green_heart: | javadoc | 0m 25s | master passed | ||| _ Patch Compile Tests _ | | +1 :green_heart: | mvninstall | 0m 44s | the patch passed | | +1 :green_heart: | compile | 0m 42s | the patch passed | | +1 :green_heart: | javac | 0m 42s | the patch passed | | +1 :green_heart: | javadoc | 0m 24s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | unit | 128m 44s | hbase-server in the patch passed. | | | | 139m 29s | | | Subsystem | Report/Notes | |--:|:-| | Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4226/1/artifact/yetus-jdk11-hadoop3-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/4226 | | Optional Tests | javac javadoc unit compile | | uname | Linux 96096196688a 5.4.0-1025-aws #25~18.04.1-Ubuntu SMP Fri Sep 11 12:03:04 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / a816204793 | | Default Java | AdoptOpenJDK-11.0.10+9 | | Test Results | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4226/1/testReport/ | | Max. process+thread count | 2437 (vs. ulimit of 3) | | modules | C: hbase-server U: hbase-server | | Console output | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4226/1/console | | versions | git=2.17.1 maven=3.6.3 | | Powered by | Apache Yetus 0.12.0 https://yetus.apache.org | This message was automatically generated. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Commented] (HBASE-26552) Introduce retry to logroller to avoid abort
[
https://issues.apache.org/jira/browse/HBASE-26552?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17506935#comment-17506935
]
Xiaolin Ha commented on HBASE-26552:
Hi, [~apurtell] , the NPE fix patch has been merged, related issue is
HBASE-26840. Thanks.
> Introduce retry to logroller to avoid abort
> ---
>
> Key: HBASE-26552
> URL: https://issues.apache.org/jira/browse/HBASE-26552
> Project: HBase
> Issue Type: Improvement
> Components: wal
>Affects Versions: 3.0.0-alpha-2, 2.4.10
>Reporter: Xiaolin Ha
>Assignee: Xiaolin Ha
>Priority: Major
> Fix For: 2.5.0, 2.6.0, 3.0.0-alpha-3, 2.4.11
>
>
> When calling RollController#rollWal in AbstractWALRoller, the regionserver
> may abort when encounters exception,
> {code:java}
> ...
> } catch (FailedLogCloseException | ConnectException e) {
> abort("Failed log close in log roller", e);
> } catch (IOException ex) {
> // Abort if we get here. We probably won't recover an IOE. HBASE-1132
> abort("IOE in log roller",
> ex instanceof RemoteException ? ((RemoteException)
> ex).unwrapRemoteException() : ex);
> } catch (Exception ex) {
> LOG.error("Log rolling failed", ex);
> abort("Log rolling failed", ex);
> } {code}
> I think we should support retry of rollWal here to avoid recovering the
> service by killing regionserver. The restart of regionserver is costly and
> very not friendly to the availability.
> I find that when creating new writer for the WAL in
> FanOutOneBlockAsyncDFSOutputHelper#createOutput, it supports retry to
> addBlock by setting this config "hbase.fs.async.create.retries". The idea of
> retry to roll WAL is similar to it, they both try best to make roll WAL
> succeed.
> But the initialization of new WAL writer also includes flushing the write
> buffer flush and waiting until it is completed by
> AsyncProtobufLogWriter#writeMagicAndWALHeader, which can also fail by some
> hardware reasons. The regionserver connected to the datanodes after addBlock,
> but that not means the magic and header can be flushed successfully.
> {code:java}
> protected long writeMagicAndWALHeader(byte[] magic, WALHeader header) throws
> IOException {
> return write(future -> {
> output.write(magic);
> try {
> header.writeDelimitedTo(asyncOutputWrapper);
> } catch (IOException e) {
> // should not happen
> throw new AssertionError(e);
> }
> addListener(output.flush(false), (len, error) -> {
> if (error != null) {
> future.completeExceptionally(error);
> } else {
> future.complete(len);
> }
> });
> });
> }{code}
> We have found that in our production clusters, there exists aborting of
> regionservers that caused by "IOE in log roller". And the practice in our
> clusters is that just one more retry of rollWal can make the WAL roll
> complete and continue serving.
>
>
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
[jira] [Updated] (HBASE-26552) Introduce retry to logroller to avoid abort
[
https://issues.apache.org/jira/browse/HBASE-26552?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Xiaolin Ha updated HBASE-26552:
---
Fix Version/s: 2.4.11
> Introduce retry to logroller to avoid abort
> ---
>
> Key: HBASE-26552
> URL: https://issues.apache.org/jira/browse/HBASE-26552
> Project: HBase
> Issue Type: Improvement
> Components: wal
>Affects Versions: 3.0.0-alpha-2, 2.4.10
>Reporter: Xiaolin Ha
>Assignee: Xiaolin Ha
>Priority: Major
> Fix For: 2.5.0, 2.6.0, 3.0.0-alpha-3, 2.4.11
>
>
> When calling RollController#rollWal in AbstractWALRoller, the regionserver
> may abort when encounters exception,
> {code:java}
> ...
> } catch (FailedLogCloseException | ConnectException e) {
> abort("Failed log close in log roller", e);
> } catch (IOException ex) {
> // Abort if we get here. We probably won't recover an IOE. HBASE-1132
> abort("IOE in log roller",
> ex instanceof RemoteException ? ((RemoteException)
> ex).unwrapRemoteException() : ex);
> } catch (Exception ex) {
> LOG.error("Log rolling failed", ex);
> abort("Log rolling failed", ex);
> } {code}
> I think we should support retry of rollWal here to avoid recovering the
> service by killing regionserver. The restart of regionserver is costly and
> very not friendly to the availability.
> I find that when creating new writer for the WAL in
> FanOutOneBlockAsyncDFSOutputHelper#createOutput, it supports retry to
> addBlock by setting this config "hbase.fs.async.create.retries". The idea of
> retry to roll WAL is similar to it, they both try best to make roll WAL
> succeed.
> But the initialization of new WAL writer also includes flushing the write
> buffer flush and waiting until it is completed by
> AsyncProtobufLogWriter#writeMagicAndWALHeader, which can also fail by some
> hardware reasons. The regionserver connected to the datanodes after addBlock,
> but that not means the magic and header can be flushed successfully.
> {code:java}
> protected long writeMagicAndWALHeader(byte[] magic, WALHeader header) throws
> IOException {
> return write(future -> {
> output.write(magic);
> try {
> header.writeDelimitedTo(asyncOutputWrapper);
> } catch (IOException e) {
> // should not happen
> throw new AssertionError(e);
> }
> addListener(output.flush(false), (len, error) -> {
> if (error != null) {
> future.completeExceptionally(error);
> } else {
> future.complete(len);
> }
> });
> });
> }{code}
> We have found that in our production clusters, there exists aborting of
> regionservers that caused by "IOE in log roller". And the practice in our
> clusters is that just one more retry of rollWal can make the WAL roll
> complete and continue serving.
>
>
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
[jira] [Resolved] (HBASE-26840) Fix NPE in the retry of logroller
[
https://issues.apache.org/jira/browse/HBASE-26840?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Xiaolin Ha resolved HBASE-26840.
Resolution: Fixed
Merged to branch-2.4+, thanks [~zhangduo] and [~vjasani] for reviewing.
> Fix NPE in the retry of logroller
> -
>
> Key: HBASE-26840
> URL: https://issues.apache.org/jira/browse/HBASE-26840
> Project: HBase
> Issue Type: Bug
> Components: wal
>Affects Versions: 2.4.10
>Reporter: Xiaolin Ha
>Assignee: Xiaolin Ha
>Priority: Minor
> Fix For: 2.5.0, 2.6.0, 3.0.0-alpha-3, 2.4.11
>
>
> {code:java}
> LOG.warn("Retry to roll log, nAttempts={}, waiting time={}ms, sleeping 1s to
> retry,"
> + " last excepiton= {}", nAttempts, waitingTime,
> ioe.getCause().getClass().getSimpleName()); {code}
> ioe.getCause() maybe null.
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
[jira] [Updated] (HBASE-26833) Avoid waiting to clear buffer usage of ReplicationSourceShipper when aborting the RS
[
https://issues.apache.org/jira/browse/HBASE-26833?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Xiaolin Ha updated HBASE-26833:
---
Resolution: Fixed
Status: Resolved (was: Patch Available)
Merged to branch-2.4+, thanks [~zhangduo] for reviewing.
> Avoid waiting to clear buffer usage of ReplicationSourceShipper when aborting
> the RS
>
>
> Key: HBASE-26833
> URL: https://issues.apache.org/jira/browse/HBASE-26833
> Project: HBase
> Issue Type: Improvement
> Components: regionserver, Replication
>Affects Versions: 2.4.10
>Reporter: Xiaolin Ha
>Assignee: Xiaolin Ha
>Priority: Major
> Fix For: 2.5.0, 2.6.0, 3.0.0-alpha-3, 2.4.12
>
>
> HBASE-24813 introduced the clear of buffer used in replication source
> shipper, but there is sleep in the method, if the variable sleepForRetries
> has a large value, and there are many wal groups, the aborting of RS may last
> a long time, but we should only do some necessary things in the aborting
> progress.
> {code:java}
> void clearWALEntryBatch() {
> long timeout = System.currentTimeMillis() + this.shipEditsTimeout;
> while(this.isAlive() || this.entryReader.isAlive()){
> try {
> if (System.currentTimeMillis() >= timeout) {
> LOG.warn("Shipper clearWALEntryBatch method timed out whilst waiting
> reader/shipper "
> + "thread to stop. Not cleaning buffer usage. Shipper alive: {};
> Reader alive: {}",
> this.source.getPeerId(), this.isAlive(),
> this.entryReader.isAlive());
> return;
> } else {
> // Wait both shipper and reader threads to stop
> Thread.sleep(this.sleepForRetries);
> }
> } catch (InterruptedException e) {
> LOG.warn("{} Interrupted while waiting {} to stop on
> clearWALEntryBatch. "
> + "Not cleaning buffer usage: {}", this.source.getPeerId(),
> this.getName(), e);
> return;
> }
> }
> .. {code}
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
[GitHub] [hbase] sunhelly merged pull request #4218: HBASE-26833 Avoid waiting to clear buffer usage of ReplicationSourceS…
sunhelly merged pull request #4218: URL: https://github.com/apache/hbase/pull/4218 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] sunhelly merged pull request #4224: HBASE-26840 Fix NPE in the retry of logroller
sunhelly merged pull request #4224: URL: https://github.com/apache/hbase/pull/4224 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Commented] (HBASE-26831) TestAdminShell2 is failing because of we referenced a deleted method
[
https://issues.apache.org/jira/browse/HBASE-26831?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17506919#comment-17506919
]
Hudson commented on HBASE-26831:
Results for branch master
[build #536 on
builds.a.o|https://ci-hbase.apache.org/job/HBase%20Nightly/job/master/536/]:
(x) *{color:red}-1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/master/536/General_20Nightly_20Build_20Report/]
(x) {color:red}-1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/master/536/JDK8_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(x) {color:red}-1 jdk11 hadoop3 checks{color}
-- For more information [see jdk11
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/master/536/JDK11_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
(/) {color:green}+1 client integration test{color}
> TestAdminShell2 is failing because of we referenced a deleted method
>
>
> Key: HBASE-26831
> URL: https://issues.apache.org/jira/browse/HBASE-26831
> Project: HBase
> Issue Type: Bug
> Components: shell
>Reporter: Duo Zhang
>Assignee: Duo Zhang
>Priority: Major
> Fix For: 3.0.0-alpha-3
>
>
> {noformat}
> src/test/ruby/hbase/admin2_test.rb:70:in `block in test_Get_detailed_status'
> /home/zhangduo/hbase/code/hbase-shell/src/test/ruby/test_helper.rb:161:in
> `capture_stdout'
> 67: end
> 68:
> 69: define_test 'Get detailed status' do
> ^[[48;5;16;38;5;226;1m => 70: output = capture_stdout {
> admin.status('detailed', '') }^[[0m
> 71: puts "Status output:\n#{output}"
> 72: # Some text which isn't in the simple output
> 73: assert output.include? 'regionsInTransition'
> src/test/ruby/hbase/admin2_test.rb:70:in `block in test_Get_detailed_status'
> uri:classloader:/hbase/admin.rb:926:in `status'
> Error: ^[[48;5;16;38;5;226;1mtest_Get_detailed_status(Hbase::StatusTest)^[[0m:
> NoMethodError: undefined method `getMaster' for
> #
> Did you mean? get_master_name
> {noformat}
> It is introduced by HBASE-26730.
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
[jira] [Commented] (HBASE-26531) Trace coprocessor exec endpoints
[
https://issues.apache.org/jira/browse/HBASE-26531?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17506917#comment-17506917
]
Hudson commented on HBASE-26531:
Results for branch master
[build #536 on
builds.a.o|https://ci-hbase.apache.org/job/HBase%20Nightly/job/master/536/]:
(x) *{color:red}-1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/master/536/General_20Nightly_20Build_20Report/]
(x) {color:red}-1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/master/536/JDK8_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(x) {color:red}-1 jdk11 hadoop3 checks{color}
-- For more information [see jdk11
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/master/536/JDK11_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
(/) {color:green}+1 client integration test{color}
> Trace coprocessor exec endpoints
>
>
> Key: HBASE-26531
> URL: https://issues.apache.org/jira/browse/HBASE-26531
> Project: HBase
> Issue Type: Sub-task
>Reporter: Nick Dimiduk
>Assignee: Nick Dimiduk
>Priority: Major
> Fix For: 3.0.0-alpha-3
>
>
> Decide how to trace Coprocessor Exec endpoints. They either look like table
> operations OR generic RPC calls... I'm guessing the latter, but will give it
> some study.
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
[jira] [Commented] (HBASE-25895) Implement a Cluster Metrics JSON endpoint
[
https://issues.apache.org/jira/browse/HBASE-25895?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17506918#comment-17506918
]
Hudson commented on HBASE-25895:
Results for branch master
[build #536 on
builds.a.o|https://ci-hbase.apache.org/job/HBase%20Nightly/job/master/536/]:
(x) *{color:red}-1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/master/536/General_20Nightly_20Build_20Report/]
(x) {color:red}-1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/master/536/JDK8_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(x) {color:red}-1 jdk11 hadoop3 checks{color}
-- For more information [see jdk11
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/master/536/JDK11_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
(/) {color:green}+1 client integration test{color}
> Implement a Cluster Metrics JSON endpoint
> -
>
> Key: HBASE-25895
> URL: https://issues.apache.org/jira/browse/HBASE-25895
> Project: HBase
> Issue Type: Sub-task
>Reporter: Nick Dimiduk
>Assignee: Nick Dimiduk
>Priority: Major
> Fix For: 2.5.0, 2.6.0, 3.0.0-alpha-3
>
>
> In order to build a dynamic visualization functionality, we need access to
> the underlying dataset. I think we can do a lot by exposing the
> ClusterMetrics already available via the {{o.a.h.h.client.Admin}} interface.
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
[GitHub] [hbase] Apache-HBase commented on pull request #4224: HBASE-26840 Fix NPE in the retry of logroller
Apache-HBase commented on pull request #4224: URL: https://github.com/apache/hbase/pull/4224#issuecomment-1067988567 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 1m 6s | Docker mode activated. | | -0 :warning: | yetus | 0m 3s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck --skip-errorprone --exclude-tests-url --build-thread --surefire-first-part-fork-count --surefire-second-part-fork-count | ||| _ Prechecks _ | ||| _ master Compile Tests _ | | +1 :green_heart: | mvninstall | 6m 59s | master passed | | +1 :green_heart: | compile | 1m 19s | master passed | | +1 :green_heart: | javadoc | 0m 41s | master passed | ||| _ Patch Compile Tests _ | | +1 :green_heart: | mvninstall | 1m 17s | the patch passed | | +1 :green_heart: | compile | 1m 12s | the patch passed | | +1 :green_heart: | javac | 1m 12s | the patch passed | | +1 :green_heart: | javadoc | 0m 39s | the patch passed | ||| _ Other Tests _ | | -1 :x: | unit | 182m 48s | hbase-server in the patch failed. | | | | 197m 16s | | | Subsystem | Report/Notes | |--:|:-| | Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4224/2/artifact/yetus-jdk11-hadoop3-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/4224 | | Optional Tests | javac javadoc unit compile | | uname | Linux 424b376e4fbd 5.4.0-90-generic #101-Ubuntu SMP Fri Oct 15 20:00:55 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / cc2c22072f | | Default Java | AdoptOpenJDK-11.0.10+9 | | unit | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4224/2/artifact/yetus-jdk11-hadoop3-check/output/patch-unit-hbase-server.txt | | Test Results | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4224/2/testReport/ | | Max. process+thread count | 1823 (vs. ulimit of 3) | | modules | C: hbase-server U: hbase-server | | Console output | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4224/2/console | | versions | git=2.17.1 maven=3.6.3 | | Powered by | Apache Yetus 0.12.0 https://yetus.apache.org | This message was automatically generated. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] Apache9 commented on a change in pull request #4221: Forward-port "HBASE-26834 Adapt ConnectionRule for both sync and async connections" to master
Apache9 commented on a change in pull request #4221:
URL: https://github.com/apache/hbase/pull/4221#discussion_r826948250
##
File path:
hbase-endpoint/src/test/java/org/apache/hadoop/hbase/coprocessor/TestCoprocessorEndpointTracing.java
##
@@ -228,7 +228,7 @@ public void onError(Throwable error) {
@Test
public void traceSyncTableEndpointCall() throws Exception {
-final Connection connection =
connectionRule.getConnection().toConnection();
+final Connection connection = connectionRule.getSyncConnection();
Review comment:
Better just call it getConnection? In ConnectionFactory we have
createConnection and createAsyncConnection.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[jira] [Commented] (HBASE-26846) Move AsyncProcess configs to HConstants
[
https://issues.apache.org/jira/browse/HBASE-26846?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17506897#comment-17506897
]
Duo Zhang commented on HBASE-26846:
---
I mean configs and configs, we do not expose configs through public API. Maybe
there are some config names can be accessed through some fields in our
IA.Public classes, but this is not a general rule.
Typically, all the public configs should be in the hbase-default.xml, but to be
honest, I'm not sure whether all the committers in the community have followed
this rule...
I think we could start a discussion thread on the dev list to see what others
in the community think.
Thanks.
> Move AsyncProcess configs to HConstants
> ---
>
> Key: HBASE-26846
> URL: https://issues.apache.org/jira/browse/HBASE-26846
> Project: HBase
> Issue Type: Improvement
>Reporter: Bryan Beaudreault
>Assignee: Bryan Beaudreault
>Priority: Minor
>
> AsyncProcess has a few public static config constants, but the class itself
> is package-private so users cannot access them. Moving them to HConstants
> will make it easier for users to change these settings:
> {code:java}
> hbase.client.primaryCallTimeout.multiget
> hbase.client.start.log.errors.counter
> hbase.client.log.batcherrors.details
> hbase.client.log.detail.period.ms
> {code}
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
[jira] [Commented] (HBASE-26847) Move ConnectionImplementation public configs to HConstants
[
https://issues.apache.org/jira/browse/HBASE-26847?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17506895#comment-17506895
]
Bryan Beaudreault commented on HBASE-26847:
---
As a user it's an uphill battle trying to keep track of the hundreds or
thousands of hbase configs that one might need or want to tweak. To me it makes
little sense to expose a config (whose intent is to make something tunable) and
then burry it in a private class. I'm happy to put them elsewhere other than
HConstants, but is there some sort of compromise we can come to to make the
constants accessible?
> Move ConnectionImplementation public configs to HConstants
> --
>
> Key: HBASE-26847
> URL: https://issues.apache.org/jira/browse/HBASE-26847
> Project: HBase
> Issue Type: Improvement
>Reporter: Bryan Beaudreault
>Assignee: Bryan Beaudreault
>Priority: Minor
>
> ConnectionImplementation has a few public static config constants, but the
> class itself is package-private so users cannot access them. Moving them to
> HConstants will make it easier for users to change these settings:
> {code:java}
> hbase.client.retries.by.server
> hbase.resolve.hostnames.on.failure {code}
> Note: AsyncConnectionImpl also separately defines the second config, so we
> can unify them both in HConstants.
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
[jira] [Comment Edited] (HBASE-26847) Move ConnectionImplementation public configs to HConstants
[
https://issues.apache.org/jira/browse/HBASE-26847?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17506895#comment-17506895
]
Bryan Beaudreault edited comment on HBASE-26847 at 3/15/22, 12:58 PM:
--
As a user it's an uphill battle trying to keep track of the hundreds of hbase
configs that one might need or want to tweak. To me it makes little sense to
expose a config (whose intent is to make something tunable) and then burry it
in a private class. I'm happy to put them elsewhere other than HConstants, but
is there some sort of compromise we can come to to make the constants
accessible?
was (Author: bbeaudreault):
As a user it's an uphill battle trying to keep track of the hundreds or
thousands of hbase configs that one might need or want to tweak. To me it makes
little sense to expose a config (whose intent is to make something tunable) and
then burry it in a private class. I'm happy to put them elsewhere other than
HConstants, but is there some sort of compromise we can come to to make the
constants accessible?
> Move ConnectionImplementation public configs to HConstants
> --
>
> Key: HBASE-26847
> URL: https://issues.apache.org/jira/browse/HBASE-26847
> Project: HBase
> Issue Type: Improvement
>Reporter: Bryan Beaudreault
>Assignee: Bryan Beaudreault
>Priority: Minor
>
> ConnectionImplementation has a few public static config constants, but the
> class itself is package-private so users cannot access them. Moving them to
> HConstants will make it easier for users to change these settings:
> {code:java}
> hbase.client.retries.by.server
> hbase.resolve.hostnames.on.failure {code}
> Note: AsyncConnectionImpl also separately defines the second config, so we
> can unify them both in HConstants.
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
[jira] [Resolved] (HBASE-26611) Changing SFT implementation on disabled table is dangerous
[ https://issues.apache.org/jira/browse/HBASE-26611?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Duo Zhang resolved HBASE-26611. --- Fix Version/s: 2.6.0 3.0.0-alpha-3 Hadoop Flags: Reviewed Resolution: Fixed Pushed to master and branch-2. Thanks [~Xiaolin Ha] for reviewing! > Changing SFT implementation on disabled table is dangerous > -- > > Key: HBASE-26611 > URL: https://issues.apache.org/jira/browse/HBASE-26611 > Project: HBase > Issue Type: Sub-task > Components: HFile, proc-v2 >Reporter: Duo Zhang >Assignee: Duo Zhang >Priority: Critical > Fix For: 2.6.0, 3.0.0-alpha-3 > > > As our current design is to have a MIGRATION SFT so when region opens, we > will mirror the store files to the dst SFT. > But for a disabled table, the region will not be reopened so we have no > chance to mirror the store files list, so after user enable the table, it may > find out that all the store files are gone... -- This message was sent by Atlassian Jira (v8.20.1#820001)
