[GitHub] [hbase] Apache-HBase commented on pull request #2741: HBASE-25364 Redo the getMidPoint() in HFileWriterImpl to get rid of t…
Apache-HBase commented on pull request #2741: URL: https://github.com/apache/hbase/pull/2741#issuecomment-739466574 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 1m 6s | Docker mode activated. | | -0 :warning: | yetus | 0m 2s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck | ||| _ Prechecks _ | ||| _ master Compile Tests _ | | +1 :green_heart: | mvninstall | 4m 12s | master passed | | +1 :green_heart: | compile | 0m 59s | master passed | | +1 :green_heart: | shadedjars | 7m 10s | branch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 0m 39s | master passed | ||| _ Patch Compile Tests _ | | +1 :green_heart: | mvninstall | 3m 50s | the patch passed | | +1 :green_heart: | compile | 1m 0s | the patch passed | | +1 :green_heart: | javac | 1m 0s | the patch passed | | +1 :green_heart: | shadedjars | 7m 9s | patch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 0m 36s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | unit | 204m 1s | hbase-server in the patch passed. | | | | 232m 29s | | | Subsystem | Report/Notes | |--:|:-| | Docker | ClientAPI=1.40 ServerAPI=1.40 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2741/1/artifact/yetus-jdk8-hadoop3-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/2741 | | Optional Tests | javac javadoc unit shadedjars compile | | uname | Linux 0e740dfd9e37 4.15.0-112-generic #113-Ubuntu SMP Thu Jul 9 23:41:39 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / 7d0a687e57 | | Default Java | AdoptOpenJDK-1.8.0_232-b09 | | Test Results | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2741/1/testReport/ | | Max. process+thread count | 3241 (vs. ulimit of 3) | | modules | C: hbase-server U: hbase-server | | Console output | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2741/1/console | | versions | git=2.17.1 maven=3.6.3 | | Powered by | Apache Yetus 0.12.0 https://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #2675: HBASE-25277 postScannerFilterRow impacts Scan performance a lot in HBase 2.x
Apache-HBase commented on pull request #2675: URL: https://github.com/apache/hbase/pull/2675#issuecomment-739462570 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 1m 47s | Docker mode activated. | ||| _ Prechecks _ | | +1 :green_heart: | dupname | 0m 0s | No case conflicting files found. | | +1 :green_heart: | hbaseanti | 0m 0s | Patch does not have any anti-patterns. | | +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. | ||| _ master Compile Tests _ | | +1 :green_heart: | mvninstall | 4m 58s | master passed | | +1 :green_heart: | checkstyle | 1m 30s | master passed | | +1 :green_heart: | spotbugs | 2m 45s | master passed | ||| _ Patch Compile Tests _ | | +1 :green_heart: | mvninstall | 4m 30s | the patch passed | | +1 :green_heart: | checkstyle | 1m 27s | hbase-server: The patch generated 0 new + 99 unchanged - 4 fixed = 99 total (was 103) | | +1 :green_heart: | whitespace | 0m 0s | The patch has no whitespace issues. | | +1 :green_heart: | hadoopcheck | 21m 23s | Patch does not cause any errors with Hadoop 3.1.2 3.2.1 3.3.0. | | +1 :green_heart: | spotbugs | 2m 10s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | asflicense | 0m 13s | The patch does not generate ASF License warnings. | | | | 48m 6s | | | Subsystem | Report/Notes | |--:|:-| | Docker | ClientAPI=1.40 ServerAPI=1.40 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2675/8/artifact/yetus-general-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/2675 | | Optional Tests | dupname asflicense spotbugs hadoopcheck hbaseanti checkstyle | | uname | Linux 2b06fd417c19 4.15.0-65-generic #74-Ubuntu SMP Tue Sep 17 17:06:04 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / 7d0a687e57 | | Max. process+thread count | 94 (vs. ulimit of 3) | | modules | C: hbase-server U: hbase-server | | Console output | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2675/8/console | | versions | git=2.17.1 maven=3.6.3 spotbugs=3.1.12 | | Powered by | Apache Yetus 0.12.0 https://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #2741: HBASE-25364 Redo the getMidPoint() in HFileWriterImpl to get rid of t…
Apache-HBase commented on pull request #2741: URL: https://github.com/apache/hbase/pull/2741#issuecomment-739461777 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 1m 6s | Docker mode activated. | | -0 :warning: | yetus | 0m 4s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck | ||| _ Prechecks _ | ||| _ master Compile Tests _ | | +1 :green_heart: | mvninstall | 4m 17s | master passed | | +1 :green_heart: | compile | 1m 5s | master passed | | +1 :green_heart: | shadedjars | 6m 42s | branch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 0m 42s | master passed | ||| _ Patch Compile Tests _ | | +1 :green_heart: | mvninstall | 4m 1s | the patch passed | | +1 :green_heart: | compile | 1m 4s | the patch passed | | +1 :green_heart: | javac | 1m 4s | the patch passed | | +1 :green_heart: | shadedjars | 6m 35s | patch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 0m 38s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | unit | 137m 39s | hbase-server in the patch passed. | | | | 165m 53s | | | Subsystem | Report/Notes | |--:|:-| | Docker | ClientAPI=1.40 ServerAPI=1.40 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2741/1/artifact/yetus-jdk11-hadoop3-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/2741 | | Optional Tests | javac javadoc unit shadedjars compile | | uname | Linux 4e341520fd43 4.15.0-65-generic #74-Ubuntu SMP Tue Sep 17 17:06:04 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / 7d0a687e57 | | Default Java | AdoptOpenJDK-11.0.6+10 | | Test Results | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2741/1/testReport/ | | Max. process+thread count | 4231 (vs. ulimit of 3) | | modules | C: hbase-server U: hbase-server | | Console output | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2741/1/console | | versions | git=2.17.1 maven=3.6.3 | | Powered by | Apache Yetus 0.12.0 https://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #2741: HBASE-25364 Redo the getMidPoint() in HFileWriterImpl to get rid of t…
Apache-HBase commented on pull request #2741: URL: https://github.com/apache/hbase/pull/2741#issuecomment-739452013 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 1m 8s | Docker mode activated. | ||| _ Prechecks _ | | +1 :green_heart: | dupname | 0m 0s | No case conflicting files found. | | +1 :green_heart: | hbaseanti | 0m 0s | Patch does not have any anti-patterns. | | +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. | ||| _ master Compile Tests _ | | +1 :green_heart: | mvninstall | 4m 9s | master passed | | +1 :green_heart: | checkstyle | 1m 12s | master passed | | +1 :green_heart: | spotbugs | 2m 11s | master passed | ||| _ Patch Compile Tests _ | | +1 :green_heart: | mvninstall | 3m 47s | the patch passed | | -0 :warning: | checkstyle | 1m 9s | hbase-server: The patch generated 2 new + 5 unchanged - 0 fixed = 7 total (was 5) | | +1 :green_heart: | whitespace | 0m 0s | The patch has no whitespace issues. | | +1 :green_heart: | hadoopcheck | 19m 7s | Patch does not cause any errors with Hadoop 3.1.2 3.2.1 3.3.0. | | +1 :green_heart: | spotbugs | 2m 18s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | asflicense | 0m 13s | The patch does not generate ASF License warnings. | | | | 43m 5s | | | Subsystem | Report/Notes | |--:|:-| | Docker | ClientAPI=1.40 ServerAPI=1.40 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2741/1/artifact/yetus-general-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/2741 | | Optional Tests | dupname asflicense spotbugs hadoopcheck hbaseanti checkstyle | | uname | Linux c64fb3ef40e9 4.15.0-112-generic #113-Ubuntu SMP Thu Jul 9 23:41:39 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / 7d0a687e57 | | checkstyle | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2741/1/artifact/yetus-general-check/output/diff-checkstyle-hbase-server.txt | | Max. process+thread count | 84 (vs. ulimit of 3) | | modules | C: hbase-server U: hbase-server | | Console output | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2741/1/console | | versions | git=2.17.1 maven=3.6.3 spotbugs=3.1.12 | | Powered by | Apache Yetus 0.12.0 https://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] nyl3532016 commented on pull request #2738: HBASE-25363 Improve performance of HFileLinkCleaner by using ReadWrit…
nyl3532016 commented on pull request #2738: URL: https://github.com/apache/hbase/pull/2738#issuecomment-739450510 LGTM This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Commented] (HBASE-23811) [OpenTracing] Add shaded JaegerTracing tracer to hbase-thirdparty
[ https://issues.apache.org/jira/browse/HBASE-23811?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17244636#comment-17244636 ] Duo Zhang commented on HBASE-23811: --- I wonder whether we should do this. The tracing system needs a backend to collect the tracing data, which is not part of HBase, so if we tie HBase together with Jaeger, what if the users want to use Apache SkyWalking? So I expect that, we just ship with the OpenTracing or OpenTelemetry's general API. Users should puts the binding jars into the install directory of HBase binaries and we will load them when starting. If no bindings found, we just use the noop tracer. Thoughts? Thanks. > [OpenTracing] Add shaded JaegerTracing tracer to hbase-thirdparty > - > > Key: HBASE-23811 > URL: https://issues.apache.org/jira/browse/HBASE-23811 > Project: HBase > Issue Type: Sub-task > Components: thirdparty >Reporter: Wei-Chiu Chuang >Assignee: Wei-Chiu Chuang >Priority: Major > > JaegerTracing pulls in lots of dependencies. Some, like libthrift (0.13.0) > conflicts the one ships in HBase (0.12.0). > Additionally, not everyone may want to use Jaeger. > I propose to shade JaegerTracing and its dependencies into an uber jar, place > it as a hbase-thirdparty artifact. As an added benefit, this makes the > management of tracers in the HBase's dependency tree much easier. Finally, we > can follow the same suit and provide Zipkin tracer support in the future. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[GitHub] [hbase] GeorryHuang opened a new pull request #2741: HBASE-25364 Redo the getMidPoint() in HFileWriterImpl to get rid of t…
GeorryHuang opened a new pull request #2741: URL: https://github.com/apache/hbase/pull/2741 …he double comparison process There is a TODO like this "TODO: Redo so only a single pass over the arrays rather than one to compare and then a second composing midpoint." in getMidpoint() of class HFileWriteImpl The old logic compares the left byte array and the right byte array twice： 1. A comparison is performed before composing MinimumMidpointArray 2. During composing of MinimumMidpointArray, bytes were comparing again My optimization combines them into one This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Commented] (HBASE-22120) Replace HTrace with OpenTracing
[ https://issues.apache.org/jira/browse/HBASE-22120?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17244634#comment-17244634 ] Duo Zhang commented on HBASE-22120: --- My concern is that whether the API in OpenTelemetry is stable enough at this point. https://github.com/open-telemetry/opentelemetry-java/releases/tag/v0.12.0 This is the newest release of OpenTelemetry, where I can see there are still plenty of breaking changes related to the tracing API. I know this is the future but I just wonder whether it is the right time to use OpenTelemetry instead of OpenTracing in production. As OpenTelemetry guarantees that it will be compatible with OpenTracing, but it does not promise that its own API will be stable enough before the 1.0.0 release... Could someone who is familiar with OpenTelemetry to better explain the current state and the plan of the OpenTelemetry project? Thanks. > Replace HTrace with OpenTracing > --- > > Key: HBASE-22120 > URL: https://issues.apache.org/jira/browse/HBASE-22120 > Project: HBase > Issue Type: New Feature > Components: tracing >Affects Versions: 3.0.0-alpha-1 >Reporter: Sergey Shelukhin >Assignee: Wei-Chiu Chuang >Priority: Major > > h2. Deprecate HTrace usage in HBase > * HBase 1.x (branch-1) > * Declare HTrace (htrace 3.x) deprecated in the user doc. > * HBase 2.x (branch-2) > * Declare HTrace deprecated in the user doc. Furthermore, state that it is > known not working. > * Either fix the trace context propagation bug in HBase 2.x, or backport > OpenTracing support from the master branch. I am inclined to the latter. > * HBase 3.x (master branch) > * Remove HTrace entirely. > * Add OpenTracing APIs. Potentially backport to HBase 2.4. > * Replace OpenTracing API with OpenTelemetry when the latter stabilizes. > h1. Milestones > # Doc -- deprecation notice > # Replace existing HTrace code with OpenTracing code in the master branch > (3.x) > # Java (a [poc|https://github.com/jojochuang/hbase/tree/HBASE-22120] is > currently under way) > # HBase shell and scripts (Ruby, shell script) > # Doc > # Add new trace instrumentation code for new features not instrumented by > the existing HTace code. > # Propagate the traces to other systems such as HDFS and MapReduce. > # Support other OpenTracing tracers. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HBASE-25364) Redo the getMidPoint() in HFileWriterImpl to get rid of the double comparison process
[ https://issues.apache.org/jira/browse/HBASE-25364?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhuoyue Huang updated HBASE-25364: -- Description: There is a TODO like this "TODO: Redo so only a single pass over the arrays rather than one to compare and then a second composing midpoint." in getMidpoint() of class HFileWriteImpl The old logic compares the left byte array and the right byte array twice： # A comparison is performed before composing MinimumMidpointArray # During composing of MinimumMidpointArray, bytes were comparing again My optimization combines them into one was: There is a TODO like this "TODO: Redo so only a single pass over the arrays rather than one to compare and then a second composing midpoint." in getMidpoint() of class HFileWriteImpl The old logic compares the left byte array and the right byte array twice： # A comparison is performed before composing MinimumMidpointArray # During composing of MinimumMidpointArray, bytes were comparing again My optimization combines them into one > Redo the getMidPoint() in HFileWriterImpl to get rid of the double comparison > process > - > > Key: HBASE-25364 > URL: https://issues.apache.org/jira/browse/HBASE-25364 > Project: HBase > Issue Type: Improvement >Reporter: Zhuoyue Huang >Assignee: Zhuoyue Huang >Priority: Minor > > There is a TODO like this "TODO: Redo so only a single pass over the arrays > rather than one to compare and then a second composing midpoint." in > getMidpoint() of class HFileWriteImpl > > The old logic compares the left byte array and the right byte array twice： > # A comparison is performed before composing MinimumMidpointArray > # During composing of MinimumMidpointArray, bytes were comparing again > > My optimization combines them into one -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (HBASE-25364) Redo the getMidPoint() in HFileWriterImpl to get rid of the double comparison process
Zhuoyue Huang created HBASE-25364: - Summary: Redo the getMidPoint() in HFileWriterImpl to get rid of the double comparison process Key: HBASE-25364 URL: https://issues.apache.org/jira/browse/HBASE-25364 Project: HBase Issue Type: Improvement Reporter: Zhuoyue Huang Assignee: Zhuoyue Huang There is a TODO like this "TODO: Redo so only a single pass over the arrays rather than one to compare and then a second composing midpoint." in getMidpoint() of class HFileWriteImpl The old logic compares the left byte array and the right byte array twice： # A comparison is performed before composing MinimumMidpointArray # During composing of MinimumMidpointArray, bytes were comparing again My optimization combines them into one -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (HBASE-25361) [Flakey Tests] branch-2 TestMetaRegionLocationCache.testStandByMetaLocations
[
https://issues.apache.org/jira/browse/HBASE-25361?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17244622#comment-17244622
]
Hudson commented on HBASE-25361:
Results for branch branch-2
[build #122 on
builds.a.o|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/branch-2/122/]:
(x) *{color:red}-1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/branch-2/122/General_20Nightly_20Build_20Report/]
(/) {color:green}+1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/branch-2/122/JDK8_20Nightly_20Build_20Report_20_28Hadoop2_29/]
(/) {color:green}+1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/branch-2/122/JDK8_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(x) {color:red}-1 jdk11 hadoop3 checks{color}
-- For more information [see jdk11
report|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/branch-2/122/JDK11_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
(/) {color:green}+1 client integration test{color}
> [Flakey Tests] branch-2 TestMetaRegionLocationCache.testStandByMetaLocations
>
>
> Key: HBASE-25361
> URL: https://issues.apache.org/jira/browse/HBASE-25361
> Project: HBase
> Issue Type: Bug
> Components: flakies
>Reporter: Michael Stack
>Assignee: Michael Stack
>Priority: Major
> Fix For: 3.0.0-alpha-1, 2.3.4, 2.5.0, 2.4.1
>
>
> This just happened to me running tests locally. I see it in the current
> flakies list
> [https://ci-hadoop.apache.org/job/HBase/job/HBase-Flaky-Tests/job/branch-2/310/testReport/junit/org.apache.hadoop.hbase.client/TestMetaRegionLocationCache/testStandByMetaLocations/]
>
> It looks like a simple timing issue. Let me put up a simple patch to see if
> it helps.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HBASE-25361) [Flakey Tests] branch-2 TestMetaRegionLocationCache.testStandByMetaLocations
[
https://issues.apache.org/jira/browse/HBASE-25361?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17244616#comment-17244616
]
Hudson commented on HBASE-25361:
Results for branch branch-2.4
[build #2 on
builds.a.o|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/branch-2.4/2/]:
(/) *{color:green}+1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/branch-2.4/2/General_20Nightly_20Build_20Report/]
(/) {color:green}+1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/branch-2.4/2/JDK8_20Nightly_20Build_20Report_20_28Hadoop2_29/]
(/) {color:green}+1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/branch-2.4/2/JDK8_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(/) {color:green}+1 jdk11 hadoop3 checks{color}
-- For more information [see jdk11
report|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/branch-2.4/2/JDK11_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
(/) {color:green}+1 client integration test{color}
> [Flakey Tests] branch-2 TestMetaRegionLocationCache.testStandByMetaLocations
>
>
> Key: HBASE-25361
> URL: https://issues.apache.org/jira/browse/HBASE-25361
> Project: HBase
> Issue Type: Bug
> Components: flakies
>Reporter: Michael Stack
>Assignee: Michael Stack
>Priority: Major
> Fix For: 3.0.0-alpha-1, 2.3.4, 2.5.0, 2.4.1
>
>
> This just happened to me running tests locally. I see it in the current
> flakies list
> [https://ci-hadoop.apache.org/job/HBase/job/HBase-Flaky-Tests/job/branch-2/310/testReport/junit/org.apache.hadoop.hbase.client/TestMetaRegionLocationCache/testStandByMetaLocations/]
>
> It looks like a simple timing issue. Let me put up a simple patch to see if
> it helps.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[GitHub] [hbase] Apache-HBase commented on pull request #2740: Fix flaky test TestClassFinder#testClassFinderDefaultsToOwnPackage
Apache-HBase commented on pull request #2740: URL: https://github.com/apache/hbase/pull/2740#issuecomment-739436968 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 6m 47s | Docker mode activated. | ||| _ Prechecks _ | | +1 :green_heart: | dupname | 0m 0s | No case conflicting files found. | | +1 :green_heart: | hbaseanti | 0m 0s | Patch does not have any anti-patterns. | | +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. | ||| _ master Compile Tests _ | | +1 :green_heart: | mvninstall | 3m 46s | master passed | | +1 :green_heart: | checkstyle | 0m 25s | master passed | | +1 :green_heart: | spotbugs | 0m 44s | master passed | ||| _ Patch Compile Tests _ | | +1 :green_heart: | mvninstall | 3m 26s | the patch passed | | +1 :green_heart: | checkstyle | 0m 23s | the patch passed | | +1 :green_heart: | whitespace | 0m 0s | The patch has no whitespace issues. | | +1 :green_heart: | hadoopcheck | 17m 13s | Patch does not cause any errors with Hadoop 3.1.2 3.2.1 3.3.0. | | +1 :green_heart: | spotbugs | 0m 52s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | asflicense | 0m 14s | The patch does not generate ASF License warnings. | | | | 41m 11s | | | Subsystem | Report/Notes | |--:|:-| | Docker | ClientAPI=1.40 ServerAPI=1.40 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2740/1/artifact/yetus-general-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/2740 | | Optional Tests | dupname asflicense spotbugs hadoopcheck hbaseanti checkstyle | | uname | Linux 1a5cb303465b 4.15.0-112-generic #113-Ubuntu SMP Thu Jul 9 23:41:39 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / 7d0a687e57 | | Max. process+thread count | 94 (vs. ulimit of 3) | | modules | C: hbase-common U: hbase-common | | Console output | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2740/1/console | | versions | git=2.17.1 maven=3.6.3 spotbugs=3.1.12 | | Powered by | Apache Yetus 0.12.0 https://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #2740: Fix flaky test TestClassFinder#testClassFinderDefaultsToOwnPackage
Apache-HBase commented on pull request #2740: URL: https://github.com/apache/hbase/pull/2740#issuecomment-739435887 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 1m 11s | Docker mode activated. | | -0 :warning: | yetus | 0m 2s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck | ||| _ Prechecks _ | ||| _ master Compile Tests _ | | +1 :green_heart: | mvninstall | 4m 9s | master passed | | +1 :green_heart: | compile | 0m 21s | master passed | | +1 :green_heart: | shadedjars | 7m 9s | branch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 0m 21s | master passed | ||| _ Patch Compile Tests _ | | +1 :green_heart: | mvninstall | 3m 54s | the patch passed | | +1 :green_heart: | compile | 0m 22s | the patch passed | | +1 :green_heart: | javac | 0m 22s | the patch passed | | +1 :green_heart: | shadedjars | 7m 10s | patch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 0m 20s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | unit | 1m 42s | hbase-common in the patch passed. | | | | 27m 45s | | | Subsystem | Report/Notes | |--:|:-| | Docker | ClientAPI=1.40 ServerAPI=1.40 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2740/1/artifact/yetus-jdk8-hadoop3-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/2740 | | Optional Tests | javac javadoc unit shadedjars compile | | uname | Linux 284055a5f02b 4.15.0-112-generic #113-Ubuntu SMP Thu Jul 9 23:41:39 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / 7d0a687e57 | | Default Java | AdoptOpenJDK-1.8.0_232-b09 | | Test Results | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2740/1/testReport/ | | Max. process+thread count | 220 (vs. ulimit of 3) | | modules | C: hbase-common U: hbase-common | | Console output | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2740/1/console | | versions | git=2.17.1 maven=3.6.3 | | Powered by | Apache Yetus 0.12.0 https://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #2740: Fix flaky test TestClassFinder#testClassFinderDefaultsToOwnPackage
Apache-HBase commented on pull request #2740: URL: https://github.com/apache/hbase/pull/2740#issuecomment-739435801 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 0m 28s | Docker mode activated. | | -0 :warning: | yetus | 0m 3s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck | ||| _ Prechecks _ | ||| _ master Compile Tests _ | | +1 :green_heart: | mvninstall | 4m 19s | master passed | | +1 :green_heart: | compile | 0m 25s | master passed | | +1 :green_heart: | shadedjars | 6m 43s | branch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 0m 26s | master passed | ||| _ Patch Compile Tests _ | | +1 :green_heart: | mvninstall | 4m 3s | the patch passed | | +1 :green_heart: | compile | 0m 26s | the patch passed | | +1 :green_heart: | javac | 0m 26s | the patch passed | | +1 :green_heart: | shadedjars | 6m 39s | patch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 0m 24s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | unit | 1m 34s | hbase-common in the patch passed. | | | | 26m 43s | | | Subsystem | Report/Notes | |--:|:-| | Docker | ClientAPI=1.40 ServerAPI=1.40 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2740/1/artifact/yetus-jdk11-hadoop3-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/2740 | | Optional Tests | javac javadoc unit shadedjars compile | | uname | Linux d016b12dc92d 4.15.0-112-generic #113-Ubuntu SMP Thu Jul 9 23:41:39 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / 7d0a687e57 | | Default Java | AdoptOpenJDK-11.0.6+10 | | Test Results | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2740/1/testReport/ | | Max. process+thread count | 274 (vs. ulimit of 3) | | modules | C: hbase-common U: hbase-common | | Console output | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2740/1/console | | versions | git=2.17.1 maven=3.6.3 | | Powered by | Apache Yetus 0.12.0 https://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #2733: HBASE-25353 [Flakey Tests] branch-2 TestShutdownBackupMaster
Apache-HBase commented on pull request #2733: URL: https://github.com/apache/hbase/pull/2733#issuecomment-739434495 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 1m 9s | Docker mode activated. | | -0 :warning: | yetus | 0m 7s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck | ||| _ Prechecks _ | ||| _ branch-2 Compile Tests _ | | +0 :ok: | mvndep | 0m 13s | Maven dependency ordering for branch | | +1 :green_heart: | mvninstall | 3m 51s | branch-2 passed | | +1 :green_heart: | compile | 1m 35s | branch-2 passed | | +1 :green_heart: | shadedjars | 6m 41s | branch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 1m 8s | branch-2 passed | ||| _ Patch Compile Tests _ | | +0 :ok: | mvndep | 0m 16s | Maven dependency ordering for patch | | +1 :green_heart: | mvninstall | 3m 57s | the patch passed | | +1 :green_heart: | compile | 1m 38s | the patch passed | | +1 :green_heart: | javac | 1m 38s | the patch passed | | +1 :green_heart: | shadedjars | 6m 38s | patch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 1m 8s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | unit | 2m 27s | hbase-client in the patch passed. | | +1 :green_heart: | unit | 135m 35s | hbase-server in the patch passed. | | | | 168m 55s | | | Subsystem | Report/Notes | |--:|:-| | Docker | ClientAPI=1.40 ServerAPI=1.40 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2733/6/artifact/yetus-jdk11-hadoop3-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/2733 | | Optional Tests | javac javadoc unit shadedjars compile | | uname | Linux 85da9176df65 4.15.0-65-generic #74-Ubuntu SMP Tue Sep 17 17:06:04 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | branch-2 / a4d5b1bb88 | | Default Java | AdoptOpenJDK-11.0.6+10 | | Test Results | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2733/6/testReport/ | | Max. process+thread count | 3620 (vs. ulimit of 12500) | | modules | C: hbase-client hbase-server U: . | | Console output | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2733/6/console | | versions | git=2.17.1 maven=3.6.3 | | Powered by | Apache Yetus 0.12.0 https://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #2733: HBASE-25353 [Flakey Tests] branch-2 TestShutdownBackupMaster
Apache-HBase commented on pull request #2733: URL: https://github.com/apache/hbase/pull/2733#issuecomment-739434364 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 0m 32s | Docker mode activated. | | -0 :warning: | yetus | 0m 8s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck | ||| _ Prechecks _ | ||| _ branch-2 Compile Tests _ | | +0 :ok: | mvndep | 0m 26s | Maven dependency ordering for branch | | +1 :green_heart: | mvninstall | 3m 14s | branch-2 passed | | +1 :green_heart: | compile | 1m 25s | branch-2 passed | | +1 :green_heart: | shadedjars | 5m 57s | branch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 1m 1s | branch-2 passed | ||| _ Patch Compile Tests _ | | +0 :ok: | mvndep | 0m 17s | Maven dependency ordering for patch | | +1 :green_heart: | mvninstall | 3m 19s | the patch passed | | +1 :green_heart: | compile | 1m 24s | the patch passed | | +1 :green_heart: | javac | 1m 24s | the patch passed | | +1 :green_heart: | shadedjars | 5m 57s | patch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 1m 2s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | unit | 2m 15s | hbase-client in the patch passed. | | +1 :green_heart: | unit | 138m 5s | hbase-server in the patch passed. | | | | 167m 30s | | | Subsystem | Report/Notes | |--:|:-| | Docker | ClientAPI=1.40 ServerAPI=1.40 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2733/6/artifact/yetus-jdk8-hadoop2-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/2733 | | Optional Tests | javac javadoc unit shadedjars compile | | uname | Linux 09178486a5f6 4.15.0-60-generic #67-Ubuntu SMP Thu Aug 22 16:55:30 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | branch-2 / a4d5b1bb88 | | Default Java | AdoptOpenJDK-1.8.0_232-b09 | | Test Results | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2733/6/testReport/ | | Max. process+thread count | 4259 (vs. ulimit of 12500) | | modules | C: hbase-client hbase-server U: . | | Console output | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2733/6/console | | versions | git=2.17.1 maven=3.6.3 | | Powered by | Apache Yetus 0.12.0 https://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] hsiangawang opened a new pull request #2740: Fix flaky test TestClassFinder#testClassFinderDefaultsToOwnPackage
hsiangawang opened a new pull request #2740: URL: https://github.com/apache/hbase/pull/2740 The test `TestClassFinder#testClassFinderDefaultsToOwnPackage` compares two sets from `allClassesFinder.findClasses`. However, the function `allClassesFinder.findClasses` does not return the deterministic order result. From code trace, the root cause is at `ClassFinder.findClassesFromJar` and `ClassFinder.findClassesFromFiles`. They use `HashSet` to store `classes` and `listFiles` to get file array. However, `HashSet` and `listFiles` do not guarantee that order, and the assertion can fail if the order differs. The PR uses `LinkedHashSet` and sorts file array to make the order deterministic. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Updated] (HBASE-25242) Add Increment/Append support to RowMutations
[ https://issues.apache.org/jira/browse/HBASE-25242?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Busbey updated HBASE-25242: Release Note: After HBASE-25242, we can add Increment/Append operations to RowMutations and perform those operations atomically in a single row. This change alters a public API so that the `mutateRow` method in both the `Table` and `AsyncTable` classes now returns a `Result` object with the result of the passed operations. Previously these methods had no return values. Code that calls these methods which was compiled against an earlier version of the HBase client libraries will fail at runtime with a `NoSuchMethodError` when used with this release. If you are going to ignore the result of the passed operations you can simply recompile your application with an updated dependency and no additional changes. was: After HBASE-25242, we can add Increment/Append operations to RowMutations and perform those operations atomically in a single row. This change alters a public API so that the `mutateRow` method in both the `Table` and `AsyncTable` classes now returns a `Result` object with the result of the passed operations. Previously these methods had no return values. Code that calls these methods which was compiled against an earlier version of the HBase client libraries will fail at runtime with a `MethodNotFoundException` when used with this release. If you are going to ignore the result of the passed operations you can simply recompile your application with an updated dependency and no additional changes. > Add Increment/Append support to RowMutations > > > Key: HBASE-25242 > URL: https://issues.apache.org/jira/browse/HBASE-25242 > Project: HBase > Issue Type: New Feature > Components: Client, regionserver >Reporter: Toshihiro Suzuki >Assignee: Toshihiro Suzuki >Priority: Critical > Fix For: 3.0.0-alpha-1, 2.4.0 > > > Currently, RowMutations supports only Put and Delete. Supporting > Increment/Append in RowMutations would be helpful for some use cases. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HBASE-25361) [Flakey Tests] branch-2 TestMetaRegionLocationCache.testStandByMetaLocations
[ https://issues.apache.org/jira/browse/HBASE-25361?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Stack updated HBASE-25361: -- Fix Version/s: 2.5.0 > [Flakey Tests] branch-2 TestMetaRegionLocationCache.testStandByMetaLocations > > > Key: HBASE-25361 > URL: https://issues.apache.org/jira/browse/HBASE-25361 > Project: HBase > Issue Type: Bug > Components: flakies >Reporter: Michael Stack >Assignee: Michael Stack >Priority: Major > Fix For: 3.0.0-alpha-1, 2.3.4, 2.5.0, 2.4.1 > > > This just happened to me running tests locally. I see it in the current > flakies list > [https://ci-hadoop.apache.org/job/HBase/job/HBase-Flaky-Tests/job/branch-2/310/testReport/junit/org.apache.hadoop.hbase.client/TestMetaRegionLocationCache/testStandByMetaLocations/] > > It looks like a simple timing issue. Let me put up a simple patch to see if > it helps. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (HBASE-25353) [Flakey Tests] branch-2 TestShutdownBackupMaster
[
https://issues.apache.org/jira/browse/HBASE-25353?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17244592#comment-17244592
]
Michael Stack commented on HBASE-25353:
---
Oh, thanks for the helpful review [~huaxiangsun]
> [Flakey Tests] branch-2 TestShutdownBackupMaster
>
>
> Key: HBASE-25353
> URL: https://issues.apache.org/jira/browse/HBASE-25353
> Project: HBase
> Issue Type: Sub-task
> Components: flakies
>Affects Versions: 2.4.0
>Reporter: Michael Stack
>Assignee: Michael Stack
>Priority: Major
> Fix For: 2.3.4, 2.5.0, 2.4.1
>
>
> Making this as a sub-issue of parent issue which fails similar to how we are
> failing now.
> Currently, I see that that TestShutdownBackupMaster test passes usually but
> it is warped in how it completes. It will do all retries just before the test
> timesout at 13minutes max...: e.g. you'll see this...
> 2020-12-02 22:07:34,200 DEBUG [master/stack:0:becomeActiveMaster]
> client.ConnectionImplementation(1009): locateRegionInMeta
> parentTable='hbase:meta', attempt=44 of 46 failed; retrying after sleep of 46
> ... so we'll do all the retries and then complete so the test looks like it
> 'succeeded' but it actually ran for Total time: 12:41 min... and the log is
> full of thread dumps because the cluster won't go down (The time is spent in
> the test shutdown).
> Often though, we won't complete the retries in time and the test fails. It is
> in the flakey list.
> Rather, we are supposed to fail out fast when we are shutting down. Below is
> the type of retry we see.
>
> {code:java}
> 2020-12-02 10:53:35,540 INFO [Listener at localhost/61609]
> util.JVMClusterUtil(348): Shutdown of 2 master(s) and 2 regionserver(s)
> complete
> 2020-12-02 10:53:35,548 DEBUG [master/stack:0:becomeActiveMaster]
> client.ConnectionImplementation(1009): locateRegionInMeta
> parentTable='hbase:meta', attempt=2 of 46 failed; retrying after sleep of 46
> org.apache.hadoop.hbase.DoNotRetryIOException: hconnection-0x1afa7f5b closed
> at
> org.apache.hadoop.hbase.client.ConnectionImplementation.checkClosed(ConnectionImplementation.java:630)
> at
> org.apache.hadoop.hbase.client.ConnectionImplementation.locateRegion(ConnectionImplementation.java:815)
> at
> org.apache.hadoop.hbase.client.ConnectionUtils$ShortCircuitingClusterConnection.locateRegion(ConnectionUtils.java:138)
> at
> org.apache.hadoop.hbase.client.ConnectionImplementation.relocateRegion(ConnectionImplementation.java:803)
> at
> org.apache.hadoop.hbase.client.ConnectionUtils$ShortCircuitingClusterConnection.relocateRegion(ConnectionUtils.java:138)
> at
> org.apache.hadoop.hbase.client.ConnectionImplementation.locateRegionInMeta(ConnectionImplementation.java:933)
> at
> org.apache.hadoop.hbase.client.ConnectionImplementation.locateRegion(ConnectionImplementation.java:823)
> at
> org.apache.hadoop.hbase.client.ConnectionUtils$ShortCircuitingClusterConnection.locateRegion(ConnectionUtils.java:138)
> at
> org.apache.hadoop.hbase.client.HRegionLocator.getRegionLocation(HRegionLocator.java:64)
> at
> org.apache.hadoop.hbase.client.RegionLocator.getRegionLocation(RegionLocator.java:70)
> at
> org.apache.hadoop.hbase.client.RegionLocator.getRegionLocation(RegionLocator.java:59)
> at
> org.apache.hadoop.hbase.client.RegionServerCallable.prepare(RegionServerCallable.java:223)
> at
> org.apache.hadoop.hbase.client.RpcRetryingCallerImpl.callWithRetries(RpcRetryingCallerImpl.java:105)
> at org.apache.hadoop.hbase.client.HTable.get(HTable.java:383)
> at org.apache.hadoop.hbase.client.HTable.get(HTable.java:357)
> at
> org.apache.hadoop.hbase.master.TableNamespaceManager.get(TableNamespaceManager.java:141)
> at
> org.apache.hadoop.hbase.master.TableNamespaceManager.isTableAvailableAndInitialized(TableNamespaceManager.java:278)
> at
> org.apache.hadoop.hbase.master.TableNamespaceManager.start(TableNamespaceManager.java:103)
> at
> org.apache.hadoop.hbase.master.ClusterSchemaServiceImpl.doStart(ClusterSchemaServiceImpl.java:63)
> at
> org.apache.hbase.thirdparty.com.google.common.util.concurrent.AbstractService.startAsync(AbstractService.java:249)
> at
> org.apache.hadoop.hbase.master.HMaster.initClusterSchemaService(HMaster.java:1224)
> at
> org.apache.hadoop.hbase.master.TestShutdownBackupMaster$MockHMaster.initClusterSchemaService(TestShutdownBackupMaster.java:68)
> at
> org.apache.hadoop.hbase.master.HMaster.finishActiveMasterInitialization(HMaster.java:1021)
> at
> org.apache.hadoop.hbase.master.HMaster.startActiveMasterManager(HMaster.java:2082)
> at
> org.apache.hadoop.hbase.master.HMaster.lambda$run$0(HMaster.java:506){code}
> See how a master is trying to become active and it won't relent trying to
> become active master even though this cluster is
[jira] [Resolved] (HBASE-25353) [Flakey Tests] branch-2 TestShutdownBackupMaster
[
https://issues.apache.org/jira/browse/HBASE-25353?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Michael Stack resolved HBASE-25353.
---
Fix Version/s: 2.5.0
2.3.4
Hadoop Flags: Reviewed
Resolution: Fixed
Merged to branch-2.3+ (There is no ConnectionImplementation on master so
skipped it)
> [Flakey Tests] branch-2 TestShutdownBackupMaster
>
>
> Key: HBASE-25353
> URL: https://issues.apache.org/jira/browse/HBASE-25353
> Project: HBase
> Issue Type: Sub-task
> Components: flakies
>Affects Versions: 2.4.0
>Reporter: Michael Stack
>Assignee: Michael Stack
>Priority: Major
> Fix For: 2.3.4, 2.5.0, 2.4.1
>
>
> Making this as a sub-issue of parent issue which fails similar to how we are
> failing now.
> Currently, I see that that TestShutdownBackupMaster test passes usually but
> it is warped in how it completes. It will do all retries just before the test
> timesout at 13minutes max...: e.g. you'll see this...
> 2020-12-02 22:07:34,200 DEBUG [master/stack:0:becomeActiveMaster]
> client.ConnectionImplementation(1009): locateRegionInMeta
> parentTable='hbase:meta', attempt=44 of 46 failed; retrying after sleep of 46
> ... so we'll do all the retries and then complete so the test looks like it
> 'succeeded' but it actually ran for Total time: 12:41 min... and the log is
> full of thread dumps because the cluster won't go down (The time is spent in
> the test shutdown).
> Often though, we won't complete the retries in time and the test fails. It is
> in the flakey list.
> Rather, we are supposed to fail out fast when we are shutting down. Below is
> the type of retry we see.
>
> {code:java}
> 2020-12-02 10:53:35,540 INFO [Listener at localhost/61609]
> util.JVMClusterUtil(348): Shutdown of 2 master(s) and 2 regionserver(s)
> complete
> 2020-12-02 10:53:35,548 DEBUG [master/stack:0:becomeActiveMaster]
> client.ConnectionImplementation(1009): locateRegionInMeta
> parentTable='hbase:meta', attempt=2 of 46 failed; retrying after sleep of 46
> org.apache.hadoop.hbase.DoNotRetryIOException: hconnection-0x1afa7f5b closed
> at
> org.apache.hadoop.hbase.client.ConnectionImplementation.checkClosed(ConnectionImplementation.java:630)
> at
> org.apache.hadoop.hbase.client.ConnectionImplementation.locateRegion(ConnectionImplementation.java:815)
> at
> org.apache.hadoop.hbase.client.ConnectionUtils$ShortCircuitingClusterConnection.locateRegion(ConnectionUtils.java:138)
> at
> org.apache.hadoop.hbase.client.ConnectionImplementation.relocateRegion(ConnectionImplementation.java:803)
> at
> org.apache.hadoop.hbase.client.ConnectionUtils$ShortCircuitingClusterConnection.relocateRegion(ConnectionUtils.java:138)
> at
> org.apache.hadoop.hbase.client.ConnectionImplementation.locateRegionInMeta(ConnectionImplementation.java:933)
> at
> org.apache.hadoop.hbase.client.ConnectionImplementation.locateRegion(ConnectionImplementation.java:823)
> at
> org.apache.hadoop.hbase.client.ConnectionUtils$ShortCircuitingClusterConnection.locateRegion(ConnectionUtils.java:138)
> at
> org.apache.hadoop.hbase.client.HRegionLocator.getRegionLocation(HRegionLocator.java:64)
> at
> org.apache.hadoop.hbase.client.RegionLocator.getRegionLocation(RegionLocator.java:70)
> at
> org.apache.hadoop.hbase.client.RegionLocator.getRegionLocation(RegionLocator.java:59)
> at
> org.apache.hadoop.hbase.client.RegionServerCallable.prepare(RegionServerCallable.java:223)
> at
> org.apache.hadoop.hbase.client.RpcRetryingCallerImpl.callWithRetries(RpcRetryingCallerImpl.java:105)
> at org.apache.hadoop.hbase.client.HTable.get(HTable.java:383)
> at org.apache.hadoop.hbase.client.HTable.get(HTable.java:357)
> at
> org.apache.hadoop.hbase.master.TableNamespaceManager.get(TableNamespaceManager.java:141)
> at
> org.apache.hadoop.hbase.master.TableNamespaceManager.isTableAvailableAndInitialized(TableNamespaceManager.java:278)
> at
> org.apache.hadoop.hbase.master.TableNamespaceManager.start(TableNamespaceManager.java:103)
> at
> org.apache.hadoop.hbase.master.ClusterSchemaServiceImpl.doStart(ClusterSchemaServiceImpl.java:63)
> at
> org.apache.hbase.thirdparty.com.google.common.util.concurrent.AbstractService.startAsync(AbstractService.java:249)
> at
> org.apache.hadoop.hbase.master.HMaster.initClusterSchemaService(HMaster.java:1224)
> at
> org.apache.hadoop.hbase.master.TestShutdownBackupMaster$MockHMaster.initClusterSchemaService(TestShutdownBackupMaster.java:68)
> at
> org.apache.hadoop.hbase.master.HMaster.finishActiveMasterInitialization(HMaster.java:1021)
> at
> org.apache.hadoop.hbase.master.HMaster.startActiveMasterManager(HMaster.java:2082)
> at
> org.apache.hadoop.hbase.master.HMaster.lambda$run$0(HMaster.java:506){code}
> See how a master is trying
[GitHub] [hbase] saintstack commented on pull request #2733: HBASE-25353 [Flakey Tests] branch-2 TestShutdownBackupMaster
saintstack commented on pull request #2733: URL: https://github.com/apache/hbase/pull/2733#issuecomment-739424492 Closing. Merged by hand. Thanks for the helpful review @huaxiangsun This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Updated] (HBASE-25242) Add Increment/Append support to RowMutations
[ https://issues.apache.org/jira/browse/HBASE-25242?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Busbey updated HBASE-25242: Release Note: After HBASE-25242, we can add Increment/Append operations to RowMutations and perform those operations atomically in a single row. This change alters a public API so that the `mutateRow` method in both the `Table` and `AsyncTable` classes now returns a `Result` object with the result of the passed operations. Previously these methods had no return values. Code that calls these methods which was compiled against an earlier version of the HBase client libraries will fail at runtime with a `MethodNotFoundException` when used with this release. If you are going to ignore the result of the passed operations you can simply recompile your application with an updated dependency and no additional changes. was: After HBASE-25242, we can add Increment/Append operations to RowMutations and perform those operations atomically in a single row. This change alters a public API so that the the mutateRow() API returns a Result object to get the result of the Increment/Append operations. > Add Increment/Append support to RowMutations > > > Key: HBASE-25242 > URL: https://issues.apache.org/jira/browse/HBASE-25242 > Project: HBase > Issue Type: New Feature > Components: Client, regionserver >Reporter: Toshihiro Suzuki >Assignee: Toshihiro Suzuki >Priority: Critical > Fix For: 3.0.0-alpha-1, 2.4.0 > > > Currently, RowMutations supports only Put and Delete. Supporting > Increment/Append in RowMutations would be helpful for some use cases. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HBASE-25242) Add Increment/Append support to RowMutations
[ https://issues.apache.org/jira/browse/HBASE-25242?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Busbey updated HBASE-25242: Release Note: After HBASE-25242, we can add Increment/Append operations to RowMutations and perform those operations atomically in a single row. This change alters a public API so that the the mutateRow() API returns a Result object to get the result of the Increment/Append operations. was: After HBASE-25242, we can add Increment/Append operations to RowMutations and perform those operations atomically in a single row. HBASE-25242 includes an API change where the mutateRow() API returns a Result object to get the result of the Increment/Append operations. > Add Increment/Append support to RowMutations > > > Key: HBASE-25242 > URL: https://issues.apache.org/jira/browse/HBASE-25242 > Project: HBase > Issue Type: New Feature > Components: Client, regionserver >Reporter: Toshihiro Suzuki >Assignee: Toshihiro Suzuki >Priority: Critical > Fix For: 3.0.0-alpha-1, 2.4.0 > > > Currently, RowMutations supports only Put and Delete. Supporting > Increment/Append in RowMutations would be helpful for some use cases. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Resolved] (HBASE-25361) [Flakey Tests] branch-2 TestMetaRegionLocationCache.testStandByMetaLocations
[ https://issues.apache.org/jira/browse/HBASE-25361?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Stack resolved HBASE-25361. --- Resolution: Fixed Re-resovling. Pushed the addendum to 2.3+ (doesn't go to 2.2) > [Flakey Tests] branch-2 TestMetaRegionLocationCache.testStandByMetaLocations > > > Key: HBASE-25361 > URL: https://issues.apache.org/jira/browse/HBASE-25361 > Project: HBase > Issue Type: Bug > Components: flakies >Reporter: Michael Stack >Assignee: Michael Stack >Priority: Major > Fix For: 3.0.0-alpha-1, 2.3.4, 2.4.1 > > > This just happened to me running tests locally. I see it in the current > flakies list > [https://ci-hadoop.apache.org/job/HBase/job/HBase-Flaky-Tests/job/branch-2/310/testReport/junit/org.apache.hadoop.hbase.client/TestMetaRegionLocationCache/testStandByMetaLocations/] > > It looks like a simple timing issue. Let me put up a simple patch to see if > it helps. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Reopened] (HBASE-25361) [Flakey Tests] branch-2 TestMetaRegionLocationCache.testStandByMetaLocations
[
https://issues.apache.org/jira/browse/HBASE-25361?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Michael Stack reopened HBASE-25361:
---
Reopen to apply addendum. Test just failed on me. Looking at my PR again, I'm
missing reset of a counter so the new wait actually doesn't happen. Here is the
addendum:
commit 7d0a687e5798a2f4ca3190b409169f7e17a75b34 (HEAD -> m, origin/master,
origin/HEAD)
Author: stack
Date: Sat Dec 5 14:00:18 2020 -0800
HBASE-25361 [Flakey Tests] branch-2
TestMetaRegionLocationCache.testStandByMetaLocations (#2736)
Addendum; Reset counter so we actually wait in the new loop added by the
above.
{{diff --git
a/hbase-server/src/test/java/org/apache/hadoop/hbase/client/TestMetaRegionLocationCache.java
b/hbase-server/src/test/java/org/apache/hadoop/hbase/client/TestMetaRegionLocationCache.java}}
{{index 577e15cedf..2bcddc9ea7 100644}}
{{---
a/hbase-server/src/test/java/org/apache/hadoop/hbase/client/TestMetaRegionLocationCache.java}}
{{+++
b/hbase-server/src/test/java/org/apache/hadoop/hbase/client/TestMetaRegionLocationCache.java}}
{{@@ -99,6 +99,7 @@ public class TestMetaRegionLocationCache {}}
{{ ZKWatcher zk = master.getZooKeeper();}}
{{ List metaZnodes = zk.getMetaReplicaNodes();}}
{{ // Wait till all replicas available.}}
{{+ retries = 0;}}
{{ while
(master.getMetaRegionLocationCache().getMetaRegionLocations().get().size() !=}}
{{ metaZnodes.size()) {}}
{{ Thread.sleep(1000);}}
> [Flakey Tests] branch-2 TestMetaRegionLocationCache.testStandByMetaLocations
>
>
> Key: HBASE-25361
> URL: https://issues.apache.org/jira/browse/HBASE-25361
> Project: HBase
> Issue Type: Bug
> Components: flakies
>Reporter: Michael Stack
>Assignee: Michael Stack
>Priority: Major
> Fix For: 3.0.0-alpha-1, 2.3.4, 2.4.1
>
>
> This just happened to me running tests locally. I see it in the current
> flakies list
> [https://ci-hadoop.apache.org/job/HBase/job/HBase-Flaky-Tests/job/branch-2/310/testReport/junit/org.apache.hadoop.hbase.client/TestMetaRegionLocationCache/testStandByMetaLocations/]
>
> It looks like a simple timing issue. Let me put up a simple patch to see if
> it helps.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[GitHub] [hbase] Apache-HBase commented on pull request #2733: HBASE-25353 [Flakey Tests] branch-2 TestShutdownBackupMaster
Apache-HBase commented on pull request #2733: URL: https://github.com/apache/hbase/pull/2733#issuecomment-739422519 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 1m 12s | Docker mode activated. | ||| _ Prechecks _ | | +1 :green_heart: | dupname | 0m 0s | No case conflicting files found. | | +1 :green_heart: | hbaseanti | 0m 0s | Patch does not have any anti-patterns. | | +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. | ||| _ branch-2 Compile Tests _ | | +0 :ok: | mvndep | 0m 16s | Maven dependency ordering for branch | | +1 :green_heart: | mvninstall | 3m 57s | branch-2 passed | | +1 :green_heart: | checkstyle | 1m 49s | branch-2 passed | | +1 :green_heart: | spotbugs | 3m 17s | branch-2 passed | ||| _ Patch Compile Tests _ | | +0 :ok: | mvndep | 0m 11s | Maven dependency ordering for patch | | +1 :green_heart: | mvninstall | 3m 35s | the patch passed | | +1 :green_heart: | checkstyle | 1m 47s | the patch passed | | +1 :green_heart: | whitespace | 0m 0s | The patch has no whitespace issues. | | +1 :green_heart: | xml | 0m 1s | The patch has no ill-formed XML file. | | +1 :green_heart: | hadoopcheck | 12m 45s | Patch does not cause any errors with Hadoop 3.1.2 3.2.1. | | +1 :green_heart: | spotbugs | 3m 33s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | asflicense | 0m 24s | The patch does not generate ASF License warnings. | | | | 40m 48s | | | Subsystem | Report/Notes | |--:|:-| | Docker | ClientAPI=1.40 ServerAPI=1.40 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2733/6/artifact/yetus-general-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/2733 | | Optional Tests | dupname asflicense spotbugs hadoopcheck hbaseanti checkstyle xml | | uname | Linux e01cbdc33ca6 4.15.0-112-generic #113-Ubuntu SMP Thu Jul 9 23:41:39 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | branch-2 / a4d5b1bb88 | | Max. process+thread count | 84 (vs. ulimit of 12500) | | modules | C: hbase-client hbase-server U: . | | Console output | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2733/6/console | | versions | git=2.17.1 maven=3.6.3 spotbugs=3.1.12 | | Powered by | Apache Yetus 0.12.0 https://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #2733: HBASE-25353 [Flakey Tests] branch-2 TestShutdownBackupMaster
Apache-HBase commented on pull request #2733: URL: https://github.com/apache/hbase/pull/2733#issuecomment-739417938 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 1m 47s | Docker mode activated. | | -0 :warning: | yetus | 0m 8s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck | ||| _ Prechecks _ | ||| _ branch-2 Compile Tests _ | | +0 :ok: | mvndep | 0m 15s | Maven dependency ordering for branch | | +1 :green_heart: | mvninstall | 4m 20s | branch-2 passed | | +1 :green_heart: | compile | 1m 35s | branch-2 passed | | +1 :green_heart: | shadedjars | 6m 47s | branch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 1m 5s | branch-2 passed | ||| _ Patch Compile Tests _ | | +0 :ok: | mvndep | 0m 14s | Maven dependency ordering for patch | | +1 :green_heart: | mvninstall | 3m 52s | the patch passed | | +1 :green_heart: | compile | 1m 28s | the patch passed | | +1 :green_heart: | javac | 1m 28s | the patch passed | | +1 :green_heart: | shadedjars | 6m 34s | patch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 1m 1s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | unit | 2m 30s | hbase-client in the patch passed. | | +1 :green_heart: | unit | 214m 10s | hbase-server in the patch passed. | | | | 247m 57s | | | Subsystem | Report/Notes | |--:|:-| | Docker | ClientAPI=1.40 ServerAPI=1.40 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2733/5/artifact/yetus-jdk8-hadoop2-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/2733 | | Optional Tests | javac javadoc unit shadedjars compile | | uname | Linux 41e32ad5c548 4.15.0-112-generic #113-Ubuntu SMP Thu Jul 9 23:41:39 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | branch-2 / a4d5b1bb88 | | Default Java | AdoptOpenJDK-1.8.0_232-b09 | | Test Results | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2733/5/testReport/ | | Max. process+thread count | 2766 (vs. ulimit of 12500) | | modules | C: hbase-client hbase-server U: . | | Console output | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2733/5/console | | versions | git=2.17.1 maven=3.6.3 | | Powered by | Apache Yetus 0.12.0 https://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #2733: HBASE-25353 [Flakey Tests] branch-2 TestShutdownBackupMaster
Apache-HBase commented on pull request #2733: URL: https://github.com/apache/hbase/pull/2733#issuecomment-739389252 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 1m 49s | Docker mode activated. | | -0 :warning: | yetus | 0m 7s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck | ||| _ Prechecks _ | ||| _ branch-2 Compile Tests _ | | +0 :ok: | mvndep | 0m 25s | Maven dependency ordering for branch | | +1 :green_heart: | mvninstall | 3m 59s | branch-2 passed | | +1 :green_heart: | compile | 1m 37s | branch-2 passed | | +1 :green_heart: | shadedjars | 6m 41s | branch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 1m 8s | branch-2 passed | ||| _ Patch Compile Tests _ | | +0 :ok: | mvndep | 0m 15s | Maven dependency ordering for patch | | +1 :green_heart: | mvninstall | 3m 56s | the patch passed | | +1 :green_heart: | compile | 1m 34s | the patch passed | | +1 :green_heart: | javac | 1m 34s | the patch passed | | +1 :green_heart: | shadedjars | 6m 41s | patch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 1m 11s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | unit | 2m 21s | hbase-client in the patch passed. | | +1 :green_heart: | unit | 138m 7s | hbase-server in the patch passed. | | | | 172m 22s | | | Subsystem | Report/Notes | |--:|:-| | Docker | ClientAPI=1.40 ServerAPI=1.40 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2733/5/artifact/yetus-jdk11-hadoop3-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/2733 | | Optional Tests | javac javadoc unit shadedjars compile | | uname | Linux 913e57904344 4.15.0-65-generic #74-Ubuntu SMP Tue Sep 17 17:06:04 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | branch-2 / a4d5b1bb88 | | Default Java | AdoptOpenJDK-11.0.6+10 | | Test Results | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2733/5/testReport/ | | Max. process+thread count | 4071 (vs. ulimit of 12500) | | modules | C: hbase-client hbase-server U: . | | Console output | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2733/5/console | | versions | git=2.17.1 maven=3.6.3 | | Powered by | Apache Yetus 0.12.0 https://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #2739: HBASE-23340 hmaster /hbase/replication/rs session expired (hbase repl…
Apache-HBase commented on pull request #2739: URL: https://github.com/apache/hbase/pull/2739#issuecomment-739344951 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 0m 27s | Docker mode activated. | | -0 :warning: | yetus | 0m 3s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck | ||| _ Prechecks _ | ||| _ master Compile Tests _ | | +1 :green_heart: | mvninstall | 3m 48s | master passed | | +1 :green_heart: | compile | 0m 59s | master passed | | +1 :green_heart: | shadedjars | 6m 31s | branch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 0m 40s | master passed | ||| _ Patch Compile Tests _ | | +1 :green_heart: | mvninstall | 3m 31s | the patch passed | | +1 :green_heart: | compile | 0m 58s | the patch passed | | +1 :green_heart: | javac | 0m 58s | the patch passed | | +1 :green_heart: | shadedjars | 6m 33s | patch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 0m 37s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | unit | 145m 2s | hbase-server in the patch passed. | | | | 171m 20s | | | Subsystem | Report/Notes | |--:|:-| | Docker | ClientAPI=1.40 ServerAPI=1.40 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2739/1/artifact/yetus-jdk8-hadoop3-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/2739 | | Optional Tests | javac javadoc unit shadedjars compile | | uname | Linux afd4f592977c 4.15.0-58-generic #64-Ubuntu SMP Tue Aug 6 11:12:41 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / b26395f3bb | | Default Java | AdoptOpenJDK-1.8.0_232-b09 | | Test Results | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2739/1/testReport/ | | Max. process+thread count | 4008 (vs. ulimit of 3) | | modules | C: hbase-server U: hbase-server | | Console output | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2739/1/console | | versions | git=2.17.1 maven=3.6.3 | | Powered by | Apache Yetus 0.12.0 https://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #2739: HBASE-23340 hmaster /hbase/replication/rs session expired (hbase repl…
Apache-HBase commented on pull request #2739: URL: https://github.com/apache/hbase/pull/2739#issuecomment-739336473 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 0m 29s | Docker mode activated. | | -0 :warning: | yetus | 0m 3s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck | ||| _ Prechecks _ | ||| _ master Compile Tests _ | | +1 :green_heart: | mvninstall | 4m 16s | master passed | | +1 :green_heart: | compile | 1m 5s | master passed | | +1 :green_heart: | shadedjars | 6m 43s | branch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 0m 42s | master passed | ||| _ Patch Compile Tests _ | | +1 :green_heart: | mvninstall | 3m 59s | the patch passed | | +1 :green_heart: | compile | 1m 10s | the patch passed | | +1 :green_heart: | javac | 1m 10s | the patch passed | | +1 :green_heart: | shadedjars | 6m 37s | patch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 0m 39s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | unit | 132m 2s | hbase-server in the patch passed. | | | | 159m 58s | | | Subsystem | Report/Notes | |--:|:-| | Docker | ClientAPI=1.40 ServerAPI=1.40 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2739/1/artifact/yetus-jdk11-hadoop3-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/2739 | | Optional Tests | javac javadoc unit shadedjars compile | | uname | Linux dfc9a6374077 4.15.0-58-generic #64-Ubuntu SMP Tue Aug 6 11:12:41 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / b26395f3bb | | Default Java | AdoptOpenJDK-11.0.6+10 | | Test Results | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2739/1/testReport/ | | Max. process+thread count | 4676 (vs. ulimit of 3) | | modules | C: hbase-server U: hbase-server | | Console output | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2739/1/console | | versions | git=2.17.1 maven=3.6.3 | | Powered by | Apache Yetus 0.12.0 https://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Commented] (HBASE-22120) Replace HTrace with OpenTracing
[ https://issues.apache.org/jira/browse/HBASE-22120?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17244558#comment-17244558 ] Yuri commented on HBASE-22120: -- At this point I am comfortable recommending OpenTelemetry for Java instead of OpenTracing. > Replace HTrace with OpenTracing > --- > > Key: HBASE-22120 > URL: https://issues.apache.org/jira/browse/HBASE-22120 > Project: HBase > Issue Type: New Feature > Components: tracing >Affects Versions: 3.0.0-alpha-1 >Reporter: Sergey Shelukhin >Assignee: Wei-Chiu Chuang >Priority: Major > > h2. Deprecate HTrace usage in HBase > * HBase 1.x (branch-1) > * Declare HTrace (htrace 3.x) deprecated in the user doc. > * HBase 2.x (branch-2) > * Declare HTrace deprecated in the user doc. Furthermore, state that it is > known not working. > * Either fix the trace context propagation bug in HBase 2.x, or backport > OpenTracing support from the master branch. I am inclined to the latter. > * HBase 3.x (master branch) > * Remove HTrace entirely. > * Add OpenTracing APIs. Potentially backport to HBase 2.4. > * Replace OpenTracing API with OpenTelemetry when the latter stabilizes. > h1. Milestones > # Doc -- deprecation notice > # Replace existing HTrace code with OpenTracing code in the master branch > (3.x) > # Java (a [poc|https://github.com/jojochuang/hbase/tree/HBASE-22120] is > currently under way) > # HBase shell and scripts (Ruby, shell script) > # Doc > # Add new trace instrumentation code for new features not instrumented by > the existing HTace code. > # Propagate the traces to other systems such as HDFS and MapReduce. > # Support other OpenTracing tracers. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (HBASE-25292) Improve InetSocketAddress usage discipline
[
https://issues.apache.org/jira/browse/HBASE-25292?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17244555#comment-17244555
]
Hudson commented on HBASE-25292:
Results for branch branch-2
[build #121 on
builds.a.o|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/branch-2/121/]:
(/) *{color:green}+1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/branch-2/121/General_20Nightly_20Build_20Report/]
(/) {color:green}+1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/branch-2/121/JDK8_20Nightly_20Build_20Report_20_28Hadoop2_29/]
(/) {color:green}+1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/branch-2/121/JDK8_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(/) {color:green}+1 jdk11 hadoop3 checks{color}
-- For more information [see jdk11
report|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/branch-2/121/JDK11_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
(/) {color:green}+1 client integration test{color}
> Improve InetSocketAddress usage discipline
> --
>
> Key: HBASE-25292
> URL: https://issues.apache.org/jira/browse/HBASE-25292
> Project: HBase
> Issue Type: Bug
> Components: Client, HFile
>Reporter: Andrew Kyle Purtell
>Assignee: Andrew Kyle Purtell
>Priority: Major
> Fix For: 3.0.0-alpha-1, 1.7.0, 2.5.0
>
>
> We sometimes cache InetSocketAddress in data structures in an attempt to
> optimize away potential nameservice (DNS) lookups. This is, in general, an
> anti-pattern, because once an InetSocketAddress is resolved, resolution is
> never attempted again. The ideal pattern for connect() is ISA instantiation
> just before the connect() call, with no reuse of the ISA instance. For bind()
> we presume the local identity won't change while the process is live so usage
> and caching can be relaxed in that case.
> If I can restate my proposal for a usage convention for InetSocketAddress, it
> would be this: Network identities should be bound late. This means addresses
> should be resolved at the last possible moment. Also, network identity
> mappings can change, so our code should not inappropriately cache them;
> otherwise we might miss a change and fail to operate normally.
> I have reviewed the code for InetSocketAddress usage and in my opinion
> sometimes we are caching ISA acceptably, and in other cases we are not.
> Correct cases:
> * We cache ISA for RPC connections, so we don't potentially do a lookup for
> every Call. However, we resolve the address earlier than we need to. The code
> can be improved by moving resolution to just before where we connect().
> * Use of ISA with bind. Typical uses like bindAddress, listenerAddress,
> initialIsa, or localAddress.
> ** (There is no harm to keep direct use of ISA for bind() but these could
> all be replaced with Address and on-demand create of ISA just before bind().
> * ClusterStatusPublisher in master.
> * Netty integration. Netty accepts and supplies ISA, no choice there, but we
> want to resolve at channel create time and cache for lifetime of the channel
> anyway. If a remote host goes away and is replaced with a new identity, a new
> channel/connection will be created with a new resolution just before
> connect() via higher layer error handling.
> Incorrect cases that can be fixed:
> * RPC stubs. Remote clients may be recycled and replaced with new instances
> where the network identities (DNS name to IP address mapping) have changed--.
> HBASE-14544 attempts to work around DNS instability in data centers of years
> past in a way that, in my opinion, is the wrong thing to do in the modern
> era. In modern datacenters, in public cloud, and especially in kubernetes
> environments, DNS mappings are dynamic and subject to frequent change. It is
> just never the right thing to do to cache them. I intend to propose a revert
> of HBASE-14544. Reverting this simplifies some code a bit. That is the only
> reason: this is in my opinion some legacy that can be dropped, one fewer
> configuration variable (yay!), but if this part of the proposal is
> controversial it can be skipped.
> * RPC stubs again. When looking up the IP address of the remote host when
> creating a stub key we also make a key even if the resolution fails. This is
> the wrong thing to do. If we can't resolve the remote address, we can't
> contact the server. Making a stub that
[GitHub] [hbase] Apache-HBase commented on pull request #2733: HBASE-25353 [Flakey Tests] branch-2 TestShutdownBackupMaster
Apache-HBase commented on pull request #2733: URL: https://github.com/apache/hbase/pull/2733#issuecomment-739328239 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 1m 51s | Docker mode activated. | ||| _ Prechecks _ | | +1 :green_heart: | dupname | 0m 0s | No case conflicting files found. | | +1 :green_heart: | hbaseanti | 0m 0s | Patch does not have any anti-patterns. | | +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. | ||| _ branch-2 Compile Tests _ | | +0 :ok: | mvndep | 0m 15s | Maven dependency ordering for branch | | +1 :green_heart: | mvninstall | 4m 17s | branch-2 passed | | +1 :green_heart: | checkstyle | 1m 56s | branch-2 passed | | +1 :green_heart: | spotbugs | 3m 26s | branch-2 passed | ||| _ Patch Compile Tests _ | | +0 :ok: | mvndep | 0m 13s | Maven dependency ordering for patch | | +1 :green_heart: | mvninstall | 3m 46s | the patch passed | | +1 :green_heart: | checkstyle | 1m 55s | the patch passed | | +1 :green_heart: | whitespace | 0m 0s | The patch has no whitespace issues. | | +1 :green_heart: | xml | 0m 1s | The patch has no ill-formed XML file. | | +1 :green_heart: | hadoopcheck | 13m 16s | Patch does not cause any errors with Hadoop 3.1.2 3.2.1. | | +1 :green_heart: | spotbugs | 4m 31s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | asflicense | 0m 28s | The patch does not generate ASF License warnings. | | | | 45m 11s | | | Subsystem | Report/Notes | |--:|:-| | Docker | ClientAPI=1.40 ServerAPI=1.40 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2733/5/artifact/yetus-general-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/2733 | | Optional Tests | dupname asflicense spotbugs hadoopcheck hbaseanti checkstyle xml | | uname | Linux 70a20bda2379 4.15.0-112-generic #113-Ubuntu SMP Thu Jul 9 23:41:39 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | branch-2 / a4d5b1bb88 | | Max. process+thread count | 84 (vs. ulimit of 12500) | | modules | C: hbase-client hbase-server U: . | | Console output | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2733/5/console | | versions | git=2.17.1 maven=3.6.3 spotbugs=3.1.12 | | Powered by | Apache Yetus 0.12.0 https://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #2738: HBASE-25363 Improve performance of HFileLinkCleaner by using ReadWrit…
Apache-HBase commented on pull request #2738: URL: https://github.com/apache/hbase/pull/2738#issuecomment-739325776 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 1m 40s | Docker mode activated. | | -0 :warning: | yetus | 0m 3s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck | ||| _ Prechecks _ | ||| _ master Compile Tests _ | | +1 :green_heart: | mvninstall | 4m 53s | master passed | | +1 :green_heart: | compile | 1m 12s | master passed | | +1 :green_heart: | shadedjars | 7m 25s | branch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 0m 45s | master passed | ||| _ Patch Compile Tests _ | | +1 :green_heart: | mvninstall | 4m 32s | the patch passed | | +1 :green_heart: | compile | 1m 11s | the patch passed | | +1 :green_heart: | javac | 1m 11s | the patch passed | | +1 :green_heart: | shadedjars | 7m 25s | patch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 0m 41s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | unit | 197m 36s | hbase-server in the patch passed. | | | | 229m 8s | | | Subsystem | Report/Notes | |--:|:-| | Docker | ClientAPI=1.40 ServerAPI=1.40 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2738/1/artifact/yetus-jdk11-hadoop3-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/2738 | | Optional Tests | javac javadoc unit shadedjars compile | | uname | Linux 4cc34011e549 4.15.0-112-generic #113-Ubuntu SMP Thu Jul 9 23:41:39 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / b26395f3bb | | Default Java | AdoptOpenJDK-11.0.6+10 | | Test Results | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2738/1/testReport/ | | Max. process+thread count | 3726 (vs. ulimit of 3) | | modules | C: hbase-server U: hbase-server | | Console output | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2738/1/console | | versions | git=2.17.1 maven=3.6.3 | | Powered by | Apache Yetus 0.12.0 https://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] saintstack commented on a change in pull request #2733: HBASE-25353 [Flakey Tests] branch-2 TestShutdownBackupMaster
saintstack commented on a change in pull request #2733:
URL: https://github.com/apache/hbase/pull/2733#discussion_r536826539
##
File path:
hbase-client/src/main/java/org/apache/hadoop/hbase/client/ConnectionImplementation.java
##
@@ -627,7 +628,16 @@ public Configuration getConfiguration() {
private void checkClosed() throws DoNotRetryIOException {
if (this.closed) {
- throw new DoNotRetryIOException(toString() + " closed");
+ throw new ConnectionClosedException(toString() + " closed");
+}
+ }
+
+ /**
+ * Thrown when connection is closed.
+ */
+ private static class ConnectionClosedException extends DoNotRetryIOException
{
+ConnectionClosedException(String message) {
+ super(message);
Review comment:
Good question. I didn't even see this class import.
Looking at it, the CCE is no good to me and I think probably no good to
anyone since it doesn't subclass DoNotRetryIOE. I left a TODO on that class and
called this private local class LocalCCE instead. Thanks @huaxiangsun . Let me
update the patch. Will merge if all good.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #2739: HBASE-23340 hmaster /hbase/replication/rs session expired (hbase repl…
Apache-HBase commented on pull request #2739: URL: https://github.com/apache/hbase/pull/2739#issuecomment-739322392 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 0m 29s | Docker mode activated. | ||| _ Prechecks _ | | +1 :green_heart: | dupname | 0m 0s | No case conflicting files found. | | +1 :green_heart: | hbaseanti | 0m 0s | Patch does not have any anti-patterns. | | +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. | ||| _ master Compile Tests _ | | +1 :green_heart: | mvninstall | 3m 47s | master passed | | +1 :green_heart: | checkstyle | 1m 6s | master passed | | +1 :green_heart: | spotbugs | 2m 3s | master passed | ||| _ Patch Compile Tests _ | | +1 :green_heart: | mvninstall | 3m 26s | the patch passed | | -0 :warning: | checkstyle | 1m 3s | hbase-server: The patch generated 1 new + 1 unchanged - 0 fixed = 2 total (was 1) | | +1 :green_heart: | whitespace | 0m 0s | The patch has no whitespace issues. | | +1 :green_heart: | hadoopcheck | 17m 14s | Patch does not cause any errors with Hadoop 3.1.2 3.2.1 3.3.0. | | +1 :green_heart: | spotbugs | 2m 7s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | asflicense | 0m 15s | The patch does not generate ASF License warnings. | | | | 38m 39s | | | Subsystem | Report/Notes | |--:|:-| | Docker | ClientAPI=1.40 ServerAPI=1.40 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2739/1/artifact/yetus-general-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/2739 | | Optional Tests | dupname asflicense spotbugs hadoopcheck hbaseanti checkstyle | | uname | Linux bdc8a7eabf5e 4.15.0-60-generic #67-Ubuntu SMP Thu Aug 22 16:55:30 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / b26395f3bb | | checkstyle | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2739/1/artifact/yetus-general-check/output/diff-checkstyle-hbase-server.txt | | Max. process+thread count | 94 (vs. ulimit of 3) | | modules | C: hbase-server U: hbase-server | | Console output | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2739/1/console | | versions | git=2.17.1 maven=3.6.3 spotbugs=3.1.12 | | Powered by | Apache Yetus 0.12.0 https://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Resolved] (HBASE-25361) [Flakey Tests] branch-2 TestMetaRegionLocationCache.testStandByMetaLocations
[ https://issues.apache.org/jira/browse/HBASE-25361?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Stack resolved HBASE-25361. --- Fix Version/s: 2.4.1 2.3.4 3.0.0-alpha-1 Hadoop Flags: Reviewed Assignee: Michael Stack Resolution: Fixed Pushed to branch-2.3+. Thanks for review [~huaxiang] . Hopefully this fixes the flakey. > [Flakey Tests] branch-2 TestMetaRegionLocationCache.testStandByMetaLocations > > > Key: HBASE-25361 > URL: https://issues.apache.org/jira/browse/HBASE-25361 > Project: HBase > Issue Type: Bug > Components: flakies >Reporter: Michael Stack >Assignee: Michael Stack >Priority: Major > Fix For: 3.0.0-alpha-1, 2.3.4, 2.4.1 > > > This just happened to me running tests locally. I see it in the current > flakies list > [https://ci-hadoop.apache.org/job/HBase/job/HBase-Flaky-Tests/job/branch-2/310/testReport/junit/org.apache.hadoop.hbase.client/TestMetaRegionLocationCache/testStandByMetaLocations/] > > It looks like a simple timing issue. Let me put up a simple patch to see if > it helps. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[GitHub] [hbase] saintstack merged pull request #2736: HBASE-25361 [Flakey Tests] branch-2 TestMetaRegionLocationCache.testS…
saintstack merged pull request #2736: URL: https://github.com/apache/hbase/pull/2736 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Commented] (HBASE-22120) Replace HTrace with OpenTracing
[ https://issues.apache.org/jira/browse/HBASE-22120?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17244539#comment-17244539 ] ramkrishna.s.vasudevan commented on HBASE-22120: We did a small POC with open telemetry changes and just updated some existing Tracing calls to use the Open telemetry way and some new changes. We were able to make it work with OpenTelemetry. JFYI. > Replace HTrace with OpenTracing > --- > > Key: HBASE-22120 > URL: https://issues.apache.org/jira/browse/HBASE-22120 > Project: HBase > Issue Type: New Feature > Components: tracing >Affects Versions: 3.0.0-alpha-1 >Reporter: Sergey Shelukhin >Assignee: Wei-Chiu Chuang >Priority: Major > > h2. Deprecate HTrace usage in HBase > * HBase 1.x (branch-1) > * Declare HTrace (htrace 3.x) deprecated in the user doc. > * HBase 2.x (branch-2) > * Declare HTrace deprecated in the user doc. Furthermore, state that it is > known not working. > * Either fix the trace context propagation bug in HBase 2.x, or backport > OpenTracing support from the master branch. I am inclined to the latter. > * HBase 3.x (master branch) > * Remove HTrace entirely. > * Add OpenTracing APIs. Potentially backport to HBase 2.4. > * Replace OpenTracing API with OpenTelemetry when the latter stabilizes. > h1. Milestones > # Doc -- deprecation notice > # Replace existing HTrace code with OpenTracing code in the master branch > (3.x) > # Java (a [poc|https://github.com/jojochuang/hbase/tree/HBASE-22120] is > currently under way) > # HBase shell and scripts (Ruby, shell script) > # Doc > # Add new trace instrumentation code for new features not instrumented by > the existing HTace code. > # Propagate the traces to other systems such as HDFS and MapReduce. > # Support other OpenTracing tracers. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[GitHub] [hbase] Apache-HBase commented on pull request #2738: HBASE-25363 Improve performance of HFileLinkCleaner by using ReadWrit…
Apache-HBase commented on pull request #2738: URL: https://github.com/apache/hbase/pull/2738#issuecomment-739318389 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 0m 27s | Docker mode activated. | | -0 :warning: | yetus | 0m 4s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck | ||| _ Prechecks _ | ||| _ master Compile Tests _ | | +1 :green_heart: | mvninstall | 3m 45s | master passed | | +1 :green_heart: | compile | 0m 55s | master passed | | +1 :green_heart: | shadedjars | 6m 30s | branch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 0m 38s | master passed | ||| _ Patch Compile Tests _ | | +1 :green_heart: | mvninstall | 3m 30s | the patch passed | | +1 :green_heart: | compile | 0m 57s | the patch passed | | +1 :green_heart: | javac | 0m 57s | the patch passed | | +1 :green_heart: | shadedjars | 6m 30s | patch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 0m 36s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | unit | 140m 13s | hbase-server in the patch passed. | | | | 166m 14s | | | Subsystem | Report/Notes | |--:|:-| | Docker | ClientAPI=1.40 ServerAPI=1.40 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2738/1/artifact/yetus-jdk8-hadoop3-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/2738 | | Optional Tests | javac javadoc unit shadedjars compile | | uname | Linux 69a215f94985 4.15.0-58-generic #64-Ubuntu SMP Tue Aug 6 11:12:41 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / b26395f3bb | | Default Java | AdoptOpenJDK-1.8.0_232-b09 | | Test Results | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2738/1/testReport/ | | Max. process+thread count | 4243 (vs. ulimit of 3) | | modules | C: hbase-server U: hbase-server | | Console output | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2738/1/console | | versions | git=2.17.1 maven=3.6.3 | | Powered by | Apache Yetus 0.12.0 https://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] cuibo01 opened a new pull request #2739: HBASE-23340 hmaster /hbase/replication/rs session expired (hbase repl…
cuibo01 opened a new pull request #2739: URL: https://github.com/apache/hbase/pull/2739 …ication default value is true, we don't use ) causes logcleaner can not clean oldWALs, which resulits in oldWALs too large (more than 2TB) This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #2738: HBASE-25363 Improve performance of HFileLinkCleaner by using ReadWrit…
Apache-HBase commented on pull request #2738: URL: https://github.com/apache/hbase/pull/2738#issuecomment-739258944 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 7m 21s | Docker mode activated. | ||| _ Prechecks _ | | +1 :green_heart: | dupname | 0m 0s | No case conflicting files found. | | +1 :green_heart: | hbaseanti | 0m 0s | Patch does not have any anti-patterns. | | +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. | ||| _ master Compile Tests _ | | +1 :green_heart: | mvninstall | 4m 10s | master passed | | +1 :green_heart: | checkstyle | 1m 12s | master passed | | +1 :green_heart: | spotbugs | 2m 12s | master passed | ||| _ Patch Compile Tests _ | | +1 :green_heart: | mvninstall | 3m 51s | the patch passed | | -0 :warning: | checkstyle | 1m 10s | hbase-server: The patch generated 5 new + 0 unchanged - 2 fixed = 5 total (was 2) | | +1 :green_heart: | whitespace | 0m 0s | The patch has no whitespace issues. | | +1 :green_heart: | hadoopcheck | 19m 10s | Patch does not cause any errors with Hadoop 3.1.2 3.2.1 3.3.0. | | +1 :green_heart: | spotbugs | 2m 19s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | asflicense | 0m 13s | The patch does not generate ASF License warnings. | | | | 49m 20s | | | Subsystem | Report/Notes | |--:|:-| | Docker | ClientAPI=1.40 ServerAPI=1.40 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2738/1/artifact/yetus-general-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/2738 | | Optional Tests | dupname asflicense spotbugs hadoopcheck hbaseanti checkstyle | | uname | Linux 02ec0bf357f8 4.15.0-112-generic #113-Ubuntu SMP Thu Jul 9 23:41:39 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / b26395f3bb | | checkstyle | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2738/1/artifact/yetus-general-check/output/diff-checkstyle-hbase-server.txt | | Max. process+thread count | 84 (vs. ulimit of 3) | | modules | C: hbase-server U: hbase-server | | Console output | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2738/1/console | | versions | git=2.17.1 maven=3.6.3 spotbugs=3.1.12 | | Powered by | Apache Yetus 0.12.0 https://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Commented] (HBASE-25322) Redundant Reference file in bottom region of split
[ https://issues.apache.org/jira/browse/HBASE-25322?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17244501#comment-17244501 ] Xiaolin Ha commented on HBASE-25322: Ping [~zhangduo] [~zghao], could you help to review this PR? Thanks. > Redundant Reference file in bottom region of split > -- > > Key: HBASE-25322 > URL: https://issues.apache.org/jira/browse/HBASE-25322 > Project: HBase > Issue Type: Improvement >Affects Versions: 3.0.0-alpha-1 >Reporter: Xiaolin Ha >Assignee: Xiaolin Ha >Priority: Major > > When we split a region ranges from (,), the bottom region should contain keys > of(,split key), and the top region should contain keys of [split key, ). > Currently, if we do the following operations: > # put rowkeys 100,101,102,103,104,105 to a table, and flush the memstore to > make a hfile with rowkeys 100,101,102,103,104,105; > # put rowkeys 200,201,202,203,204,205 to the table, and flush the memstore > to make a hfile with rowkeys 200,201,202,203,204,205; > # split the table region, using split key 200; > # then the bottom region will has two Reference files, while the top region > only has one. > But we expect the bottom region has only one Reference file as the the top > region. > That's because when generating Reference files in child region, the bottom > region used the `PrivateCellUtil.createLastOnRow(splitRow)` cell to compare > to first keys in the hfiles, while the top region used > `PrivateCellUtil.createFirstOnRow(splitRow)` cell to compare to last keys in > the hfiles. > `LastOnRow(splitRow)` means the maximum row generated by the split row, while > `FirstOnRow(splitRow)` means the minimus row generated by the split row. The > split row should be in the top region. And we should use > `FirstOnRow(splitRow)` compare to hfile first and last keys in both bottom > and top region. > Though the redundant Reference file will not be read by the bottom region, > the compaction of the redundant Reference file will result in empty file if > only this redundant Reference file participates in a compaction. > > > > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (HBASE-25363) Improve performance of HFileLinkCleaner by using ReadWriteLock instead of synchronize
[ https://issues.apache.org/jira/browse/HBASE-25363?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17244498#comment-17244498 ] Xiaolin Ha commented on HBASE-25363: HFileLinkCleaner is a required cleaner for almost all the cluster, I think improve the performance for it will be benefit for large clusters. > Improve performance of HFileLinkCleaner by using ReadWriteLock instead of > synchronize > - > > Key: HBASE-25363 > URL: https://issues.apache.org/jira/browse/HBASE-25363 > Project: HBase > Issue Type: Test >Reporter: Xiaolin Ha >Assignee: Xiaolin Ha >Priority: Major > > HFileLinkCleaner uses synchronized method isFileDeletable, as a result, to > cross the cleaner chain, all files will be processed serially. It may block > the fork-join thread which scans out some archived files. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[GitHub] [hbase] sunhelly opened a new pull request #2738: HBASE-25363 Improve performance of HFileLinkCleaner by using ReadWrit…
sunhelly opened a new pull request #2738: URL: https://github.com/apache/hbase/pull/2738 …eLock instead of synchronize This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Created] (HBASE-25363) Improve performance of HFileLinkCleaner by using ReadWriteLock instead of synchronize
Xiaolin Ha created HBASE-25363: -- Summary: Improve performance of HFileLinkCleaner by using ReadWriteLock instead of synchronize Key: HBASE-25363 URL: https://issues.apache.org/jira/browse/HBASE-25363 Project: HBase Issue Type: Test Reporter: Xiaolin Ha Assignee: Xiaolin Ha HFileLinkCleaner uses synchronized method isFileDeletable, as a result, to cross the cleaner chain, all files will be processed serially. It may block the fork-join thread which scans out some archived files. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (HBASE-25354) Update create-release scripts; rotted
[
https://issues.apache.org/jira/browse/HBASE-25354?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17244483#comment-17244483
]
Hudson commented on HBASE-25354:
Results for branch master
[build #149 on
builds.a.o|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/master/149/]:
(/) *{color:green}+1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/master/149/General_20Nightly_20Build_20Report/]
(/) {color:green}+1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/master/149/JDK8_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(/) {color:green}+1 jdk11 hadoop3 checks{color}
-- For more information [see jdk11
report|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/master/149/JDK11_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
(/) {color:green}+1 client integration test{color}
> Update create-release scripts; rotted
> -
>
> Key: HBASE-25354
> URL: https://issues.apache.org/jira/browse/HBASE-25354
> Project: HBase
> Issue Type: Bug
>Reporter: Michael Stack
>Assignee: Michael Stack
>Priority: Major
> Fix For: 3.0.0-alpha-1
>
>
> A few things noticed when trying to run them just now:
> * Some of the packages referenced are no longer available (the jdk and the
> gpg).
> * Some stuff that was checked in to test gpg works only makes sense in
> non-docker context
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HBASE-25292) Improve InetSocketAddress usage discipline
[
https://issues.apache.org/jira/browse/HBASE-25292?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17244473#comment-17244473
]
Hudson commented on HBASE-25292:
Results for branch branch-1
[build #64 on
builds.a.o|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/branch-1/64/]:
(x) *{color:red}-1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/branch-1/64//General_Nightly_Build_Report/]
(x) {color:red}-1 jdk7 checks{color}
-- For more information [see jdk7
report|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/branch-1/64//JDK7_Nightly_Build_Report/]
(x) {color:red}-1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/branch-1/64//JDK8_Nightly_Build_Report_(Hadoop2)/]
(x) {color:red}-1 source release artifact{color}
-- See build output for details.
> Improve InetSocketAddress usage discipline
> --
>
> Key: HBASE-25292
> URL: https://issues.apache.org/jira/browse/HBASE-25292
> Project: HBase
> Issue Type: Bug
> Components: Client, HFile
>Reporter: Andrew Kyle Purtell
>Assignee: Andrew Kyle Purtell
>Priority: Major
> Fix For: 3.0.0-alpha-1, 1.7.0, 2.5.0
>
>
> We sometimes cache InetSocketAddress in data structures in an attempt to
> optimize away potential nameservice (DNS) lookups. This is, in general, an
> anti-pattern, because once an InetSocketAddress is resolved, resolution is
> never attempted again. The ideal pattern for connect() is ISA instantiation
> just before the connect() call, with no reuse of the ISA instance. For bind()
> we presume the local identity won't change while the process is live so usage
> and caching can be relaxed in that case.
> If I can restate my proposal for a usage convention for InetSocketAddress, it
> would be this: Network identities should be bound late. This means addresses
> should be resolved at the last possible moment. Also, network identity
> mappings can change, so our code should not inappropriately cache them;
> otherwise we might miss a change and fail to operate normally.
> I have reviewed the code for InetSocketAddress usage and in my opinion
> sometimes we are caching ISA acceptably, and in other cases we are not.
> Correct cases:
> * We cache ISA for RPC connections, so we don't potentially do a lookup for
> every Call. However, we resolve the address earlier than we need to. The code
> can be improved by moving resolution to just before where we connect().
> * Use of ISA with bind. Typical uses like bindAddress, listenerAddress,
> initialIsa, or localAddress.
> ** (There is no harm to keep direct use of ISA for bind() but these could
> all be replaced with Address and on-demand create of ISA just before bind().
> * ClusterStatusPublisher in master.
> * Netty integration. Netty accepts and supplies ISA, no choice there, but we
> want to resolve at channel create time and cache for lifetime of the channel
> anyway. If a remote host goes away and is replaced with a new identity, a new
> channel/connection will be created with a new resolution just before
> connect() via higher layer error handling.
> Incorrect cases that can be fixed:
> * RPC stubs. Remote clients may be recycled and replaced with new instances
> where the network identities (DNS name to IP address mapping) have changed--.
> HBASE-14544 attempts to work around DNS instability in data centers of years
> past in a way that, in my opinion, is the wrong thing to do in the modern
> era. In modern datacenters, in public cloud, and especially in kubernetes
> environments, DNS mappings are dynamic and subject to frequent change. It is
> just never the right thing to do to cache them. I intend to propose a revert
> of HBASE-14544. Reverting this simplifies some code a bit. That is the only
> reason: this is in my opinion some legacy that can be dropped, one fewer
> configuration variable (yay!), but if this part of the proposal is
> controversial it can be skipped.
> * RPC stubs again. When looking up the IP address of the remote host when
> creating a stub key we also make a key even if the resolution fails. This is
> the wrong thing to do. If we can't resolve the remote address, we can't
> contact the server. Making a stub that can't communicate is pointless. Throw
> an exception instead.
> * Favored nodes. Although the HDFS API requires InetSocketAddress, we don't
> have to make up a list right away and cache them forever. We can use Address
> to record the list of favored nodes and convert from Address to
> InetSocketAddress on demand (when we go to create
