[jira] [Commented] (HBASE-25109) Add MR Counters to WALPlayer; currently hard to tell if it is doing anything
[
https://issues.apache.org/jira/browse/HBASE-25109?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17206005#comment-17206005
]
Hudson commented on HBASE-25109:
Results for branch branch-2
[build #64 on
builds.a.o|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/branch-2/64/]:
(x) *{color:red}-1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/branch-2/64/General_20Nightly_20Build_20Report/]
(x) {color:red}-1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/branch-2/64/JDK8_20Nightly_20Build_20Report_20_28Hadoop2_29/]
(x) {color:red}-1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/branch-2/64/JDK8_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(x) {color:red}-1 jdk11 hadoop3 checks{color}
-- For more information [see jdk11
report|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/branch-2/64/JDK11_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
(x) {color:red}-1 client integration test{color}
--Failed when running client tests on top of Hadoop 2. [see log for
details|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/branch-2/64//artifact/output-integration/hadoop-2.log].
(note that this means we didn't run on Hadoop 3)
> Add MR Counters to WALPlayer; currently hard to tell if it is doing anything
>
>
> Key: HBASE-25109
> URL: https://issues.apache.org/jira/browse/HBASE-25109
> Project: HBase
> Issue Type: Improvement
>Reporter: Michael Stack
>Assignee: Michael Stack
>Priority: Major
> Fix For: 3.0.0-alpha-1, 2.3.3, 2.4.0
>
>
> For example, when WALPlayer runs, it emits this:
> {code:java}
> 020-09-28 11:16:05,489 INFO [LocalJobRunner Map Task Executor #0]
> mapred.Task: Final Counters for attempt_local1916643172_0001_m_00_0:
> Counters: 20
> File System Counters
> FILE: Number of bytes read=268891453
> FILE: Number of bytes written=1018719
> FILE: Number of read operations=0
> FILE: Number of large read operations=0
> FILE: Number of write operations=0
> Map-Reduce Framework
> Map input records=4375
> Map output records=5369
> Input split bytes=245
> Spilled Records=0
> Failed Shuffles=0
> Merged Map outputs=0
> GC time elapsed (ms)=59
> Total committed heap usage (bytes)=518979584
> File Input Format Counters
> Bytes Read=0
> File Output Format Counters
> Bytes Written=0 {code}
> Change it so it does this:
> {code:java}
> 020-09-28 11:16:05,489 INFO [LocalJobRunner Map Task Executor #0]
> mapred.Task: Final Counters for attempt_local1916643172_0001_m_00_0:
> Counters: 20
> File System Counters
> FILE: Number of bytes read=268891453
> FILE: Number of bytes written=1018719
> FILE: Number of read operations=0
> FILE: Number of large read operations=0
> FILE: Number of write operations=0
> Map-Reduce Framework
> Map input records=4375
> Map output records=5369
> Input split bytes=245
> Spilled Records=0
> Failed Shuffles=0
> Merged Map outputs=0
> GC time elapsed (ms)=59
> Total committed heap usage (bytes)=518979584
> org.apache.hadoop.hbase.mapreduce.WALPlayer$Counter
> CELLS_READ=89574
> CELLS_WRITTEN=89572
> DELETES=64
> PUTS=5305
> WALEDITS=4375
> File Input Format Counters
> Bytes Read=0
> File Output Format Counters
> Bytes Written=0 {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HBASE-25132) Migrate flaky test jenkins job from Hadoop to hbase
[
https://issues.apache.org/jira/browse/HBASE-25132?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17206006#comment-17206006
]
Hudson commented on HBASE-25132:
Results for branch branch-2
[build #64 on
builds.a.o|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/branch-2/64/]:
(x) *{color:red}-1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/branch-2/64/General_20Nightly_20Build_20Report/]
(x) {color:red}-1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/branch-2/64/JDK8_20Nightly_20Build_20Report_20_28Hadoop2_29/]
(x) {color:red}-1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/branch-2/64/JDK8_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(x) {color:red}-1 jdk11 hadoop3 checks{color}
-- For more information [see jdk11
report|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/branch-2/64/JDK11_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
(x) {color:red}-1 client integration test{color}
--Failed when running client tests on top of Hadoop 2. [see log for
details|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/branch-2/64//artifact/output-integration/hadoop-2.log].
(note that this means we didn't run on Hadoop 3)
> Migrate flaky test jenkins job from Hadoop to hbase
> ---
>
> Key: HBASE-25132
> URL: https://issues.apache.org/jira/browse/HBASE-25132
> Project: HBase
> Issue Type: Sub-task
> Components: scripts
>Reporter: Duo Zhang
>Assignee: Duo Zhang
>Priority: Major
> Fix For: 3.0.0-alpha-1, 2.3.3, 1.7.0, 2.4.0, 1.4.14, 2.2.7
>
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HBASE-25134) Migrate HBase PreCommit jenkins job from Hadoop to hbase
[
https://issues.apache.org/jira/browse/HBASE-25134?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17206008#comment-17206008
]
Hudson commented on HBASE-25134:
Results for branch branch-2
[build #64 on
builds.a.o|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/branch-2/64/]:
(x) *{color:red}-1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/branch-2/64/General_20Nightly_20Build_20Report/]
(x) {color:red}-1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/branch-2/64/JDK8_20Nightly_20Build_20Report_20_28Hadoop2_29/]
(x) {color:red}-1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/branch-2/64/JDK8_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(x) {color:red}-1 jdk11 hadoop3 checks{color}
-- For more information [see jdk11
report|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/branch-2/64/JDK11_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
(x) {color:red}-1 client integration test{color}
--Failed when running client tests on top of Hadoop 2. [see log for
details|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/branch-2/64//artifact/output-integration/hadoop-2.log].
(note that this means we didn't run on Hadoop 3)
> Migrate HBase PreCommit jenkins job from Hadoop to hbase
>
>
> Key: HBASE-25134
> URL: https://issues.apache.org/jira/browse/HBASE-25134
> Project: HBase
> Issue Type: Sub-task
> Components: scripts
>Reporter: Duo Zhang
>Assignee: Duo Zhang
>Priority: Major
> Fix For: 3.0.0-alpha-1, 2.3.3, 1.7.0, 2.4.0, 1.4.14, 2.2.7
>
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HBASE-25133) Migrate HBase Nightly jenkins job from Hadoop to hbase
[
https://issues.apache.org/jira/browse/HBASE-25133?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17206007#comment-17206007
]
Hudson commented on HBASE-25133:
Results for branch branch-2
[build #64 on
builds.a.o|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/branch-2/64/]:
(x) *{color:red}-1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/branch-2/64/General_20Nightly_20Build_20Report/]
(x) {color:red}-1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/branch-2/64/JDK8_20Nightly_20Build_20Report_20_28Hadoop2_29/]
(x) {color:red}-1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/branch-2/64/JDK8_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(x) {color:red}-1 jdk11 hadoop3 checks{color}
-- For more information [see jdk11
report|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/branch-2/64/JDK11_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
(x) {color:red}-1 client integration test{color}
--Failed when running client tests on top of Hadoop 2. [see log for
details|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/branch-2/64//artifact/output-integration/hadoop-2.log].
(note that this means we didn't run on Hadoop 3)
> Migrate HBase Nightly jenkins job from Hadoop to hbase
> --
>
> Key: HBASE-25133
> URL: https://issues.apache.org/jira/browse/HBASE-25133
> Project: HBase
> Issue Type: Sub-task
> Components: scripts
>Reporter: Duo Zhang
>Assignee: Duo Zhang
>Priority: Major
> Fix For: 3.0.0-alpha-1, 2.3.3, 1.7.0, 2.4.0, 1.4.14, 2.2.7
>
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[GitHub] [hbase] Apache-HBase commented on pull request #2437: HBASE-25053 WAL replay should ignore 0-length files
Apache-HBase commented on pull request #2437: URL: https://github.com/apache/hbase/pull/2437#issuecomment-702529905 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 0m 29s | Docker mode activated. | | -0 :warning: | yetus | 0m 3s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck | ||| _ Prechecks _ | ||| _ master Compile Tests _ | | +1 :green_heart: | mvninstall | 4m 44s | master passed | | +1 :green_heart: | compile | 1m 14s | master passed | | +1 :green_heart: | shadedjars | 7m 12s | branch has no errors when building our shaded downstream artifacts. | | -0 :warning: | javadoc | 0m 47s | hbase-server in master failed. | ||| _ Patch Compile Tests _ | | +1 :green_heart: | mvninstall | 4m 58s | the patch passed | | +1 :green_heart: | compile | 1m 21s | the patch passed | | +1 :green_heart: | javac | 1m 21s | the patch passed | | +1 :green_heart: | shadedjars | 8m 38s | patch has no errors when building our shaded downstream artifacts. | | -0 :warning: | javadoc | 0m 49s | hbase-server in the patch failed. | ||| _ Other Tests _ | | +1 :green_heart: | unit | 140m 18s | hbase-server in the patch passed. | | | | 172m 40s | | | Subsystem | Report/Notes | |--:|:-| | Docker | Client=19.03.13 Server=19.03.13 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2437/4/artifact/yetus-jdk11-hadoop3-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/2437 | | Optional Tests | javac javadoc unit shadedjars compile | | uname | Linux 20b6853860ee 4.15.0-58-generic #64-Ubuntu SMP Tue Aug 6 11:12:41 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / 3b91a15183 | | Default Java | 2020-01-14 | | javadoc | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2437/4/artifact/yetus-jdk11-hadoop3-check/output/branch-javadoc-hbase-server.txt | | javadoc | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2437/4/artifact/yetus-jdk11-hadoop3-check/output/patch-javadoc-hbase-server.txt | | Test Results | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2437/4/testReport/ | | Max. process+thread count | 4114 (vs. ulimit of 3) | | modules | C: hbase-server U: hbase-server | | Console output | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2437/4/console | | versions | git=2.17.1 maven=(cecedd343002696d0abb50b32b541b8a6ba2883f) | | Powered by | Apache Yetus 0.11.1 https://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #2437: HBASE-25053 WAL replay should ignore 0-length files
Apache-HBase commented on pull request #2437: URL: https://github.com/apache/hbase/pull/2437#issuecomment-702529396 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 1m 8s | Docker mode activated. | | -0 :warning: | yetus | 0m 3s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck | ||| _ Prechecks _ | ||| _ master Compile Tests _ | | +1 :green_heart: | mvninstall | 3m 34s | master passed | | +1 :green_heart: | compile | 0m 59s | master passed | | +1 :green_heart: | shadedjars | 6m 39s | branch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 0m 36s | master passed | ||| _ Patch Compile Tests _ | | +1 :green_heart: | mvninstall | 3m 33s | the patch passed | | +1 :green_heart: | compile | 0m 56s | the patch passed | | +1 :green_heart: | javac | 0m 56s | the patch passed | | +1 :green_heart: | shadedjars | 6m 39s | patch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 0m 35s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | unit | 144m 41s | hbase-server in the patch passed. | | | | 171m 15s | | | Subsystem | Report/Notes | |--:|:-| | Docker | Client=19.03.13 Server=19.03.13 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2437/4/artifact/yetus-jdk8-hadoop3-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/2437 | | Optional Tests | javac javadoc unit shadedjars compile | | uname | Linux 28815b2c9297 4.15.0-65-generic #74-Ubuntu SMP Tue Sep 17 17:06:04 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / 3b91a15183 | | Default Java | 1.8.0_232 | | Test Results | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2437/4/testReport/ | | Max. process+thread count | 3926 (vs. ulimit of 3) | | modules | C: hbase-server U: hbase-server | | Console output | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2437/4/console | | versions | git=2.17.1 maven=(cecedd343002696d0abb50b32b541b8a6ba2883f) | | Powered by | Apache Yetus 0.11.1 https://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Commented] (HBASE-25142) Auto-fix 'Unknown Server'
[ https://issues.apache.org/jira/browse/HBASE-25142?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17205978#comment-17205978 ] Anoop Sam John commented on HBASE-25142: cc [~taklo...@gmail.com], [~zyork] > Auto-fix 'Unknown Server' > - > > Key: HBASE-25142 > URL: https://issues.apache.org/jira/browse/HBASE-25142 > Project: HBase > Issue Type: Improvement >Reporter: Michael Stack >Priority: Major > > Addressing reports of 'Unknown Server' has come up in various conversations > lately. This issue is about fixing instances of 'Unknown Server' > automatically as part of the tasks undertaken by CatalogJanitor when it runs. > First though, would like to figure a definition for 'Unknown Server' and a > list of ways in which they arise. We need this to figure how to do safe > auto-fixing. > Currently an 'Unknown Server' is a server found in hbase:meta that is not > online (no recent heartbeat) and that is not mentioned in the dead servers > list. > In outline, I'd think CatalogJanitor could schedule an expiration of the RS > znode in zk (if exists) and then an SCP if it finds an 'Unknown Server'. > Perhaps it waits for 2x or 10x the heartbeat interval just-in-case (or not). > The SCP would clean up any references in hbase:meta by reassigning Regions > assigned the 'Unknown Server' after replaying any WALs found in hdfs > attributed to the dead server. > As to how they arise: > * A contrived illustration would be a large online cluster crashes down with > a massive backlog of WAL files – zk went down for some reason say. The replay > of the WALs look like it could take a very long time (lets say the cluster > was badly configured and a bug and misconfig made it so each RS was carrying > hundreds of WALs and there are hundreds of servers). To get the service back > online, the procedure store and WALs are moved aside (for later replay with > WALPlayer). The cluster comes up. meta is onlined but refers to server > instances that are no longer around. Can schedule an SCP per server mentioned > in the 'HBCK Report' by scraping and scripting hbck2 or, better, > catalogjanitor could just do it. > * HBASE-24286 HMaster won't become healthy after after cloning... describes > starting a cluster over data that is hfile-content only. In this case the > original servers used manufacture the hfile cluster data are long dead yet > meta still refers to the old servers. They will not make the 'dead servers' > list. > Let this issue stew awhile. Meantime collect how 'Unknown Server' gets > created and best way to fix. > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (HBASE-25135) Convert the internal seperator while emitting the memstore read metrics to #
[
https://issues.apache.org/jira/browse/HBASE-25135?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17205976#comment-17205976
]
Hudson commented on HBASE-25135:
Results for branch branch-2.2
[build #76 on
builds.a.o|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/branch-2.2/76/]:
(x) *{color:red}-1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/branch-2.2/76//General_Nightly_Build_Report/]
(x) {color:red}-1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/branch-2.2/76//JDK8_Nightly_Build_Report_(Hadoop2)/]
(x) {color:red}-1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/branch-2.2/76//JDK8_Nightly_Build_Report_(Hadoop3)/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
(x) {color:red}-1 client integration test{color}
--Failed when running client tests on top of Hadoop 2. [see log for
details|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/branch-2.2/76//artifact/output-integration/hadoop-2.log].
(note that this means we didn't run on Hadoop 3)
> Convert the internal seperator while emitting the memstore read metrics to #
>
>
> Key: HBASE-25135
> URL: https://issues.apache.org/jira/browse/HBASE-25135
> Project: HBase
> Issue Type: Bug
>Reporter: ramkrishna.s.vasudevan
>Assignee: ramkrishna.s.vasudevan
>Priority: Minor
> Fix For: 3.0.0-alpha-1, 2.3.3, 2.4.0, 2.2.7
>
>
> Convert the '_' seperator while forming the metric for memstore reads and
> mixed reads to '#'.
> This will avoid cases where the column family itself might have '_' in them.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[GitHub] [hbase] Apache-HBase commented on pull request #2451: HBASE-25055 Add ReplicationSource for meta WALs; add enable/disable w…
Apache-HBase commented on pull request #2451: URL: https://github.com/apache/hbase/pull/2451#issuecomment-702516156 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 1m 2s | Docker mode activated. | ||| _ Prechecks _ | | +1 :green_heart: | dupname | 0m 0s | No case conflicting files found. | | +1 :green_heart: | hbaseanti | 0m 0s | Patch does not have any anti-patterns. | | +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. | ||| _ HBASE-18070 Compile Tests _ | | +1 :green_heart: | mvninstall | 3m 46s | HBASE-18070 passed | | +1 :green_heart: | checkstyle | 1m 8s | HBASE-18070 passed | | +1 :green_heart: | spotbugs | 2m 12s | HBASE-18070 passed | | -0 :warning: | patch | 2m 21s | Used diff version of patch file. Binary files and potentially other changes not applied. Please rebase and squash commits if necessary. | ||| _ Patch Compile Tests _ | | +1 :green_heart: | mvninstall | 3m 37s | the patch passed | | +1 :green_heart: | checkstyle | 1m 8s | hbase-server: The patch generated 0 new + 42 unchanged - 11 fixed = 42 total (was 53) | | +1 :green_heart: | whitespace | 0m 0s | The patch has no whitespace issues. | | +1 :green_heart: | hadoopcheck | 11m 52s | Patch does not cause any errors with Hadoop 3.1.2 3.2.1. | | +1 :green_heart: | spotbugs | 2m 22s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | asflicense | 0m 14s | The patch does not generate ASF License warnings. | | | | 34m 45s | | | Subsystem | Report/Notes | |--:|:-| | Docker | Client=19.03.13 Server=19.03.13 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2451/6/artifact/yetus-general-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/2451 | | Optional Tests | dupname asflicense spotbugs hadoopcheck hbaseanti checkstyle | | uname | Linux e2aabc6d26a9 4.15.0-112-generic #113-Ubuntu SMP Thu Jul 9 23:41:39 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | HBASE-18070 / 03e8e5b517 | | Max. process+thread count | 94 (vs. ulimit of 3) | | modules | C: hbase-server U: hbase-server | | Console output | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2451/6/console | | versions | git=2.17.1 maven=(cecedd343002696d0abb50b32b541b8a6ba2883f) spotbugs=3.1.12 | | Powered by | Apache Yetus 0.11.1 https://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] saintstack commented on pull request #2451: HBASE-25055 Add ReplicationSource for meta WALs; add enable/disable w…
saintstack commented on pull request #2451: URL: https://github.com/apache/hbase/pull/2451#issuecomment-702506783 @huaxiangsun I added HBASE-25145 "WALReader quits if nothing to replicate (and won't restart)" to look at the issue you found above sir. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Created] (HBASE-25145) WALReader quits if nothing to replicate (and won't restart)
Michael Stack created HBASE-25145:
-
Summary: WALReader quits if nothing to replicate (and won't
restart)
Key: HBASE-25145
URL: https://issues.apache.org/jira/browse/HBASE-25145
Project: HBase
Issue Type: Sub-task
Reporter: Michael Stack
Noticed by [~huaxiangsun] in his review of HBASE-25055 "Add ReplicationSource
for meta WALs; add enable/disable w…"
{quote}bq. Eventually, the meta wal file will be gced and there is no more logs
in the queue. In that case, the walReader thread will quit. When the meta
region is moved back, it does not seem that walReader thread will be restarted.
So it seems that something is broken.
{quote}
This issue is about writing a test to run the above scenario and fix any probs.
found.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[GitHub] [hbase] Apache-HBase commented on pull request #2437: HBASE-25053 WAL replay should ignore 0-length files
Apache-HBase commented on pull request #2437: URL: https://github.com/apache/hbase/pull/2437#issuecomment-702502601 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 1m 10s | Docker mode activated. | ||| _ Prechecks _ | | +1 :green_heart: | dupname | 0m 0s | No case conflicting files found. | | +1 :green_heart: | hbaseanti | 0m 0s | Patch does not have any anti-patterns. | | +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. | ||| _ master Compile Tests _ | | +1 :green_heart: | mvninstall | 4m 41s | master passed | | +1 :green_heart: | checkstyle | 1m 26s | master passed | | +1 :green_heart: | spotbugs | 2m 42s | master passed | ||| _ Patch Compile Tests _ | | +1 :green_heart: | mvninstall | 5m 10s | the patch passed | | +1 :green_heart: | checkstyle | 1m 25s | the patch passed | | +1 :green_heart: | whitespace | 0m 0s | The patch has no whitespace issues. | | +1 :green_heart: | hadoopcheck | 14m 42s | Patch does not cause any errors with Hadoop 3.1.2 3.2.1. | | +1 :green_heart: | spotbugs | 2m 59s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | asflicense | 0m 14s | The patch does not generate ASF License warnings. | | | | 43m 37s | | | Subsystem | Report/Notes | |--:|:-| | Docker | Client=19.03.13 Server=19.03.13 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2437/4/artifact/yetus-general-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/2437 | | Optional Tests | dupname asflicense spotbugs hadoopcheck hbaseanti checkstyle | | uname | Linux c8bcd4e717a6 4.15.0-112-generic #113-Ubuntu SMP Thu Jul 9 23:41:39 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / 3b91a15183 | | Max. process+thread count | 84 (vs. ulimit of 3) | | modules | C: hbase-server U: hbase-server | | Console output | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2437/4/console | | versions | git=2.17.1 maven=(cecedd343002696d0abb50b32b541b8a6ba2883f) spotbugs=3.1.12 | | Powered by | Apache Yetus 0.11.1 https://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] anoopsjohn commented on a change in pull request #2483: HBASE-25026 - Create a metric to track scans that have no start row and/or stop row
anoopsjohn commented on a change in pull request #2483:
URL: https://github.com/apache/hbase/pull/2483#discussion_r498601833
##
File path:
hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/RSRpcServices.java
##
@@ -3498,6 +3501,18 @@ public ScanResponse scan(final RpcController controller,
final ScanRequest reque
throw new ServiceException(e);
}
HRegion region = rsh.r;
+// If the start row is the actual start Row of the region and end row is
actual end row of region
+// or
+// If the stop is not specified or the stop Row is greater than the end
key of the current region
+if ((Bytes.equals(rsh.s.getStartRow(),
region.getRegionInfo().getStartKey())
+ && Bytes.equals(rsh.s.getStopRow(), region.getRegionInfo().getEndKey()))
+ || Bytes.equals(rsh.s.getStopRow(), HConstants.EMPTY_END_ROW)
+ || (Bytes.compareTo(region.getRegionInfo().getEndKey(),
rsh.s.getStopRow()) <= 0
+ && !Bytes.equals(region.getRegionInfo().getEndKey(),
HConstants.EMPTY_END_ROW))) {
+ if(!region.getTableDescriptor().isMetaTable()) {
Review comment:
Is this check going to happen for every scan#next calls? We should do
this only once for 1st req which create the scanner on region. Also will this
code get executed for get calls also?
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [hbase] anoopsjohn commented on a change in pull request #2483: HBASE-25026 - Create a metric to track scans that have no start row and/or stop row
anoopsjohn commented on a change in pull request #2483:
URL: https://github.com/apache/hbase/pull/2483#discussion_r498601592
##
File path:
hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/RSRpcServices.java
##
@@ -3498,6 +3501,18 @@ public ScanResponse scan(final RpcController controller,
final ScanRequest reque
throw new ServiceException(e);
}
HRegion region = rsh.r;
+// If the start row is the actual start Row of the region and end row is
actual end row of region
+// or
+// If the stop is not specified or the stop Row is greater than the end
key of the current region
+if ((Bytes.equals(rsh.s.getStartRow(),
region.getRegionInfo().getStartKey())
+ && Bytes.equals(rsh.s.getStopRow(), region.getRegionInfo().getEndKey()))
+ || Bytes.equals(rsh.s.getStopRow(), HConstants.EMPTY_END_ROW)
+ || (Bytes.compareTo(region.getRegionInfo().getEndKey(),
rsh.s.getStopRow()) <= 0
+ && !Bytes.equals(region.getRegionInfo().getEndKey(),
HConstants.EMPTY_END_ROW))) {
+ if(!region.getTableDescriptor().isMetaTable()) {
Review comment:
Ya better to avoid all system table checks
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [hbase] anoopsjohn commented on a change in pull request #2483: HBASE-25026 - Create a metric to track scans that have no start row and/or stop row
anoopsjohn commented on a change in pull request #2483:
URL: https://github.com/apache/hbase/pull/2483#discussion_r498601395
##
File path:
hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/RegionScanner.java
##
@@ -115,4 +116,12 @@
*/
boolean nextRaw(List result, ScannerContext scannerContext)
throws IOException;
+
+ default byte[] getStartRow() {
Review comment:
We should not be exposing these in this interface. This interface is
exposed to CPs possibly allowing to pollute the keys.
We can add a method which can check whether a full region scan or not. The
RegionScannerImpl having ref to Scan instance as well as Region so it can check
whether full region scan or not and return boolean
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[jira] [Commented] (HBASE-25144) Add Hadoop-3.3.0 to personality hadoopcheck
[ https://issues.apache.org/jira/browse/HBASE-25144?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17205934#comment-17205934 ] Michael Stack commented on HBASE-25144: --- What'll it do to our nightlies? You think we have to build against three hadoop versions on master? > Add Hadoop-3.3.0 to personality hadoopcheck > --- > > Key: HBASE-25144 > URL: https://issues.apache.org/jira/browse/HBASE-25144 > Project: HBase > Issue Type: Task > Components: build, community >Reporter: Nick Dimiduk >Assignee: Nick Dimiduk >Priority: Minor > Fix For: 3.0.0-alpha-1 > > > Now that Hadoop 3.3.0 is released, let's figure out where it goes in our > testing matrix. Start by adding it to precommit checks. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[GitHub] [hbase] Apache-HBase commented on pull request #2163: HBASE-24789 When table level quota is set and violated, pre existing …
Apache-HBase commented on pull request #2163: URL: https://github.com/apache/hbase/pull/2163#issuecomment-702449103 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 1m 24s | Docker mode activated. | | -0 :warning: | yetus | 0m 4s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck | ||| _ Prechecks _ | ||| _ master Compile Tests _ | | +1 :green_heart: | mvninstall | 7m 17s | master passed | | +1 :green_heart: | compile | 1m 55s | master passed | | +1 :green_heart: | shadedjars | 12m 24s | branch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 1m 11s | master passed | ||| _ Patch Compile Tests _ | | +1 :green_heart: | mvninstall | 6m 0s | the patch passed | | +1 :green_heart: | compile | 1m 44s | the patch passed | | +1 :green_heart: | javac | 1m 44s | the patch passed | | +1 :green_heart: | shadedjars | 11m 33s | patch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 0m 57s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | unit | 175m 4s | hbase-server in the patch passed. | | | | 221m 59s | | | Subsystem | Report/Notes | |--:|:-| | Docker | Client=19.03.13 Server=19.03.13 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2163/3/artifact/yetus-jdk8-hadoop3-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/2163 | | Optional Tests | javac javadoc unit shadedjars compile | | uname | Linux a24f977a410c 4.15.0-112-generic #113-Ubuntu SMP Thu Jul 9 23:41:39 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / 3b91a15183 | | Default Java | 1.8.0_232 | | Test Results | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2163/3/testReport/ | | Max. process+thread count | 3389 (vs. ulimit of 3) | | modules | C: hbase-server U: hbase-server | | Console output | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2163/3/console | | versions | git=2.17.1 maven=(cecedd343002696d0abb50b32b541b8a6ba2883f) | | Powered by | Apache Yetus 0.11.1 https://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #2483: HBASE-25026 - Create a metric to track scans that have no start row and/or stop row
Apache-HBase commented on pull request #2483: URL: https://github.com/apache/hbase/pull/2483#issuecomment-702447665 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 0m 31s | Docker mode activated. | | -0 :warning: | yetus | 0m 3s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck | ||| _ Prechecks _ | ||| _ master Compile Tests _ | | +0 :ok: | mvndep | 0m 22s | Maven dependency ordering for branch | | +1 :green_heart: | mvninstall | 4m 41s | master passed | | +1 :green_heart: | compile | 1m 17s | master passed | | +1 :green_heart: | shadedjars | 6m 57s | branch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 0m 59s | master passed | ||| _ Patch Compile Tests _ | | +0 :ok: | mvndep | 0m 16s | Maven dependency ordering for patch | | +1 :green_heart: | mvninstall | 3m 48s | the patch passed | | +1 :green_heart: | compile | 1m 16s | the patch passed | | +1 :green_heart: | javac | 1m 16s | the patch passed | | -1 :x: | shadedjars | 5m 34s | patch has 10 errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 0m 55s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | unit | 0m 32s | hbase-hadoop-compat in the patch passed. | | -1 :x: | unit | 137m 40s | hbase-server in the patch failed. | | | | 166m 56s | | | Subsystem | Report/Notes | |--:|:-| | Docker | Client=19.03.13 Server=19.03.13 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2483/2/artifact/yetus-jdk8-hadoop3-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/2483 | | Optional Tests | javac javadoc unit shadedjars compile | | uname | Linux 2b259db3ff40 4.15.0-58-generic #64-Ubuntu SMP Tue Aug 6 11:12:41 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / 3b91a15183 | | Default Java | 1.8.0_232 | | shadedjars | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2483/2/artifact/yetus-jdk8-hadoop3-check/output/patch-shadedjars.txt | | unit | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2483/2/artifact/yetus-jdk8-hadoop3-check/output/patch-unit-hbase-server.txt | | Test Results | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2483/2/testReport/ | | Max. process+thread count | 4224 (vs. ulimit of 3) | | modules | C: hbase-hadoop-compat hbase-server U: . | | Console output | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2483/2/console | | versions | git=2.17.1 maven=(cecedd343002696d0abb50b32b541b8a6ba2883f) | | Powered by | Apache Yetus 0.11.1 https://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] ndimiduk commented on a change in pull request #2163: HBASE-24789 When table level quota is set and violated, pre existing …
ndimiduk commented on a change in pull request #2163:
URL: https://github.com/apache/hbase/pull/2163#discussion_r498557910
##
File path:
hbase-server/src/main/java/org/apache/hadoop/hbase/master/MasterServices.java
##
@@ -552,4 +553,9 @@ default SplitWALManager getSplitWALManager(){
* @return The state of the load balancer, or false if the load balancer
isn't defined.
*/
boolean isBalancerOn();
+
+ /**
+ * @return the {@link QuotaObserverChore}
+ */
+ public QuotaObserverChore getQuotaObserverChore();
Review comment:
@saintstack , @apurtell , @Apache9 can you shed some light on what
criteria is used to decide what gets added to `MasterServices`? I have similar
questions for my own PRs. Thanks.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [hbase] ndimiduk commented on a change in pull request #2163: HBASE-24789 When table level quota is set and violated, pre existing …
ndimiduk commented on a change in pull request #2163:
URL: https://github.com/apache/hbase/pull/2163#discussion_r498557359
##
File path:
hbase-server/src/main/java/org/apache/hadoop/hbase/quotas/QuotaObserverChore.java
##
@@ -566,6 +566,13 @@ void setTableQuotaSnapshot(TableName table,
SpaceQuotaSnapshot snapshot) {
this.tableQuotaSnapshots.put(table, snapshot);
}
+ /**
+ * Removes the quota state for the given table.
+ */
+ void removeTableQuotasnapshot(TableName table) {
+this.tableQuotaSnapshots.remove(table);
Review comment:
I think you need some synchronization around modifications to this data
structure. If I understand correctly, the Chore will be run in it's own thread,
while quota changes would be dispatched from an IPC handler thread.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #2483: HBASE-25026 - Create a metric to track scans that have no start row and/or stop row
Apache-HBase commented on pull request #2483: URL: https://github.com/apache/hbase/pull/2483#issuecomment-702444905 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 0m 34s | Docker mode activated. | | -0 :warning: | yetus | 0m 3s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck | ||| _ Prechecks _ | ||| _ master Compile Tests _ | | +0 :ok: | mvndep | 0m 23s | Maven dependency ordering for branch | | +1 :green_heart: | mvninstall | 5m 0s | master passed | | +1 :green_heart: | compile | 1m 29s | master passed | | +1 :green_heart: | shadedjars | 7m 14s | branch has no errors when building our shaded downstream artifacts. | | -0 :warning: | javadoc | 0m 21s | hbase-hadoop-compat in master failed. | | -0 :warning: | javadoc | 0m 41s | hbase-server in master failed. | ||| _ Patch Compile Tests _ | | +0 :ok: | mvndep | 0m 15s | Maven dependency ordering for patch | | +1 :green_heart: | mvninstall | 4m 13s | the patch passed | | +1 :green_heart: | compile | 1m 27s | the patch passed | | +1 :green_heart: | javac | 1m 27s | the patch passed | | -1 :x: | shadedjars | 5m 30s | patch has 10 errors when building our shaded downstream artifacts. | | -0 :warning: | javadoc | 0m 20s | hbase-hadoop-compat in the patch failed. | | -0 :warning: | javadoc | 0m 40s | hbase-server in the patch failed. | ||| _ Other Tests _ | | +1 :green_heart: | unit | 0m 37s | hbase-hadoop-compat in the patch passed. | | +1 :green_heart: | unit | 127m 4s | hbase-server in the patch passed. | | | | 158m 15s | | | Subsystem | Report/Notes | |--:|:-| | Docker | Client=19.03.13 Server=19.03.13 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2483/2/artifact/yetus-jdk11-hadoop3-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/2483 | | Optional Tests | javac javadoc unit shadedjars compile | | uname | Linux d8c5d3f8ff0a 4.15.0-60-generic #67-Ubuntu SMP Thu Aug 22 16:55:30 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / 3b91a15183 | | Default Java | 2020-01-14 | | javadoc | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2483/2/artifact/yetus-jdk11-hadoop3-check/output/branch-javadoc-hbase-hadoop-compat.txt | | javadoc | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2483/2/artifact/yetus-jdk11-hadoop3-check/output/branch-javadoc-hbase-server.txt | | shadedjars | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2483/2/artifact/yetus-jdk11-hadoop3-check/output/patch-shadedjars.txt | | javadoc | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2483/2/artifact/yetus-jdk11-hadoop3-check/output/patch-javadoc-hbase-hadoop-compat.txt | | javadoc | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2483/2/artifact/yetus-jdk11-hadoop3-check/output/patch-javadoc-hbase-server.txt | | Test Results | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2483/2/testReport/ | | Max. process+thread count | 4522 (vs. ulimit of 3) | | modules | C: hbase-hadoop-compat hbase-server U: . | | Console output | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2483/2/console | | versions | git=2.17.1 maven=(cecedd343002696d0abb50b32b541b8a6ba2883f) | | Powered by | Apache Yetus 0.11.1 https://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #2490: HBASE-24419 Normalizer merge plans should consider more than 2 regions when possible
Apache-HBase commented on pull request #2490: URL: https://github.com/apache/hbase/pull/2490#issuecomment-702439630 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 1m 26s | Docker mode activated. | | -0 :warning: | yetus | 0m 4s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck | ||| _ Prechecks _ | ||| _ master Compile Tests _ | | +0 :ok: | mvndep | 0m 27s | Maven dependency ordering for branch | | +1 :green_heart: | mvninstall | 6m 37s | master passed | | +1 :green_heart: | compile | 1m 56s | master passed | | +1 :green_heart: | shadedjars | 12m 11s | branch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 1m 53s | master passed | ||| _ Patch Compile Tests _ | | +0 :ok: | mvndep | 0m 24s | Maven dependency ordering for patch | | +1 :green_heart: | mvninstall | 7m 4s | the patch passed | | +1 :green_heart: | compile | 2m 27s | the patch passed | | +1 :green_heart: | javac | 2m 27s | the patch passed | | +1 :green_heart: | shadedjars | 12m 9s | patch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 1m 43s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | unit | 2m 30s | hbase-common in the patch passed. | | -1 :x: | unit | 283m 40s | hbase-server in the patch failed. | | | | 337m 24s | | | Subsystem | Report/Notes | |--:|:-| | Docker | Client=19.03.13 Server=19.03.13 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2490/1/artifact/yetus-jdk8-hadoop3-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/2490 | | Optional Tests | javac javadoc unit shadedjars compile | | uname | Linux a80372a1c248 4.15.0-112-generic #113-Ubuntu SMP Thu Jul 9 23:41:39 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / 3b91a15183 | | Default Java | 1.8.0_232 | | unit | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2490/1/artifact/yetus-jdk8-hadoop3-check/output/patch-unit-hbase-server.txt | | Test Results | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2490/1/testReport/ | | Max. process+thread count | 2756 (vs. ulimit of 3) | | modules | C: hbase-common hbase-server U: . | | Console output | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2490/1/console | | versions | git=2.17.1 maven=(cecedd343002696d0abb50b32b541b8a6ba2883f) | | Powered by | Apache Yetus 0.11.1 https://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Comment Edited] (HBASE-25129) serial replication, addReplicationBarrier is writing to rep_barrier family even there is no serial replication peer.
[ https://issues.apache.org/jira/browse/HBASE-25129?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17205832#comment-17205832 ] Huaxiang Sun edited comment on HBASE-25129 at 10/1/20, 10:57 PM: - Thanks Duo for the explain. This issue has nothing to do with meta replica. We have one table, which has the replication scope set to 1 by accident, and actually, there is no replication peer (i.e, no replication turned on at that cluster). In this case, I was expecting that rep_barrier has 0 cells, and found that there are quite some cells in this family. While this is not causing serious issue so far, it will be great to avoid if it is not necessary. Some more background, the cluster was recently upgraded from 1.2 to 2.3, this was discovered while we were checking meta directory. two followup questions without knowing more details about serial replication design. # If serial replication peer is added after region open (without writing to rep_barrier first), will it break serial replication? Seem so from your description. 2. Can we introduce a config knob to turn on/off serial replication support so it can simply check this knob and replication scope to decide if it needs to write to rep_barrier family? was (Author: huaxiangsun): Thanks Duo for the explain. This issue has nothing to do with meta replica. We have one table, which has the replication scope set to 1 by accident, and actually, there is no replication peer (i.e, no replication turned on at that cluster). In this case, I was expecting that rep_barrier has 0 cells, and found that there are quite some cells in this family. While this is not causing serious issue so far, it will be great to avoid if it is not necessary. > serial replication, addReplicationBarrier is writing to rep_barrier family > even there is no serial replication peer. > > > Key: HBASE-25129 > URL: https://issues.apache.org/jira/browse/HBASE-25129 > Project: HBase > Issue Type: Bug >Reporter: Huaxiang Sun >Assignee: Huaxiang Sun >Priority: Major > > We found that there are quite some data in rep_barrier family even there is > no serial replication enabled. Checked the code, it is checking if table has > replication enabled. Think there is another check needed (i.e, is there any > serial replication peers configured). > [https://github.com/apache/hbase/blob/master/hbase-server/src/main/java/org/apache/hadoop/hbase/master/assignment/RegionStateStore.java#L215] > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[GitHub] [hbase] Apache-HBase commented on pull request #2163: HBASE-24789 When table level quota is set and violated, pre existing …
Apache-HBase commented on pull request #2163: URL: https://github.com/apache/hbase/pull/2163#issuecomment-702431445 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 1m 51s | Docker mode activated. | | -0 :warning: | yetus | 0m 3s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck | ||| _ Prechecks _ | ||| _ master Compile Tests _ | | +1 :green_heart: | mvninstall | 5m 35s | master passed | | +1 :green_heart: | compile | 1m 23s | master passed | | +1 :green_heart: | shadedjars | 8m 7s | branch has no errors when building our shaded downstream artifacts. | | -0 :warning: | javadoc | 0m 48s | hbase-server in master failed. | ||| _ Patch Compile Tests _ | | +1 :green_heart: | mvninstall | 4m 34s | the patch passed | | +1 :green_heart: | compile | 1m 17s | the patch passed | | +1 :green_heart: | javac | 1m 17s | the patch passed | | +1 :green_heart: | shadedjars | 6m 54s | patch has no errors when building our shaded downstream artifacts. | | -0 :warning: | javadoc | 0m 39s | hbase-server in the patch failed. | ||| _ Other Tests _ | | +1 :green_heart: | unit | 130m 29s | hbase-server in the patch passed. | | | | 163m 49s | | | Subsystem | Report/Notes | |--:|:-| | Docker | Client=19.03.13 Server=19.03.13 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2163/3/artifact/yetus-jdk11-hadoop3-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/2163 | | Optional Tests | javac javadoc unit shadedjars compile | | uname | Linux 9582930c65f6 4.15.0-65-generic #74-Ubuntu SMP Tue Sep 17 17:06:04 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / 3b91a15183 | | Default Java | 2020-01-14 | | javadoc | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2163/3/artifact/yetus-jdk11-hadoop3-check/output/branch-javadoc-hbase-server.txt | | javadoc | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2163/3/artifact/yetus-jdk11-hadoop3-check/output/patch-javadoc-hbase-server.txt | | Test Results | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2163/3/testReport/ | | Max. process+thread count | 4065 (vs. ulimit of 3) | | modules | C: hbase-server U: hbase-server | | Console output | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2163/3/console | | versions | git=2.17.1 maven=(cecedd343002696d0abb50b32b541b8a6ba2883f) | | Powered by | Apache Yetus 0.11.1 https://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #2483: HBASE-25026 - Create a metric to track scans that have no start row and/or stop row
Apache-HBase commented on pull request #2483: URL: https://github.com/apache/hbase/pull/2483#issuecomment-702407553 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 1m 57s | Docker mode activated. | ||| _ Prechecks _ | | +1 :green_heart: | dupname | 0m 0s | No case conflicting files found. | | +1 :green_heart: | hbaseanti | 0m 0s | Patch does not have any anti-patterns. | | +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. | ||| _ master Compile Tests _ | | +0 :ok: | mvndep | 0m 25s | Maven dependency ordering for branch | | +1 :green_heart: | mvninstall | 5m 16s | master passed | | +1 :green_heart: | checkstyle | 1m 45s | master passed | | +1 :green_heart: | spotbugs | 3m 22s | master passed | ||| _ Patch Compile Tests _ | | +0 :ok: | mvndep | 0m 14s | Maven dependency ordering for patch | | +1 :green_heart: | mvninstall | 4m 45s | the patch passed | | -0 :warning: | checkstyle | 0m 16s | hbase-hadoop-compat: The patch generated 1 new + 1 unchanged - 0 fixed = 2 total (was 1) | | -0 :warning: | checkstyle | 1m 24s | hbase-server: The patch generated 7 new + 266 unchanged - 0 fixed = 273 total (was 266) | | -0 :warning: | whitespace | 0m 0s | The patch has 1 line(s) that end in whitespace. Use git apply --whitespace=fix <>. Refer https://git-scm.com/docs/git-apply | | +1 :green_heart: | hadoopcheck | 15m 5s | Patch does not cause any errors with Hadoop 3.1.2 3.2.1. | | +1 :green_heart: | spotbugs | 3m 38s | the patch passed | ||| _ Other Tests _ | | -1 :x: | asflicense | 0m 26s | The patch generated 1 ASF License warnings. | | | | 48m 37s | | | Subsystem | Report/Notes | |--:|:-| | Docker | Client=19.03.13 Server=19.03.13 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2483/2/artifact/yetus-general-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/2483 | | Optional Tests | dupname asflicense spotbugs hadoopcheck hbaseanti checkstyle | | uname | Linux 4e34f33d39d3 4.15.0-65-generic #74-Ubuntu SMP Tue Sep 17 17:06:04 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / 3b91a15183 | | checkstyle | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2483/2/artifact/yetus-general-check/output/diff-checkstyle-hbase-hadoop-compat.txt | | checkstyle | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2483/2/artifact/yetus-general-check/output/diff-checkstyle-hbase-server.txt | | whitespace | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2483/2/artifact/yetus-general-check/output/whitespace-eol.txt | | asflicense | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2483/2/artifact/yetus-general-check/output/patch-asflicense-problems.txt | | Max. process+thread count | 94 (vs. ulimit of 3) | | modules | C: hbase-hadoop-compat hbase-server U: . | | Console output | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2483/2/console | | versions | git=2.17.1 maven=(cecedd343002696d0abb50b32b541b8a6ba2883f) spotbugs=3.1.12 | | Powered by | Apache Yetus 0.11.1 https://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Commented] (HBASE-25129) serial replication, addReplicationBarrier is writing to rep_barrier family even there is no serial replication peer.
[ https://issues.apache.org/jira/browse/HBASE-25129?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17205832#comment-17205832 ] Huaxiang Sun commented on HBASE-25129: -- Thanks Duo for the explain. This issue has nothing to do with meta replica. We have one table, which has the replication scope set to 1 by accident, and actually, there is no replication peer (i.e, no replication turned on at that cluster). In this case, I was expecting that rep_barrier has 0 cells, and found that there are quite some cells in this family. While this is not causing serious issue so far, it will be great to avoid if it is not necessary. > serial replication, addReplicationBarrier is writing to rep_barrier family > even there is no serial replication peer. > > > Key: HBASE-25129 > URL: https://issues.apache.org/jira/browse/HBASE-25129 > Project: HBase > Issue Type: Bug >Reporter: Huaxiang Sun >Assignee: Huaxiang Sun >Priority: Major > > We found that there are quite some data in rep_barrier family even there is > no serial replication enabled. Checked the code, it is checking if table has > replication enabled. Think there is another check needed (i.e, is there any > serial replication peers configured). > [https://github.com/apache/hbase/blob/master/hbase-server/src/main/java/org/apache/hadoop/hbase/master/assignment/RegionStateStore.java#L215] > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[GitHub] [hbase] Apache-HBase commented on pull request #2491: HBASE-25143 Remove branch-1.3 from precommit and docs
Apache-HBase commented on pull request #2491: URL: https://github.com/apache/hbase/pull/2491#issuecomment-702392884 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 0m 28s | Docker mode activated. | ||| _ Prechecks _ | | +1 :green_heart: | dupname | 0m 0s | No case conflicting files found. | | +0 :ok: | shelldocs | 0m 0s | Shelldocs was not available. | | +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. | ||| _ master Compile Tests _ | | +1 :green_heart: | mvninstall | 4m 5s | master passed | | +0 :ok: | refguide | 5m 39s | branch has no errors when building the reference guide. See footer for rendered docs, which you should manually inspect. | ||| _ Patch Compile Tests _ | | +1 :green_heart: | mvninstall | 3m 36s | the patch passed | | +1 :green_heart: | shellcheck | 0m 2s | There were no new shellcheck issues. | | +1 :green_heart: | whitespace | 0m 0s | The patch has no whitespace issues. | | +0 :ok: | refguide | 5m 19s | patch has no errors when building the reference guide. See footer for rendered docs, which you should manually inspect. | ||| _ Other Tests _ | | +1 :green_heart: | asflicense | 0m 18s | The patch does not generate ASF License warnings. | | | | 21m 0s | | | Subsystem | Report/Notes | |--:|:-| | Docker | Client=19.03.13 Server=19.03.13 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2491/1/artifact/yetus-general-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/2491 | | Optional Tests | dupname asflicense shellcheck shelldocs refguide | | uname | Linux d55f47be1b56 4.15.0-60-generic #67-Ubuntu SMP Thu Aug 22 16:55:30 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / 3b91a15183 | | refguide | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2491/1/artifact/yetus-general-check/output/branch-site/book.html | | refguide | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2491/1/artifact/yetus-general-check/output/patch-site/book.html | | Max. process+thread count | 78 (vs. ulimit of 3) | | modules | C: . U: . | | Console output | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2491/1/console | | versions | git=2.17.1 maven=(cecedd343002696d0abb50b32b541b8a6ba2883f) shellcheck=0.4.6 | | Powered by | Apache Yetus 0.11.1 https://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #2492: HBASE-25144 Add Hadoop-3.3.0 to precommit check list
Apache-HBase commented on pull request #2492: URL: https://github.com/apache/hbase/pull/2492#issuecomment-702390010 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 3m 1s | Docker mode activated. | | -0 :warning: | yetus | 0m 3s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck | ||| _ Prechecks _ | ||| _ master Compile Tests _ | | +0 :ok: | mvndep | 0m 7s | Maven dependency ordering for branch | ||| _ Patch Compile Tests _ | | +0 :ok: | mvndep | 0m 7s | Maven dependency ordering for patch | ||| _ Other Tests _ | | | | 4m 9s | | | Subsystem | Report/Notes | |--:|:-| | Docker | Client=19.03.13 Server=19.03.13 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2492/2/artifact/yetus-jdk8-hadoop3-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/2492 | | Optional Tests | | | uname | Linux 1551429452e8 4.15.0-112-generic #113-Ubuntu SMP Thu Jul 9 23:41:39 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / 3b91a15183 | | Max. process+thread count | 43 (vs. ulimit of 3) | | modules | C: U: | | Console output | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2492/2/console | | versions | git=2.17.1 maven=(cecedd343002696d0abb50b32b541b8a6ba2883f) | | Powered by | Apache Yetus 0.11.1 https://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #2492: HBASE-25144 Add Hadoop-3.3.0 to precommit check list
Apache-HBase commented on pull request #2492: URL: https://github.com/apache/hbase/pull/2492#issuecomment-702389502 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 1m 13s | Docker mode activated. | | -0 :warning: | yetus | 0m 3s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck | ||| _ Prechecks _ | ||| _ master Compile Tests _ | | +0 :ok: | mvndep | 0m 12s | Maven dependency ordering for branch | ||| _ Patch Compile Tests _ | | +0 :ok: | mvndep | 0m 14s | Maven dependency ordering for patch | ||| _ Other Tests _ | | | | 2m 38s | | | Subsystem | Report/Notes | |--:|:-| | Docker | Client=19.03.13 Server=19.03.13 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2492/2/artifact/yetus-jdk11-hadoop3-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/2492 | | Optional Tests | | | uname | Linux a4c944c60df1 4.15.0-112-generic #113-Ubuntu SMP Thu Jul 9 23:41:39 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / 3b91a15183 | | Max. process+thread count | 46 (vs. ulimit of 3) | | modules | C: U: | | Console output | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2492/2/console | | versions | git=2.17.1 maven=(cecedd343002696d0abb50b32b541b8a6ba2883f) | | Powered by | Apache Yetus 0.11.1 https://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #2492: HBASE-25144 Add Hadoop-3.3.0 to precommit check list
Apache-HBase commented on pull request #2492: URL: https://github.com/apache/hbase/pull/2492#issuecomment-702388925 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 0m 28s | Docker mode activated. | ||| _ Prechecks _ | | +1 :green_heart: | dupname | 0m 0s | No case conflicting files found. | | +0 :ok: | shelldocs | 0m 0s | Shelldocs was not available. | | +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. | ||| _ master Compile Tests _ | | +0 :ok: | mvndep | 0m 8s | Maven dependency ordering for branch | ||| _ Patch Compile Tests _ | | +0 :ok: | mvndep | 0m 8s | Maven dependency ordering for patch | | +1 :green_heart: | shellcheck | 0m 1s | There were no new shellcheck issues. | | +1 :green_heart: | whitespace | 0m 0s | The patch has no whitespace issues. | ||| _ Other Tests _ | | +0 :ok: | asflicense | 0m 0s | ASF License check generated no output? | | | | 1m 54s | | | Subsystem | Report/Notes | |--:|:-| | Docker | Client=19.03.13 Server=19.03.13 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2492/2/artifact/yetus-general-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/2492 | | Optional Tests | dupname asflicense shellcheck shelldocs | | uname | Linux 6c542bf3ce64 4.15.0-60-generic #67-Ubuntu SMP Thu Aug 22 16:55:30 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / 3b91a15183 | | Max. process+thread count | 48 (vs. ulimit of 3) | | modules | C: U: | | Console output | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2492/2/console | | versions | git=2.17.1 maven=(cecedd343002696d0abb50b32b541b8a6ba2883f) shellcheck=0.4.6 | | Powered by | Apache Yetus 0.11.1 https://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #2492: HBASE-25144 Add Hadoop-3.3.0 to precommit check list
Apache-HBase commented on pull request #2492: URL: https://github.com/apache/hbase/pull/2492#issuecomment-702388577 (!) A patch to the testing environment has been detected. Re-executing against the patched versions to perform further tests. The console is at https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2492/2/console in case of problems. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #2492: HBASE-25144 Add Hadoop-3.3.0 to precommit check list
Apache-HBase commented on pull request #2492: URL: https://github.com/apache/hbase/pull/2492#issuecomment-702388345 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #2492: HBASE-25144 Add Hadoop-3.3.0 to precommit check list
Apache-HBase commented on pull request #2492: URL: https://github.com/apache/hbase/pull/2492#issuecomment-702386616 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 1m 4s | Docker mode activated. | | -0 :warning: | yetus | 0m 4s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck | ||| _ Prechecks _ | ||| _ master Compile Tests _ | | +0 :ok: | mvndep | 1m 6s | Maven dependency ordering for branch | ||| _ Patch Compile Tests _ | | +0 :ok: | mvndep | 0m 13s | Maven dependency ordering for patch | ||| _ Other Tests _ | | | | 3m 19s | | | Subsystem | Report/Notes | |--:|:-| | Docker | Client=19.03.13 Server=19.03.13 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2492/1/artifact/yetus-jdk11-hadoop3-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/2492 | | Optional Tests | | | uname | Linux e5ae9a3f70eb 4.15.0-112-generic #113-Ubuntu SMP Thu Jul 9 23:41:39 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / 3b91a15183 | | Max. process+thread count | 51 (vs. ulimit of 3) | | modules | C: U: | | Console output | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2492/1/console | | versions | git=2.17.1 maven=(cecedd343002696d0abb50b32b541b8a6ba2883f) | | Powered by | Apache Yetus 0.11.1 https://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #2492: HBASE-25144 Add Hadoop-3.3.0 to precommit check list
Apache-HBase commented on pull request #2492: URL: https://github.com/apache/hbase/pull/2492#issuecomment-702386220 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 1m 29s | Docker mode activated. | | -0 :warning: | yetus | 0m 4s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck | ||| _ Prechecks _ | ||| _ master Compile Tests _ | | +0 :ok: | mvndep | 0m 33s | Maven dependency ordering for branch | ||| _ Patch Compile Tests _ | | +0 :ok: | mvndep | 0m 8s | Maven dependency ordering for patch | ||| _ Other Tests _ | | | | 3m 6s | | | Subsystem | Report/Notes | |--:|:-| | Docker | Client=19.03.13 Server=19.03.13 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2492/1/artifact/yetus-jdk8-hadoop3-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/2492 | | Optional Tests | | | uname | Linux 321a873aa520 4.15.0-112-generic #113-Ubuntu SMP Thu Jul 9 23:41:39 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / 3b91a15183 | | Max. process+thread count | 47 (vs. ulimit of 3) | | modules | C: U: | | Console output | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2492/1/console | | versions | git=2.17.1 maven=(cecedd343002696d0abb50b32b541b8a6ba2883f) | | Powered by | Apache Yetus 0.11.1 https://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #2492: HBASE-25144 Add Hadoop-3.3.0 to precommit check list
Apache-HBase commented on pull request #2492: URL: https://github.com/apache/hbase/pull/2492#issuecomment-702386102 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 0m 27s | Docker mode activated. | ||| _ Prechecks _ | | +1 :green_heart: | dupname | 0m 0s | No case conflicting files found. | | +0 :ok: | shelldocs | 0m 0s | Shelldocs was not available. | | +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. | ||| _ master Compile Tests _ | | +0 :ok: | mvndep | 0m 50s | Maven dependency ordering for branch | ||| _ Patch Compile Tests _ | | +0 :ok: | mvndep | 0m 9s | Maven dependency ordering for patch | | +1 :green_heart: | shellcheck | 0m 2s | There were no new shellcheck issues. | | +1 :green_heart: | whitespace | 0m 0s | The patch has no whitespace issues. | ||| _ Other Tests _ | | +0 :ok: | asflicense | 0m 0s | ASF License check generated no output? | | | | 2m 33s | | | Subsystem | Report/Notes | |--:|:-| | Docker | Client=19.03.13 Server=19.03.13 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2492/1/artifact/yetus-general-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/2492 | | Optional Tests | dupname asflicense shellcheck shelldocs | | uname | Linux c1f4ddbf00f8 4.15.0-60-generic #67-Ubuntu SMP Thu Aug 22 16:55:30 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / 3b91a15183 | | Max. process+thread count | 52 (vs. ulimit of 3) | | modules | C: U: | | Console output | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2492/1/console | | versions | git=2.17.1 maven=(cecedd343002696d0abb50b32b541b8a6ba2883f) shellcheck=0.4.6 | | Powered by | Apache Yetus 0.11.1 https://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Updated] (HBASE-25144) Add Hadoop-3.3.0 to personality hadoopcheck
[ https://issues.apache.org/jira/browse/HBASE-25144?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nick Dimiduk updated HBASE-25144: - Summary: Add Hadoop-3.3.0 to personality hadoopcheck (was: Add Hadoop-3.3.0 to precommit check list) > Add Hadoop-3.3.0 to personality hadoopcheck > --- > > Key: HBASE-25144 > URL: https://issues.apache.org/jira/browse/HBASE-25144 > Project: HBase > Issue Type: Task > Components: build, community >Reporter: Nick Dimiduk >Assignee: Nick Dimiduk >Priority: Minor > Fix For: 3.0.0-alpha-1 > > > Now that Hadoop 3.3.0 is released, let's figure out where it goes in our > testing matrix. Start by adding it to precommit checks. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (HBASE-25144) Add Hadoop-3.3.0 to precommit check list
[ https://issues.apache.org/jira/browse/HBASE-25144?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17205816#comment-17205816 ] Nick Dimiduk commented on HBASE-25144: -- Apologies: this will impact nightlies as well. > Add Hadoop-3.3.0 to precommit check list > > > Key: HBASE-25144 > URL: https://issues.apache.org/jira/browse/HBASE-25144 > Project: HBase > Issue Type: Task > Components: build, community >Reporter: Nick Dimiduk >Assignee: Nick Dimiduk >Priority: Minor > Fix For: 3.0.0-alpha-1 > > > Now that Hadoop 3.3.0 is released, let's figure out where it goes in our > testing matrix. Start by adding it to precommit checks. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[GitHub] [hbase] Apache-HBase commented on pull request #2492: HBASE-25144 Add Hadoop-3.3.0 to precommit check list
Apache-HBase commented on pull request #2492: URL: https://github.com/apache/hbase/pull/2492#issuecomment-702385215 (!) A patch to the testing environment has been detected. Re-executing against the patched versions to perform further tests. The console is at https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2492/1/console in case of problems. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #2492: HBASE-25144 Add Hadoop-3.3.0 to precommit check list
Apache-HBase commented on pull request #2492: URL: https://github.com/apache/hbase/pull/2492#issuecomment-702385497 (!) A patch to the testing environment has been detected. Re-executing against the patched versions to perform further tests. The console is at https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2492/1/console in case of problems. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #2492: HBASE-25144 Add Hadoop-3.3.0 to precommit check list
Apache-HBase commented on pull request #2492: URL: https://github.com/apache/hbase/pull/2492#issuecomment-702385348 (!) A patch to the testing environment has been detected. Re-executing against the patched versions to perform further tests. The console is at https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2492/1/console in case of problems. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] gkanade commented on a change in pull request #2483: HBASE-25026 - Create a metric to track scans that have no start row and/or stop row
gkanade commented on a change in pull request #2483:
URL: https://github.com/apache/hbase/pull/2483#discussion_r498500913
##
File path:
hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/RSRpcServices.java
##
@@ -3498,6 +3501,18 @@ public ScanResponse scan(final RpcController controller,
final ScanRequest reque
throw new ServiceException(e);
}
HRegion region = rsh.r;
+// If the start row is the actual start Row of the region and end row is
actual end row of region
+// or
+// If the stop is not specified or the stop Row is greater than the end
key of the current region
+if ((Bytes.equals(rsh.s.getStartRow(),
region.getRegionInfo().getStartKey())
+ && Bytes.equals(rsh.s.getStopRow(), region.getRegionInfo().getEndKey()))
+ || Bytes.equals(rsh.s.getStopRow(), HConstants.EMPTY_END_ROW)
+ || (Bytes.compareTo(region.getRegionInfo().getEndKey(),
rsh.s.getStopRow()) <= 0
+ && !Bytes.equals(region.getRegionInfo().getEndKey(),
HConstants.EMPTY_END_ROW))) {
+ if(!region.getTableDescriptor().isMetaTable()) {
Review comment:
As @ramkrish86 mentioned above this is to get insights into possible
inadvertent user errors, hence skipping meta. I will sync with Ram to figure
out if we should exclude other system tables as well and then move the check
accordingly.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[jira] [Work started] (HBASE-25143) Remove branch-1.3 from precommit and docs
[ https://issues.apache.org/jira/browse/HBASE-25143?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Work on HBASE-25143 started by Nick Dimiduk. > Remove branch-1.3 from precommit and docs > - > > Key: HBASE-25143 > URL: https://issues.apache.org/jira/browse/HBASE-25143 > Project: HBase > Issue Type: Task > Components: community >Affects Versions: 3.0.0-alpha-1 >Reporter: Nick Dimiduk >Assignee: Nick Dimiduk >Priority: Minor > > Following the > [announcement|https://lists.apache.org/thread.html/r9552e9085aaac2a43f8b26b866d34825a84a9be7f19118ac560d14de%40%3Cuser.hbase.apache.org%3E] > to EOL branch-1.3, update the precommit script to not consider this branch > any longer, and refresh mentions of this branch in the doc. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Work started] (HBASE-25144) Add Hadoop-3.3.0 to precommit check list
[ https://issues.apache.org/jira/browse/HBASE-25144?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Work on HBASE-25144 started by Nick Dimiduk. > Add Hadoop-3.3.0 to precommit check list > > > Key: HBASE-25144 > URL: https://issues.apache.org/jira/browse/HBASE-25144 > Project: HBase > Issue Type: Task > Components: build, community >Reporter: Nick Dimiduk >Assignee: Nick Dimiduk >Priority: Minor > Fix For: 3.0.0-alpha-1 > > > Now that Hadoop 3.3.0 is released, let's figure out where it goes in our > testing matrix. Start by adding it to precommit checks. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[GitHub] [hbase] ndimiduk opened a new pull request #2492: HBASE-25144 Add Hadoop-3.3.0 to precommit check list
ndimiduk opened a new pull request #2492: URL: https://github.com/apache/hbase/pull/2492 Now that Hadoop 3.3.0 is released, let's figure out where it goes in our testing matrix. Start by adding it to precommit checks. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] gkanade commented on a change in pull request #2483: HBASE-25026 - Create a metric to track scans that have no start row and/or stop row
gkanade commented on a change in pull request #2483:
URL: https://github.com/apache/hbase/pull/2483#discussion_r498500444
##
File path:
hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/HRegion.java
##
@@ -7030,6 +7030,7 @@ public String toString() {
protected Cell joinedContinuationRow = null;
private boolean filterClosed = false;
+protected final byte[] startRow;
Review comment:
added comment here, thx
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[jira] [Created] (HBASE-25144) Add Hadoop-3.3.0 to precommit check list
Nick Dimiduk created HBASE-25144: Summary: Add Hadoop-3.3.0 to precommit check list Key: HBASE-25144 URL: https://issues.apache.org/jira/browse/HBASE-25144 Project: HBase Issue Type: Task Components: build, community Reporter: Nick Dimiduk Assignee: Nick Dimiduk Fix For: 3.0.0-alpha-1 Now that Hadoop 3.3.0 is released, let's figure out where it goes in our testing matrix. Start by adding it to precommit checks. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[GitHub] [hbase] Apache-HBase commented on pull request #2491: HBASE-25143 Remove branch-1.3 from precommit and docs
Apache-HBase commented on pull request #2491: URL: https://github.com/apache/hbase/pull/2491#issuecomment-702383470 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 0m 31s | Docker mode activated. | | -0 :warning: | yetus | 0m 3s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck | ||| _ Prechecks _ | ||| _ master Compile Tests _ | ||| _ Patch Compile Tests _ | ||| _ Other Tests _ | | | | 1m 40s | | | Subsystem | Report/Notes | |--:|:-| | Docker | Client=19.03.13 Server=19.03.13 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2491/1/artifact/yetus-jdk8-hadoop3-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/2491 | | Optional Tests | | | uname | Linux 1a64407fd1d1 4.15.0-60-generic #67-Ubuntu SMP Thu Aug 22 16:55:30 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / 3b91a15183 | | Max. process+thread count | 51 (vs. ulimit of 3) | | modules | C: . U: . | | Console output | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2491/1/console | | versions | git=2.17.1 maven=(cecedd343002696d0abb50b32b541b8a6ba2883f) | | Powered by | Apache Yetus 0.11.1 https://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #2491: HBASE-25143 Remove branch-1.3 from precommit and docs
Apache-HBase commented on pull request #2491: URL: https://github.com/apache/hbase/pull/2491#issuecomment-702383628 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 0m 35s | Docker mode activated. | | -0 :warning: | yetus | 0m 3s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck | ||| _ Prechecks _ | ||| _ master Compile Tests _ | ||| _ Patch Compile Tests _ | ||| _ Other Tests _ | | | | 1m 34s | | | Subsystem | Report/Notes | |--:|:-| | Docker | Client=19.03.13 Server=19.03.13 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2491/1/artifact/yetus-jdk11-hadoop3-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/2491 | | Optional Tests | | | uname | Linux d39b3ad730f1 4.15.0-112-generic #113-Ubuntu SMP Thu Jul 9 23:41:39 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / 3b91a15183 | | Max. process+thread count | 51 (vs. ulimit of 3) | | modules | C: . U: . | | Console output | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2491/1/console | | versions | git=2.17.1 maven=(cecedd343002696d0abb50b32b541b8a6ba2883f) | | Powered by | Apache Yetus 0.11.1 https://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #2491: HBASE-25143 Remove branch-1.3 from precommit and docs
Apache-HBase commented on pull request #2491: URL: https://github.com/apache/hbase/pull/2491#issuecomment-702383183 (!) A patch to the testing environment has been detected. Re-executing against the patched versions to perform further tests. The console is at https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2491/1/console in case of problems. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Commented] (HBASE-24981) Enable table replication fails from 1.x to 2.x if table already exist at peer.
[
https://issues.apache.org/jira/browse/HBASE-24981?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17205811#comment-17205811
]
Hudson commented on HBASE-24981:
Results for branch branch-2.2
[build #75 on
builds.a.o|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/branch-2.2/75/]:
(x) *{color:red}-1 overall{color}*
details (if available):
(x) {color:red}-1 general checks{color}
-- For more information [see general
report|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/branch-2.2/75//General_Nightly_Build_Report/]
(x) {color:red}-1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/branch-2.2/75//JDK8_Nightly_Build_Report_(Hadoop2)/]
(x) {color:red}-1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/branch-2.2/75//JDK8_Nightly_Build_Report_(Hadoop3)/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
(x) {color:red}-1 client integration test{color}
--Failed when running client tests on top of Hadoop 2. [see log for
details|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/branch-2.2/75//artifact/output-integration/hadoop-2.log].
(note that this means we didn't run on Hadoop 3)
> Enable table replication fails from 1.x to 2.x if table already exist at
> peer.
> ---
>
> Key: HBASE-24981
> URL: https://issues.apache.org/jira/browse/HBASE-24981
> Project: HBase
> Issue Type: Improvement
> Components: Replication
>Affects Versions: 2.2.3
>Reporter: Sanjeet Nishad
>Assignee: Sanjeet Nishad
>Priority: Major
> Fix For: 3.0.0-alpha-1, 2.3.3, 2.4.0, 2.2.7
>
>
> If source is 1.x and peer is 2.x and a default table is created at source and
> also at peer, default table descriptors mismatch so the enable table
> replication fails with error :
>
> {code:java}
> Table t1 exists in peer cluster 1, but the table descriptors are not same
> when compared with source cluster. Thus can not enable the table's
> replication switch.{code}
> Following are the extra attributes set by default in 2.x:
> CACHE_DATA_ON_WRITE
> CACHE_INDEX_ON_WRITE
> CACHE_BLOOMS_ON_WRITE
> EVICT_BLOCKS_ON_CLOSE
> PREFETCH_BLOCKS_ON_OPEN.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HBASE-25132) Migrate flaky test jenkins job from Hadoop to hbase
[
https://issues.apache.org/jira/browse/HBASE-25132?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17205808#comment-17205808
]
Hudson commented on HBASE-25132:
Results for branch branch-2.2
[build #75 on
builds.a.o|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/branch-2.2/75/]:
(x) *{color:red}-1 overall{color}*
details (if available):
(x) {color:red}-1 general checks{color}
-- For more information [see general
report|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/branch-2.2/75//General_Nightly_Build_Report/]
(x) {color:red}-1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/branch-2.2/75//JDK8_Nightly_Build_Report_(Hadoop2)/]
(x) {color:red}-1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/branch-2.2/75//JDK8_Nightly_Build_Report_(Hadoop3)/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
(x) {color:red}-1 client integration test{color}
--Failed when running client tests on top of Hadoop 2. [see log for
details|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/branch-2.2/75//artifact/output-integration/hadoop-2.log].
(note that this means we didn't run on Hadoop 3)
> Migrate flaky test jenkins job from Hadoop to hbase
> ---
>
> Key: HBASE-25132
> URL: https://issues.apache.org/jira/browse/HBASE-25132
> Project: HBase
> Issue Type: Sub-task
> Components: scripts
>Reporter: Duo Zhang
>Assignee: Duo Zhang
>Priority: Major
> Fix For: 3.0.0-alpha-1, 2.3.3, 1.7.0, 2.4.0, 1.4.14, 2.2.7
>
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[GitHub] [hbase] Apache-HBase commented on pull request #2491: HBASE-25143 Remove branch-1.3 from precommit and docs
Apache-HBase commented on pull request #2491: URL: https://github.com/apache/hbase/pull/2491#issuecomment-702383079 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Commented] (HBASE-25133) Migrate HBase Nightly jenkins job from Hadoop to hbase
[
https://issues.apache.org/jira/browse/HBASE-25133?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17205809#comment-17205809
]
Hudson commented on HBASE-25133:
Results for branch branch-2.2
[build #75 on
builds.a.o|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/branch-2.2/75/]:
(x) *{color:red}-1 overall{color}*
details (if available):
(x) {color:red}-1 general checks{color}
-- For more information [see general
report|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/branch-2.2/75//General_Nightly_Build_Report/]
(x) {color:red}-1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/branch-2.2/75//JDK8_Nightly_Build_Report_(Hadoop2)/]
(x) {color:red}-1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/branch-2.2/75//JDK8_Nightly_Build_Report_(Hadoop3)/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
(x) {color:red}-1 client integration test{color}
--Failed when running client tests on top of Hadoop 2. [see log for
details|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/branch-2.2/75//artifact/output-integration/hadoop-2.log].
(note that this means we didn't run on Hadoop 3)
> Migrate HBase Nightly jenkins job from Hadoop to hbase
> --
>
> Key: HBASE-25133
> URL: https://issues.apache.org/jira/browse/HBASE-25133
> Project: HBase
> Issue Type: Sub-task
> Components: scripts
>Reporter: Duo Zhang
>Assignee: Duo Zhang
>Priority: Major
> Fix For: 3.0.0-alpha-1, 2.3.3, 1.7.0, 2.4.0, 1.4.14, 2.2.7
>
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HBASE-25134) Migrate HBase PreCommit jenkins job from Hadoop to hbase
[
https://issues.apache.org/jira/browse/HBASE-25134?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17205810#comment-17205810
]
Hudson commented on HBASE-25134:
Results for branch branch-2.2
[build #75 on
builds.a.o|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/branch-2.2/75/]:
(x) *{color:red}-1 overall{color}*
details (if available):
(x) {color:red}-1 general checks{color}
-- For more information [see general
report|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/branch-2.2/75//General_Nightly_Build_Report/]
(x) {color:red}-1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/branch-2.2/75//JDK8_Nightly_Build_Report_(Hadoop2)/]
(x) {color:red}-1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/branch-2.2/75//JDK8_Nightly_Build_Report_(Hadoop3)/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
(x) {color:red}-1 client integration test{color}
--Failed when running client tests on top of Hadoop 2. [see log for
details|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/branch-2.2/75//artifact/output-integration/hadoop-2.log].
(note that this means we didn't run on Hadoop 3)
> Migrate HBase PreCommit jenkins job from Hadoop to hbase
>
>
> Key: HBASE-25134
> URL: https://issues.apache.org/jira/browse/HBASE-25134
> Project: HBase
> Issue Type: Sub-task
> Components: scripts
>Reporter: Duo Zhang
>Assignee: Duo Zhang
>Priority: Major
> Fix For: 3.0.0-alpha-1, 2.3.3, 1.7.0, 2.4.0, 1.4.14, 2.2.7
>
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[GitHub] [hbase] ndimiduk opened a new pull request #2491: HBASE-25143 Remove branch-1.3 from precommit and docs
ndimiduk opened a new pull request #2491: URL: https://github.com/apache/hbase/pull/2491 Following the announcement [0] to EOL branch-1.3, update the precommit script to not consider this branch any longer, and refresh mentions of this branch in the doc. [0]: https://lists.apache.org/thread.html/r9552e9085aaac2a43f8b26b866d34825a84a9be7f19118ac560d14de%40%3Cuser.hbase.apache.org%3E This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Created] (HBASE-25143) Remove branch-1.3 from precommit and docs
Nick Dimiduk created HBASE-25143: Summary: Remove branch-1.3 from precommit and docs Key: HBASE-25143 URL: https://issues.apache.org/jira/browse/HBASE-25143 Project: HBase Issue Type: Task Components: community Affects Versions: 3.0.0-alpha-1 Reporter: Nick Dimiduk Assignee: Nick Dimiduk Following the [announcement|https://lists.apache.org/thread.html/r9552e9085aaac2a43f8b26b866d34825a84a9be7f19118ac560d14de%40%3Cuser.hbase.apache.org%3E] to EOL branch-1.3, update the precommit script to not consider this branch any longer, and refresh mentions of this branch in the doc. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[GitHub] [hbase] Apache-HBase commented on pull request #2438: HBASE-25033 : Provide an option to upload RC to home.apache.org
Apache-HBase commented on pull request #2438: URL: https://github.com/apache/hbase/pull/2438#issuecomment-702381403 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 1m 11s | Docker mode activated. | ||| _ Prechecks _ | | +1 :green_heart: | dupname | 0m 0s | No case conflicting files found. | | +0 :ok: | shelldocs | 0m 0s | Shelldocs was not available. | | +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. | ||| _ master Compile Tests _ | | +0 :ok: | mvndep | 0m 9s | Maven dependency ordering for branch | ||| _ Patch Compile Tests _ | | +0 :ok: | mvndep | 0m 8s | Maven dependency ordering for patch | | +1 :green_heart: | hadolint | 0m 2s | There were no new hadolint issues. | | -0 :warning: | shellcheck | 0m 4s | The patch generated 3 new + 11 unchanged - 0 fixed = 14 total (was 11) | | +1 :green_heart: | whitespace | 0m 0s | The patch has no whitespace issues. | ||| _ Other Tests _ | | +0 :ok: | asflicense | 0m 0s | ASF License check generated no output? | | | | 2m 49s | | | Subsystem | Report/Notes | |--:|:-| | Docker | Client=19.03.13 Server=19.03.13 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2438/3/artifact/yetus-general-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/2438 | | Optional Tests | dupname asflicense shellcheck shelldocs hadolint | | uname | Linux f9a8518ffcc5 4.15.0-60-generic #67-Ubuntu SMP Thu Aug 22 16:55:30 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / 3b91a15183 | | shellcheck | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2438/3/artifact/yetus-general-check/output/diff-patch-shellcheck.txt | | Max. process+thread count | 48 (vs. ulimit of 3) | | modules | C: U: | | Console output | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2438/3/console | | versions | git=2.17.1 maven=(cecedd343002696d0abb50b32b541b8a6ba2883f) shellcheck=0.4.6 hadolint=1.17.5-0-g443423c | | Powered by | Apache Yetus 0.11.1 https://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #2438: HBASE-25033 : Provide an option to upload RC to home.apache.org
Apache-HBase commented on pull request #2438: URL: https://github.com/apache/hbase/pull/2438#issuecomment-702381018 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #2454: HBASE-24628 Region normalizer now respects a rate limit
Apache-HBase commented on pull request #2454: URL: https://github.com/apache/hbase/pull/2454#issuecomment-702379334 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 1m 50s | Docker mode activated. | | -0 :warning: | yetus | 0m 4s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck | ||| _ Prechecks _ | ||| _ master Compile Tests _ | | +1 :green_heart: | mvninstall | 7m 8s | master passed | | +1 :green_heart: | compile | 1m 55s | master passed | | +1 :green_heart: | shadedjars | 12m 27s | branch has no errors when building our shaded downstream artifacts. | | -0 :warning: | javadoc | 1m 19s | hbase-server in master failed. | ||| _ Patch Compile Tests _ | | +1 :green_heart: | mvninstall | 6m 48s | the patch passed | | +1 :green_heart: | compile | 2m 12s | the patch passed | | +1 :green_heart: | javac | 2m 12s | the patch passed | | +1 :green_heart: | shadedjars | 12m 41s | patch has no errors when building our shaded downstream artifacts. | | -0 :warning: | javadoc | 1m 12s | hbase-server in the patch failed. | ||| _ Other Tests _ | | -1 :x: | unit | 201m 14s | hbase-server in the patch failed. | | | | 251m 7s | | | Subsystem | Report/Notes | |--:|:-| | Docker | Client=19.03.13 Server=19.03.13 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2454/5/artifact/yetus-jdk11-hadoop3-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/2454 | | Optional Tests | javac javadoc unit shadedjars compile | | uname | Linux 16b2491783b0 4.15.0-112-generic #113-Ubuntu SMP Thu Jul 9 23:41:39 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / 11a336a74a | | Default Java | 2020-01-14 | | javadoc | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2454/5/artifact/yetus-jdk11-hadoop3-check/output/branch-javadoc-hbase-server.txt | | javadoc | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2454/5/artifact/yetus-jdk11-hadoop3-check/output/patch-javadoc-hbase-server.txt | | unit | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2454/5/artifact/yetus-jdk11-hadoop3-check/output/patch-unit-hbase-server.txt | | Test Results | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2454/5/testReport/ | | Max. process+thread count | 4021 (vs. ulimit of 3) | | modules | C: hbase-server U: hbase-server | | Console output | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2454/5/console | | versions | git=2.17.1 maven=(cecedd343002696d0abb50b32b541b8a6ba2883f) | | Powered by | Apache Yetus 0.11.1 https://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #2163: HBASE-24789 When table level quota is set and violated, pre existing …
Apache-HBase commented on pull request #2163: URL: https://github.com/apache/hbase/pull/2163#issuecomment-702377901 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 1m 11s | Docker mode activated. | ||| _ Prechecks _ | | +1 :green_heart: | dupname | 0m 1s | No case conflicting files found. | | +1 :green_heart: | hbaseanti | 0m 0s | Patch does not have any anti-patterns. | | +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. | ||| _ master Compile Tests _ | | +1 :green_heart: | mvninstall | 4m 7s | master passed | | +1 :green_heart: | checkstyle | 1m 15s | master passed | | +1 :green_heart: | spotbugs | 2m 11s | master passed | ||| _ Patch Compile Tests _ | | +1 :green_heart: | mvninstall | 4m 8s | the patch passed | | -0 :warning: | checkstyle | 1m 14s | hbase-server: The patch generated 1 new + 141 unchanged - 0 fixed = 142 total (was 141) | | +1 :green_heart: | whitespace | 0m 0s | The patch has no whitespace issues. | | +1 :green_heart: | hadoopcheck | 13m 17s | Patch does not cause any errors with Hadoop 3.1.2 3.2.1. | | +1 :green_heart: | spotbugs | 2m 24s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | asflicense | 0m 13s | The patch does not generate ASF License warnings. | | | | 38m 2s | | | Subsystem | Report/Notes | |--:|:-| | Docker | Client=19.03.13 Server=19.03.13 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2163/3/artifact/yetus-general-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/2163 | | Optional Tests | dupname asflicense spotbugs hadoopcheck hbaseanti checkstyle | | uname | Linux 8721bfc3208b 4.15.0-112-generic #113-Ubuntu SMP Thu Jul 9 23:41:39 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / 3b91a15183 | | checkstyle | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2163/3/artifact/yetus-general-check/output/diff-checkstyle-hbase-server.txt | | Max. process+thread count | 84 (vs. ulimit of 3) | | modules | C: hbase-server U: hbase-server | | Console output | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2163/3/console | | versions | git=2.17.1 maven=(cecedd343002696d0abb50b32b541b8a6ba2883f) spotbugs=3.1.12 | | Powered by | Apache Yetus 0.11.1 https://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #2438: HBASE-25033 : Provide an option to upload RC to home.apache.org
Apache-HBase commented on pull request #2438: URL: https://github.com/apache/hbase/pull/2438#issuecomment-702371669 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 0m 29s | Docker mode activated. | | -0 :warning: | yetus | 0m 3s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck | ||| _ Prechecks _ | ||| _ master Compile Tests _ | | +0 :ok: | mvndep | 0m 24s | Maven dependency ordering for branch | ||| _ Patch Compile Tests _ | | +0 :ok: | mvndep | 0m 8s | Maven dependency ordering for patch | ||| _ Other Tests _ | | | | 1m 57s | | | Subsystem | Report/Notes | |--:|:-| | Docker | Client=19.03.13 Server=19.03.13 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2438/2/artifact/yetus-jdk11-hadoop3-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/2438 | | Optional Tests | | | uname | Linux 0ed932272838 4.15.0-60-generic #67-Ubuntu SMP Thu Aug 22 16:55:30 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / 3b91a15183 | | Max. process+thread count | 54 (vs. ulimit of 3) | | modules | C: U: | | Console output | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2438/2/console | | versions | git=2.17.1 maven=(cecedd343002696d0abb50b32b541b8a6ba2883f) | | Powered by | Apache Yetus 0.11.1 https://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #2438: HBASE-25033 : Provide an option to upload RC to home.apache.org
Apache-HBase commented on pull request #2438: URL: https://github.com/apache/hbase/pull/2438#issuecomment-702371827 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #2490: HBASE-24419 Normalizer merge plans should consider more than 2 regions when possible
Apache-HBase commented on pull request #2490: URL: https://github.com/apache/hbase/pull/2490#issuecomment-702370512 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 0m 28s | Docker mode activated. | | -0 :warning: | yetus | 0m 3s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck | ||| _ Prechecks _ | ||| _ master Compile Tests _ | | +0 :ok: | mvndep | 0m 24s | Maven dependency ordering for branch | | +1 :green_heart: | mvninstall | 4m 36s | master passed | | +1 :green_heart: | compile | 1m 43s | master passed | | +1 :green_heart: | shadedjars | 7m 19s | branch has no errors when building our shaded downstream artifacts. | | -0 :warning: | javadoc | 0m 21s | hbase-common in master failed. | | -0 :warning: | javadoc | 0m 45s | hbase-server in master failed. | ||| _ Patch Compile Tests _ | | +0 :ok: | mvndep | 0m 17s | Maven dependency ordering for patch | | +1 :green_heart: | mvninstall | 4m 25s | the patch passed | | +1 :green_heart: | compile | 1m 43s | the patch passed | | +1 :green_heart: | javac | 1m 43s | the patch passed | | +1 :green_heart: | shadedjars | 7m 9s | patch has no errors when building our shaded downstream artifacts. | | -0 :warning: | javadoc | 0m 19s | hbase-common in the patch failed. | | -0 :warning: | javadoc | 0m 44s | hbase-server in the patch failed. | ||| _ Other Tests _ | | +1 :green_heart: | unit | 1m 48s | hbase-common in the patch passed. | | -1 :x: | unit | 134m 12s | hbase-server in the patch failed. | | | | 168m 46s | | | Subsystem | Report/Notes | |--:|:-| | Docker | Client=19.03.13 Server=19.03.13 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2490/1/artifact/yetus-jdk11-hadoop3-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/2490 | | Optional Tests | javac javadoc unit shadedjars compile | | uname | Linux 37ed8c4be7ac 4.15.0-58-generic #64-Ubuntu SMP Tue Aug 6 11:12:41 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / 3b91a15183 | | Default Java | 2020-01-14 | | javadoc | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2490/1/artifact/yetus-jdk11-hadoop3-check/output/branch-javadoc-hbase-common.txt | | javadoc | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2490/1/artifact/yetus-jdk11-hadoop3-check/output/branch-javadoc-hbase-server.txt | | javadoc | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2490/1/artifact/yetus-jdk11-hadoop3-check/output/patch-javadoc-hbase-common.txt | | javadoc | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2490/1/artifact/yetus-jdk11-hadoop3-check/output/patch-javadoc-hbase-server.txt | | unit | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2490/1/artifact/yetus-jdk11-hadoop3-check/output/patch-unit-hbase-server.txt | | Test Results | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2490/1/testReport/ | | Max. process+thread count | 3950 (vs. ulimit of 3) | | modules | C: hbase-common hbase-server U: . | | Console output | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2490/1/console | | versions | git=2.17.1 maven=(cecedd343002696d0abb50b32b541b8a6ba2883f) | | Powered by | Apache Yetus 0.11.1 https://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] sanjeetnishad95 commented on a change in pull request #2163: HBASE-24789 When table level quota is set and violated, pre existing …
sanjeetnishad95 commented on a change in pull request #2163:
URL: https://github.com/apache/hbase/pull/2163#discussion_r498477956
##
File path:
hbase-server/src/main/java/org/apache/hadoop/hbase/quotas/MasterQuotaManager.java
##
@@ -284,6 +285,9 @@ public void delete() throws IOException {
SpaceQuotaSnapshot currSnapshotOfTable =
QuotaTableUtil.getCurrentSnapshotFromQuotaTable(masterServices.getConnection(),
table);
QuotaUtil.deleteTableQuota(masterServices.getConnection(), table);
+// Removes the tableName from the current state in QuotaObserverChore
+QuotaObserverChore quotaChore =
((HMaster)masterServices).getQuotaObserverChore();
Review comment:
@ndimiduk I have updated the PR, PTAL.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[jira] [Commented] (HBASE-25142) Auto-fix 'Unknown Server'
[ https://issues.apache.org/jira/browse/HBASE-25142?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17205766#comment-17205766 ] Michael Stack commented on HBASE-25142: --- At a minimum, we might add to 'hbck2 fixMeta' the scheduling of SCPs for all servers in 'Unknown Servers' list. > Auto-fix 'Unknown Server' > - > > Key: HBASE-25142 > URL: https://issues.apache.org/jira/browse/HBASE-25142 > Project: HBase > Issue Type: Improvement >Reporter: Michael Stack >Priority: Major > > Addressing reports of 'Unknown Server' has come up in various conversations > lately. This issue is about fixing instances of 'Unknown Server' > automatically as part of the tasks undertaken by CatalogJanitor when it runs. > First though, would like to figure a definition for 'Unknown Server' and a > list of ways in which they arise. We need this to figure how to do safe > auto-fixing. > Currently an 'Unknown Server' is a server found in hbase:meta that is not > online (no recent heartbeat) and that is not mentioned in the dead servers > list. > In outline, I'd think CatalogJanitor could schedule an expiration of the RS > znode in zk (if exists) and then an SCP if it finds an 'Unknown Server'. > Perhaps it waits for 2x or 10x the heartbeat interval just-in-case (or not). > The SCP would clean up any references in hbase:meta by reassigning Regions > assigned the 'Unknown Server' after replaying any WALs found in hdfs > attributed to the dead server. > As to how they arise: > * A contrived illustration would be a large online cluster crashes down with > a massive backlog of WAL files – zk went down for some reason say. The replay > of the WALs look like it could take a very long time (lets say the cluster > was badly configured and a bug and misconfig made it so each RS was carrying > hundreds of WALs and there are hundreds of servers). To get the service back > online, the procedure store and WALs are moved aside (for later replay with > WALPlayer). The cluster comes up. meta is onlined but refers to server > instances that are no longer around. Can schedule an SCP per server mentioned > in the 'HBCK Report' by scraping and scripting hbck2 or, better, > catalogjanitor could just do it. > * HBASE-24286 HMaster won't become healthy after after cloning... describes > starting a cluster over data that is hfile-content only. In this case the > original servers used manufacture the hfile cluster data are long dead yet > meta still refers to the old servers. They will not make the 'dead servers' > list. > Let this issue stew awhile. Meantime collect how 'Unknown Server' gets > created and best way to fix. > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (HBASE-25142) Auto-fix 'Unknown Server'
Michael Stack created HBASE-25142: - Summary: Auto-fix 'Unknown Server' Key: HBASE-25142 URL: https://issues.apache.org/jira/browse/HBASE-25142 Project: HBase Issue Type: Improvement Reporter: Michael Stack Addressing reports of 'Unknown Server' has come up in various conversations lately. This issue is about fixing instances of 'Unknown Server' automatically as part of the tasks undertaken by CatalogJanitor when it runs. First though, would like to figure a definition for 'Unknown Server' and a list of ways in which they arise. We need this to figure how to do safe auto-fixing. Currently an 'Unknown Server' is a server found in hbase:meta that is not online (no recent heartbeat) and that is not mentioned in the dead servers list. In outline, I'd think CatalogJanitor could schedule an expiration of the RS znode in zk (if exists) and then an SCP if it finds an 'Unknown Server'. Perhaps it waits for 2x or 10x the heartbeat interval just-in-case (or not). The SCP would clean up any references in hbase:meta by reassigning Regions assigned the 'Unknown Server' after replaying any WALs found in hdfs attributed to the dead server. As to how they arise: * A contrived illustration would be a large online cluster crashes down with a massive backlog of WAL files – zk went down for some reason say. The replay of the WALs look like it could take a very long time (lets say the cluster was badly configured and a bug and misconfig made it so each RS was carrying hundreds of WALs and there are hundreds of servers). To get the service back online, the procedure store and WALs are moved aside (for later replay with WALPlayer). The cluster comes up. meta is onlined but refers to server instances that are no longer around. Can schedule an SCP per server mentioned in the 'HBCK Report' by scraping and scripting hbck2 or, better, catalogjanitor could just do it. * HBASE-24286 HMaster won't become healthy after after cloning... describes starting a cluster over data that is hfile-content only. In this case the original servers used manufacture the hfile cluster data are long dead yet meta still refers to the old servers. They will not make the 'dead servers' list. Let this issue stew awhile. Meantime collect how 'Unknown Server' gets created and best way to fix. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[GitHub] [hbase] Apache-HBase commented on pull request #2490: HBASE-24419 Normalizer merge plans should consider more than 2 regions when possible
Apache-HBase commented on pull request #2490: URL: https://github.com/apache/hbase/pull/2490#issuecomment-702307206 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 1m 8s | Docker mode activated. | ||| _ Prechecks _ | | +1 :green_heart: | dupname | 0m 0s | No case conflicting files found. | | +1 :green_heart: | hbaseanti | 0m 0s | Patch does not have any anti-patterns. | | +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. | ||| _ master Compile Tests _ | | +0 :ok: | mvndep | 0m 22s | Maven dependency ordering for branch | | +1 :green_heart: | mvninstall | 5m 1s | master passed | | +1 :green_heart: | checkstyle | 1m 44s | master passed | | +1 :green_heart: | spotbugs | 3m 21s | master passed | ||| _ Patch Compile Tests _ | | +0 :ok: | mvndep | 0m 14s | Maven dependency ordering for patch | | +1 :green_heart: | mvninstall | 4m 32s | the patch passed | | -0 :warning: | checkstyle | 0m 25s | hbase-common: The patch generated 1 new + 0 unchanged - 0 fixed = 1 total (was 0) | | -0 :warning: | checkstyle | 1m 19s | hbase-server: The patch generated 6 new + 145 unchanged - 7 fixed = 151 total (was 152) | | +1 :green_heart: | whitespace | 0m 0s | The patch has no whitespace issues. | | +1 :green_heart: | hadoopcheck | 15m 2s | Patch does not cause any errors with Hadoop 3.1.2 3.2.1. | | +1 :green_heart: | spotbugs | 3m 36s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | asflicense | 0m 25s | The patch does not generate ASF License warnings. | | | | 46m 42s | | | Subsystem | Report/Notes | |--:|:-| | Docker | Client=19.03.13 Server=19.03.13 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2490/1/artifact/yetus-general-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/2490 | | Optional Tests | dupname asflicense spotbugs hadoopcheck hbaseanti checkstyle | | uname | Linux 5ac66cf7cfbd 4.15.0-112-generic #113-Ubuntu SMP Thu Jul 9 23:41:39 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / 3b91a15183 | | checkstyle | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2490/1/artifact/yetus-general-check/output/diff-checkstyle-hbase-common.txt | | checkstyle | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2490/1/artifact/yetus-general-check/output/diff-checkstyle-hbase-server.txt | | Max. process+thread count | 84 (vs. ulimit of 3) | | modules | C: hbase-common hbase-server U: . | | Console output | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2490/1/console | | versions | git=2.17.1 maven=(cecedd343002696d0abb50b32b541b8a6ba2883f) spotbugs=3.1.12 | | Powered by | Apache Yetus 0.11.1 https://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] saintstack commented on pull request #2410: HBASE-25048 [HBCK2] Bypassed parent procedures are not updated in store
saintstack commented on pull request #2410: URL: https://github.com/apache/hbase/pull/2410#issuecomment-702306714 @Joseph295 any progress on this nice fix? This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] saintstack commented on a change in pull request #2451: HBASE-25055 Add ReplicationSource for meta WALs; add enable/disable w…
saintstack commented on a change in pull request #2451:
URL: https://github.com/apache/hbase/pull/2451#discussion_r498424906
##
File path:
hbase-server/src/main/java/org/apache/hadoop/hbase/util/ServerRegionReplicaUtil.java
##
@@ -154,47 +162,60 @@ public static StoreFileInfo
getStoreFileInfo(Configuration conf, FileSystem fs,
}
/**
- * Create replication peer for replicating to region replicas if needed.
+ * Create replication peer for replicating user-space Region Read Replicas.
* @param conf configuration to use
- * @throws IOException
*/
public static void setupRegionReplicaReplication(Configuration conf) throws
IOException {
-if (!isRegionReplicaReplicationEnabled(conf)) {
+if (!conf.getBoolean(REGION_REPLICA_REPLICATION_CONF_KEY,
DEFAULT_REGION_REPLICA_REPLICATION)) {
return;
}
-
+String peerId = REGION_REPLICA_REPLICATION_PEER;
try (Connection connection = ConnectionFactory.createConnection(conf);
Admin admin = connection.getAdmin()) {
ReplicationPeerConfig peerConfig = null;
try {
-peerConfig =
admin.getReplicationPeerConfig(REGION_REPLICA_REPLICATION_PEER);
+peerConfig = admin.getReplicationPeerConfig(peerId);
} catch (ReplicationPeerNotFoundException e) {
-LOG.warn(
- "Region replica replication peer id=" +
REGION_REPLICA_REPLICATION_PEER + " not exist",
- e);
+LOG.warn("Region replica peer id={} does not exist", peerId, e);
}
-
if (peerConfig == null) {
-LOG.info("Region replica replication peer id=" +
REGION_REPLICA_REPLICATION_PEER
- + " not exist. Creating...");
-peerConfig = new ReplicationPeerConfig();
-peerConfig.setClusterKey(ZKConfig.getZooKeeperClusterKey(conf));
-
peerConfig.setReplicationEndpointImpl(RegionReplicaReplicationEndpoint.class.getName());
-admin.addReplicationPeer(REGION_REPLICA_REPLICATION_PEER, peerConfig);
+LOG.info("Region Read Replica peerId={} does not exist; creating...",
peerId);
+peerConfig = ReplicationPeerConfig.newBuilder().
+ setClusterKey(ZKConfig.getZooKeeperClusterKey(conf)).
+
setReplicationEndpointImpl(RegionReplicaReplicationEndpoint.class.getName()).build();
+admin.addReplicationPeer(peerId, peerConfig);
}
}
}
- public static boolean isRegionReplicaReplicationEnabled(Configuration conf) {
-return conf.getBoolean(REGION_REPLICA_REPLICATION_CONF_KEY,
- DEFAULT_REGION_REPLICA_REPLICATION);
+ /**
+ * @return True if Region Read Replica is enabled for tn.
+ */
+ public static boolean isRegionReplicaReplicationEnabled(Configuration conf,
TableName tn) {
+return isMetaRegionReplicaReplicationEnabled(conf, tn) ||
+ conf.getBoolean(REGION_REPLICA_REPLICATION_CONF_KEY,
DEFAULT_REGION_REPLICA_REPLICATION);
Review comment:
np. sounds good.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [hbase] ndimiduk commented on a change in pull request #2454: HBASE-24628 Region normalizer now respects a rate limit
ndimiduk commented on a change in pull request #2454:
URL: https://github.com/apache/hbase/pull/2454#discussion_r498407097
##
File path:
hbase-server/src/main/java/org/apache/hadoop/hbase/master/HMaster.java
##
@@ -1924,70 +1907,17 @@ public boolean normalizeRegions(final
NormalizeTableFilterParams ntfp) throws IO
return false;
}
-if (!normalizationInProgressLock.tryLock()) {
- // Don't run the normalizer concurrently
- LOG.info("Normalization already in progress. Skipping request.");
- return true;
-}
-
-int affectedTables = 0;
-try {
- final Set matchingTables = getTableDescriptors(new
LinkedList<>(),
-ntfp.getNamespace(), ntfp.getRegex(), ntfp.getTableNames(), false)
-.stream()
-.map(TableDescriptor::getTableName)
-.collect(Collectors.toSet());
- final Set allEnabledTables =
-tableStateManager.getTablesInStates(TableState.State.ENABLED);
- final List targetTables =
-new ArrayList<>(Sets.intersection(matchingTables, allEnabledTables));
- Collections.shuffle(targetTables);
-
- final List submittedPlanProcIds = new ArrayList<>();
- for (TableName table : targetTables) {
-if (table.isSystemTable()) {
Review comment:
All this moves to RegionNormalizerWorker.
##
File path:
hbase-server/src/main/java/org/apache/hadoop/hbase/master/MasterServices.java
##
@@ -354,6 +353,13 @@ long splitRegion(
*/
boolean isInMaintenanceMode();
+ /**
+ * Checks master state before initiating action over region topology.
+ * @param action the name of the action under consideration, for logging.
+ * @return {@code true} when the caller should exit early, {@code false}
otherwise.
+ */
+ boolean skipRegionManagementAction(final String action);
Review comment:
I don't love exposing this up here on MasterServices, but imo, it's
better to have the interface here than to expose its innards.
Suggestions welcome.
##
File path:
hbase-server/src/main/java/org/apache/hadoop/hbase/master/HMaster.java
##
@@ -1924,70 +1907,17 @@ public boolean normalizeRegions(final
NormalizeTableFilterParams ntfp) throws IO
return false;
}
-if (!normalizationInProgressLock.tryLock()) {
- // Don't run the normalizer concurrently
- LOG.info("Normalization already in progress. Skipping request.");
- return true;
-}
-
-int affectedTables = 0;
-try {
- final Set matchingTables = getTableDescriptors(new
LinkedList<>(),
Review comment:
Table selection code stays here in HMaster.
##
File path:
hbase-server/src/main/java/org/apache/hadoop/hbase/master/HMaster.java
##
@@ -464,9 +451,6 @@ public void run() {
// handle table states
private TableStateManager tableStateManager;
- private long splitPlanCount;
Review comment:
Moved to RegionNormalizerWorker.
##
File path:
hbase-server/src/main/java/org/apache/hadoop/hbase/master/normalizer/SimpleRegionNormalizer.java
##
@@ -203,16 +200,6 @@ public void setMasterServices(final MasterServices
masterServices) {
this.masterServices = masterServices;
}
- @Override
- public void planSkipped(final RegionInfo hri, final PlanType type) {
Review comment:
Moved to RegionNormalizerWorker.
##
File path:
hbase-server/src/main/java/org/apache/hadoop/hbase/master/HMaster.java
##
@@ -1924,70 +1907,17 @@ public boolean normalizeRegions(final
NormalizeTableFilterParams ntfp) throws IO
return false;
}
-if (!normalizationInProgressLock.tryLock()) {
- // Don't run the normalizer concurrently
- LOG.info("Normalization already in progress. Skipping request.");
- return true;
-}
-
-int affectedTables = 0;
-try {
- final Set matchingTables = getTableDescriptors(new
LinkedList<>(),
-ntfp.getNamespace(), ntfp.getRegex(), ntfp.getTableNames(), false)
-.stream()
-.map(TableDescriptor::getTableName)
-.collect(Collectors.toSet());
- final Set allEnabledTables =
-tableStateManager.getTablesInStates(TableState.State.ENABLED);
- final List targetTables =
-new ArrayList<>(Sets.intersection(matchingTables, allEnabledTables));
- Collections.shuffle(targetTables);
-
- final List submittedPlanProcIds = new ArrayList<>();
- for (TableName table : targetTables) {
-if (table.isSystemTable()) {
- continue;
-}
-final TableDescriptor tblDesc = getTableDescriptors().get(table);
-if (tblDesc != null && !tblDesc.isNormalizationEnabled()) {
- LOG.debug(
-"Skipping table {} because normalization is disabled in its table
properties.", table);
- continue;
-}
-
-// make one last check that the cluster isn't shutting down before
proceeding.
-if (skipRegionManagementAction("region nor
[GitHub] [hbase] virajjasani commented on a change in pull request #2438: HBASE-25033 : Provide an option to ignore svn ci for create-release
virajjasani commented on a change in pull request #2438: URL: https://github.com/apache/hbase/pull/2438#discussion_r498410782 ## File path: dev-support/create-release/do-release-docker.sh ## @@ -77,6 +77,10 @@ Options: If none specified, runs tag, then publish-dist, and then publish-release. 'publish-snapshot' is also an allowed, less used, option. -x debug. do less clean up. (env file, gpg forwarding on mac) + -i [ignore-svn-ci] Review comment: Sorry for too much back and forth here, was giving some thoughts on what seems better and flexible option for committers, and to answer my own question, I believe as long as we know of h.a.o upload as one of the available ways for committers to upload artifacts, we should utilize it (until better options are available). Will try mounting local .ssh to container and using password-less scp to upload RC over h.a.o. Will try to document as much as possible (as part of `-h`). This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Resolved] (HBASE-25091) Move LogComparator from ReplicationSource to AbstractFSWALProvider#.WALsStartTimeComparator
[ https://issues.apache.org/jira/browse/HBASE-25091?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Stack resolved HBASE-25091. --- Fix Version/s: 2.4.0 3.0.0-alpha-1 Hadoop Flags: Reviewed Assignee: Michael Stack Resolution: Fixed Merged. Thanks for reviews [~zhangduo] and [~zghao] > Move LogComparator from ReplicationSource to > AbstractFSWALProvider#.WALsStartTimeComparator > --- > > Key: HBASE-25091 > URL: https://issues.apache.org/jira/browse/HBASE-25091 > Project: HBase > Issue Type: Improvement >Reporter: Michael Stack >Assignee: Michael Stack >Priority: Minor > Fix For: 3.0.0-alpha-1, 2.4.0 > > > Minor cleanup item noticed playing over in HBASE-18070. > ReplicationSource has an inner class named LogComparator which is a pretty > generic name for the comparator only it just compares on WAL start time and > nothing else. > Also, messing in HBASE-18070 I ran into compares that included user-space > WALs and hbase:meta WALs. The LogComparator as is barfed on meta WALs. > This ticket moves the comparator to AbstractFSWALProvider, where folks will > go looking if they need WAL comparators, and it also renames it to more > clearly explain what it does (and makes it so it can compare start times even > if it a meta WAL). > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[GitHub] [hbase] ndimiduk opened a new pull request #2490: HBASE-24419 Normalizer merge plans should consider more than 2 regions when possible
ndimiduk opened a new pull request #2490: URL: https://github.com/apache/hbase/pull/2490 The core change here is to the loop in `SimpleRegionNormalizer#computeMergeNormalizationPlans`. It's a nested loop that walks the table's region chain once, looking for contiguous sequences of regions that meet the criteria for merge. The outer loop tracks the starting point of the next sequence, the inner loop looks for the end of that sequence. A single sequence becomes an instance of `MergeNormalizationPlan`. Pay mind only to the commit for this JIRA. This PR depends on [HBASE-24628](https://issues.apache.org/jira/browse/HBASE-24628)/#2454 and will need rebased once that one lands. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] saintstack merged pull request #2449: HBASE-25091 Move LogComparator from ReplicationSource to AbstractFSWA…
saintstack merged pull request #2449: URL: https://github.com/apache/hbase/pull/2449 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #2454: HBASE-24628 Region normalizer now respects a rate limit
Apache-HBase commented on pull request #2454: URL: https://github.com/apache/hbase/pull/2454#issuecomment-70226 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 0m 30s | Docker mode activated. | ||| _ Prechecks _ | | +1 :green_heart: | dupname | 0m 0s | No case conflicting files found. | | +1 :green_heart: | hbaseanti | 0m 0s | Patch does not have any anti-patterns. | | +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. | ||| _ master Compile Tests _ | | +1 :green_heart: | mvninstall | 3m 49s | master passed | | +1 :green_heart: | checkstyle | 1m 11s | master passed | | +1 :green_heart: | spotbugs | 2m 14s | master passed | ||| _ Patch Compile Tests _ | | +1 :green_heart: | mvninstall | 3m 41s | the patch passed | | +1 :green_heart: | checkstyle | 1m 19s | hbase-server: The patch generated 0 new + 145 unchanged - 7 fixed = 145 total (was 152) | | +1 :green_heart: | whitespace | 0m 0s | The patch has no whitespace issues. | | +1 :green_heart: | hadoopcheck | 11m 52s | Patch does not cause any errors with Hadoop 3.1.2 3.2.1. | | +1 :green_heart: | spotbugs | 2m 23s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | asflicense | 0m 13s | The patch does not generate ASF License warnings. | | | | 34m 43s | | | Subsystem | Report/Notes | |--:|:-| | Docker | Client=19.03.13 Server=19.03.13 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2454/5/artifact/yetus-general-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/2454 | | Optional Tests | dupname asflicense spotbugs hadoopcheck hbaseanti checkstyle | | uname | Linux 00dc08e7e30b 4.15.0-58-generic #64-Ubuntu SMP Tue Aug 6 11:12:41 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / 11a336a74a | | Max. process+thread count | 94 (vs. ulimit of 3) | | modules | C: hbase-server U: hbase-server | | Console output | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2454/5/console | | versions | git=2.17.1 maven=(cecedd343002696d0abb50b32b541b8a6ba2883f) spotbugs=3.1.12 | | Powered by | Apache Yetus 0.11.1 https://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Commented] (HBASE-24981) Enable table replication fails from 1.x to 2.x if table already exist at peer.
[
https://issues.apache.org/jira/browse/HBASE-24981?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17205656#comment-17205656
]
Hudson commented on HBASE-24981:
Results for branch branch-2.3
[build #67 on
builds.a.o|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/branch-2.3/67/]:
(x) *{color:red}-1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/branch-2.3/67/General_20Nightly_20Build_20Report/]
(/) {color:green}+1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/branch-2.3/67/JDK8_20Nightly_20Build_20Report_20_28Hadoop2_29/]
(x) {color:red}-1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/branch-2.3/67/JDK8_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(/) {color:green}+1 jdk11 hadoop3 checks{color}
-- For more information [see jdk11
report|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/branch-2.3/67/JDK11_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
(x) {color:red}-1 client integration test{color}
--Failed when running client tests on top of Hadoop 2. [see log for
details|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/branch-2.3/67//artifact/output-integration/hadoop-2.log].
(note that this means we didn't run on Hadoop 3)
> Enable table replication fails from 1.x to 2.x if table already exist at
> peer.
> ---
>
> Key: HBASE-24981
> URL: https://issues.apache.org/jira/browse/HBASE-24981
> Project: HBase
> Issue Type: Improvement
> Components: Replication
>Affects Versions: 2.2.3
>Reporter: Sanjeet Nishad
>Assignee: Sanjeet Nishad
>Priority: Major
> Fix For: 3.0.0-alpha-1, 2.3.3, 2.4.0, 2.2.7
>
>
> If source is 1.x and peer is 2.x and a default table is created at source and
> also at peer, default table descriptors mismatch so the enable table
> replication fails with error :
>
> {code:java}
> Table t1 exists in peer cluster 1, but the table descriptors are not same
> when compared with source cluster. Thus can not enable the table's
> replication switch.{code}
> Following are the extra attributes set by default in 2.x:
> CACHE_DATA_ON_WRITE
> CACHE_INDEX_ON_WRITE
> CACHE_BLOOMS_ON_WRITE
> EVICT_BLOCKS_ON_CLOSE
> PREFETCH_BLOCKS_ON_OPEN.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HBASE-25133) Migrate HBase Nightly jenkins job from Hadoop to hbase
[
https://issues.apache.org/jira/browse/HBASE-25133?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17205654#comment-17205654
]
Hudson commented on HBASE-25133:
Results for branch branch-2.3
[build #67 on
builds.a.o|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/branch-2.3/67/]:
(x) *{color:red}-1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/branch-2.3/67/General_20Nightly_20Build_20Report/]
(/) {color:green}+1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/branch-2.3/67/JDK8_20Nightly_20Build_20Report_20_28Hadoop2_29/]
(x) {color:red}-1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/branch-2.3/67/JDK8_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(/) {color:green}+1 jdk11 hadoop3 checks{color}
-- For more information [see jdk11
report|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/branch-2.3/67/JDK11_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
(x) {color:red}-1 client integration test{color}
--Failed when running client tests on top of Hadoop 2. [see log for
details|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/branch-2.3/67//artifact/output-integration/hadoop-2.log].
(note that this means we didn't run on Hadoop 3)
> Migrate HBase Nightly jenkins job from Hadoop to hbase
> --
>
> Key: HBASE-25133
> URL: https://issues.apache.org/jira/browse/HBASE-25133
> Project: HBase
> Issue Type: Sub-task
> Components: scripts
>Reporter: Duo Zhang
>Assignee: Duo Zhang
>Priority: Major
> Fix For: 3.0.0-alpha-1, 2.3.3, 1.7.0, 2.4.0, 1.4.14, 2.2.7
>
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HBASE-25134) Migrate HBase PreCommit jenkins job from Hadoop to hbase
[
https://issues.apache.org/jira/browse/HBASE-25134?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17205655#comment-17205655
]
Hudson commented on HBASE-25134:
Results for branch branch-2.3
[build #67 on
builds.a.o|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/branch-2.3/67/]:
(x) *{color:red}-1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/branch-2.3/67/General_20Nightly_20Build_20Report/]
(/) {color:green}+1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/branch-2.3/67/JDK8_20Nightly_20Build_20Report_20_28Hadoop2_29/]
(x) {color:red}-1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/branch-2.3/67/JDK8_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(/) {color:green}+1 jdk11 hadoop3 checks{color}
-- For more information [see jdk11
report|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/branch-2.3/67/JDK11_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
(x) {color:red}-1 client integration test{color}
--Failed when running client tests on top of Hadoop 2. [see log for
details|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/branch-2.3/67//artifact/output-integration/hadoop-2.log].
(note that this means we didn't run on Hadoop 3)
> Migrate HBase PreCommit jenkins job from Hadoop to hbase
>
>
> Key: HBASE-25134
> URL: https://issues.apache.org/jira/browse/HBASE-25134
> Project: HBase
> Issue Type: Sub-task
> Components: scripts
>Reporter: Duo Zhang
>Assignee: Duo Zhang
>Priority: Major
> Fix For: 3.0.0-alpha-1, 2.3.3, 1.7.0, 2.4.0, 1.4.14, 2.2.7
>
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HBASE-25109) Add MR Counters to WALPlayer; currently hard to tell if it is doing anything
[
https://issues.apache.org/jira/browse/HBASE-25109?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17205652#comment-17205652
]
Hudson commented on HBASE-25109:
Results for branch branch-2.3
[build #67 on
builds.a.o|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/branch-2.3/67/]:
(x) *{color:red}-1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/branch-2.3/67/General_20Nightly_20Build_20Report/]
(/) {color:green}+1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/branch-2.3/67/JDK8_20Nightly_20Build_20Report_20_28Hadoop2_29/]
(x) {color:red}-1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/branch-2.3/67/JDK8_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(/) {color:green}+1 jdk11 hadoop3 checks{color}
-- For more information [see jdk11
report|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/branch-2.3/67/JDK11_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
(x) {color:red}-1 client integration test{color}
--Failed when running client tests on top of Hadoop 2. [see log for
details|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/branch-2.3/67//artifact/output-integration/hadoop-2.log].
(note that this means we didn't run on Hadoop 3)
> Add MR Counters to WALPlayer; currently hard to tell if it is doing anything
>
>
> Key: HBASE-25109
> URL: https://issues.apache.org/jira/browse/HBASE-25109
> Project: HBase
> Issue Type: Improvement
>Reporter: Michael Stack
>Assignee: Michael Stack
>Priority: Major
> Fix For: 3.0.0-alpha-1, 2.3.3, 2.4.0
>
>
> For example, when WALPlayer runs, it emits this:
> {code:java}
> 020-09-28 11:16:05,489 INFO [LocalJobRunner Map Task Executor #0]
> mapred.Task: Final Counters for attempt_local1916643172_0001_m_00_0:
> Counters: 20
> File System Counters
> FILE: Number of bytes read=268891453
> FILE: Number of bytes written=1018719
> FILE: Number of read operations=0
> FILE: Number of large read operations=0
> FILE: Number of write operations=0
> Map-Reduce Framework
> Map input records=4375
> Map output records=5369
> Input split bytes=245
> Spilled Records=0
> Failed Shuffles=0
> Merged Map outputs=0
> GC time elapsed (ms)=59
> Total committed heap usage (bytes)=518979584
> File Input Format Counters
> Bytes Read=0
> File Output Format Counters
> Bytes Written=0 {code}
> Change it so it does this:
> {code:java}
> 020-09-28 11:16:05,489 INFO [LocalJobRunner Map Task Executor #0]
> mapred.Task: Final Counters for attempt_local1916643172_0001_m_00_0:
> Counters: 20
> File System Counters
> FILE: Number of bytes read=268891453
> FILE: Number of bytes written=1018719
> FILE: Number of read operations=0
> FILE: Number of large read operations=0
> FILE: Number of write operations=0
> Map-Reduce Framework
> Map input records=4375
> Map output records=5369
> Input split bytes=245
> Spilled Records=0
> Failed Shuffles=0
> Merged Map outputs=0
> GC time elapsed (ms)=59
> Total committed heap usage (bytes)=518979584
> org.apache.hadoop.hbase.mapreduce.WALPlayer$Counter
> CELLS_READ=89574
> CELLS_WRITTEN=89572
> DELETES=64
> PUTS=5305
> WALEDITS=4375
> File Input Format Counters
> Bytes Read=0
> File Output Format Counters
> Bytes Written=0 {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HBASE-25132) Migrate flaky test jenkins job from Hadoop to hbase
[
https://issues.apache.org/jira/browse/HBASE-25132?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17205653#comment-17205653
]
Hudson commented on HBASE-25132:
Results for branch branch-2.3
[build #67 on
builds.a.o|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/branch-2.3/67/]:
(x) *{color:red}-1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/branch-2.3/67/General_20Nightly_20Build_20Report/]
(/) {color:green}+1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/branch-2.3/67/JDK8_20Nightly_20Build_20Report_20_28Hadoop2_29/]
(x) {color:red}-1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/branch-2.3/67/JDK8_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(/) {color:green}+1 jdk11 hadoop3 checks{color}
-- For more information [see jdk11
report|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/branch-2.3/67/JDK11_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
(x) {color:red}-1 client integration test{color}
--Failed when running client tests on top of Hadoop 2. [see log for
details|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/branch-2.3/67//artifact/output-integration/hadoop-2.log].
(note that this means we didn't run on Hadoop 3)
> Migrate flaky test jenkins job from Hadoop to hbase
> ---
>
> Key: HBASE-25132
> URL: https://issues.apache.org/jira/browse/HBASE-25132
> Project: HBase
> Issue Type: Sub-task
> Components: scripts
>Reporter: Duo Zhang
>Assignee: Duo Zhang
>Priority: Major
> Fix For: 3.0.0-alpha-1, 2.3.3, 1.7.0, 2.4.0, 1.4.14, 2.2.7
>
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[GitHub] [hbase] saintstack commented on a change in pull request #2437: HBASE-25053 WAL replay should ignore 0-length files
saintstack commented on a change in pull request #2437:
URL: https://github.com/apache/hbase/pull/2437#discussion_r498364347
##
File path:
hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/HRegion.java
##
@@ -6423,7 +6427,11 @@ protected void restoreEdit(HStore s, Cell cell,
MemStoreSizing memstoreAccountin
* @throws IOException
*/
private static boolean isZeroLengthThenDelete(final FileSystem fs, final
FileStatus stat,
- final Path p) throws IOException {
+ final Path p, Configuration conf) throws IOException {
+if (stat.getLen() > 0) {
+ return false;
+}
+RecoverLeaseFSUtils.recoverFileLease(fs, p, conf);
Review comment:
I see now (was resolved so I missed it).
Then the patch looks good. A nit is that we could do w/ a comment here that
says it is safe to trust the zero-length in this case because we've been
through rename and lease recovery in the above. Thanks.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #2481: HBASE-25115 HFilePrettyPrinter can't seek to the row which is the first row of a hfile
Apache-HBase commented on pull request #2481: URL: https://github.com/apache/hbase/pull/2481#issuecomment-702232565 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 0m 38s | Docker mode activated. | ||| _ Prechecks _ | | +1 :green_heart: | dupname | 0m 1s | No case conflicting files found. | | +1 :green_heart: | hbaseanti | 0m 0s | Patch does not have any anti-patterns. | | +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. | | -0 :warning: | test4tests | 0m 0s | The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. | ||| _ branch-2.2 Compile Tests _ | | +1 :green_heart: | mvninstall | 3m 38s | branch-2.2 passed | | +1 :green_heart: | compile | 1m 1s | branch-2.2 passed | | +1 :green_heart: | checkstyle | 1m 21s | branch-2.2 passed | | +1 :green_heart: | shadedjars | 4m 24s | branch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 0m 41s | branch-2.2 passed | | +0 :ok: | spotbugs | 3m 47s | Used deprecated FindBugs config; considering switching to SpotBugs. | | +1 :green_heart: | findbugs | 3m 44s | branch-2.2 passed | ||| _ Patch Compile Tests _ | | +1 :green_heart: | mvninstall | 3m 8s | the patch passed | | +1 :green_heart: | compile | 1m 1s | the patch passed | | +1 :green_heart: | javac | 1m 1s | the patch passed | | +1 :green_heart: | checkstyle | 1m 22s | the patch passed | | +1 :green_heart: | whitespace | 0m 0s | The patch has no whitespace issues. | | +1 :green_heart: | shadedjars | 4m 19s | patch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | hadoopcheck | 26m 46s | Patch does not cause any errors with Hadoop 2.8.5 2.9.2 2.10.0 or 3.1.2 3.2.1. | | +1 :green_heart: | javadoc | 0m 38s | the patch passed | | +1 :green_heart: | findbugs | 3m 40s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | unit | 178m 18s | hbase-server in the patch passed. | | +1 :green_heart: | asflicense | 0m 41s | The patch does not generate ASF License warnings. | | | | 241m 49s | | | Subsystem | Report/Notes | |--:|:-| | Docker | Client=19.03.13 Server=19.03.13 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2481/2/artifact/out/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/2481 | | Optional Tests | dupname asflicense javac javadoc unit spotbugs findbugs shadedjars hadoopcheck hbaseanti checkstyle compile | | uname | Linux 4d968ae1f342 4.15.0-58-generic #64-Ubuntu SMP Tue Aug 6 11:12:41 UTC 2019 x86_64 GNU/Linux | | Build tool | maven | | Personality | /home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-2481/out/precommit/personality/provided.sh | | git revision | branch-2.2 / 27f453fb21 | | Default Java | 1.8.0_181 | | Test Results | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2481/2/testReport/ | | Max. process+thread count | 3518 (vs. ulimit of 12500) | | modules | C: hbase-server U: hbase-server | | Console output | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2481/2/console | | versions | git=2.11.0 maven=2018-06-17T18:33:14Z) findbugs=3.1.11 | | Powered by | Apache Yetus 0.11.1 https://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Updated] (HBASE-25140) HBase test mini cluster is working only with Hadoop 2.8.0 - 3.0.3
[
https://issues.apache.org/jira/browse/HBASE-25140?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Miklos Gergely updated HBASE-25140:
---
Attachment: mvn-1.log
> HBase test mini cluster is working only with Hadoop 2.8.0 - 3.0.3
> -
>
> Key: HBASE-25140
> URL: https://issues.apache.org/jira/browse/HBASE-25140
> Project: HBase
> Issue Type: Bug
> Components: documentation, hadoop2, test
>Affects Versions: 2.2.3
>Reporter: Miklos Gergely
>Priority: Major
> Attachments: mvn-1.log
>
>
> Running HBaseTestingUtility.startMiniCluster() on HBase 2.2.3 works only with
> hadoop version range 2.8.0 - 3.0.3, for example with 2.4.1 the following
> exception occurs:
>
> {code:java}
> 21:49:04,124 [RS:0;71af2d647bb3:35715] ERROR
> org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper [] -
> Couldn't properly initialize access to HDFS internals. Please update your WAL
> Provider to not make use of the 'asyncfs' provider. See HBASE-16110 for more
> information.21:49:04,124 [RS:0;71af2d647bb3:35715] ERROR
> org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper [] -
> Couldn't properly initialize access to HDFS internals. Please update your WAL
> Provider to not make use of the 'asyncfs' provider. See HBASE-16110 for more
> information.java.lang.NoSuchMethodException:
> org.apache.hadoop.hdfs.DFSClient.beginFileLease(long,
> org.apache.hadoop.hdfs.DFSOutputStream) at
> java.lang.Class.getDeclaredMethod(Class.java:2130) ~[?:1.8.0_242] at
> org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper.createLeaseManager(FanOutOneBlockAsyncDFSOutputHelper.java:198)
> ~[hbase-server-2.2.3.jar:2.2.3] at
> org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper.(FanOutOneBlockAsyncDFSOutputHelper.java:274)
> [hbase-server-2.2.3.jar:2.2.3] at java.lang.Class.forName0(Native Method)
> ~[?:1.8.0_242] at java.lang.Class.forName(Class.java:264) [?:1.8.0_242] at
> org.apache.hadoop.hbase.wal.AsyncFSWALProvider.load(AsyncFSWALProvider.java:136)
> [hbase-server-2.2.3.jar:2.2.3] at
> org.apache.hadoop.hbase.wal.WALFactory.getProviderClass(WALFactory.java:136)
> [hbase-server-2.2.3.jar:2.2.3] at
> org.apache.hadoop.hbase.wal.WALFactory.getProvider(WALFactory.java:175)
> [hbase-server-2.2.3.jar:2.2.3] at
> org.apache.hadoop.hbase.wal.WALFactory.(WALFactory.java:198)
> [hbase-server-2.2.3.jar:2.2.3] at
> org.apache.hadoop.hbase.regionserver.HRegionServer.setupWALAndReplication(HRegionServer.java:1871)
> [hbase-server-2.2.3.jar:2.2.3] at
> org.apache.hadoop.hbase.regionserver.HRegionServer.handleReportForDutyResponse(HRegionServer.java:1589)
> [hbase-server-2.2.3.jar:2.2.3] at
> org.apache.hadoop.hbase.MiniHBaseCluster$MiniHBaseClusterRegionServer.handleReportForDutyResponse(MiniHBaseCluster.java:157)
> [hbase-server-2.2.3-tests.jar:2.2.3] at
> org.apache.hadoop.hbase.regionserver.HRegionServer.run(HRegionServer.java:1001)
> [hbase-server-2.2.3.jar:2.2.3] at
> org.apache.hadoop.hbase.MiniHBaseCluster$MiniHBaseClusterRegionServer.runRegionServer(MiniHBaseCluster.java:184)
> [hbase-server-2.2.3-tests.jar:2.2.3] at
> org.apache.hadoop.hbase.MiniHBaseCluster$MiniHBaseClusterRegionServer.access$000(MiniHBaseCluster.java:130)
> [hbase-server-2.2.3-tests.jar:2.2.3] at
> org.apache.hadoop.hbase.MiniHBaseCluster$MiniHBaseClusterRegionServer$1.run(MiniHBaseCluster.java:168)
> [hbase-server-2.2.3-tests.jar:2.2.3] at
> java.security.AccessController.doPrivileged(Native Method) ~[?:1.8.0_242] at
> javax.security.auth.Subject.doAs(Subject.java:360) [?:1.8.0_242] at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1536)
> [hadoop-common-2.4.1.jar:?] at
> org.apache.hadoop.hbase.security.User$SecureHadoopUser.runAs(User.java:341)
> [hbase-common-2.2.3.jar:2.2.3] at
> org.apache.hadoop.hbase.MiniHBaseCluster$MiniHBaseClusterRegionServer.run(MiniHBaseCluster.java:165)
> [hbase-server-2.2.3-tests.jar:2.2.3] at
> java.lang.Thread.run(Thread.java:748) [?:1.8.0_242]
> {code}
> Also upon failure during maven run it would be great if the actual exception
> would be displayed, not just that "Master not initialized after 20ms".
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Comment Edited] (HBASE-25140) HBase test mini cluster is working only with Hadoop 2.8.0 - 3.0.3
[
https://issues.apache.org/jira/browse/HBASE-25140?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17205614#comment-17205614
]
Miklos Gergely edited comment on HBASE-25140 at 10/1/20, 3:34 PM:
--
Yes, I meant hadoop 2.4.1, but I as it is not supported, it is not relelvant.
At Flink our nightly tests run with hadoop 2.4.1 and 3.1.3 and they both fail,
for 3.1.3 check this:
[https://dev.azure.com/apache-flink/apache-flink/_build/results?buildId=7093&view=logs&j=ba53eb01-1462-56a3-8e98-0dd97fbcaab5&t=bfbc6239-57a0-5db0-63f3-41551b4f7d51]
This is the maven output, as you can see it dislays onlly that "Master not
initialized after 20ms". You can find the actual logs with the min cluster
failure here at the attached log file, here is the error:
{code:java}
java.lang.IncompatibleClassChangeError: Found interface
org.apache.hadoop.hdfs.protocol.HdfsFileStatus, but class was expected
at
org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper.createOutput(FanOutOneBlockAsyncDFSOutputHelper.java:496)
at
org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper.access$400(FanOutOneBlockAsyncDFSOutputHelper.java:116)
at
org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper$8.doCall(FanOutOneBlockAsyncDFSOutputHelper.java:576)
at
org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper$8.doCall(FanOutOneBlockAsyncDFSOutputHelper.java:571)
at
org.apache.hadoop.fs.FileSystemLinkResolver.resolve(FileSystemLinkResolver.java:81)
at
org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper.createOutput(FanOutOneBlockAsyncDFSOutputHelper.java:584)
at
org.apache.hadoop.hbase.io.asyncfs.AsyncFSOutputHelper.createOutput(AsyncFSOutputHelper.java:51)
at
org.apache.hadoop.hbase.regionserver.wal.AsyncProtobufLogWriter.initOutput(AsyncProtobufLogWriter.java:169)
at
org.apache.hadoop.hbase.regionserver.wal.AbstractProtobufLogWriter.init(AbstractProtobufLogWriter.java:166)
at
org.apache.hadoop.hbase.wal.AsyncFSWALProvider.createAsyncWriter(AsyncFSWALProvider.java:113)
at
org.apache.hadoop.hbase.regionserver.wal.AsyncFSWAL.createWriterInstance(AsyncFSWAL.java:643)
at
org.apache.hadoop.hbase.regionserver.wal.AsyncFSWAL.createWriterInstance(AsyncFSWAL.java:126)
at
org.apache.hadoop.hbase.regionserver.wal.AbstractFSWAL.rollWriter(AbstractFSWAL.java:767)
at
org.apache.hadoop.hbase.regionserver.wal.AbstractFSWAL.rollWriter(AbstractFSWAL.java:501)
at
org.apache.hadoop.hbase.regionserver.wal.AbstractFSWAL.init(AbstractFSWAL.java:442)
at
org.apache.hadoop.hbase.wal.AbstractFSWALProvider.getWAL(AbstractFSWALProvider.java:156)
at
org.apache.hadoop.hbase.wal.AbstractFSWALProvider.getWAL(AbstractFSWALProvider.java:61)
at org.apache.hadoop.hbase.wal.WALFactory.getWAL(WALFactory.java:284)
at
org.apache.hadoop.hbase.regionserver.HRegionServer.getWAL(HRegionServer.java:2181)
at
org.apache.hadoop.hbase.regionserver.handler.AssignRegionHandler.process(AssignRegionHandler.java:133)
at
org.apache.hadoop.hbase.executor.EventHandler.run(EventHandler.java:104)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
{code}
One way to reproduce this:
# git clone [https://github.com/apache/flink.git]
# mvn clean install -DskipTests -Dfast # this will take about 30 minutes
# edit
flink-connectors/flink-connector-hbase-2.2/src/test/java/org/apache/flink/connector/hbase2/util/HBaseTestingClusterAutoStarter.java
- modify the range at line 57 to include 3.1.3
# cd flink-connectors/flink-connector-hbase-2.2
# mvn verify -Dit.test=HBaseConnectorITCase -Dhadoop.version=3.1.3
was (Author: mgergely):
Yes, I meant hadoop 2.4.1, but I as it is not supported, it is not relelvant.
At Flink our nightly tests run with hadoop 2.4.1 and 3.1.3 and they both fail,
for 3.1.3 check this:
[https://dev.azure.com/apache-flink/apache-flink/_build/results?buildId=7093&view=logs&j=ba53eb01-1462-56a3-8e98-0dd97fbcaab5&t=bfbc6239-57a0-5db0-63f3-41551b4f7d51]
This is the maven output, as you can see it dislays onlly that "Master not
initialized after 20ms". You can find the actual logs with the min cluster
failure here at the attached log file, here is the error:
{code:java}
java.lang.IncompatibleClassChangeError: Found interface
org.apache.hadoop.hdfs.protocol.HdfsFileStatus, but class was expected
at
org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper.createOutput(FanOutOneBlockAsyncDFSOutputHelper.java:496)
at
org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper.access$40
[jira] [Commented] (HBASE-25140) HBase test mini cluster is working only with Hadoop 2.8.0 - 3.0.3
[
https://issues.apache.org/jira/browse/HBASE-25140?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17205614#comment-17205614
]
Miklos Gergely commented on HBASE-25140:
Yes, I meant hadoop 2.4.1, but I as it is not supported, it is not relelvant.
At Flink our nightly tests run with hadoop 2.4.1 and 3.1.3 and they both fail,
for 3.1.3 check this:
[https://dev.azure.com/apache-flink/apache-flink/_build/results?buildId=7093&view=logs&j=ba53eb01-1462-56a3-8e98-0dd97fbcaab5&t=bfbc6239-57a0-5db0-63f3-41551b4f7d51]
This is the maven output, as you can see it dislays onlly that "Master not
initialized after 20ms". You can find the actual logs with the min cluster
failure here at the attached log file, here is the error:
{code:java}
java.lang.IncompatibleClassChangeError: Found interface
org.apache.hadoop.hdfs.protocol.HdfsFileStatus, but class was expected
at
org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper.createOutput(FanOutOneBlockAsyncDFSOutputHelper.java:496)
at
org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper.access$400(FanOutOneBlockAsyncDFSOutputHelper.java:116)
at
org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper$8.doCall(FanOutOneBlockAsyncDFSOutputHelper.java:576)
at
org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper$8.doCall(FanOutOneBlockAsyncDFSOutputHelper.java:571)
at
org.apache.hadoop.fs.FileSystemLinkResolver.resolve(FileSystemLinkResolver.java:81)
at
org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper.createOutput(FanOutOneBlockAsyncDFSOutputHelper.java:584)
at
org.apache.hadoop.hbase.io.asyncfs.AsyncFSOutputHelper.createOutput(AsyncFSOutputHelper.java:51)
at
org.apache.hadoop.hbase.regionserver.wal.AsyncProtobufLogWriter.initOutput(AsyncProtobufLogWriter.java:169)
at
org.apache.hadoop.hbase.regionserver.wal.AbstractProtobufLogWriter.init(AbstractProtobufLogWriter.java:166)
at
org.apache.hadoop.hbase.wal.AsyncFSWALProvider.createAsyncWriter(AsyncFSWALProvider.java:113)
at
org.apache.hadoop.hbase.regionserver.wal.AsyncFSWAL.createWriterInstance(AsyncFSWAL.java:643)
at
org.apache.hadoop.hbase.regionserver.wal.AsyncFSWAL.createWriterInstance(AsyncFSWAL.java:126)
at
org.apache.hadoop.hbase.regionserver.wal.AbstractFSWAL.rollWriter(AbstractFSWAL.java:767)
at
org.apache.hadoop.hbase.regionserver.wal.AbstractFSWAL.rollWriter(AbstractFSWAL.java:501)
at
org.apache.hadoop.hbase.regionserver.wal.AbstractFSWAL.init(AbstractFSWAL.java:442)
at
org.apache.hadoop.hbase.wal.AbstractFSWALProvider.getWAL(AbstractFSWALProvider.java:156)
at
org.apache.hadoop.hbase.wal.AbstractFSWALProvider.getWAL(AbstractFSWALProvider.java:61)
at org.apache.hadoop.hbase.wal.WALFactory.getWAL(WALFactory.java:284)
at
org.apache.hadoop.hbase.regionserver.HRegionServer.getWAL(HRegionServer.java:2181)
at
org.apache.hadoop.hbase.regionserver.handler.AssignRegionHandler.process(AssignRegionHandler.java:133)
at
org.apache.hadoop.hbase.executor.EventHandler.run(EventHandler.java:104)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
{code}
One way to reproduce this:
> HBase test mini cluster is working only with Hadoop 2.8.0 - 3.0.3
> -
>
> Key: HBASE-25140
> URL: https://issues.apache.org/jira/browse/HBASE-25140
> Project: HBase
> Issue Type: Bug
> Components: documentation, hadoop2, test
>Affects Versions: 2.2.3
>Reporter: Miklos Gergely
>Priority: Major
>
> Running HBaseTestingUtility.startMiniCluster() on HBase 2.2.3 works only with
> hadoop version range 2.8.0 - 3.0.3, for example with 2.4.1 the following
> exception occurs:
>
> {code:java}
> 21:49:04,124 [RS:0;71af2d647bb3:35715] ERROR
> org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper [] -
> Couldn't properly initialize access to HDFS internals. Please update your WAL
> Provider to not make use of the 'asyncfs' provider. See HBASE-16110 for more
> information.21:49:04,124 [RS:0;71af2d647bb3:35715] ERROR
> org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper [] -
> Couldn't properly initialize access to HDFS internals. Please update your WAL
> Provider to not make use of the 'asyncfs' provider. See HBASE-16110 for more
> information.java.lang.NoSuchMethodException:
> org.apache.hadoop.hdfs.DFSClient.beginFileLease(long,
> org.apache.hadoop.hdfs.DFSOutputStream) at
> java.lang.Class.getDeclaredMethod(Class.java:2130) ~[?:
[GitHub] [hbase] virajjasani commented on a change in pull request #2438: HBASE-25033 : Provide an option to ignore svn ci for create-release
virajjasani commented on a change in pull request #2438: URL: https://github.com/apache/hbase/pull/2438#discussion_r498328243 ## File path: dev-support/create-release/do-release-docker.sh ## @@ -77,6 +77,10 @@ Options: If none specified, runs tag, then publish-dist, and then publish-release. 'publish-snapshot' is also an allowed, less used, option. -x debug. do less clean up. (env file, gpg forwarding on mac) + -i [ignore-svn-ci] Review comment: Btw if we are good with uploading to h.a.o/p.a.o, perhaps we might not need ssh-agent either. We can mount ~/.ssh directly in docker image and that should help run password-less `scp` without issues. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Updated] (HBASE-25140) HBase test mini cluster is working only with Hadoop 2.8.0 - 3.0.3
[
https://issues.apache.org/jira/browse/HBASE-25140?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sean Busbey updated HBASE-25140:
Component/s: test
hadoop2
documentation
> HBase test mini cluster is working only with Hadoop 2.8.0 - 3.0.3
> -
>
> Key: HBASE-25140
> URL: https://issues.apache.org/jira/browse/HBASE-25140
> Project: HBase
> Issue Type: Bug
> Components: documentation, hadoop2, test
>Affects Versions: 2.2.3
>Reporter: Miklos Gergely
>Priority: Major
>
> Running HBaseTestingUtility.startMiniCluster() on HBase 2.2.3 works only with
> hadoop version range 2.8.0 - 3.0.3, for example with 2.4.1 the following
> exception occurs:
>
> {code:java}
> 21:49:04,124 [RS:0;71af2d647bb3:35715] ERROR
> org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper [] -
> Couldn't properly initialize access to HDFS internals. Please update your WAL
> Provider to not make use of the 'asyncfs' provider. See HBASE-16110 for more
> information.21:49:04,124 [RS:0;71af2d647bb3:35715] ERROR
> org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper [] -
> Couldn't properly initialize access to HDFS internals. Please update your WAL
> Provider to not make use of the 'asyncfs' provider. See HBASE-16110 for more
> information.java.lang.NoSuchMethodException:
> org.apache.hadoop.hdfs.DFSClient.beginFileLease(long,
> org.apache.hadoop.hdfs.DFSOutputStream) at
> java.lang.Class.getDeclaredMethod(Class.java:2130) ~[?:1.8.0_242] at
> org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper.createLeaseManager(FanOutOneBlockAsyncDFSOutputHelper.java:198)
> ~[hbase-server-2.2.3.jar:2.2.3] at
> org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper.(FanOutOneBlockAsyncDFSOutputHelper.java:274)
> [hbase-server-2.2.3.jar:2.2.3] at java.lang.Class.forName0(Native Method)
> ~[?:1.8.0_242] at java.lang.Class.forName(Class.java:264) [?:1.8.0_242] at
> org.apache.hadoop.hbase.wal.AsyncFSWALProvider.load(AsyncFSWALProvider.java:136)
> [hbase-server-2.2.3.jar:2.2.3] at
> org.apache.hadoop.hbase.wal.WALFactory.getProviderClass(WALFactory.java:136)
> [hbase-server-2.2.3.jar:2.2.3] at
> org.apache.hadoop.hbase.wal.WALFactory.getProvider(WALFactory.java:175)
> [hbase-server-2.2.3.jar:2.2.3] at
> org.apache.hadoop.hbase.wal.WALFactory.(WALFactory.java:198)
> [hbase-server-2.2.3.jar:2.2.3] at
> org.apache.hadoop.hbase.regionserver.HRegionServer.setupWALAndReplication(HRegionServer.java:1871)
> [hbase-server-2.2.3.jar:2.2.3] at
> org.apache.hadoop.hbase.regionserver.HRegionServer.handleReportForDutyResponse(HRegionServer.java:1589)
> [hbase-server-2.2.3.jar:2.2.3] at
> org.apache.hadoop.hbase.MiniHBaseCluster$MiniHBaseClusterRegionServer.handleReportForDutyResponse(MiniHBaseCluster.java:157)
> [hbase-server-2.2.3-tests.jar:2.2.3] at
> org.apache.hadoop.hbase.regionserver.HRegionServer.run(HRegionServer.java:1001)
> [hbase-server-2.2.3.jar:2.2.3] at
> org.apache.hadoop.hbase.MiniHBaseCluster$MiniHBaseClusterRegionServer.runRegionServer(MiniHBaseCluster.java:184)
> [hbase-server-2.2.3-tests.jar:2.2.3] at
> org.apache.hadoop.hbase.MiniHBaseCluster$MiniHBaseClusterRegionServer.access$000(MiniHBaseCluster.java:130)
> [hbase-server-2.2.3-tests.jar:2.2.3] at
> org.apache.hadoop.hbase.MiniHBaseCluster$MiniHBaseClusterRegionServer$1.run(MiniHBaseCluster.java:168)
> [hbase-server-2.2.3-tests.jar:2.2.3] at
> java.security.AccessController.doPrivileged(Native Method) ~[?:1.8.0_242] at
> javax.security.auth.Subject.doAs(Subject.java:360) [?:1.8.0_242] at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1536)
> [hadoop-common-2.4.1.jar:?] at
> org.apache.hadoop.hbase.security.User$SecureHadoopUser.runAs(User.java:341)
> [hbase-common-2.2.3.jar:2.2.3] at
> org.apache.hadoop.hbase.MiniHBaseCluster$MiniHBaseClusterRegionServer.run(MiniHBaseCluster.java:165)
> [hbase-server-2.2.3-tests.jar:2.2.3] at
> java.lang.Thread.run(Thread.java:748) [?:1.8.0_242]
> {code}
> Also upon failure during maven run it would be great if the actual exception
> would be displayed, not just that "Master not initialized after 20ms".
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HBASE-25140) HBase test mini cluster is working only with Hadoop 2.8.0 - 3.0.3
[
https://issues.apache.org/jira/browse/HBASE-25140?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17205589#comment-17205589
]
Sean Busbey commented on HBASE-25140:
-
{quote}
for example with 2.4.1 the following exception occurs:
{quote}
Do you mean with Hadoop 2.4.1?
{quote}
HBaseTestingUtility.startMiniCluster() on HBase 2.2.3 works only with hadoop
version range 2.8.0 - 3.0.3
{quote}
Please see our [reference guide for the expected Hadoop
compatibility|http://hbase.apache.org/book.html#hadoop]. HBase 2.2.z releases
should be used with Hadoop 2.8, 2.9, 3.1, and 3.2 (with specific maintenance
versions mattering for some of those releases). Is the minicluster failing with
Hadoop 3.1 or 3.2? How are you setting up dependencies?
{quote}
Also upon failure during maven run it would be great if the actual exception
would be displayed, not just that "Master not initialized after 20ms".
{quote}
Given the problem with the fan-out wal writer you posted I am surprised the
entire minicluster did not fail with a clear pointer to that message. Could you
attach logs?
> HBase test mini cluster is working only with Hadoop 2.8.0 - 3.0.3
> -
>
> Key: HBASE-25140
> URL: https://issues.apache.org/jira/browse/HBASE-25140
> Project: HBase
> Issue Type: Bug
>Affects Versions: 2.2.3
>Reporter: Miklos Gergely
>Priority: Major
>
> Running HBaseTestingUtility.startMiniCluster() on HBase 2.2.3 works only with
> hadoop version range 2.8.0 - 3.0.3, for example with 2.4.1 the following
> exception occurs:
>
> {code:java}
> 21:49:04,124 [RS:0;71af2d647bb3:35715] ERROR
> org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper [] -
> Couldn't properly initialize access to HDFS internals. Please update your WAL
> Provider to not make use of the 'asyncfs' provider. See HBASE-16110 for more
> information.21:49:04,124 [RS:0;71af2d647bb3:35715] ERROR
> org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper [] -
> Couldn't properly initialize access to HDFS internals. Please update your WAL
> Provider to not make use of the 'asyncfs' provider. See HBASE-16110 for more
> information.java.lang.NoSuchMethodException:
> org.apache.hadoop.hdfs.DFSClient.beginFileLease(long,
> org.apache.hadoop.hdfs.DFSOutputStream) at
> java.lang.Class.getDeclaredMethod(Class.java:2130) ~[?:1.8.0_242] at
> org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper.createLeaseManager(FanOutOneBlockAsyncDFSOutputHelper.java:198)
> ~[hbase-server-2.2.3.jar:2.2.3] at
> org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper.(FanOutOneBlockAsyncDFSOutputHelper.java:274)
> [hbase-server-2.2.3.jar:2.2.3] at java.lang.Class.forName0(Native Method)
> ~[?:1.8.0_242] at java.lang.Class.forName(Class.java:264) [?:1.8.0_242] at
> org.apache.hadoop.hbase.wal.AsyncFSWALProvider.load(AsyncFSWALProvider.java:136)
> [hbase-server-2.2.3.jar:2.2.3] at
> org.apache.hadoop.hbase.wal.WALFactory.getProviderClass(WALFactory.java:136)
> [hbase-server-2.2.3.jar:2.2.3] at
> org.apache.hadoop.hbase.wal.WALFactory.getProvider(WALFactory.java:175)
> [hbase-server-2.2.3.jar:2.2.3] at
> org.apache.hadoop.hbase.wal.WALFactory.(WALFactory.java:198)
> [hbase-server-2.2.3.jar:2.2.3] at
> org.apache.hadoop.hbase.regionserver.HRegionServer.setupWALAndReplication(HRegionServer.java:1871)
> [hbase-server-2.2.3.jar:2.2.3] at
> org.apache.hadoop.hbase.regionserver.HRegionServer.handleReportForDutyResponse(HRegionServer.java:1589)
> [hbase-server-2.2.3.jar:2.2.3] at
> org.apache.hadoop.hbase.MiniHBaseCluster$MiniHBaseClusterRegionServer.handleReportForDutyResponse(MiniHBaseCluster.java:157)
> [hbase-server-2.2.3-tests.jar:2.2.3] at
> org.apache.hadoop.hbase.regionserver.HRegionServer.run(HRegionServer.java:1001)
> [hbase-server-2.2.3.jar:2.2.3] at
> org.apache.hadoop.hbase.MiniHBaseCluster$MiniHBaseClusterRegionServer.runRegionServer(MiniHBaseCluster.java:184)
> [hbase-server-2.2.3-tests.jar:2.2.3] at
> org.apache.hadoop.hbase.MiniHBaseCluster$MiniHBaseClusterRegionServer.access$000(MiniHBaseCluster.java:130)
> [hbase-server-2.2.3-tests.jar:2.2.3] at
> org.apache.hadoop.hbase.MiniHBaseCluster$MiniHBaseClusterRegionServer$1.run(MiniHBaseCluster.java:168)
> [hbase-server-2.2.3-tests.jar:2.2.3] at
> java.security.AccessController.doPrivileged(Native Method) ~[?:1.8.0_242] at
> javax.security.auth.Subject.doAs(Subject.java:360) [?:1.8.0_242] at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1536)
> [hadoop-common-2.4.1.jar:?] at
> org.apache.hadoop.hbase.security.User$SecureHadoopUser.runAs(User.java:341)
> [hbase-common-2.2.3.jar:2.2.3] at
> org.apache.hadoop.hbase.MiniHBaseCluster$MiniHBaseClusterRegionServer.run(MiniHBaseCluster.
[jira] [Created] (HBASE-25141) ref guide for 2.2.z builds needs to be updated to include 2.2 specific details
Sean Busbey created HBASE-25141: --- Summary: ref guide for 2.2.z builds needs to be updated to include 2.2 specific details Key: HBASE-25141 URL: https://issues.apache.org/jira/browse/HBASE-25141 Project: HBase Issue Type: Task Components: documentation, website Reporter: Sean Busbey Fix For: 2.2.7 for example, it's currently missing a column for 2.2 in the hadoop compat matrix. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HBASE-25141) ref guide for 2.2.z builds needs to be updated to include 2.2 specific details
[ https://issues.apache.org/jira/browse/HBASE-25141?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Busbey updated HBASE-25141: Priority: Minor (was: Major) > ref guide for 2.2.z builds needs to be updated to include 2.2 specific details > -- > > Key: HBASE-25141 > URL: https://issues.apache.org/jira/browse/HBASE-25141 > Project: HBase > Issue Type: Task > Components: documentation, website >Reporter: Sean Busbey >Priority: Minor > Fix For: 2.2.7 > > > for example, it's currently missing a column for 2.2 in the hadoop compat > matrix. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Comment Edited] (HBASE-25123) Add possibility to set different types of L1 cache
[ https://issues.apache.org/jira/browse/HBASE-25123?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17205575#comment-17205575 ] Sean Busbey edited comment on HBASE-25123 at 10/1/20, 2:48 PM: --- if we abstracted out a policy for deciding if a block gets cached and which blocks get evicted that would be enough to isolate the changes needed for HBASE-23887 I think? was (Author: busbey): if we abstracted out a policy for deciding if a block gets cacheing and which blocks get evicted that would be enough to isolate the changes needed for HBASE-23887 I think? > Add possibility to set different types of L1 cache > -- > > Key: HBASE-25123 > URL: https://issues.apache.org/jira/browse/HBASE-25123 > Project: HBase > Issue Type: New Feature > Components: BlockCache >Reporter: Danil Lipovoy >Priority: Minor > > The feature HBASE-23887 allow speed up to 3 times read performance but maybe > it is too complicated. So there are proposals give users possibility to > choose type of cache L1. Looks like it needs to change few classes > (CombinedBlockCache, InclusiveCombinedBlockCache, CacheConfig) if somebody > can code this it would be cool. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (HBASE-25123) Add possibility to set different types of L1 cache
[ https://issues.apache.org/jira/browse/HBASE-25123?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17205575#comment-17205575 ] Sean Busbey commented on HBASE-25123: - if we abstracted out a policy for deciding if a block gets cacheing and which blocks get evicted that would be enough to isolate the changes needed for HBASE-23887 I think? > Add possibility to set different types of L1 cache > -- > > Key: HBASE-25123 > URL: https://issues.apache.org/jira/browse/HBASE-25123 > Project: HBase > Issue Type: New Feature > Components: BlockCache >Reporter: Danil Lipovoy >Priority: Minor > > The feature HBASE-23887 allow speed up to 3 times read performance but maybe > it is too complicated. So there are proposals give users possibility to > choose type of cache L1. Looks like it needs to change few classes > (CombinedBlockCache, InclusiveCombinedBlockCache, CacheConfig) if somebody > can code this it would be cool. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HBASE-25123) Add possibility to set different types of L1 cache
[ https://issues.apache.org/jira/browse/HBASE-25123?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Busbey updated HBASE-25123: Component/s: BlockCache > Add possibility to set different types of L1 cache > -- > > Key: HBASE-25123 > URL: https://issues.apache.org/jira/browse/HBASE-25123 > Project: HBase > Issue Type: New Feature > Components: BlockCache >Reporter: Danil Lipovoy >Priority: Minor > > The feature HBASE-23887 allow speed up to 3 times read performance but maybe > it is too complicated. So there are proposals give users possibility to > choose type of cache L1. Looks like it needs to change few classes > (CombinedBlockCache, InclusiveCombinedBlockCache, CacheConfig) if somebody > can code this it would be cool. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (HBASE-25140) HBase test mini cluster is working only with Hadoop 2.8.0 - 3.0.3
Miklos Gergely created HBASE-25140:
--
Summary: HBase test mini cluster is working only with Hadoop 2.8.0
- 3.0.3
Key: HBASE-25140
URL: https://issues.apache.org/jira/browse/HBASE-25140
Project: HBase
Issue Type: Bug
Affects Versions: 2.2.3
Reporter: Miklos Gergely
Running HBaseTestingUtility.startMiniCluster() on HBase 2.2.3 works only with
hadoop version range 2.8.0 - 3.0.3, for example with 2.4.1 the following
exception occurs:
{code:java}
21:49:04,124 [RS:0;71af2d647bb3:35715] ERROR
org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper [] -
Couldn't properly initialize access to HDFS internals. Please update your WAL
Provider to not make use of the 'asyncfs' provider. See HBASE-16110 for more
information.21:49:04,124 [RS:0;71af2d647bb3:35715] ERROR
org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper [] -
Couldn't properly initialize access to HDFS internals. Please update your WAL
Provider to not make use of the 'asyncfs' provider. See HBASE-16110 for more
information.java.lang.NoSuchMethodException:
org.apache.hadoop.hdfs.DFSClient.beginFileLease(long,
org.apache.hadoop.hdfs.DFSOutputStream) at
java.lang.Class.getDeclaredMethod(Class.java:2130) ~[?:1.8.0_242] at
org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper.createLeaseManager(FanOutOneBlockAsyncDFSOutputHelper.java:198)
~[hbase-server-2.2.3.jar:2.2.3] at
org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper.(FanOutOneBlockAsyncDFSOutputHelper.java:274)
[hbase-server-2.2.3.jar:2.2.3] at java.lang.Class.forName0(Native Method)
~[?:1.8.0_242] at java.lang.Class.forName(Class.java:264) [?:1.8.0_242] at
org.apache.hadoop.hbase.wal.AsyncFSWALProvider.load(AsyncFSWALProvider.java:136)
[hbase-server-2.2.3.jar:2.2.3] at
org.apache.hadoop.hbase.wal.WALFactory.getProviderClass(WALFactory.java:136)
[hbase-server-2.2.3.jar:2.2.3] at
org.apache.hadoop.hbase.wal.WALFactory.getProvider(WALFactory.java:175)
[hbase-server-2.2.3.jar:2.2.3] at
org.apache.hadoop.hbase.wal.WALFactory.(WALFactory.java:198)
[hbase-server-2.2.3.jar:2.2.3] at
org.apache.hadoop.hbase.regionserver.HRegionServer.setupWALAndReplication(HRegionServer.java:1871)
[hbase-server-2.2.3.jar:2.2.3] at
org.apache.hadoop.hbase.regionserver.HRegionServer.handleReportForDutyResponse(HRegionServer.java:1589)
[hbase-server-2.2.3.jar:2.2.3] at
org.apache.hadoop.hbase.MiniHBaseCluster$MiniHBaseClusterRegionServer.handleReportForDutyResponse(MiniHBaseCluster.java:157)
[hbase-server-2.2.3-tests.jar:2.2.3] at
org.apache.hadoop.hbase.regionserver.HRegionServer.run(HRegionServer.java:1001)
[hbase-server-2.2.3.jar:2.2.3] at
org.apache.hadoop.hbase.MiniHBaseCluster$MiniHBaseClusterRegionServer.runRegionServer(MiniHBaseCluster.java:184)
[hbase-server-2.2.3-tests.jar:2.2.3] at
org.apache.hadoop.hbase.MiniHBaseCluster$MiniHBaseClusterRegionServer.access$000(MiniHBaseCluster.java:130)
[hbase-server-2.2.3-tests.jar:2.2.3] at
org.apache.hadoop.hbase.MiniHBaseCluster$MiniHBaseClusterRegionServer$1.run(MiniHBaseCluster.java:168)
[hbase-server-2.2.3-tests.jar:2.2.3] at
java.security.AccessController.doPrivileged(Native Method) ~[?:1.8.0_242] at
javax.security.auth.Subject.doAs(Subject.java:360) [?:1.8.0_242] at
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1536)
[hadoop-common-2.4.1.jar:?] at
org.apache.hadoop.hbase.security.User$SecureHadoopUser.runAs(User.java:341)
[hbase-common-2.2.3.jar:2.2.3] at
org.apache.hadoop.hbase.MiniHBaseCluster$MiniHBaseClusterRegionServer.run(MiniHBaseCluster.java:165)
[hbase-server-2.2.3-tests.jar:2.2.3] at java.lang.Thread.run(Thread.java:748)
[?:1.8.0_242]
{code}
Also upon failure during maven run it would be great if the actual exception
would be displayed, not just that "Master not initialized after 20ms".
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HBASE-25134) Migrate HBase PreCommit jenkins job from Hadoop to hbase
[ https://issues.apache.org/jira/browse/HBASE-25134?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17205547#comment-17205547 ] Duo Zhang commented on HBASE-25134: --- Anyway, let me rever this from master branch first so we at least we can still run our pre commit job. Will reapply after we addressed the problem. > Migrate HBase PreCommit jenkins job from Hadoop to hbase > > > Key: HBASE-25134 > URL: https://issues.apache.org/jira/browse/HBASE-25134 > Project: HBase > Issue Type: Sub-task > Components: scripts >Reporter: Duo Zhang >Assignee: Duo Zhang >Priority: Major > Fix For: 3.0.0-alpha-1, 2.3.3, 1.7.0, 2.4.0, 1.4.14, 2.2.7 > > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[GitHub] [hbase] nyl3532016 commented on a change in pull request #2437: HBASE-25053 WAL replay should ignore 0-length files
nyl3532016 commented on a change in pull request #2437:
URL: https://github.com/apache/hbase/pull/2437#discussion_r498275885
##
File path:
hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/HRegion.java
##
@@ -6423,7 +6427,11 @@ protected void restoreEdit(HStore s, Cell cell,
MemStoreSizing memstoreAccountin
* @throws IOException
*/
private static boolean isZeroLengthThenDelete(final FileSystem fs, final
FileStatus stat,
- final Path p) throws IOException {
+ final Path p, Configuration conf) throws IOException {
+if (stat.getLen() > 0) {
+ return false;
+}
+RecoverLeaseFSUtils.recoverFileLease(fs, p, conf);
Review comment:
@saintstack Here is the conversation
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [hbase] nyl3532016 commented on pull request #2437: HBASE-25053 WAL replay should ignore 0-length files
nyl3532016 commented on pull request #2437: URL: https://github.com/apache/hbase/pull/2437#issuecomment-702163306 > Files that have not been closed show as 0-length. I'm concerned looking at length will have us miss edits. Can you address my concern? Thanks. @saintstack as Duo Zhang comments in the above conversation? `Anyway, my point here is that, make sure that when we call this method, we have already finished renaming the wal directory and calling recover lease on the wal files. Calling recoverLease and then re-get the FileStatus will generate extra load to NameNode.` And I check the recoverFileLease is already invoked This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Reopened] (HBASE-25134) Migrate HBase PreCommit jenkins job from Hadoop to hbase
[ https://issues.apache.org/jira/browse/HBASE-25134?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Duo Zhang reopened HBASE-25134: --- > Migrate HBase PreCommit jenkins job from Hadoop to hbase > > > Key: HBASE-25134 > URL: https://issues.apache.org/jira/browse/HBASE-25134 > Project: HBase > Issue Type: Sub-task > Components: scripts >Reporter: Duo Zhang >Assignee: Duo Zhang >Priority: Major > Fix For: 3.0.0-alpha-1, 2.3.3, 1.7.0, 2.4.0, 1.4.14, 2.2.7 > > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Work started] (HBASE-25139) The new hbase jenkins node has something wrong when running UTs
[ https://issues.apache.org/jira/browse/HBASE-25139?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Work on HBASE-25139 started by Duo Zhang. - > The new hbase jenkins node has something wrong when running UTs > --- > > Key: HBASE-25139 > URL: https://issues.apache.org/jira/browse/HBASE-25139 > Project: HBase > Issue Type: Sub-task > Components: test >Reporter: Duo Zhang >Assignee: Duo Zhang >Priority: Major > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (HBASE-25139) The new hbase jenkins node has something wrong when running UTs
Duo Zhang created HBASE-25139: - Summary: The new hbase jenkins node has something wrong when running UTs Key: HBASE-25139 URL: https://issues.apache.org/jira/browse/HBASE-25139 Project: HBase Issue Type: Sub-task Components: test Reporter: Duo Zhang Assignee: Duo Zhang -- This message was sent by Atlassian Jira (v8.3.4#803005)
