[GitHub] [hbase] Apache-HBase commented on pull request #3204: HBASE-25741: Replication Source still having the replication metrics for peer ID which doesn't exist
Apache-HBase commented on pull request #3204: URL: https://github.com/apache/hbase/pull/3204#issuecomment-831693768 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 0m 0s | Docker mode activated. | | -1 :x: | docker | 5m 31s | Docker failed to build yetus/hbase:edccfe439a. | | Subsystem | Report/Notes | |--:|:-| | GITHUB PR | https://github.com/apache/hbase/pull/3204 | | JIRA Issue | HBASE-25741 | | Console output | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3204/11/console | | versions | git=2.17.1 | | Powered by | Apache Yetus 0.12.0 https://yetus.apache.org | This message was automatically generated. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] nyl3532016 commented on a change in pull request #3217: HBASE-25834 Remove balanceTable method from LoadBalancer interface
nyl3532016 commented on a change in pull request #3217:
URL: https://github.com/apache/hbase/pull/3217#discussion_r625513246
##
File path:
hbase-server/src/main/java/org/apache/hadoop/hbase/rsgroup/RSGroupBasedLoadBalancer.java
##
@@ -449,40 +448,6 @@ public void updateBalancerStatus(boolean status) {
internalBalancer.updateBalancerStatus(status);
}
- /**
Review comment:
Yes,this method is redundant. Originally planned to throw nosupport
exception
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #3227: HBASE-25845 [branch-1] Precommit fails to build docker due to python-…
Apache-HBase commented on pull request #3227: URL: https://github.com/apache/hbase/pull/3227#issuecomment-831672071 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 6m 28s | Docker mode activated. | ||| _ Prechecks _ | | +1 :green_heart: | dupname | 0m 0s | No case conflicting files found. | | +0 :ok: | hadolint | 0m 0s | hadolint was not available. | | +0 :ok: | shelldocs | 0m 0s | Shelldocs was not available. | | +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. | ||| _ branch-1 Compile Tests _ | | +0 :ok: | mvndep | 2m 30s | Maven dependency ordering for branch | ||| _ Patch Compile Tests _ | | +0 :ok: | mvndep | 0m 9s | Maven dependency ordering for patch | | +1 :green_heart: | shellcheck | 0m 0s | There were no new shellcheck issues. | | +1 :green_heart: | whitespace | 0m 0s | The patch has no whitespace issues. | ||| _ Other Tests _ | | +0 :ok: | asflicense | 0m 0s | ASF License check generated no output? | | | | 10m 17s | | | Subsystem | Report/Notes | |--:|:-| | Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3227/1/artifact/out/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/3227 | | Optional Tests | dupname asflicense hadolint shellcheck shelldocs | | uname | Linux 9490191e6170 4.15.0-58-generic #64-Ubuntu SMP Tue Aug 6 11:12:41 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | /home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-3227/out/precommit/personality/provided.sh | | git revision | branch-1 / 0fe1c9c | | Max. process+thread count | 41 (vs. ulimit of 1) | | modules | C: U: | | Console output | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3227/1/console | | versions | git=1.9.1 maven=3.0.5 | | Powered by | Apache Yetus 0.12.0 https://yetus.apache.org | This message was automatically generated. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Commented] (HBASE-25612) HMaster should abort if ReplicationLogCleaner is not able to delete oldWALs.
[
https://issues.apache.org/jira/browse/HBASE-25612?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17338726#comment-17338726
]
Reid Chan commented on HBASE-25612:
---
can, let me fix the precommit error first.
> HMaster should abort if ReplicationLogCleaner is not able to delete oldWALs.
>
>
> Key: HBASE-25612
> URL: https://issues.apache.org/jira/browse/HBASE-25612
> Project: HBase
> Issue Type: Improvement
>Affects Versions: 1.6.0
>Reporter: Rushabh Shah
>Assignee: Rushabh Shah
>Priority: Major
> Fix For: 1.8.0
>
>
> In our production cluster, we encountered an issue where the number of files
> within /hbase/oldWALs directory were growing exponentially from about 4000
> baseline to 15 and growing at the rate of 333 files per minute.
> On further investigation we found that ReplicatonLogCleaner thread was
> getting aborted since it was not able to talk to zookeeper. Stack trace below
> {noformat}
> 2021-02-25 23:05:01,149 WARN [an-pool3-thread-1729] zookeeper.ZKUtil -
> replicationLogCleaner-0x302e05e0d8f,
> quorum=zookeeper-0:2181,zookeeper-1:2181,zookeeper-2:2181,zookeeper-3:2181,zookeeper-4:2181,
> baseZNode=/hbase Unable to get data of znode /hbase/replication/rs
> org.apache.zookeeper.KeeperException$SessionExpiredException: KeeperErrorCode
> = Session expired for /hbase/replication/rs
> at org.apache.zookeeper.KeeperException.create(KeeperException.java:130)
> at org.apache.zookeeper.KeeperException.create(KeeperException.java:54)
> at org.apache.zookeeper.ZooKeeper.getData(ZooKeeper.java:1229)
> at
> org.apache.hadoop.hbase.zookeeper.RecoverableZooKeeper.getData(RecoverableZooKeeper.java:374)
> at org.apache.hadoop.hbase.zookeeper.ZKUtil.getDataNoWatch(ZKUtil.java:713)
> at
> org.apache.hadoop.hbase.replication.ReplicationQueuesClientZKImpl.getQueuesZNodeCversion(ReplicationQueuesClientZKImpl.java:87)
> at
> org.apache.hadoop.hbase.replication.master.ReplicationLogCleaner.loadWALsFromQueues(ReplicationLogCleaner.java:99)
> at
> org.apache.hadoop.hbase.replication.master.ReplicationLogCleaner.getDeletableFiles(ReplicationLogCleaner.java:70)
> at
> org.apache.hadoop.hbase.master.cleaner.CleanerChore.checkAndDeleteFiles(CleanerChore.java:262)
> at
> org.apache.hadoop.hbase.master.cleaner.CleanerChore.access$200(CleanerChore.java:52)
> at
> org.apache.hadoop.hbase.master.cleaner.CleanerChore$3.act(CleanerChore.java:413)
> at
> org.apache.hadoop.hbase.master.cleaner.CleanerChore$3.act(CleanerChore.java:410)
> at
> org.apache.hadoop.hbase.master.cleaner.CleanerChore.deleteAction(CleanerChore.java:481)
> at
> org.apache.hadoop.hbase.master.cleaner.CleanerChore.traverseAndDelete(CleanerChore.java:410)
> at
> org.apache.hadoop.hbase.master.cleaner.CleanerChore.access$100(CleanerChore.java:52)
> at
> org.apache.hadoop.hbase.master.cleaner.CleanerChore$1.run(CleanerChore.java:220)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
> at java.lang.Thread.run(Thread.java:748)
> 2021-02-25 23:05:01,149 WARN [an-pool3-thread-1729]
> master.ReplicationLogCleaner - ReplicationLogCleaner received abort,
> ignoring. Reason: Failed to get stat of replication rs node
> 2021-02-25 23:05:01,149 DEBUG [an-pool3-thread-1729]
> master.ReplicationLogCleaner -
> org.apache.zookeeper.KeeperException$SessionExpiredException: KeeperErrorCode
> = Session expired for /hbase/replication/rs
> 2021-02-25 23:05:01,150 WARN [an-pool3-thread-1729]
> master.ReplicationLogCleaner - Failed to read zookeeper, skipping checking
> deletable files
> {noformat}
>
> {quote} 2021-02-25 23:05:01,149 WARN [an-pool3-thread-1729]
> master.ReplicationLogCleaner - ReplicationLogCleaner received abort,
> ignoring. Reason: Failed to get stat of replication rs node
> {quote}
>
> This line is more scary where HMaster invoked Abortable but just ignored and
> HMaster was doing it business as usual.
> We have max files per directory configuration in namenode which is set to 1M
> in our clusters. If this directory reached that limit then that would have
> brought down the whole cluster.
> We shouldn't ignore Abortable and should crash the Hmaster if Abortable is
> invoked.
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[GitHub] [hbase] Apache-HBase commented on pull request #3223: HBASE-25840 CatalogJanitor warns about skipping gc of regions during RIT, but does not actually skip
Apache-HBase commented on pull request #3223: URL: https://github.com/apache/hbase/pull/3223#issuecomment-831670166 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 0m 27s | Docker mode activated. | | -0 :warning: | yetus | 0m 3s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck | ||| _ Prechecks _ | ||| _ master Compile Tests _ | | +1 :green_heart: | mvninstall | 4m 30s | master passed | | +1 :green_heart: | compile | 1m 12s | master passed | | +1 :green_heart: | shadedjars | 8m 10s | branch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 0m 43s | master passed | ||| _ Patch Compile Tests _ | | +1 :green_heart: | mvninstall | 4m 14s | the patch passed | | +1 :green_heart: | compile | 1m 13s | the patch passed | | +1 :green_heart: | javac | 1m 13s | the patch passed | | +1 :green_heart: | shadedjars | 8m 1s | patch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 0m 40s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | unit | 140m 27s | hbase-server in the patch passed. | | | | 171m 55s | | | Subsystem | Report/Notes | |--:|:-| | Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3223/2/artifact/yetus-jdk11-hadoop3-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/3223 | | Optional Tests | javac javadoc unit shadedjars compile | | uname | Linux 3da89e85 4.15.0-58-generic #64-Ubuntu SMP Tue Aug 6 11:12:41 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / 762abe3bea | | Default Java | AdoptOpenJDK-11.0.10+9 | | Test Results | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3223/2/testReport/ | | Max. process+thread count | 3781 (vs. ulimit of 3) | | modules | C: hbase-server U: hbase-server | | Console output | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3223/2/console | | versions | git=2.17.1 maven=3.6.3 | | Powered by | Apache Yetus 0.12.0 https://yetus.apache.org | This message was automatically generated. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #3227: HBASE-25845 [branch-1] Precommit fails to build docker due to python-…
Apache-HBase commented on pull request #3227: URL: https://github.com/apache/hbase/pull/3227#issuecomment-831669861 (!) A patch to the testing environment has been detected. Re-executing against the patched versions to perform further tests. The console is at https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3227/1/console in case of problems. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] Reidddddd opened a new pull request #3227: HBASE-25845 [branch-1] Precommit fails to build docker due to python-…
Reidd opened a new pull request #3227: URL: https://github.com/apache/hbase/pull/3227 …dateutil -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Created] (HBASE-25845) [branch-1] Precommit fails to build docker due to python-dateutil
Reid Chan created HBASE-25845: - Summary: [branch-1] Precommit fails to build docker due to python-dateutil Key: HBASE-25845 URL: https://issues.apache.org/jira/browse/HBASE-25845 Project: HBase Issue Type: Task Reporter: Reid Chan Assignee: Reid Chan 02:34:22 Downloading/unpacking python-dateutil 02:34:22Cannot fetch index base URL https://pypi.python.org/simple/ 02:34:22Could not find any downloads that satisfy the requirement python-dateutil 02:34:22 Cleaning up... 02:34:22 No distributions at all found for python-dateutil 02:34:22 Storing debug log for failure in /root/.pip/pip.log 02:34:22 The command '/bin/sh -c pip install python-dateutil' returned a non-zero code: 1 02:34:22 ERROR: Docker failed to build yetus/hbase:edccfe439a. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[GitHub] [hbase] Apache-HBase commented on pull request #3225: HBASE-25842 move regionserver http-related code into o.a.h.h.regionserver.http
Apache-HBase commented on pull request #3225: URL: https://github.com/apache/hbase/pull/3225#issuecomment-831657637 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 1m 12s | Docker mode activated. | | -0 :warning: | yetus | 0m 3s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck | ||| _ Prechecks _ | ||| _ master Compile Tests _ | | +1 :green_heart: | mvninstall | 5m 19s | master passed | | +1 :green_heart: | compile | 1m 24s | master passed | | +1 :green_heart: | shadedjars | 9m 52s | branch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 0m 54s | master passed | ||| _ Patch Compile Tests _ | | +1 :green_heart: | mvninstall | 5m 43s | the patch passed | | +1 :green_heart: | compile | 1m 36s | the patch passed | | +1 :green_heart: | javac | 1m 36s | the patch passed | | +1 :green_heart: | shadedjars | 10m 7s | patch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 0m 48s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | unit | 221m 18s | hbase-server in the patch passed. | | | | 260m 9s | | | Subsystem | Report/Notes | |--:|:-| | Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3225/1/artifact/yetus-jdk11-hadoop3-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/3225 | | Optional Tests | javac javadoc unit shadedjars compile | | uname | Linux b16f2ec68a56 4.15.0-136-generic #140-Ubuntu SMP Thu Jan 28 05:20:47 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / 7640134e3e | | Default Java | AdoptOpenJDK-11.0.10+9 | | Test Results | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3225/1/testReport/ | | Max. process+thread count | 3012 (vs. ulimit of 3) | | modules | C: hbase-server U: hbase-server | | Console output | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3225/1/console | | versions | git=2.17.1 maven=3.6.3 | | Powered by | Apache Yetus 0.12.0 https://yetus.apache.org | This message was automatically generated. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #3223: HBASE-25840 CatalogJanitor warns about skipping gc of regions during RIT, but does not actually skip
Apache-HBase commented on pull request #3223: URL: https://github.com/apache/hbase/pull/3223#issuecomment-831641479 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 0m 27s | Docker mode activated. | ||| _ Prechecks _ | | +1 :green_heart: | dupname | 0m 1s | No case conflicting files found. | | +1 :green_heart: | hbaseanti | 0m 0s | Patch does not have any anti-patterns. | | +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. | ||| _ master Compile Tests _ | | +1 :green_heart: | mvninstall | 3m 56s | master passed | | +1 :green_heart: | compile | 3m 9s | master passed | | +1 :green_heart: | checkstyle | 1m 1s | master passed | | +1 :green_heart: | spotbugs | 2m 5s | master passed | ||| _ Patch Compile Tests _ | | +1 :green_heart: | mvninstall | 3m 37s | the patch passed | | +1 :green_heart: | compile | 3m 9s | the patch passed | | +1 :green_heart: | javac | 3m 9s | the patch passed | | +1 :green_heart: | checkstyle | 1m 3s | the patch passed | | +1 :green_heart: | whitespace | 0m 0s | The patch has no whitespace issues. | | +1 :green_heart: | hadoopcheck | 18m 2s | Patch does not cause any errors with Hadoop 3.1.2 3.2.1 3.3.0. | | +1 :green_heart: | spotbugs | 2m 14s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | asflicense | 0m 16s | The patch does not generate ASF License warnings. | | | | 46m 37s | | | Subsystem | Report/Notes | |--:|:-| | Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3223/2/artifact/yetus-general-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/3223 | | Optional Tests | dupname asflicense javac spotbugs hadoopcheck hbaseanti checkstyle compile | | uname | Linux fa5e9c0e055d 4.15.0-112-generic #113-Ubuntu SMP Thu Jul 9 23:41:39 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / 762abe3bea | | Default Java | AdoptOpenJDK-1.8.0_282-b08 | | Max. process+thread count | 96 (vs. ulimit of 3) | | modules | C: hbase-server U: hbase-server | | Console output | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3223/2/console | | versions | git=2.17.1 maven=3.6.3 spotbugs=4.2.2 | | Powered by | Apache Yetus 0.12.0 https://yetus.apache.org | This message was automatically generated. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #3225: HBASE-25842 move regionserver http-related code into o.a.h.h.regionserver.http
Apache-HBase commented on pull request #3225: URL: https://github.com/apache/hbase/pull/3225#issuecomment-831639789 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 1m 2s | Docker mode activated. | | -0 :warning: | yetus | 0m 3s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck | ||| _ Prechecks _ | ||| _ master Compile Tests _ | | +1 :green_heart: | mvninstall | 4m 0s | master passed | | +1 :green_heart: | compile | 1m 0s | master passed | | +1 :green_heart: | shadedjars | 8m 7s | branch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 0m 40s | master passed | ||| _ Patch Compile Tests _ | | +1 :green_heart: | mvninstall | 3m 40s | the patch passed | | +1 :green_heart: | compile | 0m 59s | the patch passed | | +1 :green_heart: | javac | 0m 59s | the patch passed | | +1 :green_heart: | shadedjars | 8m 11s | patch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 0m 36s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | unit | 154m 31s | hbase-server in the patch passed. | | | | 185m 2s | | | Subsystem | Report/Notes | |--:|:-| | Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3225/1/artifact/yetus-jdk8-hadoop3-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/3225 | | Optional Tests | javac javadoc unit shadedjars compile | | uname | Linux 6fc551095545 4.15.0-112-generic #113-Ubuntu SMP Thu Jul 9 23:41:39 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / 7640134e3e | | Default Java | AdoptOpenJDK-1.8.0_282-b08 | | Test Results | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3225/1/testReport/ | | Max. process+thread count | 4940 (vs. ulimit of 3) | | modules | C: hbase-server U: hbase-server | | Console output | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3225/1/console | | versions | git=2.17.1 maven=3.6.3 | | Powered by | Apache Yetus 0.12.0 https://yetus.apache.org | This message was automatically generated. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] apurtell commented on pull request #3218: HBASE-25835 Ignore duplicate split requests from regionserver reports
apurtell commented on pull request #3218: URL: https://github.com/apache/hbase/pull/3218#issuecomment-831633712 @Apache9 please let me know if the latest patch addresses your change requests. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Updated] (HBASE-25836) RegionStates#getAssignmentsForBalancer should only care about OPEN or OPENING regions
[ https://issues.apache.org/jira/browse/HBASE-25836?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Kyle Purtell updated HBASE-25836: Hadoop Flags: Reviewed Resolution: Fixed Status: Resolved (was: Patch Available) > RegionStates#getAssignmentsForBalancer should only care about OPEN or OPENING > regions > - > > Key: HBASE-25836 > URL: https://issues.apache.org/jira/browse/HBASE-25836 > Project: HBase > Issue Type: Sub-task >Reporter: Andrew Kyle Purtell >Assignee: Andrew Kyle Purtell >Priority: Major > Fix For: 3.0.0-alpha-1, 2.5.0, 2.4.3 > > > RegionStates#getAssignmentsForBalancer is used by the HMaster to collect all > regions of interest to the balancer for the next chore iteration. We check if > a table is in disabled state to exclude regions that will not be of interest > (because disabled regions are or will be offline) or are in a state where > they shouldn't be mutated (like SPLITTING ). The current checks do not get to > the point so are not comprehensive. > We should filter out all regions that are not in OPEN or OPENING state when > building the set of interesting regions for the balancer to consider. Only > regions open (or opening) on the cluster are of interest to balancing > calculations. Regions in all other states can be expected to not be of > interest -- either offline (OFFLINE, or FAILED_*), not subject to balancer > decisions now (SPLITTING, SPLITTING_NEW, MERGING, MERGING_NEW), or will be > offline shortly (CLOSING) -- until at least the next chore iteration. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[GitHub] [hbase] apurtell merged pull request #3219: HBASE-25836 RegionStates#getAssignmentsForBalancer should only care about OPEN or OPENING regions
apurtell merged pull request #3219: URL: https://github.com/apache/hbase/pull/3219 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Updated] (HBASE-25840) CatalogJanitor warns about skipping gc of regions during RIT, but does not actually skip
[
https://issues.apache.org/jira/browse/HBASE-25840?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Andrew Kyle Purtell updated HBASE-25840:
Resolution: Fixed
Status: Resolved (was: Patch Available)
> CatalogJanitor warns about skipping gc of regions during RIT, but does not
> actually skip
>
>
> Key: HBASE-25840
> URL: https://issues.apache.org/jira/browse/HBASE-25840
> Project: HBase
> Issue Type: Sub-task
>Affects Versions: 2.4.3
>Reporter: Andrew Kyle Purtell
>Assignee: Andrew Kyle Purtell
>Priority: Minor
> Fix For: 3.0.0-alpha-1, 2.5.0, 2.4.3, 2.3.6, 2.2.7
>
>
> Found while looking for something else.
> We claim in a WARN level log line to {{"Playing-it-safe skipping merge/split
> gc'ing of regions from hbase:meta while regions-in-transition (RIT)"}} but do
> not actually skip because of a missing return.
> Playing it safe has proven unnecessary, I would say. Nonetheless here is a
> patch that adds the missing return, for discussion.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (HBASE-25840) CatalogJanitor warns about skipping gc of regions during RIT, but does not actually skip
[
https://issues.apache.org/jira/browse/HBASE-25840?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Andrew Kyle Purtell updated HBASE-25840:
Fix Version/s: 2.2.7
2.3.6
> CatalogJanitor warns about skipping gc of regions during RIT, but does not
> actually skip
>
>
> Key: HBASE-25840
> URL: https://issues.apache.org/jira/browse/HBASE-25840
> Project: HBase
> Issue Type: Sub-task
>Affects Versions: 2.4.3
>Reporter: Andrew Kyle Purtell
>Assignee: Andrew Kyle Purtell
>Priority: Minor
> Fix For: 3.0.0-alpha-1, 2.2.7, 2.5.0, 2.4.3, 2.3.6
>
>
> Found while looking for something else.
> We claim in a WARN level log line to {{"Playing-it-safe skipping merge/split
> gc'ing of regions from hbase:meta while regions-in-transition (RIT)"}} but do
> not actually skip because of a missing return.
> Playing it safe has proven unnecessary, I would say. Nonetheless here is a
> patch that adds the missing return, for discussion.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[GitHub] [hbase] Apache-HBase commented on pull request #3223: HBASE-25840 CatalogJanitor warns about skipping gc of regions during RIT, but does not actually skip
Apache-HBase commented on pull request #3223: URL: https://github.com/apache/hbase/pull/3223#issuecomment-831631054 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 1m 10s | Docker mode activated. | | -1 :x: | patch | 0m 3s | https://github.com/apache/hbase/pull/3223 does not apply to master. Rebase required? Wrong Branch? See https://yetus.apache.org/documentation/in-progress/precommit-patchnames for help. | | Subsystem | Report/Notes | |--:|:-| | Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3223/2/artifact/yetus-jdk8-hadoop3-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/3223 | | Console output | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3223/2/console | | versions | git=2.17.1 | | Powered by | Apache Yetus 0.12.0 https://yetus.apache.org | This message was automatically generated. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] apurtell merged pull request #3223: HBASE-25840 CatalogJanitor warns about skipping gc of regions during RIT, but does not actually skip
apurtell merged pull request #3223: URL: https://github.com/apache/hbase/pull/3223 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] apurtell commented on pull request #3223: HBASE-25840 CatalogJanitor warns about skipping gc of regions during RIT, but does not actually skip
apurtell commented on pull request #3223: URL: https://github.com/apache/hbase/pull/3223#issuecomment-831630411 Updated patch removes the incorrect warning. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Comment Edited] (HBASE-25829) SPLIT state detritus
[
https://issues.apache.org/jira/browse/HBASE-25829?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17338650#comment-17338650
]
Andrew Kyle Purtell edited comment on HBASE-25829 at 5/4/21, 1:08 AM:
--
Subtasks look good. Back to the main issue.
{noformat}
2021-05-03 20:30:29,964 INFO [master/ip-172-31-58-47:8100.Chore.2]
master.HbckChore: Loaded 184 regions from in-memory state of AssignmentManager
2021-05-03 20:30:29,964 INFO [master/ip-172-31-58-47:8100.Chore.2]
master.HbckChore: Loaded 133 regions from 5 regionservers' reports and found 0
orphan regions
2021-05-03 20:30:29,975 INFO [master/ip-172-31-58-47:8100.Chore.2]
master.HbckChore: Loaded 3 tables 184 regions from filesystem and found 0
orphan regions
{noformat}
The 51 extra regions are SPLIT parents, with server = null.
I notice in AssignmentManager#markRegionAsMerged we remove the merge parents
from {{regionStates}} right there, but in AssignmentManager#markRegionAsSplit
we do not. We have code in various places that account for a post-split parent
to be hanging out in {{regionStates}} in SPLIT state. CatalogJanitor is
supposed to clean it.
If I patch AssignmentManager#markRegionAsSplit to remove the parent from
{{regionStates}} for spits the same way AssignmentManager#markRegionAsMerged
does for merges, then things begin to look better:
{noformat}
2021-05-03 22:08:29,036 INFO [master/ip-172-31-58-47:8100.Chore.1]
master.HbckChore: Loaded 23 regions from in-memory state of AssignmentManager
2021-05-03 22:08:29,036 INFO [master/ip-172-31-58-47:8100.Chore.1]
master.HbckChore: Loaded 23 regions from 5 regionservers' reports and found 0
orphan regions
2021-05-03 22:08:29,043 INFO [master/ip-172-31-58-47:8100.Chore.1]
master.HbckChore: Loaded 3 tables 32 regions from filesystem and found 9 orphan
regions
{noformat}
No more junk in {{regionStates}} but those 9 split parents are found as orphan
regions. There is a simple change to HbckChore that should accompany the other
changes I have under test: The conditional for determining if a region is
orphan should become {{if (hri == null &&
!splitParentRegions.contains(encodedRegionName) &&
!mergedParentRegions.contains(encodedRegionName))}} so you can ignore the
non-zero orphan count. With the complete change it would be reported as 0.
Remains to be seen if CatalogJanitor will clean the regions. Probably. What is
happening during latest tests is because isRIT evaluates to true usually (it's
an ingest test after all) and HBASE-25840 is applied all of the cleanup is
deferred. Will update and see if region GC proceeds as expected.
was (Author: apurtell):
Subtasks look good. Back to the main issue.
{noformat}
2021-05-03 20:30:29,964 INFO [master/ip-172-31-58-47:8100.Chore.2]
master.HbckChore: Loaded 184 regions from in-memory state of AssignmentManager
2021-05-03 20:30:29,964 INFO [master/ip-172-31-58-47:8100.Chore.2]
master.HbckChore: Loaded 133 regions from 5 regionservers' reports and found 0
orphan regions
2021-05-03 20:30:29,975 INFO [master/ip-172-31-58-47:8100.Chore.2]
master.HbckChore: Loaded 3 tables 184 regions from filesystem and found 0
orphan regions
{noformat}
The 51 extra regions are SPLIT parents, with server = null.
I notice in AssignmentManager#markRegionAsMerged we remove the merge parents
from {{regionStates}} right there, but in AssignmentManager#markRegionAsSplit
we do not. We have code in various places that account for a post-split parent
to be hanging out in {{regionStates}} in SPLIT state. CatalogJanitor is
supposed to clean it.
If I patch AssignmentManager#markRegionAsSplit to remove the parent from
{{regionStates}} for spits the same way AssignmentManager#markRegionAsMerged
does for merges, then things begin to look better:
{noformat}
2021-05-03 22:08:29,036 INFO [master/ip-172-31-58-47:8100.Chore.1]
master.HbckChore: Loaded 23 regions from in-memory state of AssignmentManager
2021-05-03 22:08:29,036 INFO [master/ip-172-31-58-47:8100.Chore.1]
master.HbckChore: Loaded 23 regions from 5 regionservers' reports and found 0
orphan regions
2021-05-03 22:08:29,043 INFO [master/ip-172-31-58-47:8100.Chore.1]
master.HbckChore: Loaded 3 tables 32 regions from filesystem and found 9 orphan
regions
{noformat}
No more junk in {{regionStates}} but those 9 split parents are found as orphan
regions. There is a simple change to HbckChore that should accompany the other
changes I have under test: The conditional for determining if a region is
orphan should become {{if (hri == null &&
!splitParentRegions.contains(encodedRegionName) &&
!mergedParentRegions.contains(encodedRegionName))}} so you can ignore the
non-zero orphan count. With the complete change it would be reported as 0.
Remains to be seen if CatalogJanitor will clean the regions. Probably. What is
happening during latest tests is because isRIT evaluates to true usually (it's
an
[GitHub] [hbase] apurtell edited a comment on pull request #3223: HBASE-25840 CatalogJanitor warns about skipping gc of regions during RIT, but does not actually skip
apurtell edited a comment on pull request #3223: URL: https://github.com/apache/hbase/pull/3223#issuecomment-831628917 > So we are running like this for a long time? No problem, then maybe we could just remove this check? Yes, this is my opinion too. Let me prepare an alternate patch. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] apurtell commented on pull request #3223: HBASE-25840 CatalogJanitor warns about skipping gc of regions during RIT, but does not actually skip
apurtell commented on pull request #3223: URL: https://github.com/apache/hbase/pull/3223#issuecomment-831628917 bq. So we are running like this for a long time? No problem, then maybe we could just remove this check? Yes, this is my opinion too. Let me prepare an alternate patch. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] Apache9 merged pull request #3221: HBASE-25838 Use double instead of Double in StochasticLoadBalancer
Apache9 merged pull request #3221: URL: https://github.com/apache/hbase/pull/3221 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Updated] (HBASE-25824) IntegrationTestLoadCommonCrawl
[
https://issues.apache.org/jira/browse/HBASE-25824?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Andrew Kyle Purtell updated HBASE-25824:
Fix Version/s: 2.5.0
Hadoop Flags: Reviewed
Resolution: Fixed
Status: Resolved (was: Patch Available)
> IntegrationTestLoadCommonCrawl
> --
>
> Key: HBASE-25824
> URL: https://issues.apache.org/jira/browse/HBASE-25824
> Project: HBase
> Issue Type: Test
> Components: integration tests
>Reporter: Andrew Kyle Purtell
>Assignee: Andrew Kyle Purtell
>Priority: Minor
> Fix For: 3.0.0-alpha-1, 2.5.0
>
>
> This integration test loads successful resource retrieval records from the
> Common Crawl (https://commoncrawl.org/) public dataset into an HBase table
> and writes records that can be used to later verify the presence and
> integrity of those records.
> Run like:
> {noformat}
> ./bin/hbase org.apache.hadoop.hbase.test.IntegrationTestLoadCommonCrawl \
> -files /path/to/hadoop-aws.jar \
> -Dfs.s3n.awsAccessKeyId= \
> -Dfs.s3n.awsSecretAccessKey= \
> /path/to/test-CC-MAIN-2021-10-warc.paths.gz \
> /path/to/tmp/warc-loader-output
> {noformat}
> Access to the Common Crawl dataset in S3 is made available to anyone by
> Amazon AWS, but Hadoop's S3N filesystem still requires valid access
> credentials to initialize.
> The input path can either specify a directory or a file. The file may
> optionally be compressed with gzip. If a directory, the loader expects the
> directory to contain one or more WARC files from the Common Crawl dataset. If
> a file, the loader expects a list of Hadoop S3N URIs which point to S3
> locations for one or more WARC files from the Common Crawl dataset, one URI
> per line. Lines should be terminated with the UNIX line terminator.
> Included in hbase-it/src/test/resources/CC-MAIN-2021-10-warc.paths.gz is a
> list of all WARC files comprising the Q1 2021 crawl archive. There are 64,000
> WARC files in this data set, each containing ~1GB of gzipped data. The WARC
> files contain several record types, such as metadata, request, and response,
> but we only load the response record types. If the HBase table schema does
> not specify compression (by default) there is roughly a 10x expansion.
> Loading the complete CC-MAIN-2021-10 crawl archive results in a table
> approximately 640 TB in size.
> The hadoop-aws jar will be needed at runtime to instantiate the S3N
> filesystem. Use the -files ToolRunner argument to add it.
> You can also split the Loader and Verify stages:
> Load with:
> {noformat}
> ./bin/hbase
> 'org.apache.hadoop.hbase.test.IntegrationTestLoadCommonCrawl$Loader' \
> -files /path/to/hadoop-aws.jar \
> -Dfs.s3n.awsAccessKeyId= \
> -Dfs.s3n.awsSecretAccessKey= \
> /path/to/test-CC-MAIN-2021-10-warc.paths.gz \
> /path/to/tmp/warc-loader-output
> {noformat}
> Verify with:
> {noformat}
> ./bin/hbase
> 'org.apache.hadoop.hbase.test.IntegrationTestLoadCommonCrawl$Verify' \
> /path/to/tmp/warc-loader-output
> {noformat}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HBASE-25839) Bulk Import fails with java.io.IOException: Type mismatch in value from map
[
https://issues.apache.org/jira/browse/HBASE-25839?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17338709#comment-17338709
]
Duo Zhang commented on HBASE-25839:
---
Thank you for reporting this issue. It is definately a bug.
And I think you already have a fix, so mind opening a PR?
Thanks.
> Bulk Import fails with java.io.IOException: Type mismatch in value from map
> ---
>
> Key: HBASE-25839
> URL: https://issues.apache.org/jira/browse/HBASE-25839
> Project: HBase
> Issue Type: Bug
> Components: mapreduce
>Affects Versions: 2.2.5
>Reporter: ZFabrik
>Priority: Major
>
> Bulk Import with {{-Dimport.bulk.output=/HFILES
> -Dimport.bulk.hasLargeResult=true}} always results in
> {noformat}
> Error: java.io.IOException: Type mismatch in value from map: expected
> org.apache.hadoop.hbase.util.MapReduceExtendedCell, received
> org.apache.hadoop.hbase.IndividualBytesFieldCell
> at
> org.apache.hadoop.mapred.MapTask$MapOutputBuffer.collect(MapTask.java:1077)
> at
> org.apache.hadoop.mapred.MapTask$NewOutputCollector.write(MapTask.java:715)
> at
> org.apache.hadoop.mapreduce.task.TaskInputOutputContextImpl.write(TaskInputOutputContextImpl.java:89)
> at
> org.apache.hadoop.mapreduce.lib.map.WrappedMapper$Context.write(WrappedMapper.java:112)
> at
> org.apache.hadoop.hbase.mapreduce.Import$CellSortImporter.map(Import.java:423)
> at
> org.apache.hadoop.hbase.mapreduce.Import$CellSortImporter.map(Import.java:394)
> at org.apache.hadoop.mapreduce.Mapper.run(Mapper.java:146)
> at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:787)
> at org.apache.hadoop.mapred.MapTask.run(MapTask.java:341)
> at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:175)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:422)
> at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1844)
> at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:169)
> {noformat}
> The Problem is that in
> {{org.apache.hadoop.hbase.mapreduce.Import.CellSortImporter#map}} the
> {{Cell}} coming from {{value.rawCells()}} (which is of type
> {{org.apache.hadoop.hbase.IndividualBytesFieldCell}}) is directly written to
> the context without wrapping it into {{MapReduceExtendedCell}} (like it
> happens at {{CellImporter#map}}).
> IMHO [line
> 423|https://github.com/apache/hbase/blob/f76a601273e834267b55c0cda12474590283fd4c/hbase-mapreduce/src/main/java/org/apache/hadoop/hbase/mapreduce/Import.java#L423]
> must look like this:
> {code:java}
> context.write(new CellWritableComparable(ret), new
> MapReduceExtendedCell(ret));
> {code}
> Furthermore it seems to me that this is also wrong in all subsequent versions
>
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (HBASE-25839) Bulk Import fails with java.io.IOException: Type mismatch in value from map
[
https://issues.apache.org/jira/browse/HBASE-25839?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Duo Zhang updated HBASE-25839:
--
Component/s: mapreduce
> Bulk Import fails with java.io.IOException: Type mismatch in value from map
> ---
>
> Key: HBASE-25839
> URL: https://issues.apache.org/jira/browse/HBASE-25839
> Project: HBase
> Issue Type: Bug
> Components: mapreduce
>Affects Versions: 2.2.5
>Reporter: ZFabrik
>Priority: Major
>
> Bulk Import with {{-Dimport.bulk.output=/HFILES
> -Dimport.bulk.hasLargeResult=true}} always results in
> {noformat}
> Error: java.io.IOException: Type mismatch in value from map: expected
> org.apache.hadoop.hbase.util.MapReduceExtendedCell, received
> org.apache.hadoop.hbase.IndividualBytesFieldCell
> at
> org.apache.hadoop.mapred.MapTask$MapOutputBuffer.collect(MapTask.java:1077)
> at
> org.apache.hadoop.mapred.MapTask$NewOutputCollector.write(MapTask.java:715)
> at
> org.apache.hadoop.mapreduce.task.TaskInputOutputContextImpl.write(TaskInputOutputContextImpl.java:89)
> at
> org.apache.hadoop.mapreduce.lib.map.WrappedMapper$Context.write(WrappedMapper.java:112)
> at
> org.apache.hadoop.hbase.mapreduce.Import$CellSortImporter.map(Import.java:423)
> at
> org.apache.hadoop.hbase.mapreduce.Import$CellSortImporter.map(Import.java:394)
> at org.apache.hadoop.mapreduce.Mapper.run(Mapper.java:146)
> at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:787)
> at org.apache.hadoop.mapred.MapTask.run(MapTask.java:341)
> at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:175)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:422)
> at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1844)
> at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:169)
> {noformat}
> The Problem is that in
> {{org.apache.hadoop.hbase.mapreduce.Import.CellSortImporter#map}} the
> {{Cell}} coming from {{value.rawCells()}} (which is of type
> {{org.apache.hadoop.hbase.IndividualBytesFieldCell}}) is directly written to
> the context without wrapping it into {{MapReduceExtendedCell}} (like it
> happens at {{CellImporter#map}}).
> IMHO [line
> 423|https://github.com/apache/hbase/blob/f76a601273e834267b55c0cda12474590283fd4c/hbase-mapreduce/src/main/java/org/apache/hadoop/hbase/mapreduce/Import.java#L423]
> must look like this:
> {code:java}
> context.write(new CellWritableComparable(ret), new
> MapReduceExtendedCell(ret));
> {code}
> Furthermore it seems to me that this is also wrong in all subsequent versions
>
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[GitHub] [hbase] apurtell merged pull request #3208: HBASE-25824 IntegrationTestLoadCommonCrawl
apurtell merged pull request #3208: URL: https://github.com/apache/hbase/pull/3208 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Comment Edited] (HBASE-25829) SPLIT state detritus
[
https://issues.apache.org/jira/browse/HBASE-25829?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17338650#comment-17338650
]
Andrew Kyle Purtell edited comment on HBASE-25829 at 5/4/21, 12:57 AM:
---
Subtasks look good. Back to the main issue.
{noformat}
2021-05-03 20:30:29,964 INFO [master/ip-172-31-58-47:8100.Chore.2]
master.HbckChore: Loaded 184 regions from in-memory state of AssignmentManager
2021-05-03 20:30:29,964 INFO [master/ip-172-31-58-47:8100.Chore.2]
master.HbckChore: Loaded 133 regions from 5 regionservers' reports and found 0
orphan regions
2021-05-03 20:30:29,975 INFO [master/ip-172-31-58-47:8100.Chore.2]
master.HbckChore: Loaded 3 tables 184 regions from filesystem and found 0
orphan regions
{noformat}
The 51 extra regions are SPLIT parents, with server = null.
I notice in AssignmentManager#markRegionAsMerged we remove the merge parents
from {{regionStates}} right there, but in AssignmentManager#markRegionAsSplit
we do not. We have code in various places that account for a post-split parent
to be hanging out in {{regionStates}} in SPLIT state. CatalogJanitor is
supposed to clean it.
If I patch AssignmentManager#markRegionAsSplit to remove the parent from
{{regionStates}} for spits the same way AssignmentManager#markRegionAsMerged
does for merges, then things begin to look better:
{noformat}
2021-05-03 22:08:29,036 INFO [master/ip-172-31-58-47:8100.Chore.1]
master.HbckChore: Loaded 23 regions from in-memory state of AssignmentManager
2021-05-03 22:08:29,036 INFO [master/ip-172-31-58-47:8100.Chore.1]
master.HbckChore: Loaded 23 regions from 5 regionservers' reports and found 0
orphan regions
2021-05-03 22:08:29,043 INFO [master/ip-172-31-58-47:8100.Chore.1]
master.HbckChore: Loaded 3 tables 32 regions from filesystem and found 9 orphan
regions
{noformat}
No more junk in {{regionStates}} but those 9 split parents are found as orphan
regions. There is a simple change to HbckChore that should accompany the other
changes I have under test: The conditional for determining if a region is
orphan should become {{if (hri == null &&
!splitParentRegions.contains(encodedRegionName) &&
!mergedParentRegions.contains(encodedRegionName))}} so you can ignore the
non-zero orphan count. With the complete change it would be reported as 0.
Remains to be seen if CatalogJanitor will clean the regions. Probably. What is
happening during latest tests is because isRIT evaluates to true usually (it's
an ingest test after all) and HBASE-25840 is applied all of the cleanup is
deferred. Will wait until the end to see if region GC proceeds as expected.
was (Author: apurtell):
Subtasks look good. Back to the main issue.
{noformat}
2021-05-03 20:30:29,964 INFO [master/ip-172-31-58-47:8100.Chore.2]
master.HbckChore: Loaded 184 regions from in-memory state of AssignmentManager
2021-05-03 20:30:29,964 INFO [master/ip-172-31-58-47:8100.Chore.2]
master.HbckChore: Loaded 133 regions from 5 regionservers' reports and found 0
orphan regions
2021-05-03 20:30:29,975 INFO [master/ip-172-31-58-47:8100.Chore.2]
master.HbckChore: Loaded 3 tables 184 regions from filesystem and found 0
orphan regions
{noformat}
The 51 extra regions are SPLIT parents, with server = null.
I notice in AssignmentManager#markRegionAsMerged we remove the merge parents
from {{regionStates}} right there, but in AssignmentManager#markRegionAsSplit
we do not. We have code in various places that account for a post-split parent
to be hanging out in {{regionStates}} in SPLIT state. CatalogJanitor is
supposed to clean it, but does not!
If I patch AssignmentManager#markRegionAsSplit to remove the parent from
{{regionStates}} for spits the same way AssignmentManager#markRegionAsMerged
does for merges, then things begin to look better:
{noformat}
2021-05-03 22:08:29,036 INFO [master/ip-172-31-58-47:8100.Chore.1]
master.HbckChore: Loaded 23 regions from in-memory state of AssignmentManager
2021-05-03 22:08:29,036 INFO [master/ip-172-31-58-47:8100.Chore.1]
master.HbckChore: Loaded 23 regions from 5 regionservers' reports and found 0
orphan regions
2021-05-03 22:08:29,043 INFO [master/ip-172-31-58-47:8100.Chore.1]
master.HbckChore: Loaded 3 tables 32 regions from filesystem and found 9 orphan
regions
{noformat}
No more junk in {{regionStates}} but those 9 split parents are found as orphan
regions. There is a simple change to HbckChore that should accompany the other
changes I have under test: The conditional for determining if a region is
orphan should become {{if (hri == null &&
!splitParentRegions.contains(encodedRegionName) &&
!mergedParentRegions.contains(encodedRegionName))}} so you can ignore the
non-zero orphan count. With the complete change it would be reported as 0.
Anyway, there is more to debug. Seems CatalogJanitor or the region GC
procedures it submits are not completing their work. I have added
[GitHub] [hbase] Apache9 commented on pull request #3224: HBASE-25841 Add basic jshell support
Apache9 commented on pull request #3224: URL: https://github.com/apache/hbase/pull/3224#issuecomment-831626165 Could you please provide some examples on how to make use of this jshell? It is just a standalone process? Not inside a master or regionserver process? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] Apache9 commented on a change in pull request #3225: HBASE-25842 move regionserver http-related code into o.a.h.h.regionserver.http
Apache9 commented on a change in pull request #3225:
URL: https://github.com/apache/hbase/pull/3225#discussion_r625452587
##
File path:
hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/HRegion.java
##
@@ -6687,6 +6687,13 @@ public int getReadLockCount() {
return lockedRows;
}
+ /**
+ * Stringify a coarse "dump" of {@code RowLock} information.
+ */
+ public void dumpLocks(final StringBuilder sb) {
Review comment:
Not finished?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[jira] [Assigned] (HBASE-25797) Batch version of checkAndMutate on AsyncTable always returns false
[
https://issues.apache.org/jira/browse/HBASE-25797?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Toshihiro Suzuki reassigned HBASE-25797:
Assignee: Toshihiro Suzuki
> Batch version of checkAndMutate on AsyncTable always returns false
> --
>
> Key: HBASE-25797
> URL: https://issues.apache.org/jira/browse/HBASE-25797
> Project: HBase
> Issue Type: Bug
> Components: Client
>Affects Versions: 2.4.1
>Reporter: Hochul Shin
>Assignee: Toshihiro Suzuki
>Priority: Minor
> Labels: bug
>
> Batch version of CheckAndMutate always returns false whereas non-batch
> version works fine.
> The code is like:
>
> {code:java}
> AysncTable table = connection.getTable(tableName, executorService);
> ...
> // add r1
> table.put(Arrays.asList(
> new Put(Bytes.toBytes("r1")).addColumn(COL_FAMILY, Bytes.toBytes("q1"),
> Bytes.toBytes("v1";
> CheckAndMutate checkAndMutate1 =
> CheckAndMutate.newBuilder(Bytes.toBytes("r1"))
> .ifNotExists(COL_FAMILY, Bytes.toBytes("q1"))
> .build(new Put(Bytes.toBytes("r1")).addColumn(COL_FAMILY,
> Bytes.toBytes("q1"), Bytes.toBytes("v1")));
> CheckAndMutate checkAndMutate2 =
> CheckAndMutate.newBuilder(Bytes.toBytes("r2"))
> .ifNotExists(COL_FAMILY, Bytes.toBytes("q2"))
> .build(new Put(Bytes.toBytes("r2")).addColumn(COL_FAMILY,
> Bytes.toBytes("q2"), Bytes.toBytes("v2")));
> {code}
>
> With batch version of checkAndMutate
> {code:java}
> List> results =
> table.checkAndMutate(Arrays.asList(checkAndMutate1, checkAndMutate2));
> System.out.println("first: " + results.get(0).join().isSuccess()); // false
> System.out.println("second: " + results.get(1).join().isSuccess()); //false!
> {code}
>
>
> Even it returned false, the r1 was written to the table. (timestamp was
> updated)
>
> With non-batch version of checkAndMutate
>
> {code:java}
> CompletableFuture result1 =
> table.checkAndMutate(checkAndMutate1);
> CompletableFuture result2 =
> table.checkAndMutate(checkAndMutate2);
> System.out.println("first: " + result1.join().isSuccess()); // false
> System.out.println("second: " + result2.join().isSuccess()); // true as
> expected
> {code}
>
> As expected r1 wasn't written to the table.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HBASE-25797) Batch version of checkAndMutate on AsyncTable always returns false
[
https://issues.apache.org/jira/browse/HBASE-25797?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17338700#comment-17338700
]
Toshihiro Suzuki commented on HBASE-25797:
--
Thank you [~bryson],
Can you please try to call the get() method of the CompletableFuture object
returned by table.put() as follows:
{code}
// add r1
table.put(Arrays.asList(
new Put(Bytes.toBytes("r1")).addColumn(COL_FAMILY, Bytes.toBytes("q1"),
Bytes.toBytes("v1".get();
{code}
As the *table* is AysncTable, we need to wait for table.put() to complete
before calling table.checkAndMutate(). Thanks.
> Batch version of checkAndMutate on AsyncTable always returns false
> --
>
> Key: HBASE-25797
> URL: https://issues.apache.org/jira/browse/HBASE-25797
> Project: HBase
> Issue Type: Bug
> Components: Client
>Affects Versions: 2.4.1
>Reporter: Hochul Shin
>Priority: Minor
> Labels: bug
>
> Batch version of CheckAndMutate always returns false whereas non-batch
> version works fine.
> The code is like:
>
> {code:java}
> AysncTable table = connection.getTable(tableName, executorService);
> ...
> // add r1
> table.put(Arrays.asList(
> new Put(Bytes.toBytes("r1")).addColumn(COL_FAMILY, Bytes.toBytes("q1"),
> Bytes.toBytes("v1";
> CheckAndMutate checkAndMutate1 =
> CheckAndMutate.newBuilder(Bytes.toBytes("r1"))
> .ifNotExists(COL_FAMILY, Bytes.toBytes("q1"))
> .build(new Put(Bytes.toBytes("r1")).addColumn(COL_FAMILY,
> Bytes.toBytes("q1"), Bytes.toBytes("v1")));
> CheckAndMutate checkAndMutate2 =
> CheckAndMutate.newBuilder(Bytes.toBytes("r2"))
> .ifNotExists(COL_FAMILY, Bytes.toBytes("q2"))
> .build(new Put(Bytes.toBytes("r2")).addColumn(COL_FAMILY,
> Bytes.toBytes("q2"), Bytes.toBytes("v2")));
> {code}
>
> With batch version of checkAndMutate
> {code:java}
> List> results =
> table.checkAndMutate(Arrays.asList(checkAndMutate1, checkAndMutate2));
> System.out.println("first: " + results.get(0).join().isSuccess()); // false
> System.out.println("second: " + results.get(1).join().isSuccess()); //false!
> {code}
>
>
> Even it returned false, the r1 was written to the table. (timestamp was
> updated)
>
> With non-batch version of checkAndMutate
>
> {code:java}
> CompletableFuture result1 =
> table.checkAndMutate(checkAndMutate1);
> CompletableFuture result2 =
> table.checkAndMutate(checkAndMutate2);
> System.out.println("first: " + result1.join().isSuccess()); // false
> System.out.println("second: " + result2.join().isSuccess()); // true as
> expected
> {code}
>
> As expected r1 wasn't written to the table.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[GitHub] [hbase] Apache-HBase commented on pull request #3219: HBASE-25836 RegionStates#getAssignmentsForBalancer should only care about OPEN or OPENING regions
Apache-HBase commented on pull request #3219: URL: https://github.com/apache/hbase/pull/3219#issuecomment-831601253 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 0m 26s | Docker mode activated. | | -0 :warning: | yetus | 0m 3s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck | ||| _ Prechecks _ | ||| _ master Compile Tests _ | | +1 :green_heart: | mvninstall | 3m 37s | master passed | | +1 :green_heart: | compile | 1m 0s | master passed | | +1 :green_heart: | shadedjars | 8m 9s | branch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 0m 38s | master passed | ||| _ Patch Compile Tests _ | | +1 :green_heart: | mvninstall | 3m 37s | the patch passed | | +1 :green_heart: | compile | 1m 4s | the patch passed | | +1 :green_heart: | javac | 1m 4s | the patch passed | | +1 :green_heart: | shadedjars | 8m 12s | patch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 0m 38s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | unit | 148m 44s | hbase-server in the patch passed. | | | | 178m 20s | | | Subsystem | Report/Notes | |--:|:-| | Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3219/3/artifact/yetus-jdk8-hadoop3-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/3219 | | Optional Tests | javac javadoc unit shadedjars compile | | uname | Linux 17de50684198 4.15.0-112-generic #113-Ubuntu SMP Thu Jul 9 23:41:39 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / 7640134e3e | | Default Java | AdoptOpenJDK-1.8.0_282-b08 | | Test Results | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3219/3/testReport/ | | Max. process+thread count | 4595 (vs. ulimit of 3) | | modules | C: hbase-server U: hbase-server | | Console output | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3219/3/console | | versions | git=2.17.1 maven=3.6.3 | | Powered by | Apache Yetus 0.12.0 https://yetus.apache.org | This message was automatically generated. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #3208: HBASE-25824 IntegrationTestLoadCommonCrawl
Apache-HBase commented on pull request #3208: URL: https://github.com/apache/hbase/pull/3208#issuecomment-831600135 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 1m 38s | Docker mode activated. | | -0 :warning: | yetus | 0m 4s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck | ||| _ Prechecks _ | ||| _ master Compile Tests _ | | +0 :ok: | mvndep | 0m 34s | Maven dependency ordering for branch | | +1 :green_heart: | mvninstall | 5m 11s | master passed | | +1 :green_heart: | compile | 3m 45s | master passed | | +1 :green_heart: | shadedjars | 10m 57s | branch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 3m 6s | master passed | ||| _ Patch Compile Tests _ | | +0 :ok: | mvndep | 0m 21s | Maven dependency ordering for patch | | +1 :green_heart: | mvninstall | 4m 57s | the patch passed | | +1 :green_heart: | compile | 3m 34s | the patch passed | | +1 :green_heart: | javac | 3m 34s | the patch passed | | +1 :green_heart: | shadedjars | 10m 56s | patch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 3m 0s | the patch passed | ||| _ Other Tests _ | | -1 :x: | unit | 360m 11s | root in the patch failed. | | | | 411m 12s | | | Subsystem | Report/Notes | |--:|:-| | Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3208/8/artifact/yetus-jdk8-hadoop3-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/3208 | | Optional Tests | javac javadoc unit shadedjars compile | | uname | Linux 6c53dc6f95bf 4.15.0-65-generic #74-Ubuntu SMP Tue Sep 17 17:06:04 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / 7640134e3e | | Default Java | AdoptOpenJDK-1.8.0_282-b08 | | unit | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3208/8/artifact/yetus-jdk8-hadoop3-check/output/patch-unit-root.txt | | Test Results | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3208/8/testReport/ | | Max. process+thread count | 5598 (vs. ulimit of 3) | | modules | C: hbase-it . U: . | | Console output | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3208/8/console | | versions | git=2.17.1 maven=3.6.3 | | Powered by | Apache Yetus 0.12.0 https://yetus.apache.org | This message was automatically generated. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #3225: HBASE-25842 move regionserver http-related code into o.a.h.h.regionserver.http
Apache-HBase commented on pull request #3225: URL: https://github.com/apache/hbase/pull/3225#issuecomment-831599530 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 0m 26s | Docker mode activated. | ||| _ Prechecks _ | | +1 :green_heart: | dupname | 0m 0s | No case conflicting files found. | | +1 :green_heart: | hbaseanti | 0m 0s | Patch does not have any anti-patterns. | | +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. | ||| _ master Compile Tests _ | | +1 :green_heart: | mvninstall | 4m 9s | master passed | | +1 :green_heart: | compile | 3m 16s | master passed | | +1 :green_heart: | checkstyle | 1m 5s | master passed | | +1 :green_heart: | spotbugs | 2m 7s | master passed | ||| _ Patch Compile Tests _ | | +1 :green_heart: | mvninstall | 3m 34s | the patch passed | | +1 :green_heart: | compile | 3m 17s | the patch passed | | +1 :green_heart: | javac | 3m 17s | the patch passed | | -0 :warning: | checkstyle | 1m 3s | hbase-server: The patch generated 6 new + 214 unchanged - 6 fixed = 220 total (was 220) | | +1 :green_heart: | whitespace | 0m 0s | The patch has no whitespace issues. | | +1 :green_heart: | hadoopcheck | 18m 8s | Patch does not cause any errors with Hadoop 3.1.2 3.2.1 3.3.0. | | +1 :green_heart: | spotbugs | 2m 13s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | asflicense | 0m 16s | The patch does not generate ASF License warnings. | | | | 47m 20s | | | Subsystem | Report/Notes | |--:|:-| | Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3225/1/artifact/yetus-general-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/3225 | | Optional Tests | dupname asflicense javac spotbugs hadoopcheck hbaseanti checkstyle compile | | uname | Linux f134451b7e94 4.15.0-58-generic #64-Ubuntu SMP Tue Aug 6 11:12:41 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / 7640134e3e | | Default Java | AdoptOpenJDK-1.8.0_282-b08 | | checkstyle | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3225/1/artifact/yetus-general-check/output/diff-checkstyle-hbase-server.txt | | Max. process+thread count | 96 (vs. ulimit of 3) | | modules | C: hbase-server U: hbase-server | | Console output | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3225/1/console | | versions | git=2.17.1 maven=3.6.3 spotbugs=4.2.2 | | Powered by | Apache Yetus 0.12.0 https://yetus.apache.org | This message was automatically generated. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #3219: HBASE-25836 RegionStates#getAssignmentsForBalancer should only care about OPEN or OPENING regions
Apache-HBase commented on pull request #3219: URL: https://github.com/apache/hbase/pull/3219#issuecomment-831598724 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 0m 28s | Docker mode activated. | | -0 :warning: | yetus | 0m 3s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck | ||| _ Prechecks _ | ||| _ master Compile Tests _ | | +1 :green_heart: | mvninstall | 4m 33s | master passed | | +1 :green_heart: | compile | 1m 13s | master passed | | +1 :green_heart: | shadedjars | 8m 2s | branch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 0m 42s | master passed | ||| _ Patch Compile Tests _ | | +1 :green_heart: | mvninstall | 4m 17s | the patch passed | | +1 :green_heart: | compile | 1m 13s | the patch passed | | +1 :green_heart: | javac | 1m 13s | the patch passed | | +1 :green_heart: | shadedjars | 8m 8s | patch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 0m 41s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | unit | 138m 31s | hbase-server in the patch passed. | | | | 170m 2s | | | Subsystem | Report/Notes | |--:|:-| | Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3219/3/artifact/yetus-jdk11-hadoop3-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/3219 | | Optional Tests | javac javadoc unit shadedjars compile | | uname | Linux 3ee0a95a7154 4.15.0-112-generic #113-Ubuntu SMP Thu Jul 9 23:41:39 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / 7640134e3e | | Default Java | AdoptOpenJDK-11.0.10+9 | | Test Results | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3219/3/testReport/ | | Max. process+thread count | 3906 (vs. ulimit of 3) | | modules | C: hbase-server U: hbase-server | | Console output | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3219/3/console | | versions | git=2.17.1 maven=3.6.3 | | Powered by | Apache Yetus 0.12.0 https://yetus.apache.org | This message was automatically generated. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #3204: HBASE-25741: Replication Source still having the replication metrics for peer ID which doesn't exist
Apache-HBase commented on pull request #3204: URL: https://github.com/apache/hbase/pull/3204#issuecomment-831597958 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 0m 0s | Docker mode activated. | | -1 :x: | docker | 0m 10s | Docker failed to build yetus/hbase:edccfe439a. | | Subsystem | Report/Notes | |--:|:-| | GITHUB PR | https://github.com/apache/hbase/pull/3204 | | JIRA Issue | HBASE-25741 | | Console output | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3204/10/console | | versions | git=2.17.1 | | Powered by | Apache Yetus 0.12.0 https://yetus.apache.org | This message was automatically generated. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #3204: HBASE-25741: Replication Source still having the replication metrics for peer ID which doesn't exist
Apache-HBase commented on pull request #3204: URL: https://github.com/apache/hbase/pull/3204#issuecomment-831597179 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 0m 0s | Docker mode activated. | | -1 :x: | docker | 0m 10s | Docker failed to build yetus/hbase:edccfe439a. | | Subsystem | Report/Notes | |--:|:-| | GITHUB PR | https://github.com/apache/hbase/pull/3204 | | JIRA Issue | HBASE-25741 | | Console output | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3204/9/console | | versions | git=2.17.1 | | Powered by | Apache Yetus 0.12.0 https://yetus.apache.org | This message was automatically generated. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #3226: HBASE-25843 move master http-related code into o.a.h.h.master.http
Apache-HBase commented on pull request #3226: URL: https://github.com/apache/hbase/pull/3226#issuecomment-831596282 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 2m 15s | Docker mode activated. | | -0 :warning: | yetus | 0m 3s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck | ||| _ Prechecks _ | ||| _ master Compile Tests _ | | +1 :green_heart: | mvninstall | 5m 46s | master passed | | +1 :green_heart: | compile | 1m 33s | master passed | | +1 :green_heart: | shadedjars | 10m 54s | branch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 0m 50s | master passed | ||| _ Patch Compile Tests _ | | -1 :x: | mvninstall | 2m 27s | root in the patch failed. | | -1 :x: | compile | 0m 49s | hbase-server in the patch failed. | | -0 :warning: | javac | 0m 49s | hbase-server in the patch failed. | | -1 :x: | shadedjars | 7m 36s | patch has 14 errors when building our shaded downstream artifacts. | | -0 :warning: | javadoc | 0m 46s | hbase-server generated 1 new + 20 unchanged - 0 fixed = 21 total (was 20) | ||| _ Other Tests _ | | -1 :x: | unit | 0m 46s | hbase-server in the patch failed. | | | | 35m 4s | | | Subsystem | Report/Notes | |--:|:-| | Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3226/1/artifact/yetus-jdk8-hadoop3-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/3226 | | Optional Tests | javac javadoc unit shadedjars compile | | uname | Linux bb30915299a5 4.15.0-128-generic #131-Ubuntu SMP Wed Dec 9 06:57:35 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / 7640134e3e | | Default Java | AdoptOpenJDK-1.8.0_282-b08 | | mvninstall | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3226/1/artifact/yetus-jdk8-hadoop3-check/output/patch-mvninstall-root.txt | | compile | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3226/1/artifact/yetus-jdk8-hadoop3-check/output/patch-compile-hbase-server.txt | | javac | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3226/1/artifact/yetus-jdk8-hadoop3-check/output/patch-compile-hbase-server.txt | | shadedjars | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3226/1/artifact/yetus-jdk8-hadoop3-check/output/patch-shadedjars.txt | | javadoc | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3226/1/artifact/yetus-jdk8-hadoop3-check/output/diff-javadoc-javadoc-hbase-server.txt | | unit | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3226/1/artifact/yetus-jdk8-hadoop3-check/output/patch-unit-hbase-server.txt | | Test Results | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3226/1/testReport/ | | Max. process+thread count | 78 (vs. ulimit of 3) | | modules | C: hbase-server U: hbase-server | | Console output | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3226/1/console | | versions | git=2.17.1 maven=3.6.3 | | Powered by | Apache Yetus 0.12.0 https://yetus.apache.org | This message was automatically generated. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #3223: HBASE-25840 CatalogJanitor warns about skipping gc of regions during RIT, but does not actually skip
Apache-HBase commented on pull request #3223: URL: https://github.com/apache/hbase/pull/3223#issuecomment-831595306 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 1m 11s | Docker mode activated. | | -0 :warning: | yetus | 0m 3s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck | ||| _ Prechecks _ | ||| _ master Compile Tests _ | | +1 :green_heart: | mvninstall | 4m 19s | master passed | | +1 :green_heart: | compile | 1m 3s | master passed | | +1 :green_heart: | shadedjars | 9m 7s | branch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 0m 38s | master passed | ||| _ Patch Compile Tests _ | | +1 :green_heart: | mvninstall | 4m 24s | the patch passed | | +1 :green_heart: | compile | 1m 4s | the patch passed | | +1 :green_heart: | javac | 1m 4s | the patch passed | | +1 :green_heart: | shadedjars | 8m 58s | patch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 0m 38s | the patch passed | ||| _ Other Tests _ | | -1 :x: | unit | 215m 56s | hbase-server in the patch failed. | | | | 249m 12s | | | Subsystem | Report/Notes | |--:|:-| | Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3223/1/artifact/yetus-jdk8-hadoop3-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/3223 | | Optional Tests | javac javadoc unit shadedjars compile | | uname | Linux b5fa372f03fb 4.15.0-136-generic #140-Ubuntu SMP Thu Jan 28 05:20:47 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / 7640134e3e | | Default Java | AdoptOpenJDK-1.8.0_282-b08 | | unit | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3223/1/artifact/yetus-jdk8-hadoop3-check/output/patch-unit-hbase-server.txt | | Test Results | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3223/1/testReport/ | | Max. process+thread count | 3481 (vs. ulimit of 3) | | modules | C: hbase-server U: hbase-server | | Console output | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3223/1/console | | versions | git=2.17.1 maven=3.6.3 | | Powered by | Apache Yetus 0.12.0 https://yetus.apache.org | This message was automatically generated. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #3204: HBASE-25741: Replication Source still having the replication metrics for peer ID which doesn't exist
Apache-HBase commented on pull request #3204: URL: https://github.com/apache/hbase/pull/3204#issuecomment-831593699 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 0m 0s | Docker mode activated. | | -1 :x: | docker | 0m 10s | Docker failed to build yetus/hbase:edccfe439a. | | Subsystem | Report/Notes | |--:|:-| | GITHUB PR | https://github.com/apache/hbase/pull/3204 | | JIRA Issue | HBASE-25741 | | Console output | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3204/8/console | | versions | git=2.17.1 | | Powered by | Apache Yetus 0.12.0 https://yetus.apache.org | This message was automatically generated. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #3226: HBASE-25843 move master http-related code into o.a.h.h.master.http
Apache-HBase commented on pull request #3226: URL: https://github.com/apache/hbase/pull/3226#issuecomment-831593620 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 0m 31s | Docker mode activated. | | -0 :warning: | yetus | 0m 3s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck | ||| _ Prechecks _ | ||| _ master Compile Tests _ | | +1 :green_heart: | mvninstall | 4m 43s | master passed | | +1 :green_heart: | compile | 1m 14s | master passed | | +1 :green_heart: | shadedjars | 8m 38s | branch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 0m 44s | master passed | ||| _ Patch Compile Tests _ | | -1 :x: | mvninstall | 2m 15s | root in the patch failed. | | -1 :x: | compile | 0m 42s | hbase-server in the patch failed. | | -0 :warning: | javac | 0m 42s | hbase-server in the patch failed. | | -1 :x: | shadedjars | 6m 7s | patch has 14 errors when building our shaded downstream artifacts. | | -0 :warning: | javadoc | 0m 23s | hbase-server in the patch failed. | ||| _ Other Tests _ | | -1 :x: | unit | 0m 41s | hbase-server in the patch failed. | | | | 27m 8s | | | Subsystem | Report/Notes | |--:|:-| | Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3226/1/artifact/yetus-jdk11-hadoop3-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/3226 | | Optional Tests | javac javadoc unit shadedjars compile | | uname | Linux 0ff7f7eb88d0 4.15.0-58-generic #64-Ubuntu SMP Tue Aug 6 11:12:41 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / 7640134e3e | | Default Java | AdoptOpenJDK-11.0.10+9 | | mvninstall | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3226/1/artifact/yetus-jdk11-hadoop3-check/output/patch-mvninstall-root.txt | | compile | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3226/1/artifact/yetus-jdk11-hadoop3-check/output/patch-compile-hbase-server.txt | | javac | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3226/1/artifact/yetus-jdk11-hadoop3-check/output/patch-compile-hbase-server.txt | | shadedjars | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3226/1/artifact/yetus-jdk11-hadoop3-check/output/patch-shadedjars.txt | | javadoc | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3226/1/artifact/yetus-jdk11-hadoop3-check/output/patch-javadoc-hbase-server.txt | | unit | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3226/1/artifact/yetus-jdk11-hadoop3-check/output/patch-unit-hbase-server.txt | | Test Results | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3226/1/testReport/ | | Max. process+thread count | 102 (vs. ulimit of 3) | | modules | C: hbase-server U: hbase-server | | Console output | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3226/1/console | | versions | git=2.17.1 maven=3.6.3 | | Powered by | Apache Yetus 0.12.0 https://yetus.apache.org | This message was automatically generated. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Commented] (HBASE-25844) Fix Jersey for hbase-server processes
[
https://issues.apache.org/jira/browse/HBASE-25844?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17338659#comment-17338659
]
Nick Dimiduk commented on HBASE-25844:
--
For reference, it would be nice to be able to use jersey for our WebUI
features. Most of our servlets would be more nicely implemented as Resources
and/or a Resource + a View.
I'd also like to start exposing JSON objects for driving a more interactive UI
experience, and such development is really nice with Jersey.
> Fix Jersey for hbase-server processes
> -
>
> Key: HBASE-25844
> URL: https://issues.apache.org/jira/browse/HBASE-25844
> Project: HBase
> Issue Type: Task
> Components: master, regionserver
>Affects Versions: 3.0.0-alpha-1, 2.5.0
>Reporter: Nick Dimiduk
>Priority: Major
>
> I spent some time trying to use Jersey from within the Master and it's not
> working. To summarize, we have unshaded resources from both
> jersey-server-1.19 and jersey-server-2.32 on the hbase-server classpath.
> Jersey's initialization uses ServiceLoader to look up concrete implementation
> classes of {{javax.ws.rs}} classes at runtime. Because we do not shade
> {{javax.ws.rs}} in hbase-thirdparty-jersey, an attempt to use shaded
> jersey-2.x still results in loading unshaded jersey-1.x jars, leading to an
> error like this
> {noformat}
> java.lang.AbstractMethodError:
> javax.ws.rs.core.UriBuilder.uri(Ljava/lang/String;)Ljavax/ws/rs/core/UriBuilder;
> at javax.ws.rs.core.UriBuilder.fromUri(UriBuilder.java:96)
> at
> org.apache.hbase.thirdparty.org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:275)
> at
> org.apache.hbase.thirdparty.org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:205)
> at
> org.apache.hbase.thirdparty.org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:791)
> at
> org.apache.hbase.thirdparty.org.eclipse.jetty.servlet.ServletHandler$ChainEnd.doFilter(ServletHandler.java:1626)
> at
> org.apache.hadoop.hbase.http.lib.StaticUserWebFilter$StaticUserFilter.doFilter(StaticUserWebFilter.java:112)
> {noformat}
> We cannot override what version of these classes are loaded at runtime via
> Java property because Jersey's load order implementation checks system
> properties as a last resort, not first thing as is claimed by javadoc.
> So I can think of two solutions.
> # One is to shade {{javax.ws.rs}} in hbase-thirdparty-jersey. This would
> shade both the interfaces and the resource files that are referenced at
> runtime, allowing for an entirely isolated jersey container instantiate.
> # Another idea is to add a custom {{ClassLoader}} that is inserted before
> jersey is initialized. This would filter out resources that are "banned",
> allowing our desired implementation through.
> Between these, I think (1) is better, but I don't know what else might break.
> I've made an effort of both, but with neither approach can I get a jersey
> environment to response from my resource class... either because the solution
> is incomplete, or because I don't have the jersey environment configured
> properly.
> See also some discussion that happened over on Slack,
> https://apache-hbase.slack.com/archives/C13K8NVAM/p1618857521051700
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Created] (HBASE-25844) Fix Jersey for hbase-server processes
Nick Dimiduk created HBASE-25844:
Summary: Fix Jersey for hbase-server processes
Key: HBASE-25844
URL: https://issues.apache.org/jira/browse/HBASE-25844
Project: HBase
Issue Type: Task
Components: master, regionserver
Affects Versions: 3.0.0-alpha-1, 2.5.0
Reporter: Nick Dimiduk
I spent some time trying to use Jersey from within the Master and it's not
working. To summarize, we have unshaded resources from both jersey-server-1.19
and jersey-server-2.32 on the hbase-server classpath. Jersey's initialization
uses ServiceLoader to look up concrete implementation classes of
{{javax.ws.rs}} classes at runtime. Because we do not shade {{javax.ws.rs}} in
hbase-thirdparty-jersey, an attempt to use shaded jersey-2.x still results in
loading unshaded jersey-1.x jars, leading to an error like this
{noformat}
java.lang.AbstractMethodError:
javax.ws.rs.core.UriBuilder.uri(Ljava/lang/String;)Ljavax/ws/rs/core/UriBuilder;
at javax.ws.rs.core.UriBuilder.fromUri(UriBuilder.java:96)
at
org.apache.hbase.thirdparty.org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:275)
at
org.apache.hbase.thirdparty.org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:205)
at
org.apache.hbase.thirdparty.org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:791)
at
org.apache.hbase.thirdparty.org.eclipse.jetty.servlet.ServletHandler$ChainEnd.doFilter(ServletHandler.java:1626)
at
org.apache.hadoop.hbase.http.lib.StaticUserWebFilter$StaticUserFilter.doFilter(StaticUserWebFilter.java:112)
{noformat}
We cannot override what version of these classes are loaded at runtime via Java
property because Jersey's load order implementation checks system properties as
a last resort, not first thing as is claimed by javadoc.
So I can think of two solutions.
# One is to shade {{javax.ws.rs}} in hbase-thirdparty-jersey. This would shade
both the interfaces and the resource files that are referenced at runtime,
allowing for an entirely isolated jersey container instantiate.
# Another idea is to add a custom {{ClassLoader}} that is inserted before
jersey is initialized. This would filter out resources that are "banned",
allowing our desired implementation through.
Between these, I think (1) is better, but I don't know what else might break.
I've made an effort of both, but with neither approach can I get a jersey
environment to response from my resource class... either because the solution
is incomplete, or because I don't have the jersey environment configured
properly.
See also some discussion that happened over on Slack,
https://apache-hbase.slack.com/archives/C13K8NVAM/p1618857521051700
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[GitHub] [hbase] Apache-HBase commented on pull request #3226: HBASE-25843 move master http-related code into o.a.h.h.master.http
Apache-HBase commented on pull request #3226: URL: https://github.com/apache/hbase/pull/3226#issuecomment-831592124 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 0m 28s | Docker mode activated. | ||| _ Prechecks _ | | +1 :green_heart: | dupname | 0m 0s | No case conflicting files found. | | +1 :green_heart: | hbaseanti | 0m 0s | Patch does not have any anti-patterns. | | +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. | ||| _ master Compile Tests _ | | +1 :green_heart: | mvninstall | 3m 53s | master passed | | +1 :green_heart: | compile | 3m 12s | master passed | | +1 :green_heart: | checkstyle | 1m 7s | master passed | | +1 :green_heart: | spotbugs | 2m 7s | master passed | ||| _ Patch Compile Tests _ | | -1 :x: | mvninstall | 1m 46s | root in the patch failed. | | -1 :x: | compile | 0m 42s | hbase-server in the patch failed. | | -0 :warning: | javac | 0m 42s | hbase-server in the patch failed. | | +1 :green_heart: | checkstyle | 1m 3s | the patch passed | | -0 :warning: | whitespace | 0m 0s | The patch has 1 line(s) that end in whitespace. Use git apply --whitespace=fix <>. Refer https://git-scm.com/docs/git-apply | | -1 :x: | hadoopcheck | 2m 1s | The patch causes 14 errors with Hadoop v3.1.2. | | -1 :x: | hadoopcheck | 4m 3s | The patch causes 14 errors with Hadoop v3.2.1. | | -1 :x: | hadoopcheck | 6m 10s | The patch causes 14 errors with Hadoop v3.3.0. | | -1 :x: | spotbugs | 0m 36s | hbase-server in the patch failed. | ||| _ Other Tests _ | | +1 :green_heart: | asflicense | 0m 14s | The patch does not generate ASF License warnings. | | | | 22m 59s | | | Subsystem | Report/Notes | |--:|:-| | Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3226/1/artifact/yetus-general-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/3226 | | Optional Tests | dupname asflicense javac spotbugs hadoopcheck hbaseanti checkstyle compile | | uname | Linux ba3cf295b9ab 4.15.0-58-generic #64-Ubuntu SMP Tue Aug 6 11:12:41 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / 7640134e3e | | Default Java | AdoptOpenJDK-1.8.0_282-b08 | | mvninstall | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3226/1/artifact/yetus-general-check/output/patch-mvninstall-root.txt | | compile | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3226/1/artifact/yetus-general-check/output/patch-compile-hbase-server.txt | | javac | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3226/1/artifact/yetus-general-check/output/patch-compile-hbase-server.txt | | whitespace | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3226/1/artifact/yetus-general-check/output/whitespace-eol.txt | | hadoopcheck | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3226/1/artifact/yetus-general-check/output/patch-javac-3.1.2.txt | | hadoopcheck | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3226/1/artifact/yetus-general-check/output/patch-javac-3.2.1.txt | | hadoopcheck | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3226/1/artifact/yetus-general-check/output/patch-javac-3.3.0.txt | | spotbugs | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3226/1/artifact/yetus-general-check/output/patch-spotbugs-hbase-server.txt | | Max. process+thread count | 96 (vs. ulimit of 3) | | modules | C: hbase-server U: hbase-server | | Console output | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3226/1/console | | versions | git=2.17.1 maven=3.6.3 spotbugs=4.2.2 | | Powered by | Apache Yetus 0.12.0 https://yetus.apache.org | This message was automatically generated. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Comment Edited] (HBASE-25829) SPLIT state detritus
[
https://issues.apache.org/jira/browse/HBASE-25829?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17338650#comment-17338650
]
Andrew Kyle Purtell edited comment on HBASE-25829 at 5/3/21, 10:56 PM:
---
Subtasks look good. Back to the main issue.
{noformat}
2021-05-03 20:30:29,964 INFO [master/ip-172-31-58-47:8100.Chore.2]
master.HbckChore: Loaded 184 regions from in-memory state of AssignmentManager
2021-05-03 20:30:29,964 INFO [master/ip-172-31-58-47:8100.Chore.2]
master.HbckChore: Loaded 133 regions from 5 regionservers' reports and found 0
orphan regions
2021-05-03 20:30:29,975 INFO [master/ip-172-31-58-47:8100.Chore.2]
master.HbckChore: Loaded 3 tables 184 regions from filesystem and found 0
orphan regions
{noformat}
The 51 extra regions are SPLIT parents, with server = null.
I notice in AssignmentManager#markRegionAsMerged we remove the merge parents
from {{regionStates}} right there, but in AssignmentManager#markRegionAsSplit
we do not. We have code in various places that account for a post-split parent
to be hanging out in {{regionStates}} in SPLIT state. CatalogJanitor is
supposed to clean it, but does not!
If I patch AssignmentManager#markRegionAsSplit to remove the parent from
{{regionStates}} for spits the same way AssignmentManager#markRegionAsMerged
does for merges, then things begin to look better:
{noformat}
2021-05-03 22:08:29,036 INFO [master/ip-172-31-58-47:8100.Chore.1]
master.HbckChore: Loaded 23 regions from in-memory state of AssignmentManager
2021-05-03 22:08:29,036 INFO [master/ip-172-31-58-47:8100.Chore.1]
master.HbckChore: Loaded 23 regions from 5 regionservers' reports and found 0
orphan regions
2021-05-03 22:08:29,043 INFO [master/ip-172-31-58-47:8100.Chore.1]
master.HbckChore: Loaded 3 tables 32 regions from filesystem and found 9 orphan
regions
{noformat}
No more junk in {{regionStates}} but those 9 split parents are found as orphan
regions. There is a simple change to HbckChore that should accompany the other
changes I have under test: The conditional for determining if a region is
orphan should become {{if (hri == null &&
!splitParentRegions.contains(encodedRegionName) &&
!mergedParentRegions.contains(encodedRegionName))}} so you can ignore the
non-zero orphan count. With the complete change it would be reported as 0.
Anyway, there is more to debug. Seems CatalogJanitor or the region GC
procedures it submits are not completing their work. I have added some debug
logging to CatalogJanitor to investigate further.
was (Author: apurtell):
Subtasks look good. Back to the main issue.
{noformat}
2021-05-03 20:30:29,964 INFO [master/ip-172-31-58-47:8100.Chore.2]
master.HbckChore: Loaded 184 regions from in-memory state of AssignmentManager
2021-05-03 20:30:29,964 INFO [master/ip-172-31-58-47:8100.Chore.2]
master.HbckChore: Loaded 133 regions from 5 regionservers' reports and found 0
orphan regions
2021-05-03 20:30:29,975 INFO [master/ip-172-31-58-47:8100.Chore.2]
master.HbckChore: Loaded 3 tables 184 regions from filesystem and found 0
orphan regions
{noformat}
The 51 extra regions are SPLIT parents, with server = null.
I notice in AssignmentManager#markRegionAsMerged we remove the merge parents
from {{regionStates}} right there, but in AssignmentManager#markRegionAsSplit
we do not. We have code in various places that account for a post-split parent
to be hanging out in {{regionStates}} in SPLIT state. CatalogJanitor is
supposed to clean it, but does not!
If I patch AssignmentManager#markRegionAsSplit to remove the parent from
{{regionStates}} for spits the same way AssignmentManager#markRegionAsMerged
does for merges, then things begin to look better:
{noformat}2021-05-03 22:08:29,036 INFO [master/ip-172-31-58-47:8100.Chore.1]
master.HbckChore: Loaded 23 regions from in-memory state of AssignmentManager
2021-05-03 22:08:29,036 INFO [master/ip-172-31-58-47:8100.Chore.1]
master.HbckChore: Loaded 23 regions from 5 regionservers' reports and found 0
orphan regions
2021-05-03 22:08:29,043 INFO [master/ip-172-31-58-47:8100.Chore.1]
master.HbckChore: Loaded 3 tables 32 regions from filesystem and found 9 orphan
regions
{noformat}
No more junk in {{regionStates}} but those 9 split parents are found as orphan
regions. There is a simple change to HbckChore that should accompany the other
changes I have under test: The conditional for determining if a region is
orphan should become {{if (hri == null &&
!splitParentRegions.contains(encodedRegionName) &&
!mergedParentRegions.contains(encodedRegionName))}} so you can ignore the
non-zero orphan count. With the complete change it would be reported as 0.
Anyway, there is more to debug. CatalogJanitor or the region GC procedures it
submits may not be completing their work. I have added some debug logging to
CatalogJanitor to investigate further.
> SPLIT state detritus
>
[jira] [Comment Edited] (HBASE-25829) SPLIT state detritus
[
https://issues.apache.org/jira/browse/HBASE-25829?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17338650#comment-17338650
]
Andrew Kyle Purtell edited comment on HBASE-25829 at 5/3/21, 10:55 PM:
---
Subtasks look good. Back to the main issue.
{noformat}
2021-05-03 20:30:29,964 INFO [master/ip-172-31-58-47:8100.Chore.2]
master.HbckChore: Loaded 184 regions from in-memory state of AssignmentManager
2021-05-03 20:30:29,964 INFO [master/ip-172-31-58-47:8100.Chore.2]
master.HbckChore: Loaded 133 regions from 5 regionservers' reports and found 0
orphan regions
2021-05-03 20:30:29,975 INFO [master/ip-172-31-58-47:8100.Chore.2]
master.HbckChore: Loaded 3 tables 184 regions from filesystem and found 0
orphan regions
{noformat}
The 51 extra regions are SPLIT parents, with server = null.
I notice in AssignmentManager#markRegionAsMerged we remove the merge parents
from {{regionStates}} right there, but in AssignmentManager#markRegionAsSplit
we do not. We have code in various places that account for a post-split parent
to be hanging out in {{regionStates}} in SPLIT state. CatalogJanitor is
supposed to clean it, but does not!
If I patch AssignmentManager#markRegionAsSplit to remove the parent from
{{regionStates}} for spits the same way AssignmentManager#markRegionAsMerged
does for merges, then things begin to look better:
{noformat}2021-05-03 22:08:29,036 INFO [master/ip-172-31-58-47:8100.Chore.1]
master.HbckChore: Loaded 23 regions from in-memory state of AssignmentManager
2021-05-03 22:08:29,036 INFO [master/ip-172-31-58-47:8100.Chore.1]
master.HbckChore: Loaded 23 regions from 5 regionservers' reports and found 0
orphan regions
2021-05-03 22:08:29,043 INFO [master/ip-172-31-58-47:8100.Chore.1]
master.HbckChore: Loaded 3 tables 32 regions from filesystem and found 9 orphan
regions
{noformat}
No more junk in {{regionStates}} but those 9 split parents are found as orphan
regions. There is a simple change to HbckChore that should accompany the other
changes I have under test: The conditional for determining if a region is
orphan should become {{if (hri == null &&
!splitParentRegions.contains(encodedRegionName) &&
!mergedParentRegions.contains(encodedRegionName))}} so you can ignore the
non-zero orphan count. With the complete change it would be reported as 0.
Anyway, there is more to debug. CatalogJanitor or the region GC procedures it
submits may not be completing their work. I have added some debug logging to
CatalogJanitor to investigate further.
was (Author: apurtell):
Subtasks look good. Back to the main issue.
{noformat}
2021-05-03 20:30:29,964 INFO [master/ip-172-31-58-47:8100.Chore.2]
master.HbckChore: Loaded 184 regions from in-memory state of AssignmentManager
2021-05-03 20:30:29,964 INFO [master/ip-172-31-58-47:8100.Chore.2]
master.HbckChore: Loaded 133 regions from 5 regionservers' reports and found 0
orphan regions
2021-05-03 20:30:29,975 INFO [master/ip-172-31-58-47:8100.Chore.2]
master.HbckChore: Loaded 3 tables 184 regions from filesystem and found 0
orphan regions
{noformat}
The 51 extra regions are SPLIT parents, with server = null.
I notice in AssignmentManager#markRegionAsMerged we remove the merge parents
from {{regionStates}} right there, but in AssignmentManager#markRegionAsSplit
we do not. We have code in various places that account for a post-split parent
to be hanging out in {{regionStates}} in SPLIT state. CatalogJanitor is
supposed to clean it, but does not!
If I patch AssignmentManager#markRegionAsSplit to remove the parent from
{{regionStates}} for spits the same way AssignmentManager#markRegionAsMerged
does for merges, then things begin to look better:
{noformat}2021-05-03 22:08:29,036 INFO [master/ip-172-31-58-47:8100.Chore.1]
master.HbckChore: Loaded 23 regions from in-memory state of AssignmentManager
2021-05-03 22:08:29,036 INFO [master/ip-172-31-58-47:8100.Chore.1]
master.HbckChore: Loaded 23 regions from 5 regionservers' reports and found 0
orphan regions
2021-05-03 22:08:29,043 INFO [master/ip-172-31-58-47:8100.Chore.1]
master.HbckChore: Loaded 3 tables 32 regions from filesystem and found 9 orphan
regions
{noformat}
No more junk in {{regionStates}} but those 9 split parents are found as orphan
regions. There is a simple change to HbckChore that should accompany the other
changes I have under test: The conditional for determining if a region is
orphan should become {{ if (hri == null &&
!splitParentRegions.contains(encodedRegionName) &&
!mergedParentRegions.contains(encodedRegionName)) }} so you can ignore the
non-zero orphan count. With the complete change it would be reported as 0.
Anyway, there is more to debug. CatalogJanitor or the region GC procedures it
submits may not be completing their work. I have added some debug logging to
CatalogJanitor to investigate further.
> SPLIT state detritus
[jira] [Comment Edited] (HBASE-25829) SPLIT state detritus
[
https://issues.apache.org/jira/browse/HBASE-25829?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17338650#comment-17338650
]
Andrew Kyle Purtell edited comment on HBASE-25829 at 5/3/21, 10:54 PM:
---
Subtasks look good. Back to the main issue.
{noformat}
2021-05-03 20:30:29,964 INFO [master/ip-172-31-58-47:8100.Chore.2]
master.HbckChore: Loaded 184 regions from in-memory state of AssignmentManager
2021-05-03 20:30:29,964 INFO [master/ip-172-31-58-47:8100.Chore.2]
master.HbckChore: Loaded 133 regions from 5 regionservers' reports and found 0
orphan regions
2021-05-03 20:30:29,975 INFO [master/ip-172-31-58-47:8100.Chore.2]
master.HbckChore: Loaded 3 tables 184 regions from filesystem and found 0
orphan regions
{noformat}
The 51 extra regions are SPLIT parents, with server = null.
I notice in AssignmentManager#markRegionAsMerged we remove the merge parents
from {{regionStates}} right there, but in AssignmentManager#markRegionAsSplit
we do not. We have code in various places that account for a post-split parent
to be hanging out in {{regionStates}} in SPLIT state. CatalogJanitor is
supposed to clean it, but does not!
If I patch AssignmentManager#markRegionAsSplit to remove the parent from
{{regionStates}} for spits the same way AssignmentManager#markRegionAsMerged
does for merges, then things begin to look better:
{noformat}2021-05-03 22:08:29,036 INFO [master/ip-172-31-58-47:8100.Chore.1]
master.HbckChore: Loaded 23 regions from in-memory state of AssignmentManager
2021-05-03 22:08:29,036 INFO [master/ip-172-31-58-47:8100.Chore.1]
master.HbckChore: Loaded 23 regions from 5 regionservers' reports and found 0
orphan regions
2021-05-03 22:08:29,043 INFO [master/ip-172-31-58-47:8100.Chore.1]
master.HbckChore: Loaded 3 tables 32 regions from filesystem and found 9 orphan
regions
{noformat}
No more junk in {{regionStates}} but those 9 split parents are found as orphan
regions. There is a simple change to HbckChore that should accompany the other
changes I have under test: The conditional for determining if a region is
orphan should become {{ if (hri == null &&
!splitParentRegions.contains(encodedRegionName) &&
!mergedParentRegions.contains(encodedRegionName)) }} so you can ignore the
non-zero orphan count. With the complete change it would be reported as 0.
Anyway, there is more to debug. CatalogJanitor or the region GC procedures it
submits may not be completing their work. I have added some debug logging to
CatalogJanitor to investigate further.
was (Author: apurtell):
Subtasks look good. Back to the main issue.
{noformat}
2021-05-03 20:30:29,964 INFO [master/ip-172-31-58-47:8100.Chore.2]
master.HbckChore: Loaded 184 regions from in-memory state of AssignmentManager
2021-05-03 20:30:29,964 INFO [master/ip-172-31-58-47:8100.Chore.2]
master.HbckChore: Loaded 133 regions from 5 regionservers' reports and found 0
orphan regions
2021-05-03 20:30:29,975 INFO [master/ip-172-31-58-47:8100.Chore.2]
master.HbckChore: Loaded 3 tables 184 regions from filesystem and found 0
orphan regions
{noformat}
The 51 extra regions are SPLIT parents, with server = null.
I notice in AssignmentManager#markRegionAsMerged we remove the merge parents
from {{regionStates}} right there, but in AssignmentManager#markRegionAsSplit
we do not. We have code in various places that account for a post-split parent
to be hanging out in {{regionStates}} in SPLIT state. CatalogJanitor is
supposed to clean it, but does not!
If I patch AssignmentManager#markRegionAsSplit to remove the parent from
{{regionStates}} for spits the same way AssignmentManager#markRegionAsMerged
does for merges, then things begin to look better:
{noformat}2021-05-03 22:08:29,036 INFO [master/ip-172-31-58-47:8100.Chore.1]
master.HbckChore: Loaded 23 regions from in-memory state of AssignmentManager
2021-05-03 22:08:29,036 INFO [master/ip-172-31-58-47:8100.Chore.1]
master.HbckChore: Loaded 23 regions from 5 regionservers' reports and found 0
orphan regions
2021-05-03 22:08:29,043 INFO [master/ip-172-31-58-47:8100.Chore.1]
master.HbckChore: Loaded 3 tables 32 regions from filesystem and found 9 orphan
regions
{noformat}
No more junk in {{regionStates}} but those 9 split parents are found as orphan
regions.
I have added some debug logging to CatalogJanitor to investigate further.
> SPLIT state detritus
>
>
> Key: HBASE-25829
> URL: https://issues.apache.org/jira/browse/HBASE-25829
> Project: HBase
> Issue Type: Bug
>Affects Versions: 2.4.3
>Reporter: Andrew Kyle Purtell
>Assignee: Andrew Kyle Purtell
>Priority: Major
> Fix For: 3.0.0-alpha-1, 2.5.0, 2.4.3
>
>
> Seen after an integration test (see HBASE-25824) with 'calm' monkey, so this
> happened in the happy path.
> There were no
[jira] [Work started] (HBASE-25842) Move regionserver http-related code into o.a.h.h.regionserver.http
[ https://issues.apache.org/jira/browse/HBASE-25842?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Work on HBASE-25842 started by Nick Dimiduk. > Move regionserver http-related code into o.a.h.h.regionserver.http > -- > > Key: HBASE-25842 > URL: https://issues.apache.org/jira/browse/HBASE-25842 > Project: HBase > Issue Type: Task > Components: regionserver >Reporter: Nick Dimiduk >Assignee: Nick Dimiduk >Priority: Minor > > Minor refactor. Group code related to the Region Server's Info Page into a > common package. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Comment Edited] (HBASE-25829) SPLIT state detritus
[
https://issues.apache.org/jira/browse/HBASE-25829?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17338650#comment-17338650
]
Andrew Kyle Purtell edited comment on HBASE-25829 at 5/3/21, 10:50 PM:
---
Subtasks look good. Back to the main issue.
{noformat}
2021-05-03 20:30:29,964 INFO [master/ip-172-31-58-47:8100.Chore.2]
master.HbckChore: Loaded 184 regions from in-memory state of AssignmentManager
2021-05-03 20:30:29,964 INFO [master/ip-172-31-58-47:8100.Chore.2]
master.HbckChore: Loaded 133 regions from 5 regionservers' reports and found 0
orphan regions
2021-05-03 20:30:29,975 INFO [master/ip-172-31-58-47:8100.Chore.2]
master.HbckChore: Loaded 3 tables 184 regions from filesystem and found 0
orphan regions
{noformat}
The 51 extra regions are SPLIT parents, with server = null.
I notice in AssignmentManager#markRegionAsMerged we remove the merge parents
from {{regionStates}} right there, but in AssignmentManager#markRegionAsSplit
we do not. We have code in various places that account for a post-split parent
to be hanging out in {{regionStates}} in SPLIT state. CatalogJanitor is
supposed to clean it, but does not!
If I patch AssignmentManager#markRegionAsSplit to remove the parent from
{{regionStates}} for spits the same way AssignmentManager#markRegionAsMerged
does for merges, then things begin to look better:
{noformat}2021-05-03 22:08:29,036 INFO [master/ip-172-31-58-47:8100.Chore.1]
master.HbckChore: Loaded 23 regions from in-memory state of AssignmentManager
2021-05-03 22:08:29,036 INFO [master/ip-172-31-58-47:8100.Chore.1]
master.HbckChore: Loaded 23 regions from 5 regionservers' reports and found 0
orphan regions
2021-05-03 22:08:29,043 INFO [master/ip-172-31-58-47:8100.Chore.1]
master.HbckChore: Loaded 3 tables 32 regions from filesystem and found 9 orphan
regions
{noformat}
No more junk in {{regionStates}} but those 9 split parents are found as orphan
regions.
I have added some debug logging to CatalogJanitor to investigate further.
was (Author: apurtell):
Subtasks look good. Back to the main issue.
{noformat}
2021-05-03 20:30:29,964 INFO [master/ip-172-31-58-47:8100.Chore.2]
master.HbckChore: Loaded 184 regions from in-memory state of AssignmentManager
2021-05-03 20:30:29,964 INFO [master/ip-172-31-58-47:8100.Chore.2]
master.HbckChore: Loaded 133 regions from 5 regionservers' reports and found 0
orphan regions
2021-05-03 20:30:29,975 INFO [master/ip-172-31-58-47:8100.Chore.2]
master.HbckChore: Loaded 3 tables 184 regions from filesyetem and found 0
orphan regions
{noformat}
The 51 extra regions are SPLIT parents, with server = null.
I notice in AssignmentManager#markRegionAsMerged we remove the merge parents
from {{regionStates}} right there, but in AssignmentManager#markRegionAsSplit
we do not. We have code in various places that account for a post-split parent
to be hanging out in {{regionStates}} in SPLIT state. CatalogJanitor is
supposed to clean it, but does not!
If I patch AssignmentManager#markRegionAsSplit to remove the parent from
{{regionStates}} for spits the same way AssignmentManager#markRegionAsMerged
does for merges, then things begin to look better:
{noformat}2021-05-03 22:08:29,036 INFO [master/ip-172-31-58-47:8100.Chore.1]
master.HbckChore: Loaded 23 regions from in-memory state of AssignmentManager
2021-05-03 22:08:29,036 INFO [master/ip-172-31-58-47:8100.Chore.1]
master.HbckChore: Loaded 23 regions from 5 regionservers' reports and found 0
orphan regions
2021-05-03 22:08:29,043 INFO [master/ip-172-31-58-47:8100.Chore.1]
master.HbckChore: Loaded 3 tables 32 regions from filesyetem and found 9 orphan
regions
{noformat}
No more junk in {{regionStates}} but those 9 split parents are found as orphan
regions.
I have added some debug logging to CatalogJanitor to investigate further.
> SPLIT state detritus
>
>
> Key: HBASE-25829
> URL: https://issues.apache.org/jira/browse/HBASE-25829
> Project: HBase
> Issue Type: Bug
>Affects Versions: 2.4.3
>Reporter: Andrew Kyle Purtell
>Assignee: Andrew Kyle Purtell
>Priority: Major
> Fix For: 3.0.0-alpha-1, 2.5.0, 2.4.3
>
>
> Seen after an integration test (see HBASE-25824) with 'calm' monkey, so this
> happened in the happy path.
> There were no errors accessing all loaded table data. The integration test
> writes a log to HDFS of every cell written to HBase and the verify phase uses
> that log to read each value and confirm it. That seems fine:
> {noformat}
> 2021-04-30 02:16:33,316 INFO [main]
> test.IntegrationTestLoadCommonCrawl$Verify: REFERENCED: 154943544
> 2021-04-30 02:16:33,316 INFO [main]
> test.IntegrationTestLoadCommonCrawl$Verify: UNREFERENCED: 0
> 2021-04-30 02:16:33,316 INFO [main]
> test.IntegrationTestLoadCommonCrawl$Verify:
[jira] [Updated] (HBASE-25842) Move regionserver http-related code into o.a.h.h.regionserver.http
[ https://issues.apache.org/jira/browse/HBASE-25842?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nick Dimiduk updated HBASE-25842: - Status: Patch Available (was: In Progress) > Move regionserver http-related code into o.a.h.h.regionserver.http > -- > > Key: HBASE-25842 > URL: https://issues.apache.org/jira/browse/HBASE-25842 > Project: HBase > Issue Type: Task > Components: regionserver >Reporter: Nick Dimiduk >Assignee: Nick Dimiduk >Priority: Minor > > Minor refactor. Group code related to the Region Server's Info Page into a > common package. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HBASE-25843) Move master http-related code into o.a.h.h.master.http
[ https://issues.apache.org/jira/browse/HBASE-25843?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nick Dimiduk updated HBASE-25843: - Status: Patch Available (was: Open) > Move master http-related code into o.a.h.h.master.http > -- > > Key: HBASE-25843 > URL: https://issues.apache.org/jira/browse/HBASE-25843 > Project: HBase > Issue Type: Task > Components: master >Reporter: Nick Dimiduk >Assignee: Nick Dimiduk >Priority: Minor > > Minor refactor. Group code related to the Master's Info Page into a common > package. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (HBASE-25829) SPLIT state detritus
[
https://issues.apache.org/jira/browse/HBASE-25829?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17338650#comment-17338650
]
Andrew Kyle Purtell commented on HBASE-25829:
-
Subtasks look good. Back to the main issue.
{noformat}
2021-05-03 20:30:29,964 INFO [master/ip-172-31-58-47:8100.Chore.2]
master.HbckChore: Loaded 184 regions from in-memory state of AssignmentManager
2021-05-03 20:30:29,964 INFO [master/ip-172-31-58-47:8100.Chore.2]
master.HbckChore: Loaded 133 regions from 5 regionservers' reports and found 0
orphan regions
2021-05-03 20:30:29,975 INFO [master/ip-172-31-58-47:8100.Chore.2]
master.HbckChore: Loaded 3 tables 184 regions from filesyetem and found 0
orphan regions
{noformat}
The 51 extra regions are SPLIT parents, with server = null.
I notice in AssignmentManager#markRegionAsMerged we remove the merge parents
from {{regionStates}} right there, but in AssignmentManager#markRegionAsSplit
we do not. We have code in various places that account for a post-split parent
to be hanging out in {{regionStates}} in SPLIT state. CatalogJanitor is
supposed to clean it, but does not!
If I patch AssignmentManager#markRegionAsSplit to remove the parent from
{{regionStates}}, things begin to look better:
{noformat}2021-05-03 22:08:29,036 INFO [master/ip-172-31-58-47:8100.Chore.1]
master.HbckChore: Loaded 23 regions from in-memory state of AssignmentManager
2021-05-03 22:08:29,036 INFO [master/ip-172-31-58-47:8100.Chore.1]
master.HbckChore: Loaded 23 regions from 5 regionservers' reports and found 0
orphan regions
2021-05-03 22:08:29,043 INFO [master/ip-172-31-58-47:8100.Chore.1]
master.HbckChore: Loaded 3 tables 32 regions from filesyetem and found 9 orphan
regions
{noformat}
No more junk in {{regionStates}} but those 9 split parents are found as orphan
regions.
I have added some debug logging to CatalogJanitor to investigate further.
> SPLIT state detritus
>
>
> Key: HBASE-25829
> URL: https://issues.apache.org/jira/browse/HBASE-25829
> Project: HBase
> Issue Type: Bug
>Affects Versions: 2.4.3
>Reporter: Andrew Kyle Purtell
>Assignee: Andrew Kyle Purtell
>Priority: Major
> Fix For: 3.0.0-alpha-1, 2.5.0, 2.4.3
>
>
> Seen after an integration test (see HBASE-25824) with 'calm' monkey, so this
> happened in the happy path.
> There were no errors accessing all loaded table data. The integration test
> writes a log to HDFS of every cell written to HBase and the verify phase uses
> that log to read each value and confirm it. That seems fine:
> {noformat}
> 2021-04-30 02:16:33,316 INFO [main]
> test.IntegrationTestLoadCommonCrawl$Verify: REFERENCED: 154943544
> 2021-04-30 02:16:33,316 INFO [main]
> test.IntegrationTestLoadCommonCrawl$Verify: UNREFERENCED: 0
> 2021-04-30 02:16:33,316 INFO [main]
> test.IntegrationTestLoadCommonCrawl$Verify: CORRUPT: 0
> {noformat}
> However whenever the balancer runs there are a number of concerning INFO
> level log messages printed of the form _assignment.RegionStates: Skipping, no
> server for state=SPLIT, location=null, table=TABLENAME_
> For example:
> {noformat}
> 2021-04-30 02:02:09,286 INFO [master/ip-172-31-58-47:8100.Chore.2]
> assignment.RegionStates: Skipping, no server for state=SPLIT, location=null,
> table=IntegrationTestLoadCommonCrawl, region=087fb2f7847c2fc0a0b85eb30a97036e
> 2021-04-30 02:02:09,286 INFO [master/ip-172-31-58-47:8100.Chore.2]
> assignment.RegionStates: Skipping, no server for state=SPLIT, location=null,
> table=IntegrationTestLoadCommonCrawl, region=0952b94a920454afe9c40becbb7bf205
> 2021-04-30 02:02:09,286 INFO [master/ip-172-31-58-47:8100.Chore.2]
> assignment.RegionStates: Skipping, no server for state=SPLIT, location=null,
> table=IntegrationTestLoadCommonCrawl, region=f87a8b993f7eca2524bf2331b7ee3c06
> 2021-04-30 02:02:09,286 INFO [master/ip-172-31-58-47:8100.Chore.2]
> assignment.RegionStates: Skipping, no server for state=SPLIT, location=null,
> table=IntegrationTestLoadCommonCrawl, region=74bb28864a120decdf0f4956741df745
> 2021-04-30 02:02:09,286 INFO [master/ip-172-31-58-47:8100.Chore.2]
> assignment.RegionStates: Skipping, no server for state=SPLIT, location=null,
> table=IntegrationTestLoadCommonCrawl, region=bc918b609ade0ae4d5530f0467354cae
> 2021-04-30 02:02:09,286 INFO [master/ip-172-31-58-47:8100.Chore.2]
> assignment.RegionStates: Skipping, no server for state=SPLIT, location=null,
> table=IntegrationTestLoadCommonCrawl, region=183a199984539f3917a2f8927fe01572
> 2021-04-30 02:02:09,286 INFO [master/ip-172-31-58-47:8100.Chore.2]
> assignment.RegionStates: Skipping, no server for state=SPLIT, location=null,
> table=IntegrationTestLoadCommonCrawl, region=6cc5ce4fb4adc00445b3ec7dd8760ba8
> {noformat}
> The HBCK chore notices them but does nothing:
> "Loaded *80
[GitHub] [hbase] ndimiduk opened a new pull request #3226: HBASE-25843 move master http-related code into o.a.h.h.master.http
ndimiduk opened a new pull request #3226: URL: https://github.com/apache/hbase/pull/3226 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Comment Edited] (HBASE-25829) SPLIT state detritus
[
https://issues.apache.org/jira/browse/HBASE-25829?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17338650#comment-17338650
]
Andrew Kyle Purtell edited comment on HBASE-25829 at 5/3/21, 10:47 PM:
---
Subtasks look good. Back to the main issue.
{noformat}
2021-05-03 20:30:29,964 INFO [master/ip-172-31-58-47:8100.Chore.2]
master.HbckChore: Loaded 184 regions from in-memory state of AssignmentManager
2021-05-03 20:30:29,964 INFO [master/ip-172-31-58-47:8100.Chore.2]
master.HbckChore: Loaded 133 regions from 5 regionservers' reports and found 0
orphan regions
2021-05-03 20:30:29,975 INFO [master/ip-172-31-58-47:8100.Chore.2]
master.HbckChore: Loaded 3 tables 184 regions from filesyetem and found 0
orphan regions
{noformat}
The 51 extra regions are SPLIT parents, with server = null.
I notice in AssignmentManager#markRegionAsMerged we remove the merge parents
from {{regionStates}} right there, but in AssignmentManager#markRegionAsSplit
we do not. We have code in various places that account for a post-split parent
to be hanging out in {{regionStates}} in SPLIT state. CatalogJanitor is
supposed to clean it, but does not!
If I patch AssignmentManager#markRegionAsSplit to remove the parent from
{{regionStates}} for spits the same way AssignmentManager#markRegionAsMerged
does for merges, then things begin to look better:
{noformat}2021-05-03 22:08:29,036 INFO [master/ip-172-31-58-47:8100.Chore.1]
master.HbckChore: Loaded 23 regions from in-memory state of AssignmentManager
2021-05-03 22:08:29,036 INFO [master/ip-172-31-58-47:8100.Chore.1]
master.HbckChore: Loaded 23 regions from 5 regionservers' reports and found 0
orphan regions
2021-05-03 22:08:29,043 INFO [master/ip-172-31-58-47:8100.Chore.1]
master.HbckChore: Loaded 3 tables 32 regions from filesyetem and found 9 orphan
regions
{noformat}
No more junk in {{regionStates}} but those 9 split parents are found as orphan
regions.
I have added some debug logging to CatalogJanitor to investigate further.
was (Author: apurtell):
Subtasks look good. Back to the main issue.
{noformat}
2021-05-03 20:30:29,964 INFO [master/ip-172-31-58-47:8100.Chore.2]
master.HbckChore: Loaded 184 regions from in-memory state of AssignmentManager
2021-05-03 20:30:29,964 INFO [master/ip-172-31-58-47:8100.Chore.2]
master.HbckChore: Loaded 133 regions from 5 regionservers' reports and found 0
orphan regions
2021-05-03 20:30:29,975 INFO [master/ip-172-31-58-47:8100.Chore.2]
master.HbckChore: Loaded 3 tables 184 regions from filesyetem and found 0
orphan regions
{noformat}
The 51 extra regions are SPLIT parents, with server = null.
I notice in AssignmentManager#markRegionAsMerged we remove the merge parents
from {{regionStates}} right there, but in AssignmentManager#markRegionAsSplit
we do not. We have code in various places that account for a post-split parent
to be hanging out in {{regionStates}} in SPLIT state. CatalogJanitor is
supposed to clean it, but does not!
If I patch AssignmentManager#markRegionAsSplit to remove the parent from
{{regionStates}}, things begin to look better:
{noformat}2021-05-03 22:08:29,036 INFO [master/ip-172-31-58-47:8100.Chore.1]
master.HbckChore: Loaded 23 regions from in-memory state of AssignmentManager
2021-05-03 22:08:29,036 INFO [master/ip-172-31-58-47:8100.Chore.1]
master.HbckChore: Loaded 23 regions from 5 regionservers' reports and found 0
orphan regions
2021-05-03 22:08:29,043 INFO [master/ip-172-31-58-47:8100.Chore.1]
master.HbckChore: Loaded 3 tables 32 regions from filesyetem and found 9 orphan
regions
{noformat}
No more junk in {{regionStates}} but those 9 split parents are found as orphan
regions.
I have added some debug logging to CatalogJanitor to investigate further.
> SPLIT state detritus
>
>
> Key: HBASE-25829
> URL: https://issues.apache.org/jira/browse/HBASE-25829
> Project: HBase
> Issue Type: Bug
>Affects Versions: 2.4.3
>Reporter: Andrew Kyle Purtell
>Assignee: Andrew Kyle Purtell
>Priority: Major
> Fix For: 3.0.0-alpha-1, 2.5.0, 2.4.3
>
>
> Seen after an integration test (see HBASE-25824) with 'calm' monkey, so this
> happened in the happy path.
> There were no errors accessing all loaded table data. The integration test
> writes a log to HDFS of every cell written to HBase and the verify phase uses
> that log to read each value and confirm it. That seems fine:
> {noformat}
> 2021-04-30 02:16:33,316 INFO [main]
> test.IntegrationTestLoadCommonCrawl$Verify: REFERENCED: 154943544
> 2021-04-30 02:16:33,316 INFO [main]
> test.IntegrationTestLoadCommonCrawl$Verify: UNREFERENCED: 0
> 2021-04-30 02:16:33,316 INFO [main]
> test.IntegrationTestLoadCommonCrawl$Verify: CORRUPT: 0
> {noformat}
> However whenever the balancer runs there are a number
[GitHub] [hbase] ndimiduk opened a new pull request #3225: HBASE-25842 move regionserver http-related code into o.a.h.h.regionserver.http
ndimiduk opened a new pull request #3225: URL: https://github.com/apache/hbase/pull/3225 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Created] (HBASE-25843) Move master http-related code into o.a.h.h.master.http
Nick Dimiduk created HBASE-25843: Summary: Move master http-related code into o.a.h.h.master.http Key: HBASE-25843 URL: https://issues.apache.org/jira/browse/HBASE-25843 Project: HBase Issue Type: Task Components: master Reporter: Nick Dimiduk Assignee: Nick Dimiduk Minor refactor. Group code related to the Master's Info Page into a common package. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (HBASE-25842) Move regionserver http-related code into o.a.h.h.regionserver.http
Nick Dimiduk created HBASE-25842: Summary: Move regionserver http-related code into o.a.h.h.regionserver.http Key: HBASE-25842 URL: https://issues.apache.org/jira/browse/HBASE-25842 Project: HBase Issue Type: Task Components: regionserver Reporter: Nick Dimiduk Assignee: Nick Dimiduk Minor refactor. Group code related to the Region Server's Info Page into a common package. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[GitHub] [hbase] Apache-HBase commented on pull request #3223: HBASE-25840 CatalogJanitor warns about skipping gc of regions during RIT, but does not actually skip
Apache-HBase commented on pull request #3223: URL: https://github.com/apache/hbase/pull/3223#issuecomment-831574280 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 1m 31s | Docker mode activated. | | -0 :warning: | yetus | 0m 3s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck | ||| _ Prechecks _ | ||| _ master Compile Tests _ | | +1 :green_heart: | mvninstall | 6m 0s | master passed | | +1 :green_heart: | compile | 1m 34s | master passed | | +1 :green_heart: | shadedjars | 11m 46s | branch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 0m 58s | master passed | ||| _ Patch Compile Tests _ | | +1 :green_heart: | mvninstall | 5m 42s | the patch passed | | +1 :green_heart: | compile | 1m 35s | the patch passed | | +1 :green_heart: | javac | 1m 35s | the patch passed | | +1 :green_heart: | shadedjars | 11m 31s | patch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 0m 52s | the patch passed | ||| _ Other Tests _ | | -1 :x: | unit | 151m 1s | hbase-server in the patch failed. | | | | 194m 57s | | | Subsystem | Report/Notes | |--:|:-| | Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3223/1/artifact/yetus-jdk11-hadoop3-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/3223 | | Optional Tests | javac javadoc unit shadedjars compile | | uname | Linux c56ebc384313 4.15.0-112-generic #113-Ubuntu SMP Thu Jul 9 23:41:39 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / 7640134e3e | | Default Java | AdoptOpenJDK-11.0.10+9 | | unit | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3223/1/artifact/yetus-jdk11-hadoop3-check/output/patch-unit-hbase-server.txt | | Test Results | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3223/1/testReport/ | | Max. process+thread count | 3859 (vs. ulimit of 3) | | modules | C: hbase-server U: hbase-server | | Console output | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3223/1/console | | versions | git=2.17.1 maven=3.6.3 | | Powered by | Apache Yetus 0.12.0 https://yetus.apache.org | This message was automatically generated. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #3204: HBASE-25741: Replication Source still having the replication metrics for peer ID which doesn't exist
Apache-HBase commented on pull request #3204: URL: https://github.com/apache/hbase/pull/3204#issuecomment-831570682 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 0m 0s | Docker mode activated. | | -1 :x: | docker | 5m 59s | Docker failed to build yetus/hbase:edccfe439a. | | Subsystem | Report/Notes | |--:|:-| | GITHUB PR | https://github.com/apache/hbase/pull/3204 | | JIRA Issue | HBASE-25741 | | Console output | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3204/7/console | | versions | git=2.17.1 | | Powered by | Apache Yetus 0.12.0 https://yetus.apache.org | This message was automatically generated. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #3204: HBASE-25741: Replication Source still having the replication metrics for peer ID which doesn't exist
Apache-HBase commented on pull request #3204: URL: https://github.com/apache/hbase/pull/3204#issuecomment-831564634 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 0m 38s | Docker mode activated. | ||| _ Prechecks _ | | +1 :green_heart: | dupname | 0m 0s | No case conflicting files found. | | +1 :green_heart: | hbaseanti | 0m 0s | Patch does not have any anti-patterns. | | +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. | | +1 :green_heart: | test4tests | 0m 0s | The patch appears to include 3 new or modified test files. | ||| _ branch-1 Compile Tests _ | | -1 :x: | mvninstall | 9m 59s | root in branch-1 failed. | | +1 :green_heart: | compile | 0m 43s | branch-1 passed with JDK Azul Systems, Inc.-1.8.0_262-b19 | | +1 :green_heart: | compile | 0m 45s | branch-1 passed with JDK Azul Systems, Inc.-1.7.0_272-b10 | | +1 :green_heart: | checkstyle | 1m 39s | branch-1 passed | | +1 :green_heart: | shadedjars | 3m 2s | branch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 0m 49s | branch-1 passed with JDK Azul Systems, Inc.-1.8.0_262-b19 | | +1 :green_heart: | javadoc | 0m 40s | branch-1 passed with JDK Azul Systems, Inc.-1.7.0_272-b10 | | +0 :ok: | spotbugs | 3m 6s | Used deprecated FindBugs config; considering switching to SpotBugs. | | +1 :green_heart: | findbugs | 3m 2s | branch-1 passed | ||| _ Patch Compile Tests _ | | -1 :x: | mvninstall | 1m 46s | root in the patch failed. | | +1 :green_heart: | compile | 0m 39s | the patch passed with JDK Azul Systems, Inc.-1.8.0_262-b19 | | +1 :green_heart: | javac | 0m 39s | the patch passed | | +1 :green_heart: | compile | 0m 46s | the patch passed with JDK Azul Systems, Inc.-1.7.0_272-b10 | | +1 :green_heart: | javac | 0m 46s | the patch passed | | +1 :green_heart: | checkstyle | 1m 32s | the patch passed | | +1 :green_heart: | whitespace | 0m 0s | The patch has no whitespace issues. | | +1 :green_heart: | shadedjars | 2m 50s | patch has no errors when building our shaded downstream artifacts. | | -1 :x: | hadoopcheck | 1m 35s | The patch causes 10 errors with Hadoop v2.8.5. | | -1 :x: | hadoopcheck | 3m 51s | The patch causes 10 errors with Hadoop v2.9.2. | | +1 :green_heart: | javadoc | 0m 31s | the patch passed with JDK Azul Systems, Inc.-1.8.0_262-b19 | | +1 :green_heart: | javadoc | 0m 38s | the patch passed with JDK Azul Systems, Inc.-1.7.0_272-b10 | | +1 :green_heart: | findbugs | 2m 55s | the patch passed | ||| _ Other Tests _ | | -1 :x: | unit | 133m 40s | hbase-server in the patch failed. | | +1 :green_heart: | asflicense | 0m 38s | The patch does not generate ASF License warnings. | | | | 172m 40s | | | Reason | Tests | |---:|:--| | Failed junit tests | hadoop.hbase.namequeues.TestSlowLogAccessor | | Subsystem | Report/Notes | |--:|:-| | Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3204/6/artifact/out/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/3204 | | JIRA Issue | HBASE-25741 | | Optional Tests | dupname asflicense javac javadoc unit spotbugs findbugs shadedjars hadoopcheck hbaseanti checkstyle compile | | uname | Linux af69721d7a20 4.15.0-60-generic #67-Ubuntu SMP Thu Aug 22 16:55:30 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | /home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-3204/out/precommit/personality/provided.sh | | git revision | branch-1 / 0fe1c9c | | Default Java | Azul Systems, Inc.-1.7.0_272-b10 | | Multi-JDK versions | /usr/lib/jvm/zulu-8-amd64:Azul Systems, Inc.-1.8.0_262-b19 /usr/lib/jvm/zulu-7-amd64:Azul Systems, Inc.-1.7.0_272-b10 | | mvninstall | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3204/6/artifact/out/branch-mvninstall-root.txt | | mvninstall | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3204/6/artifact/out/patch-mvninstall-root.txt | | hadoopcheck | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3204/6/artifact/out/patch-javac-2.8.5.txt | | hadoopcheck | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3204/6/artifact/out/patch-javac-2.9.2.txt | | unit | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3204/6/artifact/out/patch-unit-hbase-server.txt | | Test Results |
[GitHub] [hbase] Apache-HBase commented on pull request #3208: HBASE-25824 IntegrationTestLoadCommonCrawl
Apache-HBase commented on pull request #3208: URL: https://github.com/apache/hbase/pull/3208#issuecomment-831563623 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 1m 39s | Docker mode activated. | | -0 :warning: | yetus | 0m 2s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck | ||| _ Prechecks _ | ||| _ master Compile Tests _ | | +0 :ok: | mvndep | 0m 35s | Maven dependency ordering for branch | | +1 :green_heart: | mvninstall | 5m 35s | master passed | | +1 :green_heart: | compile | 3m 53s | master passed | | +1 :green_heart: | shadedjars | 10m 5s | branch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 4m 16s | master passed | ||| _ Patch Compile Tests _ | | +0 :ok: | mvndep | 0m 22s | Maven dependency ordering for patch | | +1 :green_heart: | mvninstall | 5m 46s | the patch passed | | +1 :green_heart: | compile | 3m 55s | the patch passed | | +1 :green_heart: | javac | 3m 55s | the patch passed | | +1 :green_heart: | shadedjars | 10m 51s | patch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 4m 19s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | unit | 264m 36s | root in the patch passed. | | | | 318m 19s | | | Subsystem | Report/Notes | |--:|:-| | Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3208/8/artifact/yetus-jdk11-hadoop3-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/3208 | | Optional Tests | javac javadoc unit shadedjars compile | | uname | Linux 051d6a2fa71d 4.15.0-136-generic #140-Ubuntu SMP Thu Jan 28 05:20:47 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / 7640134e3e | | Default Java | AdoptOpenJDK-11.0.10+9 | | Test Results | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3208/8/testReport/ | | Max. process+thread count | 4370 (vs. ulimit of 3) | | modules | C: hbase-it . U: . | | Console output | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3208/8/console | | versions | git=2.17.1 maven=3.6.3 | | Powered by | Apache Yetus 0.12.0 https://yetus.apache.org | This message was automatically generated. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] bharathv commented on a change in pull request #3204: HBASE-25741: Replication Source still having the replication metrics for peer ID which doesn't exist
bharathv commented on a change in pull request #3204:
URL: https://github.com/apache/hbase/pull/3204#discussion_r625390467
##
File path:
hbase-server/src/main/java/org/apache/hadoop/hbase/replication/regionserver/ReplicationSourceManager.java
##
@@ -653,6 +658,30 @@ public void regionServerRemoved(String regionserver) {
transferQueues(regionserver);
}
+ /**
+ * We want to run the peer removal in a separate thread when the peer removal
+ * is called from ReplicationSource shipper thread on encountering
NoNodeException
+ * because peerRemoved terminate the source which might leave replication
source
+ * in orphaned state.
+ * See HBASE-25741.
+ * @param peerId peer ID to be removed.
+ */
+ public void schedulePeerRemoval(final String peerId) {
+final ExecutorService threadPool =
+
Executors.newSingleThreadExecutor(Threads.getNamedThreadFactory("peer-removal"));
+try {
+ Future res = threadPool.submit(new Runnable() {
+@Override public void run() {
+ peerRemoved(peerId);
+}
+ });
+ res.get();
Review comment:
> replication shipper thread calls this method and wait till it complete
the execution.
The thread is killed during wait and this thread pool is never cleaned up.
My point is we don't need to wait on it. Instead submit this task to the
"executor" in this class, that executor is shutdown eventually.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [hbase] sandeepvinayak commented on a change in pull request #3204: HBASE-25741: Replication Source still having the replication metrics for peer ID which doesn't exist
sandeepvinayak commented on a change in pull request #3204:
URL: https://github.com/apache/hbase/pull/3204#discussion_r625388141
##
File path:
hbase-server/src/main/java/org/apache/hadoop/hbase/replication/regionserver/ReplicationSourceManager.java
##
@@ -653,6 +658,30 @@ public void regionServerRemoved(String regionserver) {
transferQueues(regionserver);
}
+ /**
+ * We want to run the peer removal in a separate thread when the peer removal
+ * is called from ReplicationSource shipper thread on encountering
NoNodeException
+ * because peerRemoved terminate the source which might leave replication
source
+ * in orphaned state.
+ * See HBASE-25741.
+ * @param peerId peer ID to be removed.
+ */
+ public void schedulePeerRemoval(final String peerId) {
+final ExecutorService threadPool =
+
Executors.newSingleThreadExecutor(Threads.getNamedThreadFactory("peer-removal"));
+try {
+ Future res = threadPool.submit(new Runnable() {
+@Override public void run() {
+ peerRemoved(peerId);
+}
+ });
+ res.get();
Review comment:
@bharathv We are creating a new executor in this method and replication
shipper thread calls this method and wait till it complete the execution. As
part of this execution the shipper thread will also be terminated and this new
thread make sure that it do the clean up after termination as well. In short,
this make sure that thread calling the `peerRemoved` is not the one getting
terminated within this method. Does it make sense?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #3218: HBASE-25835 Ignore duplicate split requests from regionserver reports
Apache-HBase commented on pull request #3218: URL: https://github.com/apache/hbase/pull/3218#issuecomment-831551042 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 4m 28s | Docker mode activated. | | -0 :warning: | yetus | 0m 3s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck | ||| _ Prechecks _ | ||| _ master Compile Tests _ | | +1 :green_heart: | mvninstall | 4m 24s | master passed | | +1 :green_heart: | compile | 1m 4s | master passed | | +1 :green_heart: | shadedjars | 8m 56s | branch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 0m 38s | master passed | ||| _ Patch Compile Tests _ | | +1 :green_heart: | mvninstall | 4m 4s | the patch passed | | +1 :green_heart: | compile | 1m 4s | the patch passed | | +1 :green_heart: | javac | 1m 4s | the patch passed | | +1 :green_heart: | shadedjars | 8m 58s | patch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 0m 36s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | unit | 219m 28s | hbase-server in the patch passed. | | | | 255m 37s | | | Subsystem | Report/Notes | |--:|:-| | Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3218/2/artifact/yetus-jdk8-hadoop3-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/3218 | | Optional Tests | javac javadoc unit shadedjars compile | | uname | Linux 6f7f6083b1f1 4.15.0-101-generic #102-Ubuntu SMP Mon May 11 10:07:26 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / 7640134e3e | | Default Java | AdoptOpenJDK-1.8.0_282-b08 | | Test Results | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3218/2/testReport/ | | Max. process+thread count | 2818 (vs. ulimit of 3) | | modules | C: hbase-server U: hbase-server | | Console output | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3218/2/console | | versions | git=2.17.1 maven=3.6.3 | | Powered by | Apache Yetus 0.12.0 https://yetus.apache.org | This message was automatically generated. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #3219: HBASE-25836 RegionStates#getAssignmentsForBalancer should only care about OPEN or OPENING regions
Apache-HBase commented on pull request #3219: URL: https://github.com/apache/hbase/pull/3219#issuecomment-831548044 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 2m 23s | Docker mode activated. | ||| _ Prechecks _ | | +1 :green_heart: | dupname | 0m 0s | No case conflicting files found. | | +1 :green_heart: | hbaseanti | 0m 0s | Patch does not have any anti-patterns. | | +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. | ||| _ master Compile Tests _ | | +1 :green_heart: | mvninstall | 3m 57s | master passed | | +1 :green_heart: | compile | 3m 12s | master passed | | +1 :green_heart: | checkstyle | 1m 4s | master passed | | +1 :green_heart: | spotbugs | 2m 3s | master passed | ||| _ Patch Compile Tests _ | | +1 :green_heart: | mvninstall | 3m 34s | the patch passed | | +1 :green_heart: | compile | 3m 11s | the patch passed | | +1 :green_heart: | javac | 3m 11s | the patch passed | | +1 :green_heart: | checkstyle | 1m 2s | the patch passed | | +1 :green_heart: | whitespace | 0m 0s | The patch has no whitespace issues. | | +1 :green_heart: | hadoopcheck | 17m 57s | Patch does not cause any errors with Hadoop 3.1.2 3.2.1 3.3.0. | | +1 :green_heart: | spotbugs | 2m 14s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | asflicense | 0m 15s | The patch does not generate ASF License warnings. | | | | 48m 30s | | | Subsystem | Report/Notes | |--:|:-| | Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3219/3/artifact/yetus-general-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/3219 | | Optional Tests | dupname asflicense javac spotbugs hadoopcheck hbaseanti checkstyle compile | | uname | Linux 536024d0fbcd 4.15.0-58-generic #64-Ubuntu SMP Tue Aug 6 11:12:41 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / 7640134e3e | | Default Java | AdoptOpenJDK-1.8.0_282-b08 | | Max. process+thread count | 96 (vs. ulimit of 3) | | modules | C: hbase-server U: hbase-server | | Console output | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3219/3/console | | versions | git=2.17.1 maven=3.6.3 spotbugs=4.2.2 | | Powered by | Apache Yetus 0.12.0 https://yetus.apache.org | This message was automatically generated. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Commented] (HBASE-25833) HBase Configuration File Descriptions has some description errors or inaccurate
[ https://issues.apache.org/jira/browse/HBASE-25833?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17338609#comment-17338609 ] Nick Dimiduk commented on HBASE-25833: -- Thank you for the contribution, [~impself]! > HBase Configuration File Descriptions has some description errors or > inaccurate > --- > > Key: HBASE-25833 > URL: https://issues.apache.org/jira/browse/HBASE-25833 > Project: HBase > Issue Type: Bug > Components: documentation >Affects Versions: 3.0.0-alpha-1 >Reporter: Che Xun >Assignee: Che Xun >Priority: Minor > Fix For: 3.0.0-alpha-1 > > > In Apache HBase ™ Reference Guide > *Need to modify part1：* > _3 Configuration Files->HBase Configuration File > Descriptions->log4j.properties_ > *explanation：* > HBase 3.0.0 log frame is now log4j2，so need to update dectiption like > follows: > log4j2.xml > Configuration file for HBase logging via log4j2. > *Need to modify part2：* > _3 Configuration Files->HBase Configuration File > Descriptions->hbase-site.xml_ > You can view (but do not edit) the default configuration file at > docs/hbase-default.xml. > *explanation：* > maybe we need to update hbase-default.xml filePath to > hbase-common/src/main/resources/hbase-default.xml -- This message was sent by Atlassian Jira (v8.3.4#803005)
[GitHub] [hbase] ndimiduk commented on a change in pull request #3139: HBASE-25745 Deprecate/Rename config `hbase.normalizer.min.region.coun…
ndimiduk commented on a change in pull request #3139:
URL: https://github.com/apache/hbase/pull/3139#discussion_r625367579
##
File path:
hbase-server/src/main/java/org/apache/hadoop/hbase/master/normalizer/SimpleRegionNormalizer.java
##
@@ -101,11 +106,21 @@ public void onConfigurationChange(Configuration conf) {
setConf(conf);
}
- private static int parseMinRegionCount(final Configuration conf) {
-final int parsedValue = conf.getInt(MIN_REGION_COUNT_KEY,
DEFAULT_MIN_REGION_COUNT);
+ private static int parseMergeMinRegionCount(final Configuration conf) {
+String parsedStringValue = conf.get(MERGE_MIN_REGION_COUNT_KEY);
Review comment:
I think we need to keep the old config in hbase-defaults.xml and add the
new config there as well.
```
modified hbase-common/src/main/resources/hbase-default.xml
@@ -41,6 +41,8 @@ possible configurations would overwhelm and obscure the
important.
+ foo42deprecated; see
'bar'
+ bar42
hbase.tmp.dir
${java.io.tmpdir}/hbase-${user.name}
---
| Welcome to JShell -- Version 11.0.10
| For an introduction type: /help intro
jshell> import org.apache.hadoop.conf.Configuration
jshell> import org.apache.hadoop.hbase.HBaseConfiguration
jshell> Configuration c = HBaseConfiguration.create()
c ==> Configuration: core-default.xml, core-site.xml, h ... efault.xml,
hbase-site.xml
jshell> c.get("foo")
$4 ==> "42"
jshell> c.get("bar")
$5 ==> "42"
jshell> Configuration.addDeprecation("foo", "bar")
jshell> c.get("foo")
2021-05-03T14:03:00,038 INFO [main] Configuration.deprecation: foo is
deprecated. Instead, use bar
$7 ==> "42"
jshell> c.get("bar")
$8 ==> "42"
jshell> c.set("foo", "99")
jshell> c.get("foo")
$10 ==> "99"
jshell> c.get("bar")
$11 ==> "99"
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [hbase] bharathv commented on a change in pull request #3204: HBASE-25741: Replication Source still having the replication metrics for peer ID which doesn't exist
bharathv commented on a change in pull request #3204:
URL: https://github.com/apache/hbase/pull/3204#discussion_r625358260
##
File path:
hbase-server/src/main/java/org/apache/hadoop/hbase/replication/regionserver/ReplicationSourceManager.java
##
@@ -653,6 +658,30 @@ public void regionServerRemoved(String regionserver) {
transferQueues(regionserver);
}
+ /**
+ * We want to run the peer removal in a separate thread when the peer removal
+ * is called from ReplicationSource shipper thread on encountering
NoNodeException
+ * because peerRemoved terminate the source which might leave replication
source
+ * in orphaned state.
+ * See HBASE-25741.
+ * @param peerId peer ID to be removed.
+ */
+ public void schedulePeerRemoval(final String peerId) {
+final ExecutorService threadPool =
+
Executors.newSingleThreadExecutor(Threads.getNamedThreadFactory("peer-removal"));
+try {
+ Future res = threadPool.submit(new Runnable() {
+@Override public void run() {
+ peerRemoved(peerId);
+}
+ });
+ res.get();
Review comment:
wait.. I don't get this.. the blocking thread here is the shipper
thread, it should be killed before get() returns? How is this any different
from the current state of code.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #3219: HBASE-25836 RegionStates#getAssignmentsForBalancer should only care about OPEN or OPENING regions
Apache-HBase commented on pull request #3219: URL: https://github.com/apache/hbase/pull/3219#issuecomment-831519452 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 0m 39s | Docker mode activated. | | -0 :warning: | yetus | 0m 3s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck | ||| _ Prechecks _ | ||| _ master Compile Tests _ | | +1 :green_heart: | mvninstall | 3m 47s | master passed | | +1 :green_heart: | compile | 1m 4s | master passed | | +1 :green_heart: | shadedjars | 8m 7s | branch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 0m 37s | master passed | ||| _ Patch Compile Tests _ | | +1 :green_heart: | mvninstall | 3m 36s | the patch passed | | +1 :green_heart: | compile | 1m 1s | the patch passed | | +1 :green_heart: | javac | 1m 1s | the patch passed | | +1 :green_heart: | shadedjars | 8m 12s | patch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 0m 37s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | unit | 149m 34s | hbase-server in the patch passed. | | | | 179m 41s | | | Subsystem | Report/Notes | |--:|:-| | Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3219/2/artifact/yetus-jdk8-hadoop3-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/3219 | | Optional Tests | javac javadoc unit shadedjars compile | | uname | Linux 06d8919e373f 4.15.0-112-generic #113-Ubuntu SMP Thu Jul 9 23:41:39 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / 7640134e3e | | Default Java | AdoptOpenJDK-1.8.0_282-b08 | | Test Results | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3219/2/testReport/ | | Max. process+thread count | 4783 (vs. ulimit of 3) | | modules | C: hbase-server U: hbase-server | | Console output | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3219/2/console | | versions | git=2.17.1 maven=3.6.3 | | Powered by | Apache Yetus 0.12.0 https://yetus.apache.org | This message was automatically generated. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #3219: HBASE-25836 RegionStates#getAssignmentsForBalancer should only care about OPEN or OPENING regions
Apache-HBase commented on pull request #3219: URL: https://github.com/apache/hbase/pull/3219#issuecomment-831518568 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 0m 26s | Docker mode activated. | | -0 :warning: | yetus | 0m 3s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck | ||| _ Prechecks _ | ||| _ master Compile Tests _ | | +1 :green_heart: | mvninstall | 4m 10s | master passed | | +1 :green_heart: | compile | 1m 11s | master passed | | +1 :green_heart: | shadedjars | 8m 13s | branch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 0m 42s | master passed | ||| _ Patch Compile Tests _ | | +1 :green_heart: | mvninstall | 4m 24s | the patch passed | | +1 :green_heart: | compile | 1m 13s | the patch passed | | +1 :green_heart: | javac | 1m 13s | the patch passed | | +1 :green_heart: | shadedjars | 8m 37s | patch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 0m 41s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | unit | 146m 29s | hbase-server in the patch passed. | | | | 178m 11s | | | Subsystem | Report/Notes | |--:|:-| | Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3219/2/artifact/yetus-jdk11-hadoop3-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/3219 | | Optional Tests | javac javadoc unit shadedjars compile | | uname | Linux 36de140616e5 4.15.0-58-generic #64-Ubuntu SMP Tue Aug 6 11:12:41 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / 7640134e3e | | Default Java | AdoptOpenJDK-11.0.10+9 | | Test Results | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3219/2/testReport/ | | Max. process+thread count | 4004 (vs. ulimit of 3) | | modules | C: hbase-server U: hbase-server | | Console output | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3219/2/console | | versions | git=2.17.1 maven=3.6.3 | | Powered by | Apache Yetus 0.12.0 https://yetus.apache.org | This message was automatically generated. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #3218: HBASE-25835 Ignore duplicate split requests from regionserver reports
Apache-HBase commented on pull request #3218: URL: https://github.com/apache/hbase/pull/3218#issuecomment-831510827 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 1m 5s | Docker mode activated. | | -0 :warning: | yetus | 0m 3s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck | ||| _ Prechecks _ | ||| _ master Compile Tests _ | | +1 :green_heart: | mvninstall | 4m 27s | master passed | | +1 :green_heart: | compile | 1m 11s | master passed | | +1 :green_heart: | shadedjars | 8m 7s | branch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 0m 42s | master passed | ||| _ Patch Compile Tests _ | | +1 :green_heart: | mvninstall | 4m 17s | the patch passed | | +1 :green_heart: | compile | 1m 10s | the patch passed | | +1 :green_heart: | javac | 1m 10s | the patch passed | | +1 :green_heart: | shadedjars | 8m 9s | patch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 0m 41s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | unit | 150m 53s | hbase-server in the patch passed. | | | | 182m 48s | | | Subsystem | Report/Notes | |--:|:-| | Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3218/2/artifact/yetus-jdk11-hadoop3-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/3218 | | Optional Tests | javac javadoc unit shadedjars compile | | uname | Linux ea2b47f7811d 4.15.0-112-generic #113-Ubuntu SMP Thu Jul 9 23:41:39 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / 7640134e3e | | Default Java | AdoptOpenJDK-11.0.10+9 | | Test Results | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3218/2/testReport/ | | Max. process+thread count | 3682 (vs. ulimit of 3) | | modules | C: hbase-server U: hbase-server | | Console output | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3218/2/console | | versions | git=2.17.1 maven=3.6.3 | | Powered by | Apache Yetus 0.12.0 https://yetus.apache.org | This message was automatically generated. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #3223: HBASE-25840 CatalogJanitor warns about skipping gc of regions during RIT, but does not actually skip
Apache-HBase commented on pull request #3223: URL: https://github.com/apache/hbase/pull/3223#issuecomment-831505389 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 0m 35s | Docker mode activated. | ||| _ Prechecks _ | | +1 :green_heart: | dupname | 0m 0s | No case conflicting files found. | | +1 :green_heart: | hbaseanti | 0m 0s | Patch does not have any anti-patterns. | | +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. | ||| _ master Compile Tests _ | | +1 :green_heart: | mvninstall | 5m 11s | master passed | | +1 :green_heart: | compile | 4m 6s | master passed | | +1 :green_heart: | checkstyle | 1m 19s | master passed | | +1 :green_heart: | spotbugs | 2m 39s | master passed | ||| _ Patch Compile Tests _ | | +1 :green_heart: | mvninstall | 4m 49s | the patch passed | | +1 :green_heart: | compile | 3m 59s | the patch passed | | +1 :green_heart: | javac | 3m 59s | the patch passed | | +1 :green_heart: | checkstyle | 1m 18s | the patch passed | | +1 :green_heart: | whitespace | 0m 0s | The patch has no whitespace issues. | | +1 :green_heart: | hadoopcheck | 24m 14s | Patch does not cause any errors with Hadoop 3.1.2 3.2.1 3.3.0. | | +1 :green_heart: | spotbugs | 3m 6s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | asflicense | 0m 15s | The patch does not generate ASF License warnings. | | | | 61m 49s | | | Subsystem | Report/Notes | |--:|:-| | Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3223/1/artifact/yetus-general-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/3223 | | Optional Tests | dupname asflicense javac spotbugs hadoopcheck hbaseanti checkstyle compile | | uname | Linux 56d132028881 4.15.0-112-generic #113-Ubuntu SMP Thu Jul 9 23:41:39 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / 7640134e3e | | Default Java | AdoptOpenJDK-1.8.0_282-b08 | | Max. process+thread count | 96 (vs. ulimit of 3) | | modules | C: hbase-server U: hbase-server | | Console output | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3223/1/console | | versions | git=2.17.1 maven=3.6.3 spotbugs=4.2.2 | | Powered by | Apache Yetus 0.12.0 https://yetus.apache.org | This message was automatically generated. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase-operator-tools] Kota-SH commented on pull request #85: HBASE-25794: Fix checkstyle violations in hbase-table-reporter module
Kota-SH commented on pull request #85: URL: https://github.com/apache/hbase-operator-tools/pull/85#issuecomment-831491537 Hi @petersomogyi, @joshelser Could you please review this PR and provide feedback. Thanks, Shanmukha K -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Updated] (HBASE-25794) Fix checkstyle violations in hbase-table-reporter module
[
https://issues.apache.org/jira/browse/HBASE-25794?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Shanmukha Haripriya Kota updated HBASE-25794:
-
Status: Patch Available (was: Open)
> Fix checkstyle violations in hbase-table-reporter module
>
>
> Key: HBASE-25794
> URL: https://issues.apache.org/jira/browse/HBASE-25794
> Project: HBase
> Issue Type: Task
> Components: hbase-operator-tools
>Reporter: Peter Somogyi
>Assignee: Shanmukha Haripriya Kota
>Priority: Trivial
> Labels: beginner
>
> The hbase-table-reporter module in hbase-operator-tools has a bunch of
> checkstyle violations.
> {noformat}
> [ERROR] src/main/java/org/apache/hbase/reporter/TableReporter.java:[22]
> (imports) ImportOrder: Wrong order for 'org.apache.commons.cli.Option' import.
> [ERROR] src/main/java/org/apache/hbase/reporter/TableReporter.java:[23]
> (imports) ImportOrder: Wrong order for
> 'org.apache.commons.cli.CommandLineParser' import.
> [ERROR] src/main/java/org/apache/hbase/reporter/TableReporter.java:[24]
> (imports) ImportOrder: Wrong order for 'org.apache.commons.cli.CommandLine'
> import.
> [ERROR] src/main/java/org/apache/hbase/reporter/TableReporter.java:[34,8]
> (imports) UnusedImports: Unused import - org.apache.hadoop.hbase.HRegionInfo.
> [ERROR] src/main/java/org/apache/hbase/reporter/TableReporter.java:[37]
> (imports) AvoidStarImport: Using the '.*' form of import should be avoided -
> org.apache.hadoop.hbase.client.*.
> [ERROR] src/main/java/org/apache/hbase/reporter/TableReporter.java:[41]
> (imports) ImportOrder: Wrong order for 'java.io.BufferedWriter' import.
> [ERROR] src/main/java/org/apache/hbase/reporter/TableReporter.java:[66,1]
> (design) HideUtilityClassConstructor: Utility classes should not have a
> public or default constructor.
> [ERROR] src/main/java/org/apache/hbase/reporter/TableReporter.java:[94]
> (sizes) LineLength: Line is longer than 100 characters (found 120).
> [ERROR] src/main/java/org/apache/hbase/reporter/TableReporter.java:[126]
> (sizes) LineLength: Line is longer than 100 characters (found 105).
> [ERROR] src/main/java/org/apache/hbase/reporter/TableReporter.java:[132]
> (sizes) LineLength: Line is longer than 100 characters (found 106).
> [ERROR] src/main/java/org/apache/hbase/reporter/TableReporter.java:[247]
> (indentation) Indentation: 'throws' has incorrect indentation level 2,
> expected level should be 4.
> [ERROR] src/main/java/org/apache/hbase/reporter/TableReporter.java:[275]
> (sizes) LineLength: Line is longer than 100 characters (found 112).
> [ERROR] src/main/java/org/apache/hbase/reporter/TableReporter.java:[279]
> (sizes) LineLength: Line is longer than 100 characters (found 105).
> [ERROR] src/main/java/org/apache/hbase/reporter/TableReporter.java:[308]
> (sizes) LineLength: Line is longer than 100 characters (found 101).
> [ERROR] src/main/java/org/apache/hbase/reporter/TableReporter.java:[370]
> (sizes) LineLength: Line is longer than 100 characters (found 121).
> [ERROR] src/main/java/org/apache/hbase/reporter/TableReporter.java:[371]
> (sizes) LineLength: Line is longer than 100 characters (found 108).
> [ERROR] src/main/java/org/apache/hbase/reporter/TableReporter.java:[375]
> (sizes) LineLength: Line is longer than 100 characters (found 102).
> [ERROR] src/main/java/org/apache/hbase/reporter/TableReporter.java:[401]
> (sizes) LineLength: Line is longer than 100 characters (found 113).
> [ERROR] src/main/java/org/apache/hbase/reporter/TableReporter.java:[407]
> (sizes) LineLength: Line is longer than 100 characters (found 112).
> [ERROR] src/main/java/org/apache/hbase/reporter/TableReporter.java:[410]
> (sizes) LineLength: Line is longer than 100 characters (found 107).
> [ERROR] src/main/java/org/apache/hbase/reporter/TableReporter.java:[437]
> (sizes) LineLength: Line is longer than 100 characters (found 114).
> [ERROR] src/main/java/org/apache/hbase/reporter/TableReporter.java:[438]
> (sizes) LineLength: Line is longer than 100 characters (found 109).
> [ERROR] src/main/java/org/apache/hbase/reporter/TableReporter.java:[445]
> (indentation) Indentation: 'throws' has incorrect indentation level 2,
> expected level should be 4.
> [ERROR] src/test/java/org/apache/hbase/reporter/TestTableReporter.java:[27]
> (imports) ImportOrder: Wrong order for 'java.util.ArrayList' import.
> [ERROR] src/test/java/org/apache/hbase/reporter/TestTableReporter.java:[30]
> (imports) ImportOrder: Wrong order for
> 'org.apache.hadoop.hbase.shaded.junit.framework.TestCase.assertEquals' import.
> [ERROR] src/test/java/org/apache/hbase/reporter/TestTableReporter.java:[59]
> (sizes) LineLength: Line is longer than 100 characters (found 110).
> [ERROR]
[GitHub] [hbase] ndimiduk commented on pull request #3224: HBASE-25841 Add basic jshell support
ndimiduk commented on pull request #3224: URL: https://github.com/apache/hbase/pull/3224#issuecomment-831483181 ``` $ JAVA_HOME=/Library/Java/JavaVirtualMachines/adoptopenjdk-8.jdk/Contents/Home ./bin/hbase jshell JShell is available only with JDK9 and lated. Detected JDK version is 1.8.0_282-b08. ``` ``` $ JAVA_HOME=/Library/Java/JavaVirtualMachines/adoptopenjdk-11.jdk/Contents/Home ./bin/hbase jshell | Welcome to JShell -- Version 11.0.10 | For an introduction type: /help intro jshell> import org.apache.hadoop.hbase.HConstants jshell> HConstants.MASTER_PORT $2 ==> "hbase.master.port" jshell> ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #3224: HBASE-25841 Add basic jshell support
Apache-HBase commented on pull request #3224: URL: https://github.com/apache/hbase/pull/3224#issuecomment-831483051 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 1m 9s | Docker mode activated. | ||| _ Prechecks _ | | +1 :green_heart: | dupname | 0m 0s | No case conflicting files found. | | +0 :ok: | shelldocs | 0m 0s | Shelldocs was not available. | | +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. | ||| _ master Compile Tests _ | ||| _ Patch Compile Tests _ | | +1 :green_heart: | shellcheck | 0m 2s | There were no new shellcheck issues. | | +1 :green_heart: | whitespace | 0m 0s | The patch has no whitespace issues. | ||| _ Other Tests _ | | +1 :green_heart: | asflicense | 0m 15s | The patch does not generate ASF License warnings. | | | | 2m 46s | | | Subsystem | Report/Notes | |--:|:-| | Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3224/1/artifact/yetus-general-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/3224 | | Optional Tests | dupname asflicense shellcheck shelldocs | | uname | Linux 5787436c3214 4.15.0-128-generic #131-Ubuntu SMP Wed Dec 9 06:57:35 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / 7640134e3e | | Max. process+thread count | 43 (vs. ulimit of 3) | | modules | C: . U: . | | Console output | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3224/1/console | | versions | git=2.17.1 maven=3.6.3 shellcheck=0.4.6 | | Powered by | Apache Yetus 0.12.0 https://yetus.apache.org | This message was automatically generated. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #3224: HBASE-25841 Add basic jshell support
Apache-HBase commented on pull request #3224: URL: https://github.com/apache/hbase/pull/3224#issuecomment-831482926 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 1m 20s | Docker mode activated. | | -0 :warning: | yetus | 0m 2s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck | ||| _ Prechecks _ | ||| _ master Compile Tests _ | ||| _ Patch Compile Tests _ | ||| _ Other Tests _ | | | | 2m 31s | | | Subsystem | Report/Notes | |--:|:-| | Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3224/1/artifact/yetus-jdk11-hadoop3-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/3224 | | Optional Tests | | | uname | Linux 5bbd000f9840 4.15.0-136-generic #140-Ubuntu SMP Thu Jan 28 05:20:47 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / 7640134e3e | | Max. process+thread count | 49 (vs. ulimit of 3) | | modules | C: . U: . | | Console output | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3224/1/console | | versions | git=2.17.1 maven=3.6.3 | | Powered by | Apache Yetus 0.12.0 https://yetus.apache.org | This message was automatically generated. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #3224: HBASE-25841 Add basic jshell support
Apache-HBase commented on pull request #3224: URL: https://github.com/apache/hbase/pull/3224#issuecomment-831482480 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 0m 29s | Docker mode activated. | | -0 :warning: | yetus | 0m 3s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck | ||| _ Prechecks _ | ||| _ master Compile Tests _ | ||| _ Patch Compile Tests _ | ||| _ Other Tests _ | | | | 1m 41s | | | Subsystem | Report/Notes | |--:|:-| | Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3224/1/artifact/yetus-jdk8-hadoop3-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/3224 | | Optional Tests | | | uname | Linux 1ec05ea1cad7 4.15.0-136-generic #140-Ubuntu SMP Thu Jan 28 05:20:47 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / 7640134e3e | | Max. process+thread count | 47 (vs. ulimit of 3) | | modules | C: . U: . | | Console output | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3224/1/console | | versions | git=2.17.1 maven=3.6.3 | | Powered by | Apache Yetus 0.12.0 https://yetus.apache.org | This message was automatically generated. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] ndimiduk commented on pull request #3224: HBASE-25841 Add basic jshell support
ndimiduk commented on pull request #3224: URL: https://github.com/apache/hbase/pull/3224#issuecomment-831481842 Doesn't _do_ anything other than adding a `jshell` command to the shell script entry point. Perhaps we also want to provide a [startup script](https://docs.oracle.com/javase/9/jshell/commands.htm#JSHEL-GUID-F4881B8E-8B6C-4906-A100-E78C4AFC277D) that will automatically import some set of packages/classes? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] ndimiduk opened a new pull request #3224: HBASE-25841 Add basic jshell support
ndimiduk opened a new pull request #3224: URL: https://github.com/apache/hbase/pull/3224 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Commented] (HBASE-25841) Add basic jshell support
[
https://issues.apache.org/jira/browse/HBASE-25841?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17338578#comment-17338578
]
Nick Dimiduk commented on HBASE-25841:
--
Setting "Affects Version" for all HBase releases that support JDK9 or later.
> Add basic jshell support
>
>
> Key: HBASE-25841
> URL: https://issues.apache.org/jira/browse/HBASE-25841
> Project: HBase
> Issue Type: New Feature
> Components: shell, Usability
>Affects Versions: 3.0.0-alpha-1, 2.3.0, 2.4.0, 2.5.0
>Reporter: Nick Dimiduk
>Assignee: Nick Dimiduk
>Priority: Minor
>
> Let's make it easy to start a {{jshell}} session that includes HBase jars and
> dependencies on the class path.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Work started] (HBASE-25841) Add basic jshell support
[
https://issues.apache.org/jira/browse/HBASE-25841?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Work on HBASE-25841 started by Nick Dimiduk.
> Add basic jshell support
>
>
> Key: HBASE-25841
> URL: https://issues.apache.org/jira/browse/HBASE-25841
> Project: HBase
> Issue Type: New Feature
> Components: shell, Usability
>Reporter: Nick Dimiduk
>Assignee: Nick Dimiduk
>Priority: Minor
>
> Let's make it easy to start a {{jshell}} session that includes HBase jars and
> dependencies on the class path.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Created] (HBASE-25841) Add basic jshell support
Nick Dimiduk created HBASE-25841:
Summary: Add basic jshell support
Key: HBASE-25841
URL: https://issues.apache.org/jira/browse/HBASE-25841
Project: HBase
Issue Type: New Feature
Components: shell, Usability
Reporter: Nick Dimiduk
Assignee: Nick Dimiduk
Let's make it easy to start a {{jshell}} session that includes HBase jars and
dependencies on the class path.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (HBASE-25841) Add basic jshell support
[
https://issues.apache.org/jira/browse/HBASE-25841?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Nick Dimiduk updated HBASE-25841:
-
Affects Version/s: 2.5.0
3.0.0-alpha-1
2.3.0
2.4.0
> Add basic jshell support
>
>
> Key: HBASE-25841
> URL: https://issues.apache.org/jira/browse/HBASE-25841
> Project: HBase
> Issue Type: New Feature
> Components: shell, Usability
>Affects Versions: 3.0.0-alpha-1, 2.3.0, 2.4.0, 2.5.0
>Reporter: Nick Dimiduk
>Assignee: Nick Dimiduk
>Priority: Minor
>
> Let's make it easy to start a {{jshell}} session that includes HBase jars and
> dependencies on the class path.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[GitHub] [hbase-operator-tools] Apache-HBase commented on pull request #85: HBASE-25794: Fix checkstyle violations in hbase-table-reporter module
Apache-HBase commented on pull request #85: URL: https://github.com/apache/hbase-operator-tools/pull/85#issuecomment-831473339 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 1m 20s | Docker mode activated. | ||| _ Prechecks _ | | +1 :green_heart: | dupname | 0m 0s | No case conflicting files found. | | +0 :ok: | spotbugs | 0m 1s | spotbugs executables are not available. | | +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. | | +1 :green_heart: | test4tests | 0m 0s | The patch appears to include 1 new or modified test files. | ||| _ master Compile Tests _ | | +1 :green_heart: | mvninstall | 0m 53s | master passed | | +1 :green_heart: | compile | 0m 5s | master passed | | +1 :green_heart: | checkstyle | 0m 6s | master passed | | +1 :green_heart: | javadoc | 0m 6s | master passed | ||| _ Patch Compile Tests _ | | +1 :green_heart: | mvninstall | 0m 10s | the patch passed | | +1 :green_heart: | compile | 0m 5s | the patch passed | | +1 :green_heart: | javac | 0m 5s | the patch passed | | +1 :green_heart: | checkstyle | 0m 3s | hbase-table-reporter: The patch generated 0 new + 0 unchanged - 28 fixed = 0 total (was 28) | | +1 :green_heart: | whitespace | 0m 0s | The patch has no whitespace issues. | | +1 :green_heart: | javadoc | 0m 4s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | unit | 0m 5s | hbase-table-reporter in the patch passed. | | +1 :green_heart: | asflicense | 0m 6s | The patch does not generate ASF License warnings. | | | | 3m 12s | | | Subsystem | Report/Notes | |--:|:-| | Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-Operator-Tools-PreCommit/job/PR-85/1/artifact/yetus-precommit-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase-operator-tools/pull/85 | | Optional Tests | dupname asflicense javac javadoc unit spotbugs findbugs checkstyle compile | | uname | Linux 21321cebf4f2 5.4.0-1025-aws #25~18.04.1-Ubuntu SMP Fri Sep 11 12:03:04 UTC 2020 x86_64 GNU/Linux | | Build tool | maven | | git revision | master / 55bca70 | | Default Java | Oracle Corporation-1.8.0_282-b08 | | Test Results | https://ci-hadoop.apache.org/job/HBase/job/HBase-Operator-Tools-PreCommit/job/PR-85/1/testReport/ | | Max. process+thread count | 37 (vs. ulimit of 5000) | | modules | C: hbase-table-reporter U: hbase-table-reporter | | Console output | https://ci-hadoop.apache.org/job/HBase/job/HBase-Operator-Tools-PreCommit/job/PR-85/1/console | | versions | git=2.20.1 | | Powered by | Apache Yetus 0.12.0 https://yetus.apache.org | This message was automatically generated. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] sandeepvinayak commented on a change in pull request #3204: HBASE-25741: Replication Source still having the replication metrics for peer ID which doesn't exist
sandeepvinayak commented on a change in pull request #3204:
URL: https://github.com/apache/hbase/pull/3204#discussion_r625306198
##
File path:
hbase-server/src/main/java/org/apache/hadoop/hbase/replication/regionserver/ReplicationSourceManager.java
##
@@ -641,8 +641,8 @@ public void removePeer(String id) {
"and that replication on this peer may not be caught up. peerId="
+ id);
}
for (ReplicationSourceInterface toRemove : srcToRemove) {
-toRemove.terminate(terminateMessage);
closeQueue(toRemove);
+toRemove.terminate(terminateMessage);
Review comment:
@bharathv can you review it again? I modified the patch to call
peerRemoved from a separate thread instead if it comes from a replication
source.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[jira] [Updated] (HBASE-25840) CatalogJanitor warns about skipping gc of regions during RIT, but does not actually skip
[
https://issues.apache.org/jira/browse/HBASE-25840?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Andrew Kyle Purtell updated HBASE-25840:
Status: Patch Available (was: Open)
> CatalogJanitor warns about skipping gc of regions during RIT, but does not
> actually skip
>
>
> Key: HBASE-25840
> URL: https://issues.apache.org/jira/browse/HBASE-25840
> Project: HBase
> Issue Type: Sub-task
>Affects Versions: 2.4.3
>Reporter: Andrew Kyle Purtell
>Assignee: Andrew Kyle Purtell
>Priority: Minor
> Fix For: 3.0.0-alpha-1, 2.5.0, 2.4.3
>
>
> Found while looking for something else.
> We claim in a WARN level log line to {{"Playing-it-safe skipping merge/split
> gc'ing of regions from hbase:meta while regions-in-transition (RIT)"}} but do
> not actually skip because of a missing return.
> Playing it safe has proven unnecessary, I would say. Nonetheless here is a
> patch that adds the missing return, for discussion.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[GitHub] [hbase-operator-tools] Kota-SH opened a new pull request #85: HBASE-25794: Fix checkstyle violations in hbase-table-reporter module
Kota-SH opened a new pull request #85: URL: https://github.com/apache/hbase-operator-tools/pull/85 This PR is regarding [https://issues.apache.org/jira/browse/HBASE-25794](url) Fixed checkstyle violations in the hbase-table-reporter module. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] apurtell opened a new pull request #3223: HBASE-25840 CatalogJanitor warns about skipping gc of regions during RIT, but does not actually skip
apurtell opened a new pull request #3223: URL: https://github.com/apache/hbase/pull/3223 We claim in a WARN level log line to be "Playing-it-safe skipping merge/ split gc'ing of regions from hbase:meta while regions-in-transition (RIT)" but do not actually skip because of a missing return. Playing it safe has proven unnecessary, I would say. Nonetheless here is a patch that adds the missing return, for discussion. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Created] (HBASE-25840) CatalogJanitor warns about skipping gc of regions during RIT, but does not actually skip
Andrew Kyle Purtell created HBASE-25840:
---
Summary: CatalogJanitor warns about skipping gc of regions during
RIT, but does not actually skip
Key: HBASE-25840
URL: https://issues.apache.org/jira/browse/HBASE-25840
Project: HBase
Issue Type: Bug
Affects Versions: 2.4.3
Reporter: Andrew Kyle Purtell
Assignee: Andrew Kyle Purtell
Fix For: 3.0.0-alpha-1, 2.5.0, 2.4.3
Found while looking for something else.
We claim in a WARN level log line to {{"Playing-it-safe skipping merge/split
gc'ing of regions from hbase:meta while regions-in-transition (RIT)"}} but do
not actually skip because of a missing return.
Playing it safe has proven unnecessary, I would say. Nonetheless here is a
patch that adds the missing return, for discussion.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (HBASE-25840) CatalogJanitor warns about skipping gc of regions during RIT, but does not actually skip
[
https://issues.apache.org/jira/browse/HBASE-25840?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Andrew Kyle Purtell updated HBASE-25840:
Parent: HBASE-25829
Issue Type: Sub-task (was: Bug)
> CatalogJanitor warns about skipping gc of regions during RIT, but does not
> actually skip
>
>
> Key: HBASE-25840
> URL: https://issues.apache.org/jira/browse/HBASE-25840
> Project: HBase
> Issue Type: Sub-task
>Affects Versions: 2.4.3
>Reporter: Andrew Kyle Purtell
>Assignee: Andrew Kyle Purtell
>Priority: Minor
> Fix For: 3.0.0-alpha-1, 2.5.0, 2.4.3
>
>
> Found while looking for something else.
> We claim in a WARN level log line to {{"Playing-it-safe skipping merge/split
> gc'ing of regions from hbase:meta while regions-in-transition (RIT)"}} but do
> not actually skip because of a missing return.
> Playing it safe has proven unnecessary, I would say. Nonetheless here is a
> patch that adds the missing return, for discussion.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HBASE-25612) HMaster should abort if ReplicationLogCleaner is not able to delete oldWALs.
[
https://issues.apache.org/jira/browse/HBASE-25612?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17338555#comment-17338555
]
Rushabh Shah commented on HBASE-25612:
--
[~reidchan] Is it possible to get this in 1.7 release ? We see this error
every few days in our production cluster still running 1.3.2 and we have to
manually failover hmaster.
Cc [~apurtell]
> HMaster should abort if ReplicationLogCleaner is not able to delete oldWALs.
>
>
> Key: HBASE-25612
> URL: https://issues.apache.org/jira/browse/HBASE-25612
> Project: HBase
> Issue Type: Improvement
>Affects Versions: 1.6.0
>Reporter: Rushabh Shah
>Assignee: Rushabh Shah
>Priority: Major
> Fix For: 1.8.0
>
>
> In our production cluster, we encountered an issue where the number of files
> within /hbase/oldWALs directory were growing exponentially from about 4000
> baseline to 15 and growing at the rate of 333 files per minute.
> On further investigation we found that ReplicatonLogCleaner thread was
> getting aborted since it was not able to talk to zookeeper. Stack trace below
> {noformat}
> 2021-02-25 23:05:01,149 WARN [an-pool3-thread-1729] zookeeper.ZKUtil -
> replicationLogCleaner-0x302e05e0d8f,
> quorum=zookeeper-0:2181,zookeeper-1:2181,zookeeper-2:2181,zookeeper-3:2181,zookeeper-4:2181,
> baseZNode=/hbase Unable to get data of znode /hbase/replication/rs
> org.apache.zookeeper.KeeperException$SessionExpiredException: KeeperErrorCode
> = Session expired for /hbase/replication/rs
> at org.apache.zookeeper.KeeperException.create(KeeperException.java:130)
> at org.apache.zookeeper.KeeperException.create(KeeperException.java:54)
> at org.apache.zookeeper.ZooKeeper.getData(ZooKeeper.java:1229)
> at
> org.apache.hadoop.hbase.zookeeper.RecoverableZooKeeper.getData(RecoverableZooKeeper.java:374)
> at org.apache.hadoop.hbase.zookeeper.ZKUtil.getDataNoWatch(ZKUtil.java:713)
> at
> org.apache.hadoop.hbase.replication.ReplicationQueuesClientZKImpl.getQueuesZNodeCversion(ReplicationQueuesClientZKImpl.java:87)
> at
> org.apache.hadoop.hbase.replication.master.ReplicationLogCleaner.loadWALsFromQueues(ReplicationLogCleaner.java:99)
> at
> org.apache.hadoop.hbase.replication.master.ReplicationLogCleaner.getDeletableFiles(ReplicationLogCleaner.java:70)
> at
> org.apache.hadoop.hbase.master.cleaner.CleanerChore.checkAndDeleteFiles(CleanerChore.java:262)
> at
> org.apache.hadoop.hbase.master.cleaner.CleanerChore.access$200(CleanerChore.java:52)
> at
> org.apache.hadoop.hbase.master.cleaner.CleanerChore$3.act(CleanerChore.java:413)
> at
> org.apache.hadoop.hbase.master.cleaner.CleanerChore$3.act(CleanerChore.java:410)
> at
> org.apache.hadoop.hbase.master.cleaner.CleanerChore.deleteAction(CleanerChore.java:481)
> at
> org.apache.hadoop.hbase.master.cleaner.CleanerChore.traverseAndDelete(CleanerChore.java:410)
> at
> org.apache.hadoop.hbase.master.cleaner.CleanerChore.access$100(CleanerChore.java:52)
> at
> org.apache.hadoop.hbase.master.cleaner.CleanerChore$1.run(CleanerChore.java:220)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
> at java.lang.Thread.run(Thread.java:748)
> 2021-02-25 23:05:01,149 WARN [an-pool3-thread-1729]
> master.ReplicationLogCleaner - ReplicationLogCleaner received abort,
> ignoring. Reason: Failed to get stat of replication rs node
> 2021-02-25 23:05:01,149 DEBUG [an-pool3-thread-1729]
> master.ReplicationLogCleaner -
> org.apache.zookeeper.KeeperException$SessionExpiredException: KeeperErrorCode
> = Session expired for /hbase/replication/rs
> 2021-02-25 23:05:01,150 WARN [an-pool3-thread-1729]
> master.ReplicationLogCleaner - Failed to read zookeeper, skipping checking
> deletable files
> {noformat}
>
> {quote} 2021-02-25 23:05:01,149 WARN [an-pool3-thread-1729]
> master.ReplicationLogCleaner - ReplicationLogCleaner received abort,
> ignoring. Reason: Failed to get stat of replication rs node
> {quote}
>
> This line is more scary where HMaster invoked Abortable but just ignored and
> HMaster was doing it business as usual.
> We have max files per directory configuration in namenode which is set to 1M
> in our clusters. If this directory reached that limit then that would have
> brought down the whole cluster.
> We shouldn't ignore Abortable and should crash the Hmaster if Abortable is
> invoked.
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[GitHub] [hbase] Apache-HBase commented on pull request #3222: HBASE-25612 HMaster should abort if ReplicationLogCleaner is not able to delete oldWALs.
Apache-HBase commented on pull request #3222: URL: https://github.com/apache/hbase/pull/3222#issuecomment-831449484 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 0m 0s | Docker mode activated. | | -1 :x: | docker | 0m 6s | Docker failed to build yetus/hbase:edccfe439a. | | Subsystem | Report/Notes | |--:|:-| | GITHUB PR | https://github.com/apache/hbase/pull/3222 | | Console output | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3222/1/console | | versions | git=2.17.1 | | Powered by | Apache Yetus 0.12.0 https://yetus.apache.org | This message was automatically generated. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Updated] (HBASE-25612) HMaster should abort if ReplicationLogCleaner is not able to delete oldWALs.
[
https://issues.apache.org/jira/browse/HBASE-25612?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Rushabh Shah updated HBASE-25612:
-
Fix Version/s: (was: 2.4.3)
(was: 2.5.0)
(was: 3.0.0-alpha-1)
> HMaster should abort if ReplicationLogCleaner is not able to delete oldWALs.
>
>
> Key: HBASE-25612
> URL: https://issues.apache.org/jira/browse/HBASE-25612
> Project: HBase
> Issue Type: Improvement
>Affects Versions: 1.6.0
>Reporter: Rushabh Shah
>Assignee: Rushabh Shah
>Priority: Major
> Fix For: 1.8.0
>
>
> In our production cluster, we encountered an issue where the number of files
> within /hbase/oldWALs directory were growing exponentially from about 4000
> baseline to 15 and growing at the rate of 333 files per minute.
> On further investigation we found that ReplicatonLogCleaner thread was
> getting aborted since it was not able to talk to zookeeper. Stack trace below
> {noformat}
> 2021-02-25 23:05:01,149 WARN [an-pool3-thread-1729] zookeeper.ZKUtil -
> replicationLogCleaner-0x302e05e0d8f,
> quorum=zookeeper-0:2181,zookeeper-1:2181,zookeeper-2:2181,zookeeper-3:2181,zookeeper-4:2181,
> baseZNode=/hbase Unable to get data of znode /hbase/replication/rs
> org.apache.zookeeper.KeeperException$SessionExpiredException: KeeperErrorCode
> = Session expired for /hbase/replication/rs
> at org.apache.zookeeper.KeeperException.create(KeeperException.java:130)
> at org.apache.zookeeper.KeeperException.create(KeeperException.java:54)
> at org.apache.zookeeper.ZooKeeper.getData(ZooKeeper.java:1229)
> at
> org.apache.hadoop.hbase.zookeeper.RecoverableZooKeeper.getData(RecoverableZooKeeper.java:374)
> at org.apache.hadoop.hbase.zookeeper.ZKUtil.getDataNoWatch(ZKUtil.java:713)
> at
> org.apache.hadoop.hbase.replication.ReplicationQueuesClientZKImpl.getQueuesZNodeCversion(ReplicationQueuesClientZKImpl.java:87)
> at
> org.apache.hadoop.hbase.replication.master.ReplicationLogCleaner.loadWALsFromQueues(ReplicationLogCleaner.java:99)
> at
> org.apache.hadoop.hbase.replication.master.ReplicationLogCleaner.getDeletableFiles(ReplicationLogCleaner.java:70)
> at
> org.apache.hadoop.hbase.master.cleaner.CleanerChore.checkAndDeleteFiles(CleanerChore.java:262)
> at
> org.apache.hadoop.hbase.master.cleaner.CleanerChore.access$200(CleanerChore.java:52)
> at
> org.apache.hadoop.hbase.master.cleaner.CleanerChore$3.act(CleanerChore.java:413)
> at
> org.apache.hadoop.hbase.master.cleaner.CleanerChore$3.act(CleanerChore.java:410)
> at
> org.apache.hadoop.hbase.master.cleaner.CleanerChore.deleteAction(CleanerChore.java:481)
> at
> org.apache.hadoop.hbase.master.cleaner.CleanerChore.traverseAndDelete(CleanerChore.java:410)
> at
> org.apache.hadoop.hbase.master.cleaner.CleanerChore.access$100(CleanerChore.java:52)
> at
> org.apache.hadoop.hbase.master.cleaner.CleanerChore$1.run(CleanerChore.java:220)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
> at java.lang.Thread.run(Thread.java:748)
> 2021-02-25 23:05:01,149 WARN [an-pool3-thread-1729]
> master.ReplicationLogCleaner - ReplicationLogCleaner received abort,
> ignoring. Reason: Failed to get stat of replication rs node
> 2021-02-25 23:05:01,149 DEBUG [an-pool3-thread-1729]
> master.ReplicationLogCleaner -
> org.apache.zookeeper.KeeperException$SessionExpiredException: KeeperErrorCode
> = Session expired for /hbase/replication/rs
> 2021-02-25 23:05:01,150 WARN [an-pool3-thread-1729]
> master.ReplicationLogCleaner - Failed to read zookeeper, skipping checking
> deletable files
> {noformat}
>
> {quote} 2021-02-25 23:05:01,149 WARN [an-pool3-thread-1729]
> master.ReplicationLogCleaner - ReplicationLogCleaner received abort,
> ignoring. Reason: Failed to get stat of replication rs node
> {quote}
>
> This line is more scary where HMaster invoked Abortable but just ignored and
> HMaster was doing it business as usual.
> We have max files per directory configuration in namenode which is set to 1M
> in our clusters. If this directory reached that limit then that would have
> brought down the whole cluster.
> We shouldn't ignore Abortable and should crash the Hmaster if Abortable is
> invoked.
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[GitHub] [hbase] shahrs87 opened a new pull request #3222: HBASE-25612 HMaster should abort if ReplicationLogCleaner is not able to delete oldWALs.
shahrs87 opened a new pull request #3222: URL: https://github.com/apache/hbase/pull/3222 This is mainly a back port of [HBASE-23340](https://issues.apache.org/jira/browse/HBASE-23340) / [PR-2779 ](https://github.com/apache/hbase/pull/2779/files) with some test related changes and adding test case. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #3219: HBASE-25836 RegionStates#getAssignmentsForBalancer should only care about OPEN or OPENING regions
Apache-HBase commented on pull request #3219: URL: https://github.com/apache/hbase/pull/3219#issuecomment-83187 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 0m 27s | Docker mode activated. | ||| _ Prechecks _ | | +1 :green_heart: | dupname | 0m 0s | No case conflicting files found. | | +1 :green_heart: | hbaseanti | 0m 0s | Patch does not have any anti-patterns. | | +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. | ||| _ master Compile Tests _ | | +1 :green_heart: | mvninstall | 3m 48s | master passed | | +1 :green_heart: | compile | 3m 13s | master passed | | +1 :green_heart: | checkstyle | 1m 2s | master passed | | +1 :green_heart: | spotbugs | 1m 58s | master passed | ||| _ Patch Compile Tests _ | | +1 :green_heart: | mvninstall | 3m 37s | the patch passed | | +1 :green_heart: | compile | 3m 9s | the patch passed | | +1 :green_heart: | javac | 3m 9s | the patch passed | | +1 :green_heart: | checkstyle | 1m 1s | the patch passed | | +1 :green_heart: | whitespace | 0m 0s | The patch has no whitespace issues. | | +1 :green_heart: | hadoopcheck | 17m 50s | Patch does not cause any errors with Hadoop 3.1.2 3.2.1 3.3.0. | | +1 :green_heart: | spotbugs | 2m 13s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | asflicense | 0m 13s | The patch does not generate ASF License warnings. | | | | 46m 13s | | | Subsystem | Report/Notes | |--:|:-| | Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3219/2/artifact/yetus-general-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/3219 | | Optional Tests | dupname asflicense javac spotbugs hadoopcheck hbaseanti checkstyle compile | | uname | Linux 3356dcb3d800 4.15.0-60-generic #67-Ubuntu SMP Thu Aug 22 16:55:30 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / 7640134e3e | | Default Java | AdoptOpenJDK-1.8.0_282-b08 | | Max. process+thread count | 96 (vs. ulimit of 3) | | modules | C: hbase-server U: hbase-server | | Console output | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3219/2/console | | versions | git=2.17.1 maven=3.6.3 spotbugs=4.2.2 | | Powered by | Apache Yetus 0.12.0 https://yetus.apache.org | This message was automatically generated. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #3218: HBASE-25835 Ignore duplicate split requests from regionserver reports
Apache-HBase commented on pull request #3218: URL: https://github.com/apache/hbase/pull/3218#issuecomment-831436077 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 1m 6s | Docker mode activated. | ||| _ Prechecks _ | | +1 :green_heart: | dupname | 0m 0s | No case conflicting files found. | | +1 :green_heart: | hbaseanti | 0m 0s | Patch does not have any anti-patterns. | | +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. | ||| _ master Compile Tests _ | | +1 :green_heart: | mvninstall | 3m 59s | master passed | | +1 :green_heart: | compile | 3m 17s | master passed | | +1 :green_heart: | checkstyle | 1m 9s | master passed | | +1 :green_heart: | spotbugs | 2m 11s | master passed | ||| _ Patch Compile Tests _ | | +1 :green_heart: | mvninstall | 4m 0s | the patch passed | | +1 :green_heart: | compile | 3m 18s | the patch passed | | +1 :green_heart: | javac | 3m 18s | the patch passed | | +1 :green_heart: | checkstyle | 1m 10s | the patch passed | | +1 :green_heart: | whitespace | 0m 0s | The patch has no whitespace issues. | | +1 :green_heart: | hadoopcheck | 20m 0s | Patch does not cause any errors with Hadoop 3.1.2 3.2.1 3.3.0. | | +1 :green_heart: | spotbugs | 2m 19s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | asflicense | 0m 11s | The patch does not generate ASF License warnings. | | | | 51m 2s | | | Subsystem | Report/Notes | |--:|:-| | Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3218/2/artifact/yetus-general-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/3218 | | Optional Tests | dupname asflicense javac spotbugs hadoopcheck hbaseanti checkstyle compile | | uname | Linux e2b6cf98bc37 4.15.0-136-generic #140-Ubuntu SMP Thu Jan 28 05:20:47 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / 7640134e3e | | Default Java | AdoptOpenJDK-1.8.0_282-b08 | | Max. process+thread count | 86 (vs. ulimit of 3) | | modules | C: hbase-server U: hbase-server | | Console output | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3218/2/console | | versions | git=2.17.1 maven=3.6.3 spotbugs=4.2.2 | | Powered by | Apache Yetus 0.12.0 https://yetus.apache.org | This message was automatically generated. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #3208: HBASE-25824 IntegrationTestLoadCommonCrawl
Apache-HBase commented on pull request #3208: URL: https://github.com/apache/hbase/pull/3208#issuecomment-831431170 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 0m 30s | Docker mode activated. | ||| _ Prechecks _ | | +1 :green_heart: | dupname | 0m 0s | No case conflicting files found. | | +1 :green_heart: | hbaseanti | 0m 0s | Patch does not have any anti-patterns. | | +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. | ||| _ master Compile Tests _ | | +0 :ok: | mvndep | 0m 20s | Maven dependency ordering for branch | | +1 :green_heart: | mvninstall | 3m 49s | master passed | | +1 :green_heart: | compile | 8m 58s | master passed | | +1 :green_heart: | checkstyle | 2m 6s | master passed | | +1 :green_heart: | spotbugs | 10m 29s | master passed | ||| _ Patch Compile Tests _ | | +0 :ok: | mvndep | 0m 20s | Maven dependency ordering for patch | | +1 :green_heart: | mvninstall | 3m 40s | the patch passed | | +1 :green_heart: | compile | 8m 31s | the patch passed | | +1 :green_heart: | javac | 8m 31s | the patch passed | | +1 :green_heart: | checkstyle | 1m 54s | the patch passed | | +1 :green_heart: | whitespace | 0m 0s | The patch has no whitespace issues. | | +1 :green_heart: | xml | 0m 1s | The patch has no ill-formed XML file. | | +1 :green_heart: | hadoopcheck | 18m 25s | Patch does not cause any errors with Hadoop 3.1.2 3.2.1 3.3.0. | | +1 :green_heart: | spotbugs | 10m 58s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | asflicense | 0m 26s | The patch does not generate ASF License warnings. | | | | 78m 44s | | | Subsystem | Report/Notes | |--:|:-| | Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3208/8/artifact/yetus-general-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/3208 | | Optional Tests | dupname asflicense javac spotbugs hadoopcheck hbaseanti checkstyle compile xml | | uname | Linux b7aa9c653277 4.15.0-112-generic #113-Ubuntu SMP Thu Jul 9 23:41:39 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / 7640134e3e | | Default Java | AdoptOpenJDK-1.8.0_282-b08 | | Max. process+thread count | 141 (vs. ulimit of 3) | | modules | C: hbase-it . U: . | | Console output | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3208/8/console | | versions | git=2.17.1 maven=3.6.3 spotbugs=4.2.2 | | Powered by | Apache Yetus 0.12.0 https://yetus.apache.org | This message was automatically generated. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] bharathv commented on a change in pull request #3219: HBASE-25836 RegionStates#getAssignmentsForBalancer should only care about OPEN or OPENING regions
bharathv commented on a change in pull request #3219:
URL: https://github.com/apache/hbase/pull/3219#discussion_r625261529
##
File path:
hbase-server/src/main/java/org/apache/hadoop/hbase/master/assignment/RegionStates.java
##
@@ -571,23 +571,37 @@ public ServerName getRegionServerOfRegion(RegionInfo
regionInfo) {
* wants to iterate this exported list. We need to synchronize on regions
* since all access to this.servers is under a lock on this.regions.
*
- * @return A clone of current assignments.
+ * @return A clone of current open or opening assignments.
*/
public Map>>
getAssignmentsForBalancer(
TableStateManager tableStateManager, List onlineServers) {
final Map>> result = new
HashMap<>();
for (RegionStateNode node : regionsMap.values()) {
+ // DisableTableProcedure first sets the table state to DISABLED and then
force unassigns
+ // the regions in a loop. The balancer should ignore all regions for
tables in DISABLED
+ // state because even if still currently open we expect them to be
offlined very soon.
if (isTableDisabled(tableStateManager, node.getTable())) {
+if (LOG.isTraceEnabled()) {
+ LOG.trace("Ignoring {} because table is disabled", node);
+}
continue;
}
- if (node.getRegionInfo().isSplitParent()) {
+ // When balancing, we are only interested in OPEN or OPENING regions.
These can be
+ // expected to remain online until the next balancer iteration or unless
the balancer
+ // decides to move it. Regions in other states are not eligible for
balancing, because
+ // they are closing, splitting, merging, or otherwise already in
transition.
+ if (!node.isInState(State.OPEN, State.OPENING)) {
+if (LOG.isTraceEnabled()) {
Review comment:
Doing both isTraceEnabled() check and parameterized logging seems
wasteful, its usually either one or the other.. I think the recommendation from
log4j is to do the latter for concise code and avoid unnecessary temporary
string objects...
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
