[jira] [Commented] (HBASE-21519) Namespace region is never assigned in a HM failover scenario when multiwal is enabled
[
https://issues.apache.org/jira/browse/HBASE-21519?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16918338#comment-16918338
]
Pankaj Kumar commented on HBASE-21519:
--
Adding this UT scenario will be better idea, will attach the test patch soon.
> Namespace region is never assigned in a HM failover scenario when multiwal is
> enabled
> -
>
> Key: HBASE-21519
> URL: https://issues.apache.org/jira/browse/HBASE-21519
> Project: HBase
> Issue Type: Bug
> Components: master, wal
>Affects Versions: 2.1.1
>Reporter: Pankaj Kumar
>Assignee: Pankaj Kumar
>Priority: Critical
> Attachments: HBASE-21519.branch-2.patch, providerId_value.png
>
>
> In our test env we found that namespace region is never be assigned on HM
> failover scenario when multiwal feature is enabled,
> {noformat}
> 2018-11-28 01:38:28,085 WARN [master/HM-1:16000:becomeActiveMaster]
> master.HMaster:
> hbase:namespace,,1543339859614.31f6d3383af09e18e1e81ca02a93de15. is NOT
> online; state=\{31f6d3383af09e18e1e81ca02a93de15 state=OPEN,
> ts=1543340156928, server=RS-2,16020,1543339824397};
> ServerCrashProcedures=false. Master startup cannot progress, in
> holding-pattern until region onlined.
> {noformat}
> And finally HM abort with following error,
> {noformat}
> 2018-11-28 01:39:16,858 ERROR
> [ActiveMasterInitializationMonitor-1543338648565] master.HMaster: Master
> failed to complete initialization after 24ms. Please consider submitting
> a bug report including a thread dump of this process.
> 2018-11-28 01:39:18,980 ERROR
> [ActiveMasterInitializationMonitor-1543338648565] master.HMaster: Zombie
> Master exiting. Thread dump to stdout
> {noformat}
> Stack trace:
> {noformat}
> Thread 102 (master/HM-1:16000:becomeActiveMaster):
> State: TIMED_WAITING
> Blocked count: 100
> Waited count: 246
> Stack:
> java.lang.Thread.sleep(Native Method)
> org.apache.hadoop.hbase.util.Threads.sleep(Threads.java:148)
> org.apache.hadoop.hbase.master.HMaster.isRegionOnline(HMaster.java:1166)
>
> org.apache.hadoop.hbase.master.HMaster.waitForNamespaceOnline(HMaster.java:1187)
>
> org.apache.hadoop.hbase.master.HMaster.finishActiveMasterInitialization(HMaster.java:1044)
>
> org.apache.hadoop.hbase.master.HMaster.startActiveMasterManager(HMaster.java:2285)
> org.apache.hadoop.hbase.master.HMaster.lambda$run$0(HMaster.java:590)
> org.apache.hadoop.hbase.master.HMaster$$Lambda$40/1078246575.run(Unknown
> Source)
> java.lang.Thread.run(Thread.java:745)
> {noformat}
>
> Step to reproduce:
> 1) Setup a HBase cluster with 1/2 HM (say HM-1) and 2 RS(say RS-1 & RS-2)
> 2) Enable multiwal feature with following configuration setting and start
> the cluster,
> {noformat}
>
> hbase.wal.provider
> multiwal
>
>
> hbase.wal.regiongrouping.strategy
> identity
>
> {noformat}
> 3) Make sure meta and namespace regions are assigned on different RS, suppose
> RS-1 & RS-2 respectively.
> 4) Create table 't1'
> 5) Flush the meta table explicitly
> 6) Kill the RS-2, so during RS-2 SCP all regions including namespace region
> will be assigned to RS-1.
> 7) Now Kill RS-1 before meta flush happen. Here both RS-2 & RS-1 are
> shutdown now.
> 8) Stop the HM and start RS-1 & RS-2.
> 9) Now start the HM.
> Meta region is assigned successfully but HM is keep waiting for the namespace
> region onlline (Master startup cannot progress, in holding-pattern until
> region onlined) and abort with timeout.
> Observation:
> 1) After step-3 namespace region was assigned to RS-2 and meta entry was as
> follows,
> {noformat}
> hbase:namespace,,1543339859614.31f6d3383af09e18e1e81ca02a93de15.
> column=info:server, timestamp=1543339860920, value=RS-2:16020
> hbase:namespace,,1543339859614.31f6d3383af09e18e1e81ca02a93de15.
> column=info:serverstartcode, timestamp=1543339860920, value=1543339824397
> {noformat}
> 2) After step-6 namespace region was assigned to RS-1 and meta entry was as
> follows,
> {noformat}
> hbase:namespace,,1543339859614.31f6d3383af09e18e1e81ca02a93de15.
> column=info:server, timestamp=1543339880920, value=RS-1:16020
> hbase:namespace,,1543339859614.31f6d3383af09e18e1e81ca02a93de15.
> column=info:serverstartcode, timestamp=1543339880920, value=1543339829288
> {noformat}
> 3) After Step-9, meta entry for namespace region was as follows,
> {noformat}
> hbase:namespace,,1543339859614.31f6d3383af09e18e1e81ca02a93de15.
> column=info:server, timestamp=1543339860920, value=RS-2:16020
> hbase:namespace,,1543339859614.31f6d3383af09e18e1e81ca02a93de15.
> column=info:serverstartcode, timestamp=1543339860920, value=1543339824397
> {noformat}
> During SCP we do meta log split based on filter,
> {noformat}
> /**
> * Specialized method to handle the
[GitHub] [hbase] Apache-HBase commented on issue #494: HBASE-22380 break circle replication when doing bulkload
Apache-HBase commented on issue #494: HBASE-22380 break circle replication when doing bulkload URL: https://github.com/apache/hbase/pull/494#issuecomment-526049792 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | 0 | reexec | 127 | Docker mode activated. | ||| _ Prechecks _ | | +1 | dupname | 0 | No case conflicting files found. | | +1 | hbaseanti | 0 | Patch does not have any anti-patterns. | | +1 | @author | 0 | The patch does not contain any @author tags. | | +1 | test4tests | 0 | The patch appears to include 2 new or modified test files. | ||| _ branch-2 Compile Tests _ | | 0 | mvndep | 19 | Maven dependency ordering for branch | | +1 | mvninstall | 384 | branch-2 passed | | +1 | compile | 128 | branch-2 passed | | +1 | checkstyle | 141 | branch-2 passed | | +1 | shadedjars | 256 | branch has no errors when building our shaded downstream artifacts. | | +1 | javadoc | 71 | branch-2 passed | | 0 | spotbugs | 213 | Used deprecated FindBugs config; considering switching to SpotBugs. | | +1 | findbugs | 433 | branch-2 passed | ||| _ Patch Compile Tests _ | | 0 | mvndep | 17 | Maven dependency ordering for patch | | +1 | mvninstall | 310 | the patch passed | | +1 | compile | 114 | the patch passed | | +1 | cc | 114 | the patch passed | | +1 | javac | 114 | the patch passed | | -1 | checkstyle | 39 | hbase-client: The patch generated 1 new + 314 unchanged - 0 fixed = 315 total (was 314) | | -1 | checkstyle | 84 | hbase-server: The patch generated 3 new + 300 unchanged - 0 fixed = 303 total (was 300) | | +1 | whitespace | 0 | The patch has no whitespace issues. | | +1 | shadedjars | 262 | patch has no errors when building our shaded downstream artifacts. | | +1 | hadoopcheck | 997 | Patch does not cause any errors with Hadoop 2.8.5 2.9.2 or 3.1.2. | | +1 | hbaseprotoc | 112 | the patch passed | | +1 | javadoc | 74 | the patch passed | | -1 | findbugs | 201 | hbase-server generated 1 new + 0 unchanged - 0 fixed = 1 total (was 0) | ||| _ Other Tests _ | | +1 | unit | 37 | hbase-protocol-shaded in the patch passed. | | +1 | unit | 196 | hbase-client in the patch passed. | | -1 | unit | 16905 | hbase-server in the patch failed. | | +1 | asflicense | 74 | The patch does not generate ASF License warnings. | | | | 21686 | | | Reason | Tests | |---:|:--| | FindBugs | module:hbase-server | | | org.apache.hadoop.hbase.replication.regionserver.ReplicationSink.replicateEntries(List, CellScanner, String, String, String) makes inefficient use of keySet iterator instead of entrySet iterator At ReplicationSink.java:makes inefficient use of keySet iterator instead of entrySet iterator At ReplicationSink.java:[line 256] | | Failed junit tests | hadoop.hbase.regionserver.TestSplitTransactionOnCluster | | Subsystem | Report/Notes | |--:|:-| | Docker | Client=19.03.1 Server=19.03.1 base: https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-494/38/artifact/out/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/494 | | Optional Tests | dupname asflicense javac javadoc unit spotbugs findbugs shadedjars hadoopcheck hbaseanti checkstyle compile cc hbaseprotoc | | uname | Linux f6e784ec7309 4.15.0-48-generic #51-Ubuntu SMP Wed Apr 3 08:28:49 UTC 2019 x86_64 GNU/Linux | | Build tool | maven | | Personality | /home/jenkins/jenkins-slave/workspace/HBase-PreCommit-GitHub-PR_PR-494/out/precommit/personality/provided.sh | | git revision | branch-2 / 84b4a61e88 | | Default Java | 1.8.0_181 | | checkstyle | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-494/38/artifact/out/diff-checkstyle-hbase-client.txt | | checkstyle | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-494/38/artifact/out/diff-checkstyle-hbase-server.txt | | findbugs | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-494/38/artifact/out/new-findbugs-hbase-server.html | | unit | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-494/38/artifact/out/patch-unit-hbase-server.txt | | Test Results | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-494/38/testReport/ | | Max. process+thread count | 4676 (vs. ulimit of 1) | | modules | C: hbase-protocol-shaded hbase-client hbase-server U: . | | Console output | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-494/38/console | | versions | git=2.11.0 maven=2018-06-17T18:33:14Z) findbugs=3.1.11 | | Powered by | Apache Yetus 0.10.0 http://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please
[jira] [Commented] (HBASE-22944) TableNotFoundException: hbase:quota is thrown when region server is restarted.
[
https://issues.apache.org/jira/browse/HBASE-22944?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16918323#comment-16918323
]
Pankaj Kumar commented on HBASE-22944:
--
This is little tricky which will happen in below scenario,
# HM & RSs are restarted after enabling Space Quota feature.
# SpaceQuotaRefresher chore interval is set as very low say 5 sec
# So by the time HMaster creates quota table SpaceQuotaRefresherChore executes
and throws TNFE.
Right [~shardulsingh]?
> TableNotFoundException: hbase:quota is thrown when region server is
> restarted.
> ---
>
> Key: HBASE-22944
> URL: https://issues.apache.org/jira/browse/HBASE-22944
> Project: HBase
> Issue Type: Bug
> Components: Quotas
>Reporter: Shardul Singh
>Assignee: Shardul Singh
>Priority: Minor
>
> During Master startup if quota feature is enabled and region server is
> running TableNotFoundException occurs in regionServer logs
> SpaceQuotaRefresherChore does not checks whether hbase:quota table is
> present,before starting its operation which sometimes may result to
> TableNotFoundExceptions like below.
> This is because master has not created the hbase:quota table and
> SpaceQuotaRefresherChore is running.
> {code:java}
> java.io.UncheckedIOException: org.apache.hadoop.hbase.
> TableNotFoundException: hbase:quota
> at
> org.apache.hadoop.hbase.client.ResultScanner$1.hasNext(ResultScanner.java:55)
> at
> org.apache.hadoop.hbase.quotas.SpaceQuotaRefresherChore.fetchSnapshotsFromQuotaTable(SpaceQuotaRefresherChore.java:170)
> at
> org.apache.hadoop.hbase.quotas.SpaceQuotaRefresherChore.chore(SpaceQuotaRefresherChore.java:84)
> at org.apache.hadoop.hbase.ScheduledChore.run(ScheduledChore.java:186)
> at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
> at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308)
> at
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180)
> at
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294)
> at
> org.apache.hadoop.hbase.JitterScheduledThreadPoolExecutorImpl$JitteredRunnableScheduledFuture.run(JitterScheduledThreadPoolExecutorImpl.java:111)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> at java.lang.Thread.run(Thread.java:748)
> {code}
>
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
[jira] [Updated] (HBASE-22944) TableNotFoundException: hbase:quota is thrown when region server is restarted.
[
https://issues.apache.org/jira/browse/HBASE-22944?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Shardul Singh updated HBASE-22944:
--
Description:
During Master startup if quota feature is enabled and region server is running
TableNotFoundException occurs in regionServer logs
SpaceQuotaRefresherChore does not checks whether hbase:quota table is
present,before starting its operation which sometimes may result to
TableNotFoundExceptions like below.
This is because master has not created the hbase:quota table and
SpaceQuotaRefresherChore is running.
{code:java}
java.io.UncheckedIOException: org.apache.hadoop.hbase.
TableNotFoundException: hbase:quota
at org.apache.hadoop.hbase.client.ResultScanner$1.hasNext(ResultScanner.java:55)
at
org.apache.hadoop.hbase.quotas.SpaceQuotaRefresherChore.fetchSnapshotsFromQuotaTable(SpaceQuotaRefresherChore.java:170)
at
org.apache.hadoop.hbase.quotas.SpaceQuotaRefresherChore.chore(SpaceQuotaRefresherChore.java:84)
at org.apache.hadoop.hbase.ScheduledChore.run(ScheduledChore.java:186)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308)
at
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180)
at
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294)
at
org.apache.hadoop.hbase.JitterScheduledThreadPoolExecutorImpl$JitteredRunnableScheduledFuture.run(JitterScheduledThreadPoolExecutorImpl.java:111)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:748)
{code}
was:
SpaceQuotaRefresherChore does not checks whether hbase:quota table is
present,before starting its operation which sometimes may result to
TableNotFoundExceptions like below.
This is because master has not created the hbase:quota table and
SpaceQuotaRefresherChore is running.
{code:java}
java.io.UncheckedIOException: org.apache.hadoop.hbase.
TableNotFoundException: hbase:quota
at org.apache.hadoop.hbase.client.ResultScanner$1.hasNext(ResultScanner.java:55)
at
org.apache.hadoop.hbase.quotas.SpaceQuotaRefresherChore.fetchSnapshotsFromQuotaTable(SpaceQuotaRefresherChore.java:170)
at
org.apache.hadoop.hbase.quotas.SpaceQuotaRefresherChore.chore(SpaceQuotaRefresherChore.java:84)
at org.apache.hadoop.hbase.ScheduledChore.run(ScheduledChore.java:186)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308)
at
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180)
at
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294)
at
org.apache.hadoop.hbase.JitterScheduledThreadPoolExecutorImpl$JitteredRunnableScheduledFuture.run(JitterScheduledThreadPoolExecutorImpl.java:111)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:748)
{code}
> TableNotFoundException: hbase:quota is thrown when region server is
> restarted.
> ---
>
> Key: HBASE-22944
> URL: https://issues.apache.org/jira/browse/HBASE-22944
> Project: HBase
> Issue Type: Bug
> Components: Quotas
>Reporter: Shardul Singh
>Assignee: Shardul Singh
>Priority: Minor
>
> During Master startup if quota feature is enabled and region server is
> running TableNotFoundException occurs in regionServer logs
> SpaceQuotaRefresherChore does not checks whether hbase:quota table is
> present,before starting its operation which sometimes may result to
> TableNotFoundExceptions like below.
> This is because master has not created the hbase:quota table and
> SpaceQuotaRefresherChore is running.
> {code:java}
> java.io.UncheckedIOException: org.apache.hadoop.hbase.
> TableNotFoundException: hbase:quota
> at
> org.apache.hadoop.hbase.client.ResultScanner$1.hasNext(ResultScanner.java:55)
> at
> org.apache.hadoop.hbase.quotas.SpaceQuotaRefresherChore.fetchSnapshotsFromQuotaTable(SpaceQuotaRefresherChore.java:170)
> at
> org.apache.hadoop.hbase.quotas.SpaceQuotaRefresherChore.chore(SpaceQuotaRefresherChore.java:84)
> at org.apache.hadoop.hbase.ScheduledChore.run(ScheduledChore.java:186)
> at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
> at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308)
> at
> java.util.concurrent.ScheduledThreadPoolExec
[GitHub] [hbase] shardul-cr7 opened a new pull request #559: HBASE-22944 TableNotFoundException: hbase:quota is thrown when region server is restarted.
shardul-cr7 opened a new pull request #559: HBASE-22944 TableNotFoundException: hbase:quota is thrown when region server is restarted. URL: https://github.com/apache/hbase/pull/559 Sometimes during startup TableNotFoundException occurs in regionServer logs. SpaceQuotaRefresherChore does not checks whether hbase:quota table is present,before starting its operation which sometimes may result to TableNotFoundExceptions like below. This is because master has not created the hbase:quota table and SpaceQuotaRefresherChore is running. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Commented] (HBASE-22568) [2.1] Upgrade to Jetty 9.3.latest and Jackson 2.9.latest
[ https://issues.apache.org/jira/browse/HBASE-22568?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16918313#comment-16918313 ] Pankaj Kumar commented on HBASE-22568: -- My bad, by mistake it got assigned to me. Reverted back. > [2.1] Upgrade to Jetty 9.3.latest and Jackson 2.9.latest > > > Key: HBASE-22568 > URL: https://issues.apache.org/jira/browse/HBASE-22568 > Project: HBase > Issue Type: Task > Components: dependencies >Reporter: Josh Elser >Assignee: Josh Elser >Priority: Major > Fix For: 2.1.6 > > Attachments: HBASE-22568.001.branch-2.1.patch, > HBASE-22568.002.branch-2.1.patch > > > Worked through some static analysis scans and these two popped up as a result. > Upgrades are: > * Jetty 9.3.25->9.3.27 > * Jackson 2.9.2->2.9.9 > Not expecting any pain with these, but we'll find out what QA thinks! :) -- This message was sent by Atlassian Jira (v8.3.2#803003)
[jira] [Assigned] (HBASE-22568) [2.1] Upgrade to Jetty 9.3.latest and Jackson 2.9.latest
[ https://issues.apache.org/jira/browse/HBASE-22568?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Pankaj Kumar reassigned HBASE-22568: Assignee: Josh Elser (was: Pankaj Kumar) > [2.1] Upgrade to Jetty 9.3.latest and Jackson 2.9.latest > > > Key: HBASE-22568 > URL: https://issues.apache.org/jira/browse/HBASE-22568 > Project: HBase > Issue Type: Task > Components: dependencies >Reporter: Josh Elser >Assignee: Josh Elser >Priority: Major > Fix For: 2.1.6 > > Attachments: HBASE-22568.001.branch-2.1.patch, > HBASE-22568.002.branch-2.1.patch > > > Worked through some static analysis scans and these two popped up as a result. > Upgrades are: > * Jetty 9.3.25->9.3.27 > * Jackson 2.9.2->2.9.9 > Not expecting any pain with these, but we'll find out what QA thinks! :) -- This message was sent by Atlassian Jira (v8.3.2#803003)
[jira] [Commented] (HBASE-22881) Fix non-daemon threads in hbase server implementation
[
https://issues.apache.org/jira/browse/HBASE-22881?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16918312#comment-16918312
]
Hudson commented on HBASE-22881:
Results for branch branch-2.2
[build #550 on
builds.a.o|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.2/550/]:
(x) *{color:red}-1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.2/550//General_Nightly_Build_Report/]
(x) {color:red}-1 jdk8 hadoop2 checks{color}
-- Something went wrong running this stage, please [check relevant console
output|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.2/550//console].
(/) {color:green}+1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.2/550//JDK8_Nightly_Build_Report_(Hadoop3)/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
(/) {color:green}+1 client integration test{color}
> Fix non-daemon threads in hbase server implementation
> -
>
> Key: HBASE-22881
> URL: https://issues.apache.org/jira/browse/HBASE-22881
> Project: HBase
> Issue Type: Bug
> Components: master
>Reporter: Xiaolin Ha
>Assignee: Xiaolin Ha
>Priority: Major
> Fix For: 3.0.0, 2.3.0, 2.2.2
>
>
> "pool-8-thread-3" #7252 prio=5 os_prio=0 tid=0x7f91040044c0 nid=0xd71e
> waiting on condition [0x7f8f4d209000]
>java.lang.Thread.State: WAITING (parking)
> at sun.misc.Unsafe.park(Native Method)
> - parking to wait for <0x0005c0e49ed0> (a
> java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
> at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
> at
> java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
> at
> java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
> at
> java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
> at java.lang.Thread.run(Thread.java:748)
>Locked ownable synchronizers:
> - None
> "pool-8-thread-2" #7251 prio=5 os_prio=0 tid=0x7f910c010be0 nid=0xd71d
> waiting on condition [0x7f8f4daab000]
>java.lang.Thread.State: WAITING (parking)
> at sun.misc.Unsafe.park(Native Method)
> - parking to wait for <0x0005c0e49ed0> (a
> java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
> at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
> at
> java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
> at
> java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
> at
> java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
> at java.lang.Thread.run(Thread.java:748)
>Locked ownable synchronizers:
> - None
> "pool-8-thread-1" #7250 prio=5 os_prio=0 tid=0x7f9119d0 nid=0xd71c
> waiting on condition [0x7f8f4da6a000]
>java.lang.Thread.State: WAITING (parking)
> at sun.misc.Unsafe.park(Native Method)
> - parking to wait for <0x0005c0e49ed0> (a
> java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
> at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
> at
> java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
> at
> java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
> at
> java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
> at java.lang.Thread.run(Thread.java:748)
>Locked ownable synchronizers:
> - None
> "pool-5-thread-3" #7248 prio=5 os_prio=0 tid=0x7f9238005ad0 nid=0xd71a
> waiting on condition [0x7f8f4cb65000]
>java.lang.Thread.State: WAITING (parking)
> at sun.misc.Unsafe.park(Native Method)
> - parking to wait for
[jira] [Updated] (HBASE-22944) TableNotFoundException: hbase:quota is thrown when region server is restarted.
[
https://issues.apache.org/jira/browse/HBASE-22944?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Shardul Singh updated HBASE-22944:
--
Description:
SpaceQuotaRefresherChore does not checks whether hbase:quota table is
present,before starting its operation which sometimes may result to
TableNotFoundExceptions like below.
This is because master has not created the hbase:quota table and
SpaceQuotaRefresherChore is running.
{code:java}
java.io.UncheckedIOException: org.apache.hadoop.hbase.
TableNotFoundException: hbase:quota
at org.apache.hadoop.hbase.client.ResultScanner$1.hasNext(ResultScanner.java:55)
at
org.apache.hadoop.hbase.quotas.SpaceQuotaRefresherChore.fetchSnapshotsFromQuotaTable(SpaceQuotaRefresherChore.java:170)
at
org.apache.hadoop.hbase.quotas.SpaceQuotaRefresherChore.chore(SpaceQuotaRefresherChore.java:84)
at org.apache.hadoop.hbase.ScheduledChore.run(ScheduledChore.java:186)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308)
at
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180)
at
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294)
at
org.apache.hadoop.hbase.JitterScheduledThreadPoolExecutorImpl$JitteredRunnableScheduledFuture.run(JitterScheduledThreadPoolExecutorImpl.java:111)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:748)
{code}
was:
SpaceQuotaRefresherChore does not checks whether hbase:quota table is
present,before starting its operation which sometimes may result to
TableNotFoundExceptions like below.
This is because master has not created the hbase:quota table and
SpaceQuotaRefresherChore is running.
{code:java}
java.io.UncheckedIOException: org.apache.hadoop.hbase.
{quote}TableNotFoundException: hbase:quota
{quote}
at org.apache.hadoop.hbase.client.ResultScanner$1.hasNext(ResultScanner.java:55)
at
org.apache.hadoop.hbase.quotas.SpaceQuotaRefresherChore.fetchSnapshotsFromQuotaTable(SpaceQuotaRefresherChore.java:170)
at
org.apache.hadoop.hbase.quotas.SpaceQuotaRefresherChore.chore(SpaceQuotaRefresherChore.java:84)
at org.apache.hadoop.hbase.ScheduledChore.run(ScheduledChore.java:186)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308)
at
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180)
at
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294)
at
org.apache.hadoop.hbase.JitterScheduledThreadPoolExecutorImpl$JitteredRunnableScheduledFuture.run(JitterScheduledThreadPoolExecutorImpl.java:111)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:748)
{code}
> TableNotFoundException: hbase:quota is thrown when region server is
> restarted.
> ---
>
> Key: HBASE-22944
> URL: https://issues.apache.org/jira/browse/HBASE-22944
> Project: HBase
> Issue Type: Bug
> Components: Quotas
>Reporter: Shardul Singh
>Assignee: Shardul Singh
>Priority: Minor
>
> SpaceQuotaRefresherChore does not checks whether hbase:quota table is
> present,before starting its operation which sometimes may result to
> TableNotFoundExceptions like below.
> This is because master has not created the hbase:quota table and
> SpaceQuotaRefresherChore is running.
> {code:java}
> java.io.UncheckedIOException: org.apache.hadoop.hbase.
> TableNotFoundException: hbase:quota
> at
> org.apache.hadoop.hbase.client.ResultScanner$1.hasNext(ResultScanner.java:55)
> at
> org.apache.hadoop.hbase.quotas.SpaceQuotaRefresherChore.fetchSnapshotsFromQuotaTable(SpaceQuotaRefresherChore.java:170)
> at
> org.apache.hadoop.hbase.quotas.SpaceQuotaRefresherChore.chore(SpaceQuotaRefresherChore.java:84)
> at org.apache.hadoop.hbase.ScheduledChore.run(ScheduledChore.java:186)
> at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
> at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308)
> at
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180)
> at
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294)
> at
> org.apache.hadoop.hbase.JitterScheduledThreadPool
[jira] [Assigned] (HBASE-22568) [2.1] Upgrade to Jetty 9.3.latest and Jackson 2.9.latest
[ https://issues.apache.org/jira/browse/HBASE-22568?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Pankaj Kumar reassigned HBASE-22568: Assignee: Pankaj Kumar (was: Josh Elser) > [2.1] Upgrade to Jetty 9.3.latest and Jackson 2.9.latest > > > Key: HBASE-22568 > URL: https://issues.apache.org/jira/browse/HBASE-22568 > Project: HBase > Issue Type: Task > Components: dependencies >Reporter: Josh Elser >Assignee: Pankaj Kumar >Priority: Major > Fix For: 2.1.6 > > Attachments: HBASE-22568.001.branch-2.1.patch, > HBASE-22568.002.branch-2.1.patch > > > Worked through some static analysis scans and these two popped up as a result. > Upgrades are: > * Jetty 9.3.25->9.3.27 > * Jackson 2.9.2->2.9.9 > Not expecting any pain with these, but we'll find out what QA thinks! :) -- This message was sent by Atlassian Jira (v8.3.2#803003)
[jira] [Updated] (HBASE-22944) TableNotFoundException: hbase:quota is thrown when region server is restarted.
[
https://issues.apache.org/jira/browse/HBASE-22944?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Shardul Singh updated HBASE-22944:
--
Description:
SpaceQuotaRefresherChore does not checks whether hbase:quota table is
present,before starting its operation which sometimes may result to
TableNotFoundExceptions like below.
This is because master has not created the hbase:quota table and
SpaceQuotaRefresherChore is running.
{code:java}
java.io.UncheckedIOException: org.apache.hadoop.hbase.
{quote}TableNotFoundException: hbase:quota
{quote}
at org.apache.hadoop.hbase.client.ResultScanner$1.hasNext(ResultScanner.java:55)
at
org.apache.hadoop.hbase.quotas.SpaceQuotaRefresherChore.fetchSnapshotsFromQuotaTable(SpaceQuotaRefresherChore.java:170)
at
org.apache.hadoop.hbase.quotas.SpaceQuotaRefresherChore.chore(SpaceQuotaRefresherChore.java:84)
at org.apache.hadoop.hbase.ScheduledChore.run(ScheduledChore.java:186)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308)
at
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180)
at
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294)
at
org.apache.hadoop.hbase.JitterScheduledThreadPoolExecutorImpl$JitteredRunnableScheduledFuture.run(JitterScheduledThreadPoolExecutorImpl.java:111)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:748)
{code}
was:
SpaceQuotaRefresherChore does not checks whether hbase:quota table is
present,before starting its operation which sometimes may result to
TableNotFoundExceptions like below.
This is because master has not created the hbase:quota table and
SpaceQuotaRefresherChore is running.
```
java.io.UncheckedIOException: org.apache.hadoop.hbase.
{quote}TableNotFoundException: hbase:quota
{quote}
at org.apache.hadoop.hbase.client.ResultScanner$1.hasNext(ResultScanner.java:55)
at
org.apache.hadoop.hbase.quotas.SpaceQuotaRefresherChore.fetchSnapshotsFromQuotaTable(SpaceQuotaRefresherChore.java:170)
at
org.apache.hadoop.hbase.quotas.SpaceQuotaRefresherChore.chore(SpaceQuotaRefresherChore.java:84)
at org.apache.hadoop.hbase.ScheduledChore.run(ScheduledChore.java:186)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308)
at
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180)
at
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294)
at
org.apache.hadoop.hbase.JitterScheduledThreadPoolExecutorImpl$JitteredRunnableScheduledFuture.run(JitterScheduledThreadPoolExecutorImpl.java:111)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:748)
```
> TableNotFoundException: hbase:quota is thrown when region server is
> restarted.
> ---
>
> Key: HBASE-22944
> URL: https://issues.apache.org/jira/browse/HBASE-22944
> Project: HBase
> Issue Type: Bug
> Components: Quotas
>Reporter: Shardul Singh
>Assignee: Shardul Singh
>Priority: Minor
>
> SpaceQuotaRefresherChore does not checks whether hbase:quota table is
> present,before starting its operation which sometimes may result to
> TableNotFoundExceptions like below.
> This is because master has not created the hbase:quota table and
> SpaceQuotaRefresherChore is running.
> {code:java}
> java.io.UncheckedIOException: org.apache.hadoop.hbase.
> {quote}TableNotFoundException: hbase:quota
> {quote}
> at
> org.apache.hadoop.hbase.client.ResultScanner$1.hasNext(ResultScanner.java:55)
> at
> org.apache.hadoop.hbase.quotas.SpaceQuotaRefresherChore.fetchSnapshotsFromQuotaTable(SpaceQuotaRefresherChore.java:170)
> at
> org.apache.hadoop.hbase.quotas.SpaceQuotaRefresherChore.chore(SpaceQuotaRefresherChore.java:84)
> at org.apache.hadoop.hbase.ScheduledChore.run(ScheduledChore.java:186)
> at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
> at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308)
> at
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180)
> at
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294)
> at
> org.apache.hadoop.hbase.Jit
[jira] [Updated] (HBASE-22944) TableNotFoundException: hbase:quota is thrown when region server is restarted.
[
https://issues.apache.org/jira/browse/HBASE-22944?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Shardul Singh updated HBASE-22944:
--
Description:
SpaceQuotaRefresherChore does not checks whether hbase:quota table is
present,before starting its operation which sometimes may result to
TableNotFoundExceptions like below.
This is because master has not created the hbase:quota table and
SpaceQuotaRefresherChore is running.
```
java.io.UncheckedIOException: org.apache.hadoop.hbase.
{quote}TableNotFoundException: hbase:quota
{quote}
at org.apache.hadoop.hbase.client.ResultScanner$1.hasNext(ResultScanner.java:55)
at
org.apache.hadoop.hbase.quotas.SpaceQuotaRefresherChore.fetchSnapshotsFromQuotaTable(SpaceQuotaRefresherChore.java:170)
at
org.apache.hadoop.hbase.quotas.SpaceQuotaRefresherChore.chore(SpaceQuotaRefresherChore.java:84)
at org.apache.hadoop.hbase.ScheduledChore.run(ScheduledChore.java:186)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308)
at
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180)
at
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294)
at
org.apache.hadoop.hbase.JitterScheduledThreadPoolExecutorImpl$JitteredRunnableScheduledFuture.run(JitterScheduledThreadPoolExecutorImpl.java:111)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:748)
```
was:
SpaceQuotaRefresherChore does not checks whether hbase:quota table is
present,before starting its operation which sometimes may result to
TableNotFoundExceptions like below.
This is becuase master has not created the hbase:quota table and
SpaceQuotaRefresherChore is running.
```
java.io.UncheckedIOException: org.apache.hadoop.hbase.
{quote}TableNotFoundException: hbase:quota
{quote}
at
org.apache.hadoop.hbase.client.ResultScanner$1.hasNext(ResultScanner.java:55)
at
org.apache.hadoop.hbase.quotas.SpaceQuotaRefresherChore.fetchSnapshotsFromQuotaTable(SpaceQuotaRefresherChore.java:170)
at
org.apache.hadoop.hbase.quotas.SpaceQuotaRefresherChore.chore(SpaceQuotaRefresherChore.java:84)
at org.apache.hadoop.hbase.ScheduledChore.run(ScheduledChore.java:186)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308)
at
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180)
at
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294)
at
org.apache.hadoop.hbase.JitterScheduledThreadPoolExecutorImpl$JitteredRunnableScheduledFuture.run(JitterScheduledThreadPoolExecutorImpl.java:111)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:748)
```
> TableNotFoundException: hbase:quota is thrown when region server is
> restarted.
> ---
>
> Key: HBASE-22944
> URL: https://issues.apache.org/jira/browse/HBASE-22944
> Project: HBase
> Issue Type: Bug
> Components: Quotas
>Reporter: Shardul Singh
>Assignee: Shardul Singh
>Priority: Minor
>
> SpaceQuotaRefresherChore does not checks whether hbase:quota table is
> present,before starting its operation which sometimes may result to
> TableNotFoundExceptions like below.
> This is because master has not created the hbase:quota table and
> SpaceQuotaRefresherChore is running.
> ```
> java.io.UncheckedIOException: org.apache.hadoop.hbase.
> {quote}TableNotFoundException: hbase:quota
> {quote}
> at
> org.apache.hadoop.hbase.client.ResultScanner$1.hasNext(ResultScanner.java:55)
> at
> org.apache.hadoop.hbase.quotas.SpaceQuotaRefresherChore.fetchSnapshotsFromQuotaTable(SpaceQuotaRefresherChore.java:170)
> at
> org.apache.hadoop.hbase.quotas.SpaceQuotaRefresherChore.chore(SpaceQuotaRefresherChore.java:84)
> at org.apache.hadoop.hbase.ScheduledChore.run(ScheduledChore.java:186)
> at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
> at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308)
> at
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180)
> at
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294)
> at
> org.apache.hadoop.hbase.JitterScheduledThr
[jira] [Created] (HBASE-22944) TableNotFoundException: hbase:quota is thrown when region server is restarted.
Shardul Singh created HBASE-22944:
-
Summary: TableNotFoundException: hbase:quota is thrown when
region server is restarted.
Key: HBASE-22944
URL: https://issues.apache.org/jira/browse/HBASE-22944
Project: HBase
Issue Type: Bug
Components: Quotas
Reporter: Shardul Singh
Assignee: Shardul Singh
SpaceQuotaRefresherChore does not checks whether hbase:quota table is
present,before starting its operation which sometimes may result to
TableNotFoundExceptions like below.
This is becuase master has not created the hbase:quota table and
SpaceQuotaRefresherChore is running.
```
java.io.UncheckedIOException: org.apache.hadoop.hbase.
{quote}TableNotFoundException: hbase:quota
{quote}
at
org.apache.hadoop.hbase.client.ResultScanner$1.hasNext(ResultScanner.java:55)
at
org.apache.hadoop.hbase.quotas.SpaceQuotaRefresherChore.fetchSnapshotsFromQuotaTable(SpaceQuotaRefresherChore.java:170)
at
org.apache.hadoop.hbase.quotas.SpaceQuotaRefresherChore.chore(SpaceQuotaRefresherChore.java:84)
at org.apache.hadoop.hbase.ScheduledChore.run(ScheduledChore.java:186)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308)
at
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180)
at
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294)
at
org.apache.hadoop.hbase.JitterScheduledThreadPoolExecutorImpl$JitteredRunnableScheduledFuture.run(JitterScheduledThreadPoolExecutorImpl.java:111)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:748)
```
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
[jira] [Commented] (HBASE-22881) Fix non-daemon threads in hbase server implementation
[
https://issues.apache.org/jira/browse/HBASE-22881?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16918303#comment-16918303
]
Hudson commented on HBASE-22881:
Results for branch branch-2
[build #2203 on
builds.a.o|https://builds.apache.org/job/HBase%20Nightly/job/branch-2/2203/]:
(x) *{color:red}-1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2/2203//General_Nightly_Build_Report/]
(x) {color:red}-1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2/2203//JDK8_Nightly_Build_Report_(Hadoop2)/]
(x) {color:red}-1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2/2203//JDK8_Nightly_Build_Report_(Hadoop3)/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
(/) {color:green}+1 client integration test{color}
> Fix non-daemon threads in hbase server implementation
> -
>
> Key: HBASE-22881
> URL: https://issues.apache.org/jira/browse/HBASE-22881
> Project: HBase
> Issue Type: Bug
> Components: master
>Reporter: Xiaolin Ha
>Assignee: Xiaolin Ha
>Priority: Major
> Fix For: 3.0.0, 2.3.0, 2.2.2
>
>
> "pool-8-thread-3" #7252 prio=5 os_prio=0 tid=0x7f91040044c0 nid=0xd71e
> waiting on condition [0x7f8f4d209000]
>java.lang.Thread.State: WAITING (parking)
> at sun.misc.Unsafe.park(Native Method)
> - parking to wait for <0x0005c0e49ed0> (a
> java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
> at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
> at
> java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
> at
> java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
> at
> java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
> at java.lang.Thread.run(Thread.java:748)
>Locked ownable synchronizers:
> - None
> "pool-8-thread-2" #7251 prio=5 os_prio=0 tid=0x7f910c010be0 nid=0xd71d
> waiting on condition [0x7f8f4daab000]
>java.lang.Thread.State: WAITING (parking)
> at sun.misc.Unsafe.park(Native Method)
> - parking to wait for <0x0005c0e49ed0> (a
> java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
> at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
> at
> java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
> at
> java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
> at
> java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
> at java.lang.Thread.run(Thread.java:748)
>Locked ownable synchronizers:
> - None
> "pool-8-thread-1" #7250 prio=5 os_prio=0 tid=0x7f9119d0 nid=0xd71c
> waiting on condition [0x7f8f4da6a000]
>java.lang.Thread.State: WAITING (parking)
> at sun.misc.Unsafe.park(Native Method)
> - parking to wait for <0x0005c0e49ed0> (a
> java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
> at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
> at
> java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
> at
> java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
> at
> java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
> at java.lang.Thread.run(Thread.java:748)
>Locked ownable synchronizers:
> - None
> "pool-5-thread-3" #7248 prio=5 os_prio=0 tid=0x7f9238005ad0 nid=0xd71a
> waiting on condition [0x7f8f4cb65000]
>java.lang.Thread.State: WAITING (parking)
> at sun.misc.Unsafe.park(Native Method)
> - parking to wait for <0x0
[jira] [Commented] (HBASE-20821) Re-creating a dropped namespace and contained table inherits previously set space quota settings
[
https://issues.apache.org/jira/browse/HBASE-20821?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16918285#comment-16918285
]
Shardul Singh commented on HBASE-20821:
---
Yeah sure. no problem. Will raise a PR for each one. Thanks!
> Re-creating a dropped namespace and contained table inherits previously set
> space quota settings
>
>
> Key: HBASE-20821
> URL: https://issues.apache.org/jira/browse/HBASE-20821
> Project: HBase
> Issue Type: Bug
>Affects Versions: 3.0.0, 2.1.0
>Reporter: Nihal Jain
>Assignee: Shardul Singh
>Priority: Major
> Labels: Quota, Space
> Fix For: 3.0.0, 2.1.0
>
> Attachments: HBASE-20821.master.001.patch,
> HBASE-20821.master.002.patch, HBASE-20821.master.003.patch
>
>
> As demonstarted in
> [HBASE-20662.master.002.patch|https://issues.apache.org/jira/secure/attachment/12927187/HBASE-20662.master.002.patch]
> re-creating a dropped namespace and contained table inherits previously set
> space quota settings.
> *Steps:*
> * Create a namespace and a table in it
> * Set space quota on namespace
> * Violate namespace quota
> * Drop table and then namespace
> * Re create same namespace and same table
> * Put data into the table (more than the previosuly set namespace quota
> limit)
> {code:java}
> private void setQuotaAndThenDropNamespace(final String namespace,
> SpaceViolationPolicy policy)
> throws Exception {
> Put put = new Put(Bytes.toBytes("to_reject"));
> put.addColumn(Bytes.toBytes(SpaceQuotaHelperForTests.F1),
> Bytes.toBytes("to"),
> Bytes.toBytes("reject"));
> createNamespaceIfNotExist(TEST_UTIL.getAdmin(), namespace);
> // Do puts until we violate space policy
> final TableName tn =
> writeUntilNSSpaceViolationAndVerifyViolation(namespace, policy, put);
> // Now, drop the table
> TEST_UTIL.deleteTable(tn);
> LOG.debug("Successfully deleted table ", tn);
> // Now, drop the namespace
> TEST_UTIL.getAdmin().deleteNamespace(namespace);
> LOG.debug("Successfully deleted the namespace ", namespace);
> // Now re-create the namespace
> createNamespaceIfNotExist(TEST_UTIL.getAdmin(), namespace);
> LOG.debug("Successfully re-created the namespace ", namespace);
> TEST_UTIL.createTable(tn, Bytes.toBytes(SpaceQuotaHelperForTests.F1));
> LOG.debug("Successfully re-created table ", tn);
> // Put some rows now: should not violate as namespace/quota was dropped
> verifyNoViolation(policy, tn, put);
> }
> {code}
> *Expected*: SpaceQuota settings should not exist on the newly re-created
> table and we should be able to put limit less data into the table
> *Actual:* We fail to put data into newly created table as SpaceQuota settings
> (systematically created due to previously added namespace space quota) exist
> on table
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
[GitHub] [hbase] jatsakthi commented on issue #553: HBASE-22939 SpaceQuotas- Bulkload from different hdfs failed when space quotas are turned on.
jatsakthi commented on issue #553: HBASE-22939 SpaceQuotas- Bulkload from different hdfs failed when space quotas are turned on. URL: https://github.com/apache/hbase/pull/553#issuecomment-526023090 Also, may I ask what kind of testing did you perform with the patch? Thanks. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [hbase] Apache-HBase commented on issue #547: HBASE-22899 logging improvements for snapshot operations w/large manifests
Apache-HBase commented on issue #547: HBASE-22899 logging improvements for snapshot operations w/large manifests URL: https://github.com/apache/hbase/pull/547#issuecomment-526017461 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | 0 | reexec | 70 | Docker mode activated. | ||| _ Prechecks _ | | +1 | dupname | 0 | No case conflicting files found. | | +1 | hbaseanti | 0 | Patch does not have any anti-patterns. | | +1 | @author | 0 | The patch does not contain any @author tags. | | -0 | test4tests | 0 | The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. | ||| _ master Compile Tests _ | | +1 | mvninstall | 354 | master passed | | +1 | compile | 55 | master passed | | +1 | checkstyle | 89 | master passed | | +1 | shadedjars | 300 | branch has no errors when building our shaded downstream artifacts. | | +1 | javadoc | 36 | master passed | | 0 | spotbugs | 254 | Used deprecated FindBugs config; considering switching to SpotBugs. | | +1 | findbugs | 252 | master passed | ||| _ Patch Compile Tests _ | | +1 | mvninstall | 330 | the patch passed | | +1 | compile | 57 | the patch passed | | +1 | javac | 57 | the patch passed | | +1 | checkstyle | 89 | the patch passed | | +1 | whitespace | 0 | The patch has no whitespace issues. | | +1 | shadedjars | 300 | patch has no errors when building our shaded downstream artifacts. | | +1 | hadoopcheck | 1035 | Patch does not cause any errors with Hadoop 2.8.5 2.9.2 or 3.1.2. | | +1 | javadoc | 36 | the patch passed | | +1 | findbugs | 267 | the patch passed | ||| _ Other Tests _ | | -1 | unit | 15151 | hbase-server in the patch failed. | | +1 | asflicense | 27 | The patch does not generate ASF License warnings. | | | | 18862 | | | Reason | Tests | |---:|:--| | Failed junit tests | hadoop.hbase.client.TestFromClientSide3 | | Subsystem | Report/Notes | |--:|:-| | Docker | Client=18.09.7 Server=18.09.7 base: https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-547/4/artifact/out/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/547 | | Optional Tests | dupname asflicense javac javadoc unit spotbugs findbugs shadedjars hadoopcheck hbaseanti checkstyle compile | | uname | Linux 1e4cf2f40257 4.15.0-52-generic #56-Ubuntu SMP Tue Jun 4 22:49:08 UTC 2019 x86_64 GNU/Linux | | Build tool | maven | | Personality | /home/jenkins/jenkins-slave/workspace/HBase-PreCommit-GitHub-PR_PR-547/out/precommit/personality/provided.sh | | git revision | master / 56980fb097 | | Default Java | 1.8.0_181 | | unit | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-547/4/artifact/out/patch-unit-hbase-server.txt | | Test Results | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-547/4/testReport/ | | Max. process+thread count | 4771 (vs. ulimit of 1) | | modules | C: hbase-server U: hbase-server | | Console output | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-547/4/console | | versions | git=2.11.0 maven=2018-06-17T18:33:14Z) findbugs=3.1.11 | | Powered by | Apache Yetus 0.10.0 http://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [hbase-operator-tools] asf-ci commented on issue #16: HBASE-22825 [HBCK2] Add a client-side to hbase-operator-tools that ca…
asf-ci commented on issue #16: HBASE-22825 [HBCK2] Add a client-side to hbase-operator-tools that ca… URL: https://github.com/apache/hbase-operator-tools/pull/16#issuecomment-526013709 Refer to this link for build results (access rights to CI server needed): https://builds.apache.org/job/PreCommit-HBASE-OPERATOR-TOOLS-Build/67/ This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Commented] (HBASE-20821) Re-creating a dropped namespace and contained table inherits previously set space quota settings
[
https://issues.apache.org/jira/browse/HBASE-20821?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16918252#comment-16918252
]
Sakthi commented on HBASE-20821:
[~shardulsingh], as we are trying to encourage people to use PRs in hbase, do
you mind creating PRs for this one including the other 3 that you are working
on? (HBASE-22342, HBASE-22141, HBASE-22013)? I feel PRs are better for
reviewing & modifications. :)
> Re-creating a dropped namespace and contained table inherits previously set
> space quota settings
>
>
> Key: HBASE-20821
> URL: https://issues.apache.org/jira/browse/HBASE-20821
> Project: HBase
> Issue Type: Bug
>Affects Versions: 3.0.0, 2.1.0
>Reporter: Nihal Jain
>Assignee: Shardul Singh
>Priority: Major
> Labels: Quota, Space
> Fix For: 3.0.0, 2.1.0
>
> Attachments: HBASE-20821.master.001.patch,
> HBASE-20821.master.002.patch, HBASE-20821.master.003.patch
>
>
> As demonstarted in
> [HBASE-20662.master.002.patch|https://issues.apache.org/jira/secure/attachment/12927187/HBASE-20662.master.002.patch]
> re-creating a dropped namespace and contained table inherits previously set
> space quota settings.
> *Steps:*
> * Create a namespace and a table in it
> * Set space quota on namespace
> * Violate namespace quota
> * Drop table and then namespace
> * Re create same namespace and same table
> * Put data into the table (more than the previosuly set namespace quota
> limit)
> {code:java}
> private void setQuotaAndThenDropNamespace(final String namespace,
> SpaceViolationPolicy policy)
> throws Exception {
> Put put = new Put(Bytes.toBytes("to_reject"));
> put.addColumn(Bytes.toBytes(SpaceQuotaHelperForTests.F1),
> Bytes.toBytes("to"),
> Bytes.toBytes("reject"));
> createNamespaceIfNotExist(TEST_UTIL.getAdmin(), namespace);
> // Do puts until we violate space policy
> final TableName tn =
> writeUntilNSSpaceViolationAndVerifyViolation(namespace, policy, put);
> // Now, drop the table
> TEST_UTIL.deleteTable(tn);
> LOG.debug("Successfully deleted table ", tn);
> // Now, drop the namespace
> TEST_UTIL.getAdmin().deleteNamespace(namespace);
> LOG.debug("Successfully deleted the namespace ", namespace);
> // Now re-create the namespace
> createNamespaceIfNotExist(TEST_UTIL.getAdmin(), namespace);
> LOG.debug("Successfully re-created the namespace ", namespace);
> TEST_UTIL.createTable(tn, Bytes.toBytes(SpaceQuotaHelperForTests.F1));
> LOG.debug("Successfully re-created table ", tn);
> // Put some rows now: should not violate as namespace/quota was dropped
> verifyNoViolation(policy, tn, put);
> }
> {code}
> *Expected*: SpaceQuota settings should not exist on the newly re-created
> table and we should be able to put limit less data into the table
> *Actual:* We fail to put data into newly created table as SpaceQuota settings
> (systematically created due to previously added namespace space quota) exist
> on table
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
[jira] [Commented] (HBASE-22013) SpaceQuotas - getNumRegions() returning wrong number of regions due to region replicas
[ https://issues.apache.org/jira/browse/HBASE-22013?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16918249#comment-16918249 ] Sakthi commented on HBASE-22013: [~shardulsingh], the change looks good. A suggestion: I think as we are adding 16 new tests (in 4 test functions), this qualifies to be present in a differnt file. Maybe TestSpaceQuotasWithRegionReplicas? As this adds around ~5 mins to TestBasic... > SpaceQuotas - getNumRegions() returning wrong number of regions due to region > replicas > -- > > Key: HBASE-22013 > URL: https://issues.apache.org/jira/browse/HBASE-22013 > Project: HBase > Issue Type: Bug >Reporter: Ajeet Rai >Assignee: Shardul Singh >Priority: Major > Labels: Quota, Space > Fix For: 2.1.0 > > Attachments: HBASE-22013.branch-2.1.001.patch, > HBASE-22013.master.001.patch, HBASE-22013.master.002.patch, > HBASE-22013.master.003.patch, hbase-22013.branch-2.001.patch, > hbase-22013.branch-2.2.001.patch > > > Space Quota: Space Quota Issue: If a table is created with region replica > then quota calculation is not happening > Steps: > 1: Create a table with 100 regions with region replica 3 > 2: Observe that 'hbase:quota' table doesn't have entry of usage for this > table So In UI only policy Limit and Policy is shown but not Usage and State. > Reason: > It looks like File system utilization core is sending data of 100 reasons > but not the size of region replicas. > But in quota observer chore, it is considering total region(actual regions+ > replica reasons) > So the ratio of reported regions is less then configured > percentRegionsReportedThreshold. > SO quota calculation is not happening -- This message was sent by Atlassian Jira (v8.3.2#803003)
[jira] [Updated] (HBASE-22760) Stop/Resume Snapshot Auto-Cleanup activity with shell command
[ https://issues.apache.org/jira/browse/HBASE-22760?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Viraj Jasani updated HBASE-22760: - Affects Version/s: (was: 1.4.11) (was: 2.2.1) > Stop/Resume Snapshot Auto-Cleanup activity with shell command > - > > Key: HBASE-22760 > URL: https://issues.apache.org/jira/browse/HBASE-22760 > Project: HBase > Issue Type: Improvement > Components: Admin, shell, snapshots >Affects Versions: 3.0.0, 1.5.0, 2.3.0 >Reporter: Viraj Jasani >Assignee: Viraj Jasani >Priority: Major > Fix For: 3.0.0, 1.5.0, 2.3.0, 2.2.1, 1.4.11, 2.1.7 > > Attachments: HBASE-22760.master.003.patch, > HBASE-22760.master.004.patch, HBASE-22760.master.005.patch, > HBASE-22760.master.008.patch, HBASE-22760.master.009.patch > > > For any scheduled snapshot backup activity, we would like to disable > auto-cleaner for snapshot based on TTL. However, as per HBASE-22648 we have a > config to disable snapshot auto-cleaner: > hbase.master.cleaner.snapshot.disable, which would take effect only upon > HMaster restart just similar to any other hbase-site configs. > For any running cluster, we should be able to stop/resume auto-cleanup > activity for snapshot based on shell command. Something similar to below > command should be able to stop/start cleanup chore: > hbase(main):001:0> snapshot_auto_cleanup_switch false (disable > auto-cleaner) > hbase(main):001:0> snapshot_auto_cleanup_switch true (enable auto-cleaner) -- This message was sent by Atlassian Jira (v8.3.2#803003)
[jira] [Updated] (HBASE-22760) Stop/Resume Snapshot Auto-Cleanup activity with shell command
[ https://issues.apache.org/jira/browse/HBASE-22760?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Viraj Jasani updated HBASE-22760: - Fix Version/s: (was: 2.1.6) 2.1.7 > Stop/Resume Snapshot Auto-Cleanup activity with shell command > - > > Key: HBASE-22760 > URL: https://issues.apache.org/jira/browse/HBASE-22760 > Project: HBase > Issue Type: Improvement > Components: Admin, shell, snapshots >Affects Versions: 3.0.0, 1.5.0, 2.3.0, 2.2.1, 1.4.11 >Reporter: Viraj Jasani >Assignee: Viraj Jasani >Priority: Major > Fix For: 3.0.0, 1.5.0, 2.3.0, 2.2.1, 1.4.11, 2.1.7 > > Attachments: HBASE-22760.master.003.patch, > HBASE-22760.master.004.patch, HBASE-22760.master.005.patch, > HBASE-22760.master.008.patch, HBASE-22760.master.009.patch > > > For any scheduled snapshot backup activity, we would like to disable > auto-cleaner for snapshot based on TTL. However, as per HBASE-22648 we have a > config to disable snapshot auto-cleaner: > hbase.master.cleaner.snapshot.disable, which would take effect only upon > HMaster restart just similar to any other hbase-site configs. > For any running cluster, we should be able to stop/resume auto-cleanup > activity for snapshot based on shell command. Something similar to below > command should be able to stop/start cleanup chore: > hbase(main):001:0> snapshot_auto_cleanup_switch false (disable > auto-cleaner) > hbase(main):001:0> snapshot_auto_cleanup_switch true (enable auto-cleaner) -- This message was sent by Atlassian Jira (v8.3.2#803003)
[jira] [Updated] (HBASE-22760) Stop/Resume Snapshot Auto-Cleanup activity with shell command
[ https://issues.apache.org/jira/browse/HBASE-22760?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Viraj Jasani updated HBASE-22760: - Fix Version/s: 2.1.6 > Stop/Resume Snapshot Auto-Cleanup activity with shell command > - > > Key: HBASE-22760 > URL: https://issues.apache.org/jira/browse/HBASE-22760 > Project: HBase > Issue Type: Improvement > Components: Admin, shell, snapshots >Affects Versions: 3.0.0, 1.5.0, 2.3.0, 2.2.1, 1.4.11 >Reporter: Viraj Jasani >Assignee: Viraj Jasani >Priority: Major > Fix For: 3.0.0, 1.5.0, 2.3.0, 2.2.1, 2.1.6, 1.4.11 > > Attachments: HBASE-22760.master.003.patch, > HBASE-22760.master.004.patch, HBASE-22760.master.005.patch, > HBASE-22760.master.008.patch, HBASE-22760.master.009.patch > > > For any scheduled snapshot backup activity, we would like to disable > auto-cleaner for snapshot based on TTL. However, as per HBASE-22648 we have a > config to disable snapshot auto-cleaner: > hbase.master.cleaner.snapshot.disable, which would take effect only upon > HMaster restart just similar to any other hbase-site configs. > For any running cluster, we should be able to stop/resume auto-cleanup > activity for snapshot based on shell command. Something similar to below > command should be able to stop/start cleanup chore: > hbase(main):001:0> snapshot_auto_cleanup_switch false (disable > auto-cleaner) > hbase(main):001:0> snapshot_auto_cleanup_switch true (enable auto-cleaner) -- This message was sent by Atlassian Jira (v8.3.2#803003)
[GitHub] [hbase] jatsakthi commented on issue #552: HBASE-22893
jatsakthi commented on issue #552: HBASE-22893 URL: https://github.com/apache/hbase/pull/552#issuecomment-526004427 @ravowlga123 as the original commit wasn't reverted & the plan is to include this commit as addendum, the patch isn't applicable. Could you please just include the doc change in this patch, so that it's applicable? This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Commented] (HBASE-22932) Add rs group management methods in Admin and AsyncAdmin
[ https://issues.apache.org/jira/browse/HBASE-22932?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16918241#comment-16918241 ] Xiaolin Ha commented on HBASE-22932: Can I work on this? > Add rs group management methods in Admin and AsyncAdmin > --- > > Key: HBASE-22932 > URL: https://issues.apache.org/jira/browse/HBASE-22932 > Project: HBase > Issue Type: Sub-task > Components: Admin, asyncclient, Client >Reporter: Duo Zhang >Priority: Major > -- This message was sent by Atlassian Jira (v8.3.2#803003)
[GitHub] [hbase] brfrn169 commented on a change in pull request #476: HBASE-11062 hbtop
brfrn169 commented on a change in pull request #476: HBASE-11062 hbtop
URL: https://github.com/apache/hbase/pull/476#discussion_r318866763
##
File path:
hbase-hbtop/src/main/java/org/apache/hadoop/hbase/hbtop/terminal/KeyPress.java
##
@@ -0,0 +1,128 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.hadoop.hbase.hbtop.terminal;
+
+import edu.umd.cs.findbugs.annotations.Nullable;
+import java.util.Objects;
+import org.apache.yetus.audience.InterfaceAudience;
+
+
+/**
+ * Represents the user pressing a key on the keyboard.
+ */
+@InterfaceAudience.Private
+public class KeyPress {
+ public enum Type {
+Character,
+Escape,
+Backspace,
+ArrowLeft,
+ArrowRight,
+ArrowUp,
+ArrowDown,
+Insert,
+Delete,
+Home,
+End,
+PageUp,
+PageDown,
+ReverseTab,
+Tab,
+Enter,
+F1,
+F2,
+F3,
+F4,
+F5,
+F6,
+F7,
+F8,
+F9,
+F10,
+F11,
+F12,
+Unknown
+ }
Review comment:
I'm going to retry to find a good terminal lib after it's committed.
Resolving this.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [hbase] sunhelly commented on issue #387: HBASE-22642 Make move operations of RSGroup idempotent
sunhelly commented on issue #387: HBASE-22642 Make move operations of RSGroup idempotent URL: https://github.com/apache/hbase/pull/387#issuecomment-525997893 Hi @infraio , is there any other concerns about this PR? Thanks. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Commented] (HBASE-22881) Fix non-daemon threads in hbase server implementation
[ https://issues.apache.org/jira/browse/HBASE-22881?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16918228#comment-16918228 ] Xiaolin Ha commented on HBASE-22881: Hi，[~stack], I have added a PR to branch-2.1. Please review it. Thanks. > Fix non-daemon threads in hbase server implementation > - > > Key: HBASE-22881 > URL: https://issues.apache.org/jira/browse/HBASE-22881 > Project: HBase > Issue Type: Bug > Components: master >Reporter: Xiaolin Ha >Assignee: Xiaolin Ha >Priority: Major > Fix For: 3.0.0, 2.3.0, 2.2.2 > > > "pool-8-thread-3" #7252 prio=5 os_prio=0 tid=0x7f91040044c0 nid=0xd71e > waiting on condition [0x7f8f4d209000] >java.lang.Thread.State: WAITING (parking) > at sun.misc.Unsafe.park(Native Method) > - parking to wait for <0x0005c0e49ed0> (a > java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject) > at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) > at > java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) > at > java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) > at > java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) > at > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) > at > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) > at java.lang.Thread.run(Thread.java:748) >Locked ownable synchronizers: > - None > "pool-8-thread-2" #7251 prio=5 os_prio=0 tid=0x7f910c010be0 nid=0xd71d > waiting on condition [0x7f8f4daab000] >java.lang.Thread.State: WAITING (parking) > at sun.misc.Unsafe.park(Native Method) > - parking to wait for <0x0005c0e49ed0> (a > java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject) > at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) > at > java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) > at > java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) > at > java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) > at > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) > at > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) > at java.lang.Thread.run(Thread.java:748) >Locked ownable synchronizers: > - None > "pool-8-thread-1" #7250 prio=5 os_prio=0 tid=0x7f9119d0 nid=0xd71c > waiting on condition [0x7f8f4da6a000] >java.lang.Thread.State: WAITING (parking) > at sun.misc.Unsafe.park(Native Method) > - parking to wait for <0x0005c0e49ed0> (a > java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject) > at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) > at > java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) > at > java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) > at > java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) > at > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) > at > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) > at java.lang.Thread.run(Thread.java:748) >Locked ownable synchronizers: > - None > "pool-5-thread-3" #7248 prio=5 os_prio=0 tid=0x7f9238005ad0 nid=0xd71a > waiting on condition [0x7f8f4cb65000] >java.lang.Thread.State: WAITING (parking) > at sun.misc.Unsafe.park(Native Method) > - parking to wait for <0x0005c0ec51e0> (a > java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject) > at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) > at > java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) > at > java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) > at > java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) > at > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) > at > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) > at java.lang.Thread.run(Thread.java:748) -- This message was sent by Atlassian Jira (v8.3.2#803003)
[GitHub] [hbase] sunhelly opened a new pull request #558: HBASE-22881 Fix non-daemon threads in hbase server implementation (#512)
sunhelly opened a new pull request #558: HBASE-22881 Fix non-daemon threads in hbase server implementation (#512) URL: https://github.com/apache/hbase/pull/558 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [hbase] Apache-HBase commented on issue #557: HBASE-22943 Various procedures should not cache log trace level
Apache-HBase commented on issue #557: HBASE-22943 Various procedures should not cache log trace level URL: https://github.com/apache/hbase/pull/557#issuecomment-525995732 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | 0 | reexec | 48 | Docker mode activated. | ||| _ Prechecks _ | | +1 | dupname | 0 | No case conflicting files found. | | +1 | hbaseanti | 0 | Patch does not have any anti-patterns. | | +1 | @author | 0 | The patch does not contain any @author tags. | | -0 | test4tests | 0 | The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. | ||| _ master Compile Tests _ | | +1 | mvninstall | 381 | master passed | | +1 | compile | 59 | master passed | | +1 | checkstyle | 84 | master passed | | +1 | shadedjars | 297 | branch has no errors when building our shaded downstream artifacts. | | +1 | javadoc | 42 | master passed | | 0 | spotbugs | 271 | Used deprecated FindBugs config; considering switching to SpotBugs. | | +1 | findbugs | 269 | master passed | ||| _ Patch Compile Tests _ | | +1 | mvninstall | 353 | the patch passed | | +1 | compile | 58 | the patch passed | | +1 | javac | 58 | the patch passed | | +1 | checkstyle | 76 | the patch passed | | +1 | whitespace | 0 | The patch has no whitespace issues. | | +1 | shadedjars | 270 | patch has no errors when building our shaded downstream artifacts. | | +1 | hadoopcheck | 944 | Patch does not cause any errors with Hadoop 2.8.5 2.9.2 or 3.1.2. | | +1 | javadoc | 38 | the patch passed | | +1 | findbugs | 246 | the patch passed | ||| _ Other Tests _ | | +1 | unit | 9613 | hbase-server in the patch passed. | | +1 | asflicense | 31 | The patch does not generate ASF License warnings. | | | | 13208 | | | Subsystem | Report/Notes | |--:|:-| | Docker | Client=19.03.1 Server=19.03.1 base: https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-557/1/artifact/out/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/557 | | Optional Tests | dupname asflicense javac javadoc unit spotbugs findbugs shadedjars hadoopcheck hbaseanti checkstyle compile | | uname | Linux 910f604e2f9f 4.15.0-58-generic #64-Ubuntu SMP Tue Aug 6 11:12:41 UTC 2019 x86_64 GNU/Linux | | Build tool | maven | | Personality | /home/jenkins/jenkins-slave/workspace/HBase-PreCommit-GitHub-PR_PR-557/out/precommit/personality/provided.sh | | git revision | master / 56980fb097 | | Default Java | 1.8.0_181 | | Test Results | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-557/1/testReport/ | | Max. process+thread count | 4805 (vs. ulimit of 1) | | modules | C: hbase-server U: hbase-server | | Console output | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-557/1/console | | versions | git=2.11.0 maven=2018-06-17T18:33:14Z) findbugs=3.1.11 | | Powered by | Apache Yetus 0.10.0 http://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [hbase] openinx commented on issue #549: HBASE-22912 [Backport] HBASE-22867 to branch-1 to avoid ForkJoinPool to spawn thousands of threads
openinx commented on issue #549: HBASE-22912 [Backport] HBASE-22867 to branch-1 to avoid ForkJoinPool to spawn thousands of threads URL: https://github.com/apache/hbase/pull/549#issuecomment-525990107 Ping @Reidd @apurtell @Apache9 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [hbase] Apache-HBase commented on issue #556: HBASE-22941 merge operation returns parent regions in random order
Apache-HBase commented on issue #556: HBASE-22941 merge operation returns parent regions in random order URL: https://github.com/apache/hbase/pull/556#issuecomment-525984991 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | 0 | reexec | 130 | Docker mode activated. | ||| _ Prechecks _ | | +1 | dupname | 1 | No case conflicting files found. | | +1 | hbaseanti | 0 | Patch does not have any anti-patterns. | | +1 | @author | 0 | The patch does not contain any @author tags. | | +1 | test4tests | 0 | The patch appears to include 1 new or modified test files. | ||| _ master Compile Tests _ | | +1 | mvninstall | 512 | master passed | | +1 | compile | 87 | master passed | | +1 | checkstyle | 124 | master passed | | +1 | shadedjars | 406 | branch has no errors when building our shaded downstream artifacts. | | +1 | javadoc | 47 | master passed | | 0 | spotbugs | 342 | Used deprecated FindBugs config; considering switching to SpotBugs. | | +1 | findbugs | 338 | master passed | ||| _ Patch Compile Tests _ | | +1 | mvninstall | 433 | the patch passed | | +1 | compile | 79 | the patch passed | | +1 | javac | 79 | the patch passed | | -1 | checkstyle | 102 | hbase-server: The patch generated 1 new + 17 unchanged - 0 fixed = 18 total (was 17) | | -1 | whitespace | 0 | The patch has 1 line(s) that end in whitespace. Use git apply --whitespace=fix <>. Refer https://git-scm.com/docs/git-apply | | +1 | shadedjars | 409 | patch has no errors when building our shaded downstream artifacts. | | +1 | hadoopcheck | 1436 | Patch does not cause any errors with Hadoop 2.8.5 2.9.2 or 3.1.2. | | +1 | javadoc | 50 | the patch passed | | +1 | findbugs | 377 | the patch passed | ||| _ Other Tests _ | | -1 | unit | 16348 | hbase-server in the patch failed. | | +1 | asflicense | 29 | The patch does not generate ASF License warnings. | | | | 21493 | | | Reason | Tests | |---:|:--| | Failed junit tests | hadoop.hbase.client.TestAdmin1 | | | hadoop.hbase.client.TestFromClientSide3 | | | hadoop.hbase.client.TestAsyncRegionAdminApi2 | | Subsystem | Report/Notes | |--:|:-| | Docker | Client=19.03.1 Server=19.03.1 base: https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-556/1/artifact/out/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/556 | | Optional Tests | dupname asflicense javac javadoc unit spotbugs findbugs shadedjars hadoopcheck hbaseanti checkstyle compile | | uname | Linux 34466454383b 4.15.0-54-generic #58-Ubuntu SMP Mon Jun 24 10:55:24 UTC 2019 x86_64 GNU/Linux | | Build tool | maven | | Personality | /home/jenkins/jenkins-slave/workspace/HBase-PreCommit-GitHub-PR_PR-556/out/precommit/personality/provided.sh | | git revision | master / 56980fb097 | | Default Java | 1.8.0_181 | | checkstyle | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-556/1/artifact/out/diff-checkstyle-hbase-server.txt | | whitespace | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-556/1/artifact/out/whitespace-eol.txt | | unit | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-556/1/artifact/out/patch-unit-hbase-server.txt | | Test Results | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-556/1/testReport/ | | Max. process+thread count | 4717 (vs. ulimit of 1) | | modules | C: hbase-server U: hbase-server | | Console output | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-556/1/console | | versions | git=2.11.0 maven=2018-06-17T18:33:14Z) findbugs=3.1.11 | | Powered by | Apache Yetus 0.10.0 http://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [hbase] Apache-HBase commented on a change in pull request #556: HBASE-22941 merge operation returns parent regions in random order
Apache-HBase commented on a change in pull request #556: HBASE-22941 merge
operation returns parent regions in random order
URL: https://github.com/apache/hbase/pull/556#discussion_r318855531
##
File path:
hbase-server/src/test/java/org/apache/hadoop/hbase/TestSplitMerge.java
##
@@ -104,4 +106,50 @@ public String explainFailure() throws Exception {
.getRegionLocation(Bytes.toBytes(1), true).get().getServerName());
}
}
+
+ @Test
+ public void testMergeRegionOrder() throws Exception {
+
+int regionCount= 20;
+
+TableName tableName = TableName.valueOf("MergeRegionOrder");
+byte[] family = Bytes.toBytes("CF");
+TableDescriptor td = TableDescriptorBuilder.newBuilder(tableName)
+.setColumnFamily(ColumnFamilyDescriptorBuilder.of(family)).build();
+
+byte[][] splitKeys = new byte[regionCount-1][];
+
+for (int c = 0; c < regionCount-1; c++) {
+ splitKeys[c] = Bytes.toBytes(c+1 * 1000);
+}
+
+UTIL.getAdmin().createTable(td, splitKeys);
+UTIL.waitTableAvailable(tableName);
+
+List regions = UTIL.getAdmin().getRegions(tableName);
+
+byte[][] regionNames = new byte[20][];
+for (int c = 0; c < regionCount; c++) {
+ regionNames[c] = regions.get(c).getRegionName();
+}
+
+UTIL.getAdmin().mergeRegionsAsync(regionNames, false).get(60,
TimeUnit.SECONDS);
+
+List mergedRegions =
+MetaTableAccessor.getTableRegions(UTIL.getConnection(), tableName);
+
+assertEquals(1, mergedRegions.size());
+
+RegionInfo mergedRegion = mergedRegions.get(0);
+
+List mergeParentRegions =
MetaTableAccessor.getMergeRegions(UTIL.getConnection(),
+ mergedRegion.getEncodedNameAsBytes());
+
+assertEquals(mergeParentRegions.size(), 20);
+
+for (int c = 0; c < 19; c++) {
+ assertTrue(Bytes.compareTo(mergeParentRegions.get(c).getStartKey(),
mergeParentRegions.get(c+1).getStartKey())<0);
+}
+
Review comment:
whitespace:end of line
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[jira] [Commented] (HBASE-22724) Add a emoji on the vote table for pre commit result on github
[
https://issues.apache.org/jira/browse/HBASE-22724?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16918207#comment-16918207
]
Hudson commented on HBASE-22724:
Results for branch HBASE-22724
[build #7 on
builds.a.o|https://builds.apache.org/job/HBase%20Nightly/job/HBASE-22724/7/]:
(x) *{color:red}-1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://builds.apache.org/job/HBase%20Nightly/job/HBASE-22724/7//General_Nightly_Build_Report/]
(x) {color:red}-1 jdk8 hadoop2 checks{color}
-- Something went wrong running this stage, please [check relevant console
output|https://builds.apache.org/job/HBase%20Nightly/job/HBASE-22724/7//console].
(x) {color:red}-1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://builds.apache.org/job/HBase%20Nightly/job/HBASE-22724/7//JDK8_Nightly_Build_Report_(Hadoop3)/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
(/) {color:green}+1 client integration test{color}
> Add a emoji on the vote table for pre commit result on github
> -
>
> Key: HBASE-22724
> URL: https://issues.apache.org/jira/browse/HBASE-22724
> Project: HBase
> Issue Type: Improvement
>Reporter: Duo Zhang
>Assignee: Duo Zhang
>Priority: Major
>
> [~zghaobac] said that the current vote table on github is not good enough, as
> the colors are almost the same, it is not easy to find out which line is
> broken.
> Since github can not change the color of the text, he suggested that we add a
> column at the left most with some emojis to better notify the developpers.
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
[GitHub] [hbase] VicoWu edited a comment on issue #509: HBASE-22877 WebHDFS based export snapshot will fail if hfile is in archive directory
VicoWu edited a comment on issue #509: HBASE-22877 WebHDFS based export
snapshot will fail if hfile is in archive directory
URL: https://github.com/apache/hbase/pull/509#issuecomment-525982956
@shahrs87
Thanks for you comments;
My current branch is a CDH version hadoop2.6.0-CDH5.16.1;
I previously thought that the hadoop2.6.0-CDH5.16.1; is totallly based on
the apache hadoop 2..6.0, but from this case it seems not;
This is my code of the `HDFSFileSystem.open() for `hadoop2.6.0-CDH5.16.1`:
```
@Override
public FSDataInputStream open(final Path f, final int bufferSize
) throws IOException {
statistics.incrementReadOps(1);
return new FSDataInputStream(new WebHdfsInputStream(f, bufferSize));
}
```
And I find the first occurrence for the `WebHdfsInputSteam.open()` method
occurs in Apache hadoop
https://github.com/apache/hadoop/commit/eda4dbbb7628ce336377d3ce5f26ecbe47d3fe3d
It seems that they are already different from scratch;
Maybe I should communicate with Cloudera ;
Thanks again;
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [hbase] VicoWu commented on issue #509: HBASE-22877 WebHDFS based export snapshot will fail if hfile is in archive directory
VicoWu commented on issue #509: HBASE-22877 WebHDFS based export snapshot will
fail if hfile is in archive directory
URL: https://github.com/apache/hbase/pull/509#issuecomment-525982956
@shahrs87
Thanks for you comments;
My current branch is a CDH version hadoop2.6.0-CDH5.16.1;
I previously thought that the hadoop2.6.0-CDH5.16.1; is totallly based on
the apache hadoop 2..6.0, but from this case it seems not;
This is my code of the `HDFSFileSystem.open() for `hadoop2.6.0-CDH5.16.1`:
@Override
public FSDataInputStream open(final Path f, final int bufferSize
) throws IOException {
statistics.incrementReadOps(1);
return new FSDataInputStream(new WebHdfsInputStream(f, bufferSize));
}
```
And I find the first occurrence for the `WebHdfsInputSteam.open()` method
occurs in Apache hadoop
https://github.com/apache/hadoop/commit/eda4dbbb7628ce336377d3ce5f26ecbe47d3fe3d
It seems that they are already different from scratch;
Maybe I should communicate with Cloudera ;
Thanks again;
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [hbase] Apache-HBase commented on issue #361: HBase-22027: Split non-MR related parts of TokenUtil off into a Clien…
Apache-HBase commented on issue #361: HBase-22027: Split non-MR related parts of TokenUtil off into a Clien… URL: https://github.com/apache/hbase/pull/361#issuecomment-525980273 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | 0 | reexec | 336 | Docker mode activated. | ||| _ Prechecks _ | | +1 | dupname | 0 | No case conflicting files found. | | +1 | hbaseanti | 0 | Patch does not have any anti-patterns. | | +1 | @author | 0 | The patch does not contain any @author tags. | | +1 | test4tests | 0 | The patch appears to include 2 new or modified test files. | ||| _ master Compile Tests _ | | 0 | mvndep | 49 | Maven dependency ordering for branch | | +1 | mvninstall | 630 | master passed | | +1 | compile | 128 | master passed | | +1 | checkstyle | 150 | master passed | | +1 | shadedjars | 433 | branch has no errors when building our shaded downstream artifacts. | | +1 | javadoc | 93 | master passed | | 0 | spotbugs | 377 | Used deprecated FindBugs config; considering switching to SpotBugs. | | +1 | findbugs | 480 | master passed | ||| _ Patch Compile Tests _ | | 0 | mvndep | 22 | Maven dependency ordering for patch | | +1 | mvninstall | 510 | the patch passed | | +1 | compile | 128 | the patch passed | | +1 | javac | 128 | the patch passed | | +1 | checkstyle | 142 | the patch passed | | +1 | whitespace | 0 | The patch has no whitespace issues. | | +1 | shadedjars | 399 | patch has no errors when building our shaded downstream artifacts. | | +1 | hadoopcheck | 1543 | Patch does not cause any errors with Hadoop 2.8.5 2.9.2 or 3.1.2. | | +1 | javadoc | 78 | the patch passed | | +1 | findbugs | 468 | the patch passed | ||| _ Other Tests _ | | +1 | unit | 133 | hbase-client in the patch passed. | | -1 | unit | 16811 | hbase-server in the patch failed. | | +1 | asflicense | 55 | The patch does not generate ASF License warnings. | | | | 23156 | | | Reason | Tests | |---:|:--| | Failed junit tests | hadoop.hbase.replication.TestReplicationKillSlaveRS | | | hadoop.hbase.replication.TestReplicationChangingPeerRegionservers | | | hadoop.hbase.replication.TestSyncReplicationRemoveRemoteWAL | | | hadoop.hbase.replication.TestReplicationSmallTests | | | hadoop.hbase.replication.TestMasterReplication | | | hadoop.hbase.tool.TestBulkLoadHFiles | | | hadoop.hbase.replication.TestNamespaceReplication | | | hadoop.hbase.io.hfile.bucket.TestBucketCache | | | hadoop.hbase.master.TestMasterShutdown | | | hadoop.hbase.replication.TestReplicationDisableInactivePeer | | | hadoop.hbase.tool.TestSecureBulkLoadHFiles | | Subsystem | Report/Notes | |--:|:-| | Docker | Client=19.03.0 Server=19.03.0 base: https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-361/143/artifact/out/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/361 | | Optional Tests | dupname asflicense javac javadoc unit spotbugs findbugs shadedjars hadoopcheck hbaseanti checkstyle compile | | uname | Linux c0cdd61c8980 4.15.0-52-generic #56-Ubuntu SMP Tue Jun 4 22:49:08 UTC 2019 x86_64 GNU/Linux | | Build tool | maven | | Personality | /home/jenkins/jenkins-slave/workspace/HBase-PreCommit-GitHub-PR_PR-361/out/precommit/personality/provided.sh | | git revision | master / 56980fb097 | | Default Java | 1.8.0_181 | | unit | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-361/143/artifact/out/patch-unit-hbase-server.txt | | Test Results | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-361/143/testReport/ | | Max. process+thread count | 4669 (vs. ulimit of 1) | | modules | C: hbase-client hbase-server U: . | | Console output | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-361/143/console | | versions | git=2.11.0 maven=2018-06-17T18:33:14Z) findbugs=3.1.11 | | Powered by | Apache Yetus 0.10.0 http://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [hbase] Apache-HBase commented on issue #510: HBASE-22878 Show table throttle quotas in table jsp
Apache-HBase commented on issue #510: HBASE-22878 Show table throttle quotas in table jsp URL: https://github.com/apache/hbase/pull/510#issuecomment-525970212 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | 0 | reexec | 88 | Docker mode activated. | ||| _ Prechecks _ | | +1 | dupname | 0 | No case conflicting files found. | | +1 | hbaseanti | 0 | Patch does not have any anti-patterns. | | +1 | @author | 0 | The patch does not contain any @author tags. | | -0 | test4tests | 0 | The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. | ||| _ master Compile Tests _ | | 0 | mvndep | 62 | Maven dependency ordering for branch | | +1 | mvninstall | 385 | master passed | | +1 | compile | 96 | master passed | | +1 | checkstyle | 127 | master passed | | +1 | shadedjars | 313 | branch has no errors when building our shaded downstream artifacts. | | +1 | javadoc | 68 | master passed | | 0 | spotbugs | 258 | Used deprecated FindBugs config; considering switching to SpotBugs. | | +1 | findbugs | 333 | master passed | ||| _ Patch Compile Tests _ | | 0 | mvndep | 17 | Maven dependency ordering for patch | | +1 | mvninstall | 338 | the patch passed | | +1 | compile | 85 | the patch passed | | +1 | javac | 85 | the patch passed | | +1 | checkstyle | 125 | the patch passed | | +1 | whitespace | 0 | The patch has no whitespace issues. | | +1 | shadedjars | 322 | patch has no errors when building our shaded downstream artifacts. | | +1 | hadoopcheck | 1071 | Patch does not cause any errors with Hadoop 2.8.5 2.9.2 or 3.1.2. | | +1 | javadoc | 57 | the patch passed | | +1 | findbugs | 344 | the patch passed | ||| _ Other Tests _ | | +1 | unit | 109 | hbase-client in the patch passed. | | -1 | unit | 15196 | hbase-server in the patch failed. | | +1 | asflicense | 78 | The patch does not generate ASF License warnings. | | | | 19646 | | | Reason | Tests | |---:|:--| | Failed junit tests | hadoop.hbase.master.procedure.TestSCPWithReplicasWithoutZKCoordinated | | | hadoop.hbase.master.procedure.TestSCPWithReplicas | | Subsystem | Report/Notes | |--:|:-| | Docker | Client=19.03.1 Server=19.03.1 base: https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-510/34/artifact/out/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/510 | | Optional Tests | dupname asflicense javac javadoc unit spotbugs findbugs shadedjars hadoopcheck hbaseanti checkstyle compile | | uname | Linux 302f2a0e8aab 4.15.0-54-generic #58-Ubuntu SMP Mon Jun 24 10:55:24 UTC 2019 x86_64 GNU/Linux | | Build tool | maven | | Personality | /home/jenkins/jenkins-slave/workspace/HBase-PreCommit-GitHub-PR_PR-510/out/precommit/personality/provided.sh | | git revision | master / 56980fb097 | | Default Java | 1.8.0_181 | | unit | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-510/34/artifact/out/patch-unit-hbase-server.txt | | Test Results | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-510/34/testReport/ | | Max. process+thread count | 5022 (vs. ulimit of 1) | | modules | C: hbase-client hbase-server U: . | | Console output | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-510/34/console | | versions | git=2.11.0 maven=2018-06-17T18:33:14Z) findbugs=3.1.11 | | Powered by | Apache Yetus 0.10.0 http://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [hbase] Apache-HBase commented on issue #549: HBASE-22912 [Backport] HBASE-22867 to branch-1 to avoid ForkJoinPool to spawn thousands of threads
Apache-HBase commented on issue #549: HBASE-22912 [Backport] HBASE-22867 to branch-1 to avoid ForkJoinPool to spawn thousands of threads URL: https://github.com/apache/hbase/pull/549#issuecomment-525968997 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | 0 | reexec | 984 | Docker mode activated. | ||| _ Prechecks _ | | +1 | dupname | 0 | No case conflicting files found. | | +1 | hbaseanti | 0 | Patch does not have any anti-patterns. | | +1 | @author | 0 | The patch does not contain any @author tags. | | -0 | test4tests | 0 | The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. | ||| _ branch-1 Compile Tests _ | | +1 | mvninstall | 484 | branch-1 passed | | +1 | compile | 39 | branch-1 passed | | +1 | checkstyle | 89 | branch-1 passed | | -1 | shadedjars | 13 | branch has 10 errors when building our shaded downstream artifacts. | | +1 | javadoc | 38 | branch-1 passed | | 0 | spotbugs | 140 | Used deprecated FindBugs config; considering switching to SpotBugs. | | +1 | findbugs | 137 | branch-1 passed | ||| _ Patch Compile Tests _ | | +1 | mvninstall | 112 | the patch passed | | +1 | compile | 39 | the patch passed | | +1 | javac | 39 | the patch passed | | +1 | checkstyle | 88 | the patch passed | | +1 | whitespace | 0 | The patch has no whitespace issues. | | -1 | shadedjars | 12 | patch has 10 errors when building our shaded downstream artifacts. | | +1 | hadoopcheck | 277 | Patch does not cause any errors with Hadoop 2.8.5 2.9.2. | | +1 | javadoc | 30 | the patch passed | | +1 | findbugs | 133 | the patch passed | ||| _ Other Tests _ | | +1 | unit | 6723 | hbase-server in the patch passed. | | +1 | asflicense | 30 | The patch does not generate ASF License warnings. | | | | 9432 | | | Subsystem | Report/Notes | |--:|:-| | Docker | Client=19.03.1 Server=19.03.1 base: https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-549/4/artifact/out/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/549 | | Optional Tests | dupname asflicense javac javadoc unit spotbugs findbugs shadedjars hadoopcheck hbaseanti checkstyle compile | | uname | Linux 06d7d0f1f464 4.15.0-58-generic #64-Ubuntu SMP Tue Aug 6 11:12:41 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | /home/jenkins/jenkins-slave/workspace/HBase-PreCommit-GitHub-PR_PR-549/out/precommit/personality/provided.sh | | git revision | branch-1 / 73e0304 | | Default Java | 1.8.0_222 | | shadedjars | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-549/4/artifact/out/branch-shadedjars.txt | | shadedjars | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-549/4/artifact/out/patch-shadedjars.txt | | Test Results | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-549/4/testReport/ | | Max. process+thread count | 4414 (vs. ulimit of 1) | | modules | C: hbase-server U: hbase-server | | Console output | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-549/4/console | | versions | git=1.9.1 maven=3.0.5 findbugs=3.1.0-RC1 | | Powered by | Apache Yetus 0.10.0 http://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [hbase] Apache-HBase commented on issue #476: HBASE-11062 hbtop
Apache-HBase commented on issue #476: HBASE-11062 hbtop URL: https://github.com/apache/hbase/pull/476#issuecomment-525958112 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | 0 | reexec | 80 | Docker mode activated. | ||| _ Prechecks _ | | +1 | dupname | 1 | No case conflicting files found. | | 0 | shelldocs | 1 | Shelldocs was not available. | | +1 | hbaseanti | 0 | Patch does not have any anti-patterns. | | +1 | @author | 1 | The patch does not contain any @author tags. | | +1 | test4tests | 0 | The patch appears to include 22 new or modified test files. | ||| _ master Compile Tests _ | | 0 | mvndep | 33 | Maven dependency ordering for branch | | +1 | mvninstall | 345 | master passed | | +1 | compile | 208 | master passed | | +1 | checkstyle | 186 | master passed | | +1 | shadedjars | 320 | branch has no errors when building our shaded downstream artifacts. | | +1 | javadoc | 193 | master passed | | 0 | spotbugs | 31 | Used deprecated FindBugs config; considering switching to SpotBugs. | | 0 | findbugs | 31 | branch/hbase-assembly no findbugs output file (findbugsXml.xml) | ||| _ Patch Compile Tests _ | | 0 | mvndep | 14 | Maven dependency ordering for patch | | +1 | mvninstall | 329 | the patch passed | | +1 | compile | 194 | the patch passed | | +1 | javac | 194 | the patch passed | | +1 | checkstyle | 171 | the patch passed | | +1 | shellcheck | 1 | There were no new shellcheck issues. | | +1 | whitespace | 0 | The patch has no whitespace issues. | | +1 | xml | 5 | The patch has no ill-formed XML file. | | +1 | shadedjars | 296 | patch has no errors when building our shaded downstream artifacts. | | +1 | hadoopcheck | 1038 | Patch does not cause any errors with Hadoop 2.8.5 2.9.2 or 3.1.2. | | +1 | javadoc | 199 | the patch passed | | 0 | findbugs | 31 | hbase-assembly has no data from findbugs | ||| _ Other Tests _ | | +1 | unit | 19078 | root in the patch passed. | | +1 | asflicense | 95 | The patch does not generate ASF License warnings. | | | | 25108 | | | Subsystem | Report/Notes | |--:|:-| | Docker | Client=18.09.7 Server=18.09.7 base: https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-476/60/artifact/out/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/476 | | Optional Tests | dupname asflicense shellcheck shelldocs javac javadoc unit shadedjars hadoopcheck xml compile spotbugs findbugs hbaseanti checkstyle | | uname | Linux 670b8cbda9b5 4.15.0-52-generic #56-Ubuntu SMP Tue Jun 4 22:49:08 UTC 2019 x86_64 GNU/Linux | | Build tool | maven | | Personality | /home/jenkins/jenkins-slave/workspace/HBase-PreCommit-GitHub-PR_PR-476/out/precommit/personality/provided.sh | | git revision | master / 56980fb097 | | Default Java | 1.8.0_181 | | Test Results | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-476/60/testReport/ | | Max. process+thread count | 4793 (vs. ulimit of 1) | | modules | C: hbase-hbtop . hbase-assembly U: . | | Console output | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-476/60/console | | versions | git=2.11.0 maven=2018-06-17T18:33:14Z) shellcheck=0.4.4 findbugs=3.1.11 | | Powered by | Apache Yetus 0.10.0 http://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [hbase] Apache-HBase commented on issue #555: HBASE-22729 Start RSGroupInfoManager as default
Apache-HBase commented on issue #555: HBASE-22729 Start RSGroupInfoManager as default URL: https://github.com/apache/hbase/pull/555#issuecomment-525951904 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | 0 | reexec | 41 | Docker mode activated. | ||| _ Prechecks _ | | +1 | dupname | 0 | No case conflicting files found. | | +1 | hbaseanti | 0 | Patch does not have any anti-patterns. | | +1 | @author | 0 | The patch does not contain any @author tags. | | +1 | test4tests | 0 | The patch appears to include 1 new or modified test files. | ||| _ HBASE-22514 Compile Tests _ | | +1 | mvninstall | 324 | HBASE-22514 passed | | +1 | compile | 61 | HBASE-22514 passed | | +1 | checkstyle | 84 | HBASE-22514 passed | | +1 | shadedjars | 304 | branch has no errors when building our shaded downstream artifacts. | | +1 | javadoc | 38 | HBASE-22514 passed | | 0 | spotbugs | 261 | Used deprecated FindBugs config; considering switching to SpotBugs. | | +1 | findbugs | 259 | HBASE-22514 passed | ||| _ Patch Compile Tests _ | | +1 | mvninstall | 323 | the patch passed | | +1 | compile | 61 | the patch passed | | +1 | javac | 61 | the patch passed | | +1 | checkstyle | 83 | the patch passed | | +1 | whitespace | 0 | The patch has no whitespace issues. | | +1 | shadedjars | 288 | patch has no errors when building our shaded downstream artifacts. | | +1 | hadoopcheck | 977 | Patch does not cause any errors with Hadoop 2.8.5 2.9.2 or 3.1.2. | | +1 | javadoc | 36 | the patch passed | | +1 | findbugs | 283 | the patch passed | ||| _ Other Tests _ | | -1 | unit | 9915 | hbase-server in the patch failed. | | +1 | asflicense | 30 | The patch does not generate ASF License warnings. | | | | 13510 | | | Reason | Tests | |---:|:--| | Failed junit tests | hadoop.hbase.master.replication.TestTransitPeerSyncReplicationStateProcedureRetry | | Subsystem | Report/Notes | |--:|:-| | Docker | Client=19.03.1 Server=19.03.1 base: https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-555/1/artifact/out/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/555 | | Optional Tests | dupname asflicense javac javadoc unit spotbugs findbugs shadedjars hadoopcheck hbaseanti checkstyle compile | | uname | Linux e766f3832c53 4.15.0-58-generic #64-Ubuntu SMP Tue Aug 6 11:12:41 UTC 2019 x86_64 GNU/Linux | | Build tool | maven | | Personality | /home/jenkins/jenkins-slave/workspace/HBase-PreCommit-GitHub-PR_PR-555/out/precommit/personality/provided.sh | | git revision | HBASE-22514 / 548e8c4ed8 | | Default Java | 1.8.0_181 | | unit | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-555/1/artifact/out/patch-unit-hbase-server.txt | | Test Results | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-555/1/testReport/ | | Max. process+thread count | 4597 (vs. ulimit of 1) | | modules | C: hbase-server U: hbase-server | | Console output | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-555/1/console | | versions | git=2.11.0 maven=2018-06-17T18:33:14Z) findbugs=3.1.11 | | Powered by | Apache Yetus 0.10.0 http://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [hbase] saintstack commented on a change in pull request #556: HBASE-22941 merge operation returns parent regions in random order
saintstack commented on a change in pull request #556: HBASE-22941 merge
operation returns parent regions in random order
URL: https://github.com/apache/hbase/pull/556#discussion_r318818139
##
File path:
hbase-server/src/main/java/org/apache/hadoop/hbase/master/assignment/RegionStateStore.java
##
@@ -263,7 +263,7 @@ public void mergeRegions(RegionInfo child, RegionInfo []
parents, ServerName ser
throws IOException {
TableDescriptor htd = getDescriptor(child.getTable());
boolean globalScope = htd.hasGlobalReplicationScope();
-Map parentSeqNums = new HashMap<>(parents.length);
+Map parentSeqNums = new TreeMap<>();
Review comment:
Better
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[jira] [Commented] (HBASE-22618) Provide a way to have Heterogeneous deployment
[
https://issues.apache.org/jira/browse/HBASE-22618?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16918147#comment-16918147
]
Hudson commented on HBASE-22618:
Results for branch branch-1
[build #1037 on
builds.a.o|https://builds.apache.org/job/HBase%20Nightly/job/branch-1/1037/]:
(x) *{color:red}-1 overall{color}*
details (if available):
(x) {color:red}-1 general checks{color}
-- For more information [see general
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-1/1037//General_Nightly_Build_Report/]
(x) {color:red}-1 jdk7 checks{color}
-- Something went wrong running this stage, please [check relevant console
output|https://builds.apache.org/job/HBase%20Nightly/job/branch-1/1037//console].
(x) {color:red}-1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-1/1037//JDK8_Nightly_Build_Report_(Hadoop2)/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
> Provide a way to have Heterogeneous deployment
> --
>
> Key: HBASE-22618
> URL: https://issues.apache.org/jira/browse/HBASE-22618
> Project: HBase
> Issue Type: Improvement
>Affects Versions: 3.0.0, 2.2.0, 2.2.1, 2.1.6, 1.4.11, 2.1.7
>Reporter: Pierre Zemb
>Assignee: Pierre Zemb
>Priority: Major
> Fix For: 3.0.0, 1.5.0, 2.3.0
>
> Attachments: HBASE-22618.branch-1.001.patch,
> HBASE-22618.branch-1.002.patch, HBASE-22618.branch-2.001.patch,
> HBASE-22618.master.001.patch
>
>
> Hi,
> We wouls like to open the discussion about bringing the possibility to have
> regions deployed on {color:#22}Heterogeneous deployment{color}, i.e Hbase
> cluster running different kind of hardware.
> h2. Why?
> * Cloud deployments means that we may not be able to have the same hardware
> throughout the years
> * Some tables may need special requirements such as SSD whereas others
> should be using hard-drives
> * {color:#22} {color}*in our usecase*{color:#22}(single table,
> dedicated HBase and Hadoop tuned for our usecase, good key
> distribution){color}*, the number of regions per RS was the real limit for
> us*{color:#22}.{color}
> h2. Our usecase
> We found out that *in our usecase*(single table, dedicated HBase and Hadoop
> tuned for our usecase, good key distribution)*, the number of regions per RS
> was the real limit for us*.
> Over the years, due to historical reasons and also the need to benchmark new
> machines, we ended-up with differents groups of hardware: some servers can
> handle only 180 regions, whereas the biggest can handle more than 900.
> Because of such a difference, we had to disable the LoadBalancing to avoid
> the {{roundRobinAssigmnent}}. We developed some internal tooling which are
> responsible for load balancing regions across RegionServers. That was 1.5
> year ago.
> h2. Our Proof-of-concept
> We did work on a Proof-of-concept
> [here|https://github.com/PierreZ/hbase/blob/dev/hbase14/balancer/hbase-server/src/main/java/org/apache/hadoop/hbase/master/balancer/HeterogeneousBalancer.java],
> and some early tests
> [here|https://github.com/PierreZ/hbase/blob/dev/hbase14/balancer/hbase-server/src/main/java/org/apache/hadoop/hbase/master/balancer/HeterogeneousBalancer.java],
>
> [here|https://github.com/PierreZ/hbase/blob/dev/hbase14/balancer/hbase-server/src/test/java/org/apache/hadoop/hbase/master/balancer/TestHeterogeneousBalancerBalance.java],
> and
> [here|https://github.com/PierreZ/hbase/blob/dev/hbase14/balancer/hbase-server/src/test/java/org/apache/hadoop/hbase/master/balancer/TestHeterogeneousBalancerRules.java].
> We wrote the balancer for our use-case, which means that:
> * there is one table
> * there is no region-replica
> * good key dispersion
> * there is no regions on master
> A rule file is loaded before balancing. It contains lines of rules. A rule is
> composed of a regexp for hostname, and a limit. For example, we could have:
>
> {quote}rs[0-9] 200
> rs1[0-9] 50
> {quote}
>
> RegionServers with hostname matching the first rules will have a limit of
> 200, and the others 50. If there's no match, a default is set.
> Thanks to the rule, we have two informations: the max number of regions for
> this cluster, and the rules for each servers. {{HeterogeneousBalancer}} will
> try to balance regions according to their capacity.
> Let's take an example. Let's say that we have 20 RS:
> * 10 RS, named through {{rs0}} to {{rs9}} loaded with 60 regions each, and
> each can handle 200 regions.
> * 10 RS, named through {{rs10}} to {{rs19}} loaded with 60 regions each, and
> each can support 50 regions.
> Based on the following rules:
>
> {quote}rs[0-9] 200
> rs1[0-9] 50
> {quote}
>
> The secon
[GitHub] [hbase] Apache-HBase commented on a change in pull request #551: HBASE-22896 TestHRegion.testFlushMarkersWALFail is flaky
Apache-HBase commented on a change in pull request #551: HBASE-22896
TestHRegion.testFlushMarkersWALFail is flaky
URL: https://github.com/apache/hbase/pull/551#discussion_r318815917
##
File path:
hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestHRegion.java
##
@@ -1233,10 +1232,10 @@ public long getLength() {
this.region = initHRegion(tableName, HConstants.EMPTY_START_ROW,
HConstants.EMPTY_END_ROW, false, Durability.USE_DEFAULT, wal, family);
region.put(put);
-
+
Review comment:
whitespace:end of line
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [hbase] Apache-HBase commented on issue #551: HBASE-22896 TestHRegion.testFlushMarkersWALFail is flaky
Apache-HBase commented on issue #551: HBASE-22896 TestHRegion.testFlushMarkersWALFail is flaky URL: https://github.com/apache/hbase/pull/551#issuecomment-525943627 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | 0 | reexec | 159 | Docker mode activated. | ||| _ Prechecks _ | | +1 | dupname | 0 | No case conflicting files found. | | +1 | hbaseanti | 0 | Patch does not have any anti-patterns. | | +1 | @author | 0 | The patch does not contain any @author tags. | | +1 | test4tests | 0 | The patch appears to include 1 new or modified test files. | ||| _ master Compile Tests _ | | +1 | mvninstall | 432 | master passed | | +1 | compile | 73 | master passed | | +1 | checkstyle | 105 | master passed | | +1 | shadedjars | 357 | branch has no errors when building our shaded downstream artifacts. | | +1 | javadoc | 39 | master passed | | 0 | spotbugs | 276 | Used deprecated FindBugs config; considering switching to SpotBugs. | | +1 | findbugs | 275 | master passed | ||| _ Patch Compile Tests _ | | +1 | mvninstall | 336 | the patch passed | | +1 | compile | 59 | the patch passed | | +1 | javac | 59 | the patch passed | | +1 | checkstyle | 92 | the patch passed | | -1 | whitespace | 0 | The patch has 1 line(s) that end in whitespace. Use git apply --whitespace=fix <>. Refer https://git-scm.com/docs/git-apply | | +1 | shadedjars | 326 | patch has no errors when building our shaded downstream artifacts. | | +1 | hadoopcheck | 1119 | Patch does not cause any errors with Hadoop 2.8.5 2.9.2 or 3.1.2. | | +1 | javadoc | 36 | the patch passed | | +1 | findbugs | 277 | the patch passed | ||| _ Other Tests _ | | -1 | unit | 16557 | hbase-server in the patch failed. | | +1 | asflicense | 25 | The patch does not generate ASF License warnings. | | | | 20688 | | | Reason | Tests | |---:|:--| | Failed junit tests | hadoop.hbase.client.TestSnapshotCloneIndependence | | | hadoop.hbase.client.TestAsyncRegionAdminApi2 | | | hadoop.hbase.replication.TestReplicationKillSlaveRS | | Subsystem | Report/Notes | |--:|:-| | Docker | Client=19.03.1 Server=19.03.1 base: https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-551/2/artifact/out/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/551 | | Optional Tests | dupname asflicense javac javadoc unit spotbugs findbugs shadedjars hadoopcheck hbaseanti checkstyle compile | | uname | Linux e9951fefa59a 4.15.0-48-generic #51-Ubuntu SMP Wed Apr 3 08:28:49 UTC 2019 x86_64 GNU/Linux | | Build tool | maven | | Personality | /home/jenkins/jenkins-slave/workspace/HBase-PreCommit-GitHub-PR_PR-551/out/precommit/personality/provided.sh | | git revision | master / 56980fb097 | | Default Java | 1.8.0_181 | | whitespace | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-551/2/artifact/out/whitespace-eol.txt | | unit | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-551/2/artifact/out/patch-unit-hbase-server.txt | | Test Results | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-551/2/testReport/ | | Max. process+thread count | 4636 (vs. ulimit of 1) | | modules | C: hbase-server U: hbase-server | | Console output | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-551/2/console | | versions | git=2.11.0 maven=2018-06-17T18:33:14Z) findbugs=3.1.11 | | Powered by | Apache Yetus 0.10.0 http://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [hbase] Apache-HBase commented on issue #251: HBASE-22114 Port HBASE-15560 (TinyLFU-based BlockCache) to branch-1
Apache-HBase commented on issue #251: HBASE-22114 Port HBASE-15560 (TinyLFU-based BlockCache) to branch-1 URL: https://github.com/apache/hbase/pull/251#issuecomment-525940997 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | 0 | reexec | 51 | Docker mode activated. | ||| _ Prechecks _ | | +1 | dupname | 0 | No case conflicting files found. | | -1 | hbaseanti | 0 | The patch appears use Hadoop classification instead of HBase. | | +1 | @author | 0 | The patch does not contain any @author tags. | | +1 | test4tests | 0 | The patch appears to include 2 new or modified test files. | ||| _ branch-1 Compile Tests _ | | 0 | mvndep | 355 | Maven dependency ordering for branch | | +1 | mvninstall | 215 | branch-1 passed | | +1 | compile | 97 | branch-1 passed | | +1 | checkstyle | 461 | branch-1 passed | | 0 | refguide | 188 | branch has no errors when building the reference guide. See footer for rendered docs, which you should manually inspect. | | -1 | shadedjars | 13 | branch has 10 errors when building our shaded downstream artifacts. | | +1 | javadoc | 189 | branch-1 passed | | 0 | spotbugs | 437 | Used deprecated FindBugs config; considering switching to SpotBugs. | | 0 | findbugs | 17 | branch/hbase-resource-bundle no findbugs output file (findbugsXml.xml) | ||| _ Patch Compile Tests _ | | 0 | mvndep | 17 | Maven dependency ordering for patch | | +1 | mvninstall | 108 | the patch passed | | +1 | compile | 103 | the patch passed | | +1 | javac | 103 | the patch passed | | +1 | checkstyle | 9 | The patch passed checkstyle in hbase-resource-bundle | | +1 | checkstyle | 24 | The patch passed checkstyle in hbase-common | | +1 | checkstyle | 91 | hbase-server: The patch generated 0 new + 83 unchanged - 11 fixed = 83 total (was 94) | | -1 | checkstyle | 10 | hbase-tinylfu-blockcache: The patch generated 1 new + 0 unchanged - 0 fixed = 1 total (was 0) | | -1 | checkstyle | 312 | root: The patch generated 1 new + 92 unchanged - 11 fixed = 93 total (was 103) | | +1 | checkstyle | 23 | The patch passed checkstyle in hbase-it | | +1 | whitespace | 0 | The patch has no whitespace issues. | | +1 | xml | 7 | The patch has no ill-formed XML file. | | 0 | refguide | 167 | patch has no errors when building the reference guide. See footer for rendered docs, which you should manually inspect. | | -1 | shadedjars | 12 | patch has 10 errors when building our shaded downstream artifacts. | | +1 | hadoopcheck | 278 | Patch does not cause any errors with Hadoop 2.8.5 2.9.2. | | +1 | javadoc | 206 | the patch passed | | 0 | findbugs | 10 | hbase-resource-bundle has no data from findbugs | ||| _ Other Tests _ | | +1 | unit | 9753 | root in the patch passed. | | +1 | asflicense | 151 | The patch does not generate ASF License warnings. | | | | 14383 | | | Subsystem | Report/Notes | |--:|:-| | Docker | Client=19.03.1 Server=19.03.1 base: https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-251/18/artifact/out/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/251 | | Optional Tests | dupname asflicense javac javadoc unit spotbugs findbugs shadedjars hadoopcheck hbaseanti checkstyle compile refguide xml | | uname | Linux 76e17b426759 4.15.0-58-generic #64-Ubuntu SMP Tue Aug 6 11:12:41 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | /home/jenkins/jenkins-slave/workspace/HBase-PreCommit-GitHub-PR_PR-251/out/precommit/personality/provided.sh | | git revision | branch-1 / 73e0304 | | Default Java | 1.8.0_222 | | refguide | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-251/18/artifact/out/branch-site/book.html | | shadedjars | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-251/18/artifact/out/branch-shadedjars.txt | | checkstyle | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-251/18/artifact/out/diff-checkstyle-hbase-tinylfu-blockcache.txt | | checkstyle | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-251/18/artifact/out/diff-checkstyle-root.txt | | refguide | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-251/18/artifact/out/patch-site/book.html | | shadedjars | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-251/18/artifact/out/patch-shadedjars.txt | | Test Results | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-251/18/testReport/ | | Max. process+thread count | 4814 (vs. ulimit of 1) | | modules | C: hbase-resource-bundle hbase-common hbase-server hbase-tinylfu-blockcache . hbase-it U: . | | Console output | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-251/18/console | | versions | git=1.9.1 maven=3.0.5 findbugs=3.1
[GitHub] [hbase] Apache-HBase commented on issue #548: HBASE-22930 Set unique name to longCompactions/shortCompactions threads
Apache-HBase commented on issue #548: HBASE-22930 Set unique name to longCompactions/shortCompactions threads URL: https://github.com/apache/hbase/pull/548#issuecomment-525930204 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | 0 | reexec | 49 | Docker mode activated. | ||| _ Prechecks _ | | +1 | dupname | 0 | No case conflicting files found. | | +1 | hbaseanti | 1 | Patch does not have any anti-patterns. | | +1 | @author | 0 | The patch does not contain any @author tags. | | -0 | test4tests | 0 | The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. | ||| _ master Compile Tests _ | | +1 | mvninstall | 434 | master passed | | +1 | compile | 68 | master passed | | +1 | checkstyle | 98 | master passed | | +1 | shadedjars | 329 | branch has no errors when building our shaded downstream artifacts. | | +1 | javadoc | 43 | master passed | | 0 | spotbugs | 304 | Used deprecated FindBugs config; considering switching to SpotBugs. | | +1 | findbugs | 301 | master passed | ||| _ Patch Compile Tests _ | | +1 | mvninstall | 422 | the patch passed | | +1 | compile | 70 | the patch passed | | +1 | javac | 70 | the patch passed | | +1 | checkstyle | 101 | the patch passed | | +1 | whitespace | 0 | The patch has no whitespace issues. | | +1 | shadedjars | 363 | patch has no errors when building our shaded downstream artifacts. | | +1 | hadoopcheck | 1294 | Patch does not cause any errors with Hadoop 2.8.5 2.9.2 or 3.1.2. | | +1 | javadoc | 42 | the patch passed | | +1 | findbugs | 311 | the patch passed | ||| _ Other Tests _ | | -1 | unit | 10250 | hbase-server in the patch failed. | | +1 | asflicense | 30 | The patch does not generate ASF License warnings. | | | | 14693 | | | Reason | Tests | |---:|:--| | Failed junit tests | hadoop.hbase.replication.TestReplicationDroppedTables | | Subsystem | Report/Notes | |--:|:-| | Docker | Client=19.03.1 Server=19.03.1 base: https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-548/3/artifact/out/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/548 | | Optional Tests | dupname asflicense javac javadoc unit spotbugs findbugs shadedjars hadoopcheck hbaseanti checkstyle compile | | uname | Linux 72fd170c811d 4.15.0-58-generic #64-Ubuntu SMP Tue Aug 6 11:12:41 UTC 2019 x86_64 GNU/Linux | | Build tool | maven | | Personality | /home/jenkins/jenkins-slave/workspace/HBase-PreCommit-GitHub-PR_PR-548/out/precommit/personality/provided.sh | | git revision | master / 56980fb097 | | Default Java | 1.8.0_181 | | unit | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-548/3/artifact/out/patch-unit-hbase-server.txt | | Test Results | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-548/3/testReport/ | | Max. process+thread count | 5705 (vs. ulimit of 1) | | modules | C: hbase-server U: hbase-server | | Console output | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-548/3/console | | versions | git=2.11.0 maven=2018-06-17T18:33:14Z) findbugs=3.1.11 | | Powered by | Apache Yetus 0.10.0 http://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [hbase] Apache-HBase commented on issue #553: HBASE-22939 SpaceQuotas- Bulkload from different hdfs failed when space quotas are turned on.
Apache-HBase commented on issue #553: HBASE-22939 SpaceQuotas- Bulkload from different hdfs failed when space quotas are turned on. URL: https://github.com/apache/hbase/pull/553#issuecomment-525927604 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | 0 | reexec | 92 | Docker mode activated. | ||| _ Prechecks _ | | +1 | dupname | 0 | No case conflicting files found. | | +1 | hbaseanti | 0 | Patch does not have any anti-patterns. | | +1 | @author | 0 | The patch does not contain any @author tags. | | -0 | test4tests | 0 | The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. | ||| _ master Compile Tests _ | | +1 | mvninstall | 446 | master passed | | +1 | compile | 69 | master passed | | +1 | checkstyle | 98 | master passed | | +1 | shadedjars | 348 | branch has no errors when building our shaded downstream artifacts. | | +1 | javadoc | 38 | master passed | | 0 | spotbugs | 241 | Used deprecated FindBugs config; considering switching to SpotBugs. | | +1 | findbugs | 239 | master passed | ||| _ Patch Compile Tests _ | | +1 | mvninstall | 363 | the patch passed | | +1 | compile | 71 | the patch passed | | +1 | javac | 71 | the patch passed | | +1 | checkstyle | 93 | the patch passed | | +1 | whitespace | 0 | The patch has no whitespace issues. | | +1 | shadedjars | 312 | patch has no errors when building our shaded downstream artifacts. | | +1 | hadoopcheck | 1034 | Patch does not cause any errors with Hadoop 2.8.5 2.9.2 or 3.1.2. | | +1 | javadoc | 42 | the patch passed | | +1 | findbugs | 294 | the patch passed | ||| _ Other Tests _ | | +1 | unit | 10092 | hbase-server in the patch passed. | | +1 | asflicense | 32 | The patch does not generate ASF License warnings. | | | | 14154 | | | Subsystem | Report/Notes | |--:|:-| | Docker | Client=19.03.1 Server=19.03.1 base: https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-553/1/artifact/out/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/553 | | Optional Tests | dupname asflicense javac javadoc unit spotbugs findbugs shadedjars hadoopcheck hbaseanti checkstyle compile | | uname | Linux c355376da5c0 4.15.0-58-generic #64-Ubuntu SMP Tue Aug 6 11:12:41 UTC 2019 x86_64 GNU/Linux | | Build tool | maven | | Personality | /home/jenkins/jenkins-slave/workspace/HBase-PreCommit-GitHub-PR_PR-553/out/precommit/personality/provided.sh | | git revision | master / 56980fb097 | | Default Java | 1.8.0_181 | | Test Results | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-553/1/testReport/ | | Max. process+thread count | 4803 (vs. ulimit of 1) | | modules | C: hbase-server U: hbase-server | | Console output | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-553/1/console | | versions | git=2.11.0 maven=2018-06-17T18:33:14Z) findbugs=3.1.11 | | Powered by | Apache Yetus 0.10.0 http://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [hbase] Apache-HBase commented on issue #494: HBASE-22380 break circle replication when doing bulkload
Apache-HBase commented on issue #494: HBASE-22380 break circle replication when doing bulkload URL: https://github.com/apache/hbase/pull/494#issuecomment-525923026 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | 0 | reexec | 38 | Docker mode activated. | ||| _ Prechecks _ | | +1 | dupname | 1 | No case conflicting files found. | | +1 | hbaseanti | 0 | Patch does not have any anti-patterns. | | +1 | @author | 0 | The patch does not contain any @author tags. | | +1 | test4tests | 0 | The patch appears to include 2 new or modified test files. | ||| _ branch-2 Compile Tests _ | | 0 | mvndep | 13 | Maven dependency ordering for branch | | +1 | mvninstall | 350 | branch-2 passed | | +1 | compile | 130 | branch-2 passed | | +1 | checkstyle | 152 | branch-2 passed | | +1 | shadedjars | 283 | branch has no errors when building our shaded downstream artifacts. | | +1 | javadoc | 74 | branch-2 passed | | 0 | spotbugs | 223 | Used deprecated FindBugs config; considering switching to SpotBugs. | | +1 | findbugs | 465 | branch-2 passed | ||| _ Patch Compile Tests _ | | 0 | mvndep | 16 | Maven dependency ordering for patch | | +1 | mvninstall | 328 | the patch passed | | +1 | compile | 127 | the patch passed | | +1 | cc | 127 | the patch passed | | +1 | javac | 127 | the patch passed | | -1 | checkstyle | 97 | hbase-server: The patch generated 3 new + 300 unchanged - 0 fixed = 303 total (was 300) | | +1 | whitespace | 0 | The patch has no whitespace issues. | | +1 | shadedjars | 279 | patch has no errors when building our shaded downstream artifacts. | | +1 | hadoopcheck | 1070 | Patch does not cause any errors with Hadoop 2.8.5 2.9.2 or 3.1.2. | | +1 | hbaseprotoc | 133 | the patch passed | | +1 | javadoc | 81 | the patch passed | | -1 | findbugs | 269 | hbase-server generated 1 new + 0 unchanged - 0 fixed = 1 total (was 0) | ||| _ Other Tests _ | | +1 | unit | 49 | hbase-protocol-shaded in the patch passed. | | +1 | unit | 240 | hbase-client in the patch passed. | | -1 | unit | 9448 | hbase-server in the patch failed. | | +1 | asflicense | 68 | The patch does not generate ASF License warnings. | | | | 14584 | | | Reason | Tests | |---:|:--| | FindBugs | module:hbase-server | | | org.apache.hadoop.hbase.replication.regionserver.ReplicationSink.replicateEntries(List, CellScanner, String, String, String) makes inefficient use of keySet iterator instead of entrySet iterator At ReplicationSink.java:makes inefficient use of keySet iterator instead of entrySet iterator At ReplicationSink.java:[line 257] | | Failed junit tests | hadoop.hbase.regionserver.TestBulkLoadReplication | | Subsystem | Report/Notes | |--:|:-| | Docker | Client=19.03.1 Server=19.03.1 base: https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-494/37/artifact/out/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/494 | | Optional Tests | dupname asflicense javac javadoc unit spotbugs findbugs shadedjars hadoopcheck hbaseanti checkstyle compile cc hbaseprotoc | | uname | Linux 6cac864672d6 4.15.0-58-generic #64-Ubuntu SMP Tue Aug 6 11:12:41 UTC 2019 x86_64 GNU/Linux | | Build tool | maven | | Personality | /home/jenkins/jenkins-slave/workspace/HBase-PreCommit-GitHub-PR_PR-494/out/precommit/personality/provided.sh | | git revision | branch-2 / 84b4a61e88 | | Default Java | 1.8.0_181 | | checkstyle | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-494/37/artifact/out/diff-checkstyle-hbase-server.txt | | findbugs | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-494/37/artifact/out/new-findbugs-hbase-server.html | | unit | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-494/37/artifact/out/patch-unit-hbase-server.txt | | Test Results | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-494/37/testReport/ | | Max. process+thread count | 4563 (vs. ulimit of 1) | | modules | C: hbase-protocol-shaded hbase-client hbase-server U: . | | Console output | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-494/37/console | | versions | git=2.11.0 maven=2018-06-17T18:33:14Z) findbugs=3.1.11 | | Powered by | Apache Yetus 0.10.0 http://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [hbase] Apache-HBase commented on issue #435: HBASE-22767 System table RIT STUCK if their RSGroup has no highest ve…
Apache-HBase commented on issue #435: HBASE-22767 System table RIT STUCK if their RSGroup has no highest ve… URL: https://github.com/apache/hbase/pull/435#issuecomment-525911453 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | 0 | reexec | 75 | Docker mode activated. | ||| _ Prechecks _ | | +1 | dupname | 0 | No case conflicting files found. | | 0 | shelldocs | 0 | Shelldocs was not available. | | +1 | hbaseanti | 0 | Patch does not have any anti-patterns. | | +1 | @author | 0 | The patch does not contain any @author tags. | | +1 | test4tests | 0 | The patch appears to include 1 new or modified test files. | ||| _ master Compile Tests _ | | 0 | mvndep | 36 | Maven dependency ordering for branch | | +1 | mvninstall | 412 | master passed | | +1 | compile | 127 | master passed | | +1 | checkstyle | 155 | master passed | | +1 | shadedjars | 364 | branch has no errors when building our shaded downstream artifacts. | | +1 | javadoc | 91 | master passed | | 0 | spotbugs | 302 | Used deprecated FindBugs config; considering switching to SpotBugs. | | +1 | findbugs | 412 | master passed | ||| _ Patch Compile Tests _ | | 0 | mvndep | 19 | Maven dependency ordering for patch | | +1 | mvninstall | 389 | the patch passed | | +1 | compile | 134 | the patch passed | | +1 | javac | 134 | the patch passed | | -1 | checkstyle | 101 | hbase-server: The patch generated 1 new + 9 unchanged - 0 fixed = 10 total (was 9) | | +1 | shellcheck | 1 | There were no new shellcheck issues. | | +1 | whitespace | 0 | The patch has no whitespace issues. | | +1 | shadedjars | 370 | patch has no errors when building our shaded downstream artifacts. | | +1 | hadoopcheck | 1209 | Patch does not cause any errors with Hadoop 2.8.5 2.9.2 or 3.1.2. | | +1 | javadoc | 81 | the patch passed | | -1 | findbugs | 60 | hbase-common generated 1 new + 0 unchanged - 0 fixed = 1 total (was 0) | ||| _ Other Tests _ | | +1 | unit | 194 | hbase-common in the patch passed. | | +1 | unit | 9787 | hbase-server in the patch passed. | | +1 | unit | 333 | hbase-rsgroup in the patch passed. | | +1 | asflicense | 83 | The patch does not generate ASF License warnings. | | | | 15337 | | | Reason | Tests | |---:|:--| | FindBugs | module:hbase-common | | | org.apache.hadoop.hbase.Version.() invokes inefficient new String(String) constructor At Version.java:inefficient new String(String) constructor At Version.java:[line 10] | | Subsystem | Report/Notes | |--:|:-| | Docker | Client=19.03.1 Server=19.03.1 base: https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-435/88/artifact/out/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/435 | | Optional Tests | dupname asflicense shellcheck shelldocs javac javadoc unit spotbugs findbugs shadedjars hadoopcheck hbaseanti checkstyle compile | | uname | Linux cabaede1d498 4.15.0-58-generic #64-Ubuntu SMP Tue Aug 6 11:12:41 UTC 2019 x86_64 GNU/Linux | | Build tool | maven | | Personality | /home/jenkins/jenkins-slave/workspace/HBase-PreCommit-GitHub-PR_PR-435/out/precommit/personality/provided.sh | | git revision | master / 56980fb097 | | Default Java | 1.8.0_181 | | checkstyle | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-435/88/artifact/out/diff-checkstyle-hbase-server.txt | | findbugs | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-435/88/artifact/out/new-findbugs-hbase-common.html | | Test Results | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-435/88/testReport/ | | Max. process+thread count | 4629 (vs. ulimit of 1) | | modules | C: hbase-common hbase-server hbase-rsgroup U: . | | Console output | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-435/88/console | | versions | git=2.11.0 maven=2018-06-17T18:33:14Z) shellcheck=0.4.4 findbugs=3.1.11 | | Powered by | Apache Yetus 0.10.0 http://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Comment Edited] (HBASE-22938) Fold all the system tables to hbase:meta
[
https://issues.apache.org/jira/browse/HBASE-22938?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16917901#comment-16917901
]
stack edited comment on HBASE-22938 at 8/28/19 7:59 PM:
Which system tables? What schema does each system table need?
* Namespace table has been integrated already
* ACL table
* Replication Peers (and history)
What else?
Should make note of loadings -- e.g. updating offsets per RS for peers will
mean lots of writing -- and then too, in any new design, should try and spread
loading so when split, load is distributed not lumped.
was (Author: stack):
Which system tables? What schema does each system table need?
* Namespace table has been integrated already
* ACL table
* Replication Peers (and history)
What else?
> Fold all the system tables to hbase:meta
>
>
> Key: HBASE-22938
> URL: https://issues.apache.org/jira/browse/HBASE-22938
> Project: HBase
> Issue Type: Brainstorming
>Reporter: Duo Zhang
>Priority: Major
>
> Quote my post on HBASE-15867 here, on how to deal with the dead lock when we
> want to store replication queues to hbase:replication table.
> {quote}
> We could add a special prefix in the row key for different system tables, and
> make a special family for it. For example, for all the records in hbase:acl,
> we could introduce a prefix like ':::acl:::', since we do not allow ':' in
> either namespace or table name, so it will not conflict with the existing
> table related records. And the family could be namd as 'acl'.
> And we could make a special split policy that only splits at these special
> prefixs, so it will not break any assumptions so far, as all the records for
> the 'system table' are in the same region.
> {quote}
> And I think there are also other advantages, for example the start up logic
> can be greatly simplified.
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
[jira] [Commented] (HBASE-22567) HBCK2 addMissingRegionsToMeta
[
https://issues.apache.org/jira/browse/HBASE-22567?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16918039#comment-16918039
]
stack commented on HBASE-22567:
---
Took another look. I think you need to put up a new patch [~wchevreuil] because
at a minimum time has gone by and there new commands in hbck2 that interact
with this one. There are other comments that I think can be resolved but you'd
be best judge. Thanks.
> HBCK2 addMissingRegionsToMeta
> -
>
> Key: HBASE-22567
> URL: https://issues.apache.org/jira/browse/HBASE-22567
> Project: HBase
> Issue Type: New Feature
> Components: hbck2
>Reporter: Wellington Chevreuil
>Assignee: Wellington Chevreuil
>Priority: Major
>
> Following latest discussion on HBASE-21745, this proposes an hbck2 command
> that allows for inserting back regions missing in META that still have
> *regioninfo* available in HDFS. Although this is still an interactive and
> simpler version than the old _OfflineMetaRepair_, it still relies on hdfs
> state as the source of truth, and performs META updates mostly independently
> from Master (apart from requiring Meta table been online).
> For a more detailed explanation on this command behaviour, pasting _command
> usage_ text:
> {noformat}
> To be used for scenarios where some regions may be missing in META,
> but there's still a valid 'regioninfo' metadata file on HDFS.
> This is a lighter version of 'OfflineMetaRepair' tool commonly used for
> similar issues on 1.x release line.
> This command needs META to be online. For each table name passed as
> parameter, it performs a diff between regions available in META,
> against existing regions dirs on HDFS. Then, for region dirs with
> no matches in META, it reads regioninfo metadata file and
> re-creates given region in META. Regions are re-created in 'CLOSED'
> state at META table only, but not in Masters' cache, and are not
> assigned either. A rolling Masters restart, followed by a
> hbck2 'assigns' command with all re-inserted regions is required.
> This hbck2 'assigns' command is printed for user convenience.
> WARNING: To avoid potential region overlapping problems due to ongoing
> splits, this command disables given tables while re-inserting regions.
> An example adding missing regions for tables 'table_1' and 'table_2':
> $ HBCK2 addMissingRegionsInMeta table_1 table_2
> Returns hbck2 'assigns' command with all re-inserted regions.{noformat}
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
[GitHub] [hbase] Apache-HBase commented on issue #547: HBASE-22899 logging improvements for snapshot operations w/large manifests
Apache-HBase commented on issue #547: HBASE-22899 logging improvements for snapshot operations w/large manifests URL: https://github.com/apache/hbase/pull/547#issuecomment-525890992 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | 0 | reexec | 38 | Docker mode activated. | ||| _ Prechecks _ | | +1 | dupname | 0 | No case conflicting files found. | | +1 | hbaseanti | 0 | Patch does not have any anti-patterns. | | +1 | @author | 0 | The patch does not contain any @author tags. | | -0 | test4tests | 0 | The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. | ||| _ master Compile Tests _ | | +1 | mvninstall | 339 | master passed | | +1 | compile | 56 | master passed | | +1 | checkstyle | 81 | master passed | | +1 | shadedjars | 286 | branch has no errors when building our shaded downstream artifacts. | | +1 | javadoc | 38 | master passed | | 0 | spotbugs | 242 | Used deprecated FindBugs config; considering switching to SpotBugs. | | +1 | findbugs | 240 | master passed | ||| _ Patch Compile Tests _ | | +1 | mvninstall | 302 | the patch passed | | +1 | compile | 54 | the patch passed | | +1 | javac | 54 | the patch passed | | +1 | checkstyle | 75 | the patch passed | | +1 | whitespace | 0 | The patch has no whitespace issues. | | +1 | shadedjars | 266 | patch has no errors when building our shaded downstream artifacts. | | +1 | hadoopcheck | 935 | Patch does not cause any errors with Hadoop 2.8.5 2.9.2 or 3.1.2. | | +1 | javadoc | 36 | the patch passed | | +1 | findbugs | 258 | the patch passed | ||| _ Other Tests _ | | +1 | unit | 9461 | hbase-server in the patch passed. | | +1 | asflicense | 27 | The patch does not generate ASF License warnings. | | | | 12889 | | | Subsystem | Report/Notes | |--:|:-| | Docker | Client=19.03.1 Server=19.03.1 base: https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-547/3/artifact/out/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/547 | | Optional Tests | dupname asflicense javac javadoc unit spotbugs findbugs shadedjars hadoopcheck hbaseanti checkstyle compile | | uname | Linux 5a6b56fd9b51 4.15.0-58-generic #64-Ubuntu SMP Tue Aug 6 11:12:41 UTC 2019 x86_64 GNU/Linux | | Build tool | maven | | Personality | /home/jenkins/jenkins-slave/workspace/HBase-PreCommit-GitHub-PR_PR-547/out/precommit/personality/provided.sh | | git revision | master / 56980fb097 | | Default Java | 1.8.0_181 | | Test Results | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-547/3/testReport/ | | Max. process+thread count | 4491 (vs. ulimit of 1) | | modules | C: hbase-server U: hbase-server | | Console output | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-547/3/console | | versions | git=2.11.0 maven=2018-06-17T18:33:14Z) findbugs=3.1.11 | | Powered by | Apache Yetus 0.10.0 http://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [hbase] busbey opened a new pull request #557: HBASE-22943 Various procedures should not cache log trace level
busbey opened a new pull request #557: HBASE-22943 Various procedures should not cache log trace level URL: https://github.com/apache/hbase/pull/557 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Commented] (HBASE-22857) Fix the failed ut TestHRegion and TestHRegionWithInMemoryFlush
[
https://issues.apache.org/jira/browse/HBASE-22857?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16918012#comment-16918012
]
Hudson commented on HBASE-22857:
Results for branch master
[build #1371 on
builds.a.o|https://builds.apache.org/job/HBase%20Nightly/job/master/1371/]: (x)
*{color:red}-1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://builds.apache.org/job/HBase%20Nightly/job/master/1371//General_Nightly_Build_Report/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
(/) {color:green}+1 client integration test{color}
> Fix the failed ut TestHRegion and TestHRegionWithInMemoryFlush
> --
>
> Key: HBASE-22857
> URL: https://issues.apache.org/jira/browse/HBASE-22857
> Project: HBase
> Issue Type: Bug
>Affects Versions: 2.2.1
>Reporter: Guanghao Zhang
>Assignee: Guanghao Zhang
>Priority: Major
> Fix For: 2.3.0, 2.2.1
>
>
> Duplicate code in unit test...
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
[jira] [Commented] (HBASE-21879) Read HFile's block to ByteBuffer directly instead of to byte for reducing young gc purpose
[
https://issues.apache.org/jira/browse/HBASE-21879?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16918009#comment-16918009
]
Hudson commented on HBASE-21879:
Results for branch HBASE-21879
[build #223 on
builds.a.o|https://builds.apache.org/job/HBase%20Nightly/job/HBASE-21879/223/]:
(x) *{color:red}-1 overall{color}*
details (if available):
(x) {color:red}-1 general checks{color}
-- For more information [see general
report|https://builds.apache.org/job/HBase%20Nightly/job/HBASE-21879/223//General_Nightly_Build_Report/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
(/) {color:green}+1 client integration test{color}
> Read HFile's block to ByteBuffer directly instead of to byte for reducing
> young gc purpose
> --
>
> Key: HBASE-21879
> URL: https://issues.apache.org/jira/browse/HBASE-21879
> Project: HBase
> Issue Type: Improvement
>Reporter: Zheng Hu
>Assignee: Zheng Hu
>Priority: Major
> Fix For: 3.0.0, 2.3.0
>
> Attachments: HBASE-21879.v1.patch, HBASE-21879.v1.patch,
> QPS-latencies-before-HBASE-21879.png, gc-data-before-HBASE-21879.png
>
>
> In HFileBlock#readBlockDataInternal, we have the following:
> {code}
> @VisibleForTesting
> protected HFileBlock readBlockDataInternal(FSDataInputStream is, long offset,
> long onDiskSizeWithHeaderL, boolean pread, boolean verifyChecksum,
> boolean updateMetrics)
> throws IOException {
> // .
> // TODO: Make this ByteBuffer-based. Will make it easier to go to HDFS with
> BBPool (offheap).
> byte [] onDiskBlock = new byte[onDiskSizeWithHeader + hdrSize];
> int nextBlockOnDiskSize = readAtOffset(is, onDiskBlock, preReadHeaderSize,
> onDiskSizeWithHeader - preReadHeaderSize, true, offset +
> preReadHeaderSize, pread);
> if (headerBuf != null) {
> // ...
> }
> // ...
> }
> {code}
> In the read path, we still read the block from hfile to on-heap byte[], then
> copy the on-heap byte[] to offheap bucket cache asynchronously, and in my
> 100% get performance test, I also observed some frequent young gc, The
> largest memory footprint in the young gen should be the on-heap block byte[].
> In fact, we can read HFile's block to ByteBuffer directly instead of to
> byte[] for reducing young gc purpose. we did not implement this before,
> because no ByteBuffer reading interface in the older HDFS client, but 2.7+
> has supported this now, so we can fix this now. I think.
> Will provide an patch and some perf-comparison for this.
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
[GitHub] [hbase] Apache-HBase commented on issue #554: HBASE-22887 Fix HFileOutputFormat2 writer roll
Apache-HBase commented on issue #554: HBASE-22887 Fix HFileOutputFormat2 writer roll URL: https://github.com/apache/hbase/pull/554#issuecomment-525872189 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | 0 | reexec | 76 | Docker mode activated. | ||| _ Prechecks _ | | +1 | dupname | 0 | No case conflicting files found. | | +1 | hbaseanti | 0 | Patch does not have any anti-patterns. | | +1 | @author | 0 | The patch does not contain any @author tags. | | -0 | test4tests | 0 | The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. | ||| _ master Compile Tests _ | | +1 | mvninstall | 360 | master passed | | +1 | compile | 27 | master passed | | +1 | checkstyle | 21 | master passed | | +1 | shadedjars | 300 | branch has no errors when building our shaded downstream artifacts. | | +1 | javadoc | 18 | master passed | | 0 | spotbugs | 54 | Used deprecated FindBugs config; considering switching to SpotBugs. | | +1 | findbugs | 52 | master passed | ||| _ Patch Compile Tests _ | | +1 | mvninstall | 336 | the patch passed | | +1 | compile | 27 | the patch passed | | +1 | javac | 27 | the patch passed | | +1 | checkstyle | 21 | the patch passed | | +1 | whitespace | 0 | The patch has no whitespace issues. | | +1 | shadedjars | 308 | patch has no errors when building our shaded downstream artifacts. | | +1 | hadoopcheck | 1036 | Patch does not cause any errors with Hadoop 2.8.5 2.9.2 or 3.1.2. | | +1 | javadoc | 17 | the patch passed | | +1 | findbugs | 60 | the patch passed | ||| _ Other Tests _ | | +1 | unit | 1392 | hbase-mapreduce in the patch passed. | | +1 | asflicense | 17 | The patch does not generate ASF License warnings. | | | | 4493 | | | Subsystem | Report/Notes | |--:|:-| | Docker | Client=19.03.1 Server=19.03.1 base: https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-554/1/artifact/out/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/554 | | Optional Tests | dupname asflicense javac javadoc unit spotbugs findbugs shadedjars hadoopcheck hbaseanti checkstyle compile | | uname | Linux 6678b9bd62b5 4.15.0-54-generic #58-Ubuntu SMP Mon Jun 24 10:55:24 UTC 2019 x86_64 GNU/Linux | | Build tool | maven | | Personality | /home/jenkins/jenkins-slave/workspace/HBase-PreCommit-GitHub-PR_PR-554/out/precommit/personality/provided.sh | | git revision | master / 56980fb097 | | Default Java | 1.8.0_181 | | Test Results | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-554/1/testReport/ | | Max. process+thread count | 5036 (vs. ulimit of 1) | | modules | C: hbase-mapreduce U: hbase-mapreduce | | Console output | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-554/1/console | | versions | git=2.11.0 maven=2018-06-17T18:33:14Z) findbugs=3.1.11 | | Powered by | Apache Yetus 0.10.0 http://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Commented] (HBASE-22760) Stop/Resume Snapshot Auto-Cleanup activity with shell command
[ https://issues.apache.org/jira/browse/HBASE-22760?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16918004#comment-16918004 ] Andrew Purtell commented on HBASE-22760: +1 > Stop/Resume Snapshot Auto-Cleanup activity with shell command > - > > Key: HBASE-22760 > URL: https://issues.apache.org/jira/browse/HBASE-22760 > Project: HBase > Issue Type: Improvement > Components: Admin, shell, snapshots >Affects Versions: 3.0.0, 1.5.0, 2.3.0, 2.2.1, 1.4.11 >Reporter: Viraj Jasani >Assignee: Viraj Jasani >Priority: Major > Fix For: 3.0.0, 1.5.0, 2.3.0, 2.2.1, 1.4.11 > > Attachments: HBASE-22760.master.003.patch, > HBASE-22760.master.004.patch, HBASE-22760.master.005.patch, > HBASE-22760.master.008.patch, HBASE-22760.master.009.patch > > > For any scheduled snapshot backup activity, we would like to disable > auto-cleaner for snapshot based on TTL. However, as per HBASE-22648 we have a > config to disable snapshot auto-cleaner: > hbase.master.cleaner.snapshot.disable, which would take effect only upon > HMaster restart just similar to any other hbase-site configs. > For any running cluster, we should be able to stop/resume auto-cleanup > activity for snapshot based on shell command. Something similar to below > command should be able to stop/start cleanup chore: > hbase(main):001:0> snapshot_auto_cleanup_switch false (disable > auto-cleaner) > hbase(main):001:0> snapshot_auto_cleanup_switch true (enable auto-cleaner) -- This message was sent by Atlassian Jira (v8.3.2#803003)
[jira] [Commented] (HBASE-22380) break circle replication when doing bulkload
[ https://issues.apache.org/jira/browse/HBASE-22380?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16918003#comment-16918003 ] Andrew Purtell commented on HBASE-22380: A few days is fine. It's not looking good this week or early next either, a few other things, and the labor day holiday in the US. > break circle replication when doing bulkload > > > Key: HBASE-22380 > URL: https://issues.apache.org/jira/browse/HBASE-22380 > Project: HBase > Issue Type: Bug > Components: Replication >Affects Versions: 3.0.0, 1.5.0, 2.2.0, 1.4.10, 2.0.5, 2.3.0, 2.1.5, 1.3.5 >Reporter: chenxu >Assignee: Wellington Chevreuil >Priority: Critical > Labels: bulkload > Fix For: 3.0.0, 1.5.0, 2.3.0, 1.4.11, 2.1.7, 2.2.2 > > > when enabled master-master bulkload replication, HFiles will be replicated > circularly between two clusters -- This message was sent by Atlassian Jira (v8.3.2#803003)
[jira] [Updated] (HBASE-22935) TaskMonitor warns MonitoredRPCHandler task may be stuck when it recently started
[
https://issues.apache.org/jira/browse/HBASE-22935?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Andrew Purtell updated HBASE-22935:
---
Fix Version/s: 1.4.11
1.3.6
2.1.6
2.2.1
2.3.0
1.5.0
3.0.0
Hadoop Flags: Reviewed
Resolution: Fixed
Status: Resolved (was: Patch Available)
> TaskMonitor warns MonitoredRPCHandler task may be stuck when it recently
> started
>
>
> Key: HBASE-22935
> URL: https://issues.apache.org/jira/browse/HBASE-22935
> Project: HBase
> Issue Type: Bug
> Components: logging
>Affects Versions: 3.0.0, 1.4.0, 1.5.0, 1.3.3, 2.0.0
>Reporter: David Manning
>Assignee: David Manning
>Priority: Minor
> Fix For: 3.0.0, 1.5.0, 2.3.0, 2.2.1, 2.1.6, 1.3.6, 1.4.11
>
> Attachments: HBASE-22935.master.001.patch
>
>
> After setting {{hbase.taskmonitor.rpc.warn.time}} to 18, the logs show
> WARN messages such as these
> {noformat}
> 2019-08-08 21:50:02,601 WARN [read for TaskMonitor] monitoring.TaskMonitor -
> Task may be stuck: RpcServer.FifoWFPBQ.default.handler=4,queue=4,port=60020:
> status=Servicing call from :55164: Scan, state=RUNNING,
> startTime=1563305858103, completionTime=-1, queuetimems=1565301002599,
> starttimems=1565301002599, clientaddress=, remoteport=55164,
> packetlength=370, rpcMethod=Scan
> {noformat}
> Notice that the first {{starttimems}} is far in the past. The second
> {{starttimems}} and the {{queuetimems}} are much closer to the log timestamp
> than 180 seconds. I think this is because the warnTime is initialized to the
> time that MonitoredTaskImpl is created, but never updated until we write a
> warn message to the log.
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
[jira] [Commented] (HBASE-22567) HBCK2 addMissingRegionsToMeta
[
https://issues.apache.org/jira/browse/HBASE-22567?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16917999#comment-16917999
]
stack commented on HBASE-22567:
---
Let me go over the PR again. I left a load of mess up there. Let me try cleanup
and see if can +1 it.
> HBCK2 addMissingRegionsToMeta
> -
>
> Key: HBASE-22567
> URL: https://issues.apache.org/jira/browse/HBASE-22567
> Project: HBase
> Issue Type: New Feature
> Components: hbck2
>Reporter: Wellington Chevreuil
>Assignee: Wellington Chevreuil
>Priority: Major
>
> Following latest discussion on HBASE-21745, this proposes an hbck2 command
> that allows for inserting back regions missing in META that still have
> *regioninfo* available in HDFS. Although this is still an interactive and
> simpler version than the old _OfflineMetaRepair_, it still relies on hdfs
> state as the source of truth, and performs META updates mostly independently
> from Master (apart from requiring Meta table been online).
> For a more detailed explanation on this command behaviour, pasting _command
> usage_ text:
> {noformat}
> To be used for scenarios where some regions may be missing in META,
> but there's still a valid 'regioninfo' metadata file on HDFS.
> This is a lighter version of 'OfflineMetaRepair' tool commonly used for
> similar issues on 1.x release line.
> This command needs META to be online. For each table name passed as
> parameter, it performs a diff between regions available in META,
> against existing regions dirs on HDFS. Then, for region dirs with
> no matches in META, it reads regioninfo metadata file and
> re-creates given region in META. Regions are re-created in 'CLOSED'
> state at META table only, but not in Masters' cache, and are not
> assigned either. A rolling Masters restart, followed by a
> hbck2 'assigns' command with all re-inserted regions is required.
> This hbck2 'assigns' command is printed for user convenience.
> WARNING: To avoid potential region overlapping problems due to ongoing
> splits, this command disables given tables while re-inserting regions.
> An example adding missing regions for tables 'table_1' and 'table_2':
> $ HBCK2 addMissingRegionsInMeta table_1 table_2
> Returns hbck2 'assigns' command with all re-inserted regions.{noformat}
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
[GitHub] [hbase] saintstack commented on issue #476: HBASE-11062 hbtop
saintstack commented on issue #476: HBASE-11062 hbtop URL: https://github.com/apache/hbase/pull/476#issuecomment-525868863 @brfrn169 Had a chat w/ @busbey and agree with his comment that whatever you think easiest for you maintaining this stuff is way to go. One other note is that 'hbase hbck' is like this: ``` hbck Run the HBase 'fsck' tool. Defaults read-only hbck1. Pass '-j /path/to/HBCK2.jar' to run hbase-2.x HBCK2. ``` We could do a hbase-operator-tools release and then when you type 'hbase hbtop', you'd get what was in the CLASSPATH. If you wanted newer version, you could point at newer jar in similar manner. Just a suggestion. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [hbase] stoty commented on a change in pull request #556: HBASE-22941 merge operation returns parent regions in random order
stoty commented on a change in pull request #556: HBASE-22941 merge operation
returns parent regions in random order
URL: https://github.com/apache/hbase/pull/556#discussion_r318731471
##
File path:
hbase-server/src/main/java/org/apache/hadoop/hbase/master/assignment/RegionStateStore.java
##
@@ -263,7 +263,7 @@ public void mergeRegions(RegionInfo child, RegionInfo []
parents, ServerName ser
throws IOException {
TableDescriptor htd = getDescriptor(child.getTable());
boolean globalScope = htd.hasGlobalReplicationScope();
-Map parentSeqNums = new HashMap<>(parents.length);
+Map parentSeqNums = new TreeMap<>();
Review comment:
I've changed it to SortedMap to signify that we are using it for the sorted
quality.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [hbase] busbey commented on issue #476: HBASE-11062 hbtop
busbey commented on issue #476: HBASE-11062 hbtop URL: https://github.com/apache/hbase/pull/476#issuecomment-525865176 One thing that occurred to me I haven't said out loud here. My views on ease of maintenance when the code is in the main repo, operator-tools repo, on branches or not, etc is all well and good, but it's unlikely I personally will be doing much maintenance of this code once it lands. @brfrn169 if you think it'll be easier to maintain this if it lives in the main repo and you're likely to be doing follow on work, then I think we're better off just keeping it in the main repo. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Commented] (HBASE-22618) Provide a way to have Heterogeneous deployment
[
https://issues.apache.org/jira/browse/HBASE-22618?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16917988#comment-16917988
]
Pierre Zemb commented on HBASE-22618:
-
Hi [~wchevreuil] ! Thanks for the help getting my patch into the different
branches! I do have some questions:
* 1.5 is the only target for 1.X? Can we target 1.4?
* The second patch is missing, making the Jira not resolved. First patch was
about opening the possibility to add cost functions, but we want to also
provide an example of a cost function, by adding one that is allowing for
Heterogeneous cluster, as stated by the ticket. Should I create another Jira?
> Provide a way to have Heterogeneous deployment
> --
>
> Key: HBASE-22618
> URL: https://issues.apache.org/jira/browse/HBASE-22618
> Project: HBase
> Issue Type: Improvement
>Affects Versions: 3.0.0, 2.2.0, 2.2.1, 2.1.6, 1.4.11, 2.1.7
>Reporter: Pierre Zemb
>Assignee: Pierre Zemb
>Priority: Major
> Fix For: 3.0.0, 1.5.0, 2.3.0
>
> Attachments: HBASE-22618.branch-1.001.patch,
> HBASE-22618.branch-1.002.patch, HBASE-22618.branch-2.001.patch,
> HBASE-22618.master.001.patch
>
>
> Hi,
> We wouls like to open the discussion about bringing the possibility to have
> regions deployed on {color:#22}Heterogeneous deployment{color}, i.e Hbase
> cluster running different kind of hardware.
> h2. Why?
> * Cloud deployments means that we may not be able to have the same hardware
> throughout the years
> * Some tables may need special requirements such as SSD whereas others
> should be using hard-drives
> * {color:#22} {color}*in our usecase*{color:#22}(single table,
> dedicated HBase and Hadoop tuned for our usecase, good key
> distribution){color}*, the number of regions per RS was the real limit for
> us*{color:#22}.{color}
> h2. Our usecase
> We found out that *in our usecase*(single table, dedicated HBase and Hadoop
> tuned for our usecase, good key distribution)*, the number of regions per RS
> was the real limit for us*.
> Over the years, due to historical reasons and also the need to benchmark new
> machines, we ended-up with differents groups of hardware: some servers can
> handle only 180 regions, whereas the biggest can handle more than 900.
> Because of such a difference, we had to disable the LoadBalancing to avoid
> the {{roundRobinAssigmnent}}. We developed some internal tooling which are
> responsible for load balancing regions across RegionServers. That was 1.5
> year ago.
> h2. Our Proof-of-concept
> We did work on a Proof-of-concept
> [here|https://github.com/PierreZ/hbase/blob/dev/hbase14/balancer/hbase-server/src/main/java/org/apache/hadoop/hbase/master/balancer/HeterogeneousBalancer.java],
> and some early tests
> [here|https://github.com/PierreZ/hbase/blob/dev/hbase14/balancer/hbase-server/src/main/java/org/apache/hadoop/hbase/master/balancer/HeterogeneousBalancer.java],
>
> [here|https://github.com/PierreZ/hbase/blob/dev/hbase14/balancer/hbase-server/src/test/java/org/apache/hadoop/hbase/master/balancer/TestHeterogeneousBalancerBalance.java],
> and
> [here|https://github.com/PierreZ/hbase/blob/dev/hbase14/balancer/hbase-server/src/test/java/org/apache/hadoop/hbase/master/balancer/TestHeterogeneousBalancerRules.java].
> We wrote the balancer for our use-case, which means that:
> * there is one table
> * there is no region-replica
> * good key dispersion
> * there is no regions on master
> A rule file is loaded before balancing. It contains lines of rules. A rule is
> composed of a regexp for hostname, and a limit. For example, we could have:
>
> {quote}rs[0-9] 200
> rs1[0-9] 50
> {quote}
>
> RegionServers with hostname matching the first rules will have a limit of
> 200, and the others 50. If there's no match, a default is set.
> Thanks to the rule, we have two informations: the max number of regions for
> this cluster, and the rules for each servers. {{HeterogeneousBalancer}} will
> try to balance regions according to their capacity.
> Let's take an example. Let's say that we have 20 RS:
> * 10 RS, named through {{rs0}} to {{rs9}} loaded with 60 regions each, and
> each can handle 200 regions.
> * 10 RS, named through {{rs10}} to {{rs19}} loaded with 60 regions each, and
> each can support 50 regions.
> Based on the following rules:
>
> {quote}rs[0-9] 200
> rs1[0-9] 50
> {quote}
>
> The second group is overloaded, whereas the first group has plenty of space.
> We know that we can handle at maximum *2500 regions* (200*10 + 50*10) and we
> have currently *1200 regions* (60*20). {{HeterogeneousBalancer}} will
> understand that the cluster is *full at 48.0%* (1200/2500). Based on this
> information, we will then *try to put all the RegionServers to ~48% of load
> acc
[jira] [Commented] (HBASE-22567) HBCK2 addMissingRegionsToMeta
[
https://issues.apache.org/jira/browse/HBASE-22567?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16917983#comment-16917983
]
Wellington Chevreuil commented on HBASE-22567:
--
Yep, I think this one would be the ideal to run, as it does lighter
intervention, with master on. Latest PR commit should be addressing most of the
outstanding comments. The ones that I haven't addressed, I had replied in the
PR with the reasons for such. I'm happy to move forward on this, either by
applying additional suggestions or, if it currently looks like +1, can get it
committed.
> HBCK2 addMissingRegionsToMeta
> -
>
> Key: HBASE-22567
> URL: https://issues.apache.org/jira/browse/HBASE-22567
> Project: HBase
> Issue Type: New Feature
> Components: hbck2
>Reporter: Wellington Chevreuil
>Assignee: Wellington Chevreuil
>Priority: Major
>
> Following latest discussion on HBASE-21745, this proposes an hbck2 command
> that allows for inserting back regions missing in META that still have
> *regioninfo* available in HDFS. Although this is still an interactive and
> simpler version than the old _OfflineMetaRepair_, it still relies on hdfs
> state as the source of truth, and performs META updates mostly independently
> from Master (apart from requiring Meta table been online).
> For a more detailed explanation on this command behaviour, pasting _command
> usage_ text:
> {noformat}
> To be used for scenarios where some regions may be missing in META,
> but there's still a valid 'regioninfo' metadata file on HDFS.
> This is a lighter version of 'OfflineMetaRepair' tool commonly used for
> similar issues on 1.x release line.
> This command needs META to be online. For each table name passed as
> parameter, it performs a diff between regions available in META,
> against existing regions dirs on HDFS. Then, for region dirs with
> no matches in META, it reads regioninfo metadata file and
> re-creates given region in META. Regions are re-created in 'CLOSED'
> state at META table only, but not in Masters' cache, and are not
> assigned either. A rolling Masters restart, followed by a
> hbck2 'assigns' command with all re-inserted regions is required.
> This hbck2 'assigns' command is printed for user convenience.
> WARNING: To avoid potential region overlapping problems due to ongoing
> splits, this command disables given tables while re-inserting regions.
> An example adding missing regions for tables 'table_1' and 'table_2':
> $ HBCK2 addMissingRegionsInMeta table_1 table_2
> Returns hbck2 'assigns' command with all re-inserted regions.{noformat}
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
[GitHub] [hbase] Apache-HBase commented on issue #546: HBASE-22928 ScanMetrics counter update may not happen in case of exception in TableRecordReaderImpl
Apache-HBase commented on issue #546: HBASE-22928 ScanMetrics counter update may not happen in case of exception in TableRecordReaderImpl URL: https://github.com/apache/hbase/pull/546#issuecomment-525858002 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | 0 | reexec | 54 | Docker mode activated. | ||| _ Prechecks _ | | +1 | dupname | 0 | No case conflicting files found. | | +1 | hbaseanti | 0 | Patch does not have any anti-patterns. | | +1 | @author | 0 | The patch does not contain any @author tags. | | -0 | test4tests | 0 | The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. | ||| _ master Compile Tests _ | | +1 | mvninstall | 380 | master passed | | +1 | compile | 31 | master passed | | +1 | checkstyle | 25 | master passed | | +1 | shadedjars | 344 | branch has no errors when building our shaded downstream artifacts. | | +1 | javadoc | 23 | master passed | | 0 | spotbugs | 61 | Used deprecated FindBugs config; considering switching to SpotBugs. | | +1 | findbugs | 58 | master passed | ||| _ Patch Compile Tests _ | | +1 | mvninstall | 390 | the patch passed | | +1 | compile | 33 | the patch passed | | +1 | javac | 33 | the patch passed | | +1 | checkstyle | 24 | the patch passed | | +1 | whitespace | 0 | The patch has no whitespace issues. | | +1 | shadedjars | 353 | patch has no errors when building our shaded downstream artifacts. | | +1 | hadoopcheck | 1240 | Patch does not cause any errors with Hadoop 2.8.5 2.9.2 or 3.1.2. | | +1 | javadoc | 19 | the patch passed | | +1 | findbugs | 66 | the patch passed | ||| _ Other Tests _ | | +1 | unit | 1346 | hbase-mapreduce in the patch passed. | | +1 | asflicense | 18 | The patch does not generate ASF License warnings. | | | | 4889 | | | Subsystem | Report/Notes | |--:|:-| | Docker | Client=19.03.1 Server=19.03.1 base: https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-546/4/artifact/out/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/546 | | Optional Tests | dupname asflicense javac javadoc unit spotbugs findbugs shadedjars hadoopcheck hbaseanti checkstyle compile | | uname | Linux 0ac1b4422676 4.15.0-58-generic #64-Ubuntu SMP Tue Aug 6 11:12:41 UTC 2019 x86_64 GNU/Linux | | Build tool | maven | | Personality | /home/jenkins/jenkins-slave/workspace/HBase-PreCommit-GitHub-PR_PR-546/out/precommit/personality/provided.sh | | git revision | master / 56980fb097 | | Default Java | 1.8.0_181 | | Test Results | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-546/4/testReport/ | | Max. process+thread count | 5616 (vs. ulimit of 1) | | modules | C: hbase-mapreduce U: hbase-mapreduce | | Console output | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-546/4/console | | versions | git=2.11.0 maven=2018-06-17T18:33:14Z) findbugs=3.1.11 | | Powered by | Apache Yetus 0.10.0 http://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [hbase] joshelser commented on a change in pull request #476: HBASE-11062 hbtop
joshelser commented on a change in pull request #476: HBASE-11062 hbtop
URL: https://github.com/apache/hbase/pull/476#discussion_r318717875
##
File path: hbase-hbtop/src/main/java/org/apache/hadoop/hbase/hbtop/HBTop.java
##
@@ -0,0 +1,141 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.hadoop.hbase.hbtop;
+
+import java.io.InputStream;
+import java.util.Objects;
+import org.apache.hadoop.conf.Configuration;
+import org.apache.hadoop.conf.Configured;
+import org.apache.hadoop.hbase.HBaseConfiguration;
+import org.apache.hadoop.hbase.hbtop.mode.Mode;
+import org.apache.hadoop.hbase.hbtop.screen.Screen;
+import org.apache.hadoop.util.Tool;
+import org.apache.hadoop.util.ToolRunner;
+import org.apache.log4j.PropertyConfigurator;
+import org.apache.yetus.audience.InterfaceAudience;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+import org.apache.hbase.thirdparty.org.apache.commons.cli.CommandLine;
+import org.apache.hbase.thirdparty.org.apache.commons.cli.DefaultParser;
+import org.apache.hbase.thirdparty.org.apache.commons.cli.HelpFormatter;
+import org.apache.hbase.thirdparty.org.apache.commons.cli.Options;
+
+
+/**
+ * A real-time monitoring tool for HBase like Unix top command.
+ */
+@InterfaceAudience.Private
+public class HBTop extends Configured implements Tool {
+
+ private static final Logger LOGGER = LoggerFactory.getLogger(HBTop.class);
+
+ public HBTop() {
+this(HBaseConfiguration.create());
+ }
+
+ public HBTop(Configuration conf) {
+super(Objects.requireNonNull(conf));
+ }
+
+ @Override
+ public int run(String[] args) throws Exception {
+// In order to change the log level forcibly, read a custom log4j
configuration file
+InputStream log4jConfigInputStream = this.getClass().getClassLoader()
+ .getResourceAsStream("log4j-hbtop.properties");
+PropertyConfigurator.configure(log4jConfigInputStream);
Review comment:
It's built into the resulting jar, you mean?
Ok, if that's the case, it would be good to just throw an exception if we
ever get a null `InputStream` back from this.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [hbase] joshelser commented on a change in pull request #476: HBASE-11062 hbtop
joshelser commented on a change in pull request #476: HBASE-11062 hbtop
URL: https://github.com/apache/hbase/pull/476#discussion_r318717560
##
File path:
hbase-hbtop/src/main/java/org/apache/hadoop/hbase/hbtop/terminal/TerminalSize.java
##
@@ -0,0 +1,61 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.hadoop.hbase.hbtop.terminal;
+
+import java.util.Objects;
+import org.apache.yetus.audience.InterfaceAudience;
+
+
+/**
+ * Terminal dimensions in 2-d space, measured in number of rows and columns.
+ */
+@InterfaceAudience.Private
+public class TerminalSize {
Review comment:
Nah, I don't think you need to unify. I was going to suggest that, but then
noticed they were distinct. I think it's OK as-is, just wanted to mention :)
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [hbase] wchevreuil commented on a change in pull request #494: HBASE-22380 break circle replication when doing bulkload
wchevreuil commented on a change in pull request #494: HBASE-22380 break circle
replication when doing bulkload
URL: https://github.com/apache/hbase/pull/494#discussion_r318716787
##
File path:
hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/RSRpcServices.java
##
@@ -2364,6 +2364,14 @@ public UpdateFavoredNodesResponse
updateFavoredNodes(RpcController controller,
public BulkLoadHFileResponse bulkLoadHFile(final RpcController controller,
final BulkLoadHFileRequest request) throws ServiceException {
long start = EnvironmentEdgeManager.currentTime();
+if(request.getClusterId() != null &&
+request.getClusterId().equals(this.regionServer.clusterId)){
+ return BulkLoadHFileResponse.newBuilder().setLoaded(true).build();
+}
+String clusterId = request.getClusterId();
Review comment:
Yep, have addressed it on the last changes.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [hbase] wchevreuil commented on a change in pull request #494: HBASE-22380 break circle replication when doing bulkload
wchevreuil commented on a change in pull request #494: HBASE-22380 break circle
replication when doing bulkload
URL: https://github.com/apache/hbase/pull/494#discussion_r318716683
##
File path:
hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestBulkLoadReplication.java
##
@@ -0,0 +1,237 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.hadoop.hbase.regionserver;
+
+import static org.apache.hadoop.hbase.HConstants.REPLICATION_CLUSTER_ID;
+import static org.apache.hadoop.hbase.HConstants.REPLICATION_CONF_DIR;
+import static org.junit.Assert.assertEquals;
+import static org.junit.Assert.assertTrue;
+import java.io.File;
+import java.io.FileOutputStream;
+import java.io.IOException;
+import java.util.List;
+import java.util.Optional;
+import java.util.concurrent.atomic.AtomicInteger;
+import org.apache.hadoop.conf.Configuration;
+import org.apache.hadoop.fs.FSDataOutputStream;
+import org.apache.hadoop.fs.Path;
+import org.apache.hadoop.hbase.Cell;
+import org.apache.hadoop.hbase.CellBuilder;
+import org.apache.hadoop.hbase.CellBuilderFactory;
+import org.apache.hadoop.hbase.CellBuilderType;
+import org.apache.hadoop.hbase.HBaseClassTestRule;
+import org.apache.hadoop.hbase.HBaseTestingUtility;
+import org.apache.hadoop.hbase.HConstants;
+import org.apache.hadoop.hbase.KeyValue;
+import org.apache.hadoop.hbase.client.Get;
+import org.apache.hadoop.hbase.client.Result;
+import org.apache.hadoop.hbase.client.Table;
+import org.apache.hadoop.hbase.coprocessor.ObserverContext;
+import org.apache.hadoop.hbase.coprocessor.RegionCoprocessor;
+import org.apache.hadoop.hbase.coprocessor.RegionCoprocessorEnvironment;
+import org.apache.hadoop.hbase.coprocessor.RegionObserver;
+import org.apache.hadoop.hbase.io.hfile.HFile;
+import org.apache.hadoop.hbase.io.hfile.HFileContext;
+import org.apache.hadoop.hbase.replication.ReplicationPeerConfig;
+import org.apache.hadoop.hbase.replication.TestReplicationBase;
+import org.apache.hadoop.hbase.testclassification.MediumTests;
+import org.apache.hadoop.hbase.testclassification.ReplicationTests;
+import org.apache.hadoop.hbase.tool.BulkLoadHFilesTool;
+import org.apache.hadoop.hbase.util.Bytes;
+import org.apache.hadoop.hbase.util.Pair;
+import org.apache.hadoop.hdfs.MiniDFSCluster;
+import org.junit.After;
+import org.junit.Before;
+import org.junit.BeforeClass;
+import org.junit.ClassRule;
+import org.junit.Rule;
+import org.junit.Test;
+import org.junit.experimental.categories.Category;
+import org.junit.rules.TemporaryFolder;
+import org.junit.rules.TestName;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+/**
+ * Integration test for bulk load replication. Defines two clusters, with two
way replication.
+ * Performs a bulk load on cluster defined by UTIL1 first, asserts the Cell on
the bulk loaded file
+ * gets into the related table in UTIL1, then also validates the same got
replicated to cluster
+ * UTIL2. Then, bulk loads another file into UTIL2, and checks if related
values are present on
+ * UTIL2, and also gets replicated to UTIL1.
+ * It also defines a preBulkLoad coprocessor that is added to all test table
regions on each of the
+ * clusters, in order to count amount of times bulk load actually gets
invoked. This is to certify
+ * we are not entered in the infinite loop condition addressed by HBASE-22380.
+ */
+@Category({ ReplicationTests.class, MediumTests.class})
+public class TestBulkLoadReplication extends TestReplicationBase {
+
+ @ClassRule
+ public static final HBaseClassTestRule CLASS_RULE =
+HBaseClassTestRule.forClass(TestBulkLoadReplication.class);
+
+ protected static final Logger LOG =
+LoggerFactory.getLogger(TestBulkLoadReplication.class);
+
+ private static final String SRC_REP_CLUSTER_ID = "src";
+ private static final String DEST_REP_CLUSTER_ID = "dest";
+ private static final String PEER_ID = "1";
+ private static final AtomicInteger BULK_LOADS_COUNT = new AtomicInteger(0);
+
+ @Rule
+ public TestName name = new TestName();
+
+ @ClassRule
+ public static TemporaryFolder testFolder = new TemporaryFolder();
+
+ @BeforeClass
+ public static void setUpBeforeClass() throws Exception {
+
[jira] [Commented] (HBASE-22872) Don't create normalization plan unnecesarily when split and merge both are disabled
[
https://issues.apache.org/jira/browse/HBASE-22872?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16917955#comment-16917955
]
Hudson commented on HBASE-22872:
Results for branch master
[build #1370 on
builds.a.o|https://builds.apache.org/job/HBase%20Nightly/job/master/1370/]: (x)
*{color:red}-1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://builds.apache.org/job/HBase%20Nightly/job/master/1370//General_Nightly_Build_Report/]
(x) {color:red}-1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://builds.apache.org/job/HBase%20Nightly/job/master/1359//JDK8_Nightly_Build_Report_(Hadoop2)/]
(x) {color:red}-1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://builds.apache.org/job/HBase%20Nightly/job/master/1348//JDK8_Nightly_Build_Report_(Hadoop3)/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
(/) {color:green}+1 client integration test{color}
> Don't create normalization plan unnecesarily when split and merge both are
> disabled
> ---
>
> Key: HBASE-22872
> URL: https://issues.apache.org/jira/browse/HBASE-22872
> Project: HBase
> Issue Type: Improvement
>Affects Versions: 1.4.10
>Reporter: Aman Poonia
>Assignee: Aman Poonia
>Priority: Minor
> Fix For: 3.0.0, 1.5.0, 2.2.1, 1.3.6, 2.1.7
>
> Attachments: HBASE-22872.branch-1.4.001.patch,
> HBASE-22872.branch-1.4.002.patch, HBASE-22872.branch-1.4.003.patch,
> HBASE-22872.branch-1.4.004.patch, HBASE-22872.branch-1.4.005.patch,
> HBASE-22872.branch-2.patch, HBASE-22872.master.001.patch,
> HBASE-22872.master.v01.patch
>
>
> We should not proceed futher in normalization plan creation if split and
> merge both are disabled on a table.
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
[GitHub] [hbase] Apache-HBase commented on issue #549: HBASE-22912 [Backport] HBASE-22867 to branch-1 to avoid ForkJoinPool to spawn thousands of threads
Apache-HBase commented on issue #549: HBASE-22912 [Backport] HBASE-22867 to branch-1 to avoid ForkJoinPool to spawn thousands of threads URL: https://github.com/apache/hbase/pull/549#issuecomment-525842810 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | 0 | reexec | 1979 | Docker mode activated. | ||| _ Prechecks _ | | +1 | dupname | 0 | No case conflicting files found. | | +1 | hbaseanti | 0 | Patch does not have any anti-patterns. | | +1 | @author | 0 | The patch does not contain any @author tags. | | -0 | test4tests | 0 | The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. | ||| _ branch-1 Compile Tests _ | | +1 | mvninstall | 532 | branch-1 passed | | +1 | compile | 53 | branch-1 passed | | +1 | checkstyle | 119 | branch-1 passed | | -1 | shadedjars | 19 | branch has 10 errors when building our shaded downstream artifacts. | | +1 | javadoc | 46 | branch-1 passed | | 0 | spotbugs | 173 | Used deprecated FindBugs config; considering switching to SpotBugs. | | +1 | findbugs | 168 | branch-1 passed | ||| _ Patch Compile Tests _ | | +1 | mvninstall | 137 | the patch passed | | +1 | compile | 45 | the patch passed | | +1 | javac | 45 | the patch passed | | +1 | checkstyle | 114 | the patch passed | | +1 | whitespace | 0 | The patch has no whitespace issues. | | -1 | shadedjars | 16 | patch has 10 errors when building our shaded downstream artifacts. | | +1 | hadoopcheck | 349 | Patch does not cause any errors with Hadoop 2.8.5 2.9.2. | | +1 | javadoc | 40 | the patch passed | | +1 | findbugs | 168 | the patch passed | ||| _ Other Tests _ | | +1 | unit | 8625 | hbase-server in the patch passed. | | +1 | asflicense | 27 | The patch does not generate ASF License warnings. | | | | 12702 | | | Subsystem | Report/Notes | |--:|:-| | Docker | Client=19.03.1 Server=19.03.1 base: https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-549/3/artifact/out/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/549 | | Optional Tests | dupname asflicense javac javadoc unit spotbugs findbugs shadedjars hadoopcheck hbaseanti checkstyle compile | | uname | Linux b98a32557c21 4.15.0-54-generic #58-Ubuntu SMP Mon Jun 24 10:55:24 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | /home/jenkins/jenkins-slave/workspace/HBase-PreCommit-GitHub-PR_PR-549/out/precommit/personality/provided.sh | | git revision | branch-1 / 73e0304 | | Default Java | 1.8.0_222 | | shadedjars | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-549/3/artifact/out/branch-shadedjars.txt | | shadedjars | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-549/3/artifact/out/patch-shadedjars.txt | | Test Results | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-549/3/testReport/ | | Max. process+thread count | 4634 (vs. ulimit of 1) | | modules | C: hbase-server U: hbase-server | | Console output | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-549/3/console | | versions | git=1.9.1 maven=3.0.5 findbugs=3.1.0-RC1 | | Powered by | Apache Yetus 0.10.0 http://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [hbase] Apache-HBase commented on issue #528: HBASE-22890 Verify the files when RegionServer is starting and BucketCache is in file mode
Apache-HBase commented on issue #528: HBASE-22890 Verify the files when RegionServer is starting and BucketCache is in file mode URL: https://github.com/apache/hbase/pull/528#issuecomment-525841943 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | 0 | reexec | 66 | Docker mode activated. | ||| _ Prechecks _ | | +1 | dupname | 0 | No case conflicting files found. | | +1 | hbaseanti | 0 | Patch does not have any anti-patterns. | | +1 | @author | 0 | The patch does not contain any @author tags. | | +1 | test4tests | 0 | The patch appears to include 1 new or modified test files. | ||| _ branch-1 Compile Tests _ | | +1 | mvninstall | 512 | branch-1 passed | | +1 | compile | 53 | branch-1 passed | | +1 | checkstyle | 127 | branch-1 passed | | -1 | shadedjars | 15 | branch has 10 errors when building our shaded downstream artifacts. | | +1 | javadoc | 44 | branch-1 passed | | 0 | spotbugs | 181 | Used deprecated FindBugs config; considering switching to SpotBugs. | | +1 | findbugs | 178 | branch-1 passed | ||| _ Patch Compile Tests _ | | +1 | mvninstall | 136 | the patch passed | | +1 | compile | 47 | the patch passed | | +1 | javac | 47 | the patch passed | | -1 | checkstyle | 114 | hbase-server: The patch generated 10 new + 51 unchanged - 0 fixed = 61 total (was 51) | | +1 | whitespace | 0 | The patch has no whitespace issues. | | -1 | shadedjars | 14 | patch has 10 errors when building our shaded downstream artifacts. | | +1 | hadoopcheck | 345 | Patch does not cause any errors with Hadoop 2.8.5 2.9.2. | | +1 | javadoc | 35 | the patch passed | | -1 | findbugs | 179 | hbase-server generated 1 new + 0 unchanged - 0 fixed = 1 total (was 0) | ||| _ Other Tests _ | | -1 | unit | 1848 | hbase-server in the patch failed. | | -1 | asflicense | 20 | The patch generated 1 ASF License warnings. | | | | 3976 | | | Reason | Tests | |---:|:--| | FindBugs | module:hbase-server | | | Write to static field org.apache.hadoop.hbase.io.hfile.bucket.BucketCache.ALGORITHNAME from instance method new org.apache.hadoop.hbase.io.hfile.bucket.BucketCache(String, long, int, int[], int, int, String, int, Configuration) At BucketCache.java:from instance method new org.apache.hadoop.hbase.io.hfile.bucket.BucketCache(String, long, int, int[], int, int, String, int, Configuration) At BucketCache.java:[line 297] | | Failed junit tests | hadoop.hbase.io.hfile.bucket.TestVerifyBucketCacheFile | | | hadoop.hbase.io.hfile.bucket.TestBucketCache | | Subsystem | Report/Notes | |--:|:-| | Docker | Client=19.03.1 Server=19.03.1 base: https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-528/16/artifact/out/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/528 | | Optional Tests | dupname asflicense javac javadoc unit spotbugs findbugs shadedjars hadoopcheck hbaseanti checkstyle compile | | uname | Linux da21a8f0e924 4.15.0-58-generic #64-Ubuntu SMP Tue Aug 6 11:12:41 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | /home/jenkins/jenkins-slave/workspace/HBase-PreCommit-GitHub-PR_PR-528/out/precommit/personality/provided.sh | | git revision | branch-1 / 73e0304 | | Default Java | 1.8.0_222 | | shadedjars | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-528/16/artifact/out/branch-shadedjars.txt | | checkstyle | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-528/16/artifact/out/diff-checkstyle-hbase-server.txt | | shadedjars | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-528/16/artifact/out/patch-shadedjars.txt | | findbugs | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-528/16/artifact/out/new-findbugs-hbase-server.html | | unit | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-528/16/artifact/out/patch-unit-hbase-server.txt | | Test Results | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-528/16/testReport/ | | asflicense | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-528/16/artifact/out/patch-asflicense-problems.txt | | Max. process+thread count | 758 (vs. ulimit of 1) | | modules | C: hbase-server U: hbase-server | | Console output | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-528/16/console | | versions | git=1.9.1 maven=3.0.5 findbugs=3.1.0-RC1 | | Powered by | Apache Yetus 0.10.0 http://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries abo
[jira] [Commented] (HBASE-22380) break circle replication when doing bulkload
[ https://issues.apache.org/jira/browse/HBASE-22380?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16917945#comment-16917945 ] Wellington Chevreuil commented on HBASE-22380: -- Pushed another commit with the fix to use list of cluster ids for tracking clusters that had already processed given bulkload requests. > break circle replication when doing bulkload > > > Key: HBASE-22380 > URL: https://issues.apache.org/jira/browse/HBASE-22380 > Project: HBase > Issue Type: Bug > Components: Replication >Affects Versions: 3.0.0, 1.5.0, 2.2.0, 1.4.10, 2.0.5, 2.3.0, 2.1.5, 1.3.5 >Reporter: chenxu >Assignee: Wellington Chevreuil >Priority: Critical > Labels: bulkload > Fix For: 3.0.0, 1.5.0, 2.3.0, 1.4.11, 2.1.7, 2.2.2 > > > when enabled master-master bulkload replication, HFiles will be replicated > circularly between two clusters -- This message was sent by Atlassian Jira (v8.3.2#803003)
[jira] [Commented] (HBASE-22893) Change the comment in HBaseClassTestRule to reflect change in default test timeouts
[
https://issues.apache.org/jira/browse/HBASE-22893?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16917929#comment-16917929
]
HBase QA commented on HBASE-22893:
--
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
0s{color} | {color:blue} Docker mode activated. {color} |
| {color:red}-1{color} | {color:red} patch {color} | {color:red} 0m 8s{color}
| {color:red} https://github.com/apache/hbase/pull/552 does not apply to
master. Rebase required? Wrong Branch? See
https://yetus.apache.org/documentation/in-progress/precommit-patchnames for
help. {color} |
\\
\\
|| Subsystem || Report/Notes ||
| GITHUB PR | https://github.com/apache/hbase/pull/552 |
| JIRA Issue | HBASE-22893 |
| Console output |
https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-552/1/console |
| versions | git=2.17.1 |
| Powered by | Apache Yetus 0.10.0 http://yetus.apache.org |
This message was automatically generated.
> Change the comment in HBaseClassTestRule to reflect change in default test
> timeouts
> ---
>
> Key: HBASE-22893
> URL: https://issues.apache.org/jira/browse/HBASE-22893
> Project: HBase
> Issue Type: Bug
>Reporter: Sakthi
>Assignee: Rabi Kumar K C
>Priority: Trivial
> Labels: beginner, noob
> Fix For: 3.0.0, 2.3.0, 2.2.1, 2.1.6
>
>
> The default timeout is 13 minutes. Change the comment accordingly.
> {code}
> // All tests have a 10minute timeout.
> {code}
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
[GitHub] [hbase] Apache-HBase commented on issue #552: HBASE-22893
Apache-HBase commented on issue #552: HBASE-22893 URL: https://github.com/apache/hbase/pull/552#issuecomment-525834601 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | 0 | reexec | 0 | Docker mode activated. | | -1 | patch | 8 | https://github.com/apache/hbase/pull/552 does not apply to master. Rebase required? Wrong Branch? See https://yetus.apache.org/documentation/in-progress/precommit-patchnames for help. | | Subsystem | Report/Notes | |--:|:-| | GITHUB PR | https://github.com/apache/hbase/pull/552 | | JIRA Issue | HBASE-22893 | | Console output | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-552/1/console | | versions | git=2.17.1 | | Powered by | Apache Yetus 0.10.0 http://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Commented] (HBASE-22514) Move rsgroup feature into core of HBase
[
https://issues.apache.org/jira/browse/HBASE-22514?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16917927#comment-16917927
]
Hudson commented on HBASE-22514:
Results for branch HBASE-22514
[build #80 on
builds.a.o|https://builds.apache.org/job/HBase%20Nightly/job/HBASE-22514/80/]:
(x) *{color:red}-1 overall{color}*
details (if available):
(x) {color:red}-1 general checks{color}
-- Something went wrong running this stage, please [check relevant console
output|https://builds.apache.org/job/HBase%20Nightly/job/HBASE-22514/80//console].
(x) {color:red}-1 jdk8 hadoop2 checks{color}
-- Something went wrong running this stage, please [check relevant console
output|https://builds.apache.org/job/HBase%20Nightly/job/HBASE-22514/80//console].
(x) {color:red}-1 jdk8 hadoop3 checks{color}
-- Something went wrong running this stage, please [check relevant console
output|https://builds.apache.org/job/HBase%20Nightly/job/HBASE-22514/80//console].
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
(/) {color:green}+1 client integration test{color}
> Move rsgroup feature into core of HBase
> ---
>
> Key: HBASE-22514
> URL: https://issues.apache.org/jira/browse/HBASE-22514
> Project: HBase
> Issue Type: Umbrella
> Components: Admin, Client, rsgroup
>Reporter: Yechao Chen
>Assignee: Duo Zhang
>Priority: Major
> Attachments: HBASE-22514.master.001.patch,
> image-2019-05-31-18-25-38-217.png
>
>
> The class RSGroupAdminClient is not public
> we need to use java api RSGroupAdminClient to manager RSG
> so RSGroupAdminClient should be public
>
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
[jira] [Commented] (HBASE-22881) Fix non-daemon threads in hbase server implementation
[ https://issues.apache.org/jira/browse/HBASE-22881?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16917918#comment-16917918 ] stack commented on HBASE-22881: --- Merged to branch-2 and branch-2.2. Conflicts when trying to cherry-pick to branch-2.1. You up for a patch for branch-2.1 [~Xiaolin Ha]? Thanks. Leaving open till hear from you. > Fix non-daemon threads in hbase server implementation > - > > Key: HBASE-22881 > URL: https://issues.apache.org/jira/browse/HBASE-22881 > Project: HBase > Issue Type: Bug > Components: master >Reporter: Xiaolin Ha >Assignee: Xiaolin Ha >Priority: Major > Fix For: 3.0.0, 2.3.0, 2.2.2 > > > "pool-8-thread-3" #7252 prio=5 os_prio=0 tid=0x7f91040044c0 nid=0xd71e > waiting on condition [0x7f8f4d209000] >java.lang.Thread.State: WAITING (parking) > at sun.misc.Unsafe.park(Native Method) > - parking to wait for <0x0005c0e49ed0> (a > java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject) > at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) > at > java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) > at > java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) > at > java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) > at > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) > at > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) > at java.lang.Thread.run(Thread.java:748) >Locked ownable synchronizers: > - None > "pool-8-thread-2" #7251 prio=5 os_prio=0 tid=0x7f910c010be0 nid=0xd71d > waiting on condition [0x7f8f4daab000] >java.lang.Thread.State: WAITING (parking) > at sun.misc.Unsafe.park(Native Method) > - parking to wait for <0x0005c0e49ed0> (a > java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject) > at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) > at > java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) > at > java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) > at > java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) > at > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) > at > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) > at java.lang.Thread.run(Thread.java:748) >Locked ownable synchronizers: > - None > "pool-8-thread-1" #7250 prio=5 os_prio=0 tid=0x7f9119d0 nid=0xd71c > waiting on condition [0x7f8f4da6a000] >java.lang.Thread.State: WAITING (parking) > at sun.misc.Unsafe.park(Native Method) > - parking to wait for <0x0005c0e49ed0> (a > java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject) > at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) > at > java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) > at > java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) > at > java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) > at > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) > at > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) > at java.lang.Thread.run(Thread.java:748) >Locked ownable synchronizers: > - None > "pool-5-thread-3" #7248 prio=5 os_prio=0 tid=0x7f9238005ad0 nid=0xd71a > waiting on condition [0x7f8f4cb65000] >java.lang.Thread.State: WAITING (parking) > at sun.misc.Unsafe.park(Native Method) > - parking to wait for <0x0005c0ec51e0> (a > java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject) > at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) > at > java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) > at > java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) > at > java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) > at > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) > at > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) > at java.lang.Thread.run(Thread.java:748) -- This message wa
[jira] [Updated] (HBASE-22881) Fix non-daemon threads in hbase server implementation
[ https://issues.apache.org/jira/browse/HBASE-22881?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack updated HBASE-22881: -- Fix Version/s: 2.2.2 2.3.0 3.0.0 > Fix non-daemon threads in hbase server implementation > - > > Key: HBASE-22881 > URL: https://issues.apache.org/jira/browse/HBASE-22881 > Project: HBase > Issue Type: Bug > Components: master >Reporter: Xiaolin Ha >Assignee: Xiaolin Ha >Priority: Major > Fix For: 3.0.0, 2.3.0, 2.2.2 > > > "pool-8-thread-3" #7252 prio=5 os_prio=0 tid=0x7f91040044c0 nid=0xd71e > waiting on condition [0x7f8f4d209000] >java.lang.Thread.State: WAITING (parking) > at sun.misc.Unsafe.park(Native Method) > - parking to wait for <0x0005c0e49ed0> (a > java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject) > at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) > at > java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) > at > java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) > at > java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) > at > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) > at > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) > at java.lang.Thread.run(Thread.java:748) >Locked ownable synchronizers: > - None > "pool-8-thread-2" #7251 prio=5 os_prio=0 tid=0x7f910c010be0 nid=0xd71d > waiting on condition [0x7f8f4daab000] >java.lang.Thread.State: WAITING (parking) > at sun.misc.Unsafe.park(Native Method) > - parking to wait for <0x0005c0e49ed0> (a > java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject) > at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) > at > java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) > at > java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) > at > java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) > at > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) > at > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) > at java.lang.Thread.run(Thread.java:748) >Locked ownable synchronizers: > - None > "pool-8-thread-1" #7250 prio=5 os_prio=0 tid=0x7f9119d0 nid=0xd71c > waiting on condition [0x7f8f4da6a000] >java.lang.Thread.State: WAITING (parking) > at sun.misc.Unsafe.park(Native Method) > - parking to wait for <0x0005c0e49ed0> (a > java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject) > at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) > at > java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) > at > java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) > at > java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) > at > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) > at > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) > at java.lang.Thread.run(Thread.java:748) >Locked ownable synchronizers: > - None > "pool-5-thread-3" #7248 prio=5 os_prio=0 tid=0x7f9238005ad0 nid=0xd71a > waiting on condition [0x7f8f4cb65000] >java.lang.Thread.State: WAITING (parking) > at sun.misc.Unsafe.park(Native Method) > - parking to wait for <0x0005c0ec51e0> (a > java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject) > at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) > at > java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) > at > java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) > at > java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) > at > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) > at > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) > at java.lang.Thread.run(Thread.java:748) -- This message was sent by Atlassian Jira (v8.3.2#803003)
[jira] [Updated] (HBASE-22881) Fix non-daemon threads in hbase server implementation
[ https://issues.apache.org/jira/browse/HBASE-22881?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack updated HBASE-22881: -- Priority: Major (was: Minor) > Fix non-daemon threads in hbase server implementation > - > > Key: HBASE-22881 > URL: https://issues.apache.org/jira/browse/HBASE-22881 > Project: HBase > Issue Type: Bug > Components: master >Reporter: Xiaolin Ha >Assignee: Xiaolin Ha >Priority: Major > > "pool-8-thread-3" #7252 prio=5 os_prio=0 tid=0x7f91040044c0 nid=0xd71e > waiting on condition [0x7f8f4d209000] >java.lang.Thread.State: WAITING (parking) > at sun.misc.Unsafe.park(Native Method) > - parking to wait for <0x0005c0e49ed0> (a > java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject) > at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) > at > java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) > at > java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) > at > java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) > at > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) > at > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) > at java.lang.Thread.run(Thread.java:748) >Locked ownable synchronizers: > - None > "pool-8-thread-2" #7251 prio=5 os_prio=0 tid=0x7f910c010be0 nid=0xd71d > waiting on condition [0x7f8f4daab000] >java.lang.Thread.State: WAITING (parking) > at sun.misc.Unsafe.park(Native Method) > - parking to wait for <0x0005c0e49ed0> (a > java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject) > at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) > at > java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) > at > java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) > at > java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) > at > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) > at > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) > at java.lang.Thread.run(Thread.java:748) >Locked ownable synchronizers: > - None > "pool-8-thread-1" #7250 prio=5 os_prio=0 tid=0x7f9119d0 nid=0xd71c > waiting on condition [0x7f8f4da6a000] >java.lang.Thread.State: WAITING (parking) > at sun.misc.Unsafe.park(Native Method) > - parking to wait for <0x0005c0e49ed0> (a > java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject) > at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) > at > java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) > at > java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) > at > java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) > at > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) > at > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) > at java.lang.Thread.run(Thread.java:748) >Locked ownable synchronizers: > - None > "pool-5-thread-3" #7248 prio=5 os_prio=0 tid=0x7f9238005ad0 nid=0xd71a > waiting on condition [0x7f8f4cb65000] >java.lang.Thread.State: WAITING (parking) > at sun.misc.Unsafe.park(Native Method) > - parking to wait for <0x0005c0ec51e0> (a > java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject) > at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) > at > java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) > at > java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) > at > java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) > at > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) > at > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) > at java.lang.Thread.run(Thread.java:748) -- This message was sent by Atlassian Jira (v8.3.2#803003)
[GitHub] [hbase] saintstack merged pull request #536: HBASE-22881 Fix non-daemon threads in hbase server implementation
saintstack merged pull request #536: HBASE-22881 Fix non-daemon threads in hbase server implementation URL: https://github.com/apache/hbase/pull/536 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Commented] (HBASE-22935) TaskMonitor warns MonitoredRPCHandler task may be stuck when it recently started
[
https://issues.apache.org/jira/browse/HBASE-22935?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16917912#comment-16917912
]
stack commented on HBASE-22935:
---
+1
> TaskMonitor warns MonitoredRPCHandler task may be stuck when it recently
> started
>
>
> Key: HBASE-22935
> URL: https://issues.apache.org/jira/browse/HBASE-22935
> Project: HBase
> Issue Type: Bug
> Components: logging
>Affects Versions: 3.0.0, 1.4.0, 1.5.0, 1.3.3, 2.0.0
>Reporter: David Manning
>Assignee: David Manning
>Priority: Minor
> Attachments: HBASE-22935.master.001.patch
>
>
> After setting {{hbase.taskmonitor.rpc.warn.time}} to 18, the logs show
> WARN messages such as these
> {noformat}
> 2019-08-08 21:50:02,601 WARN [read for TaskMonitor] monitoring.TaskMonitor -
> Task may be stuck: RpcServer.FifoWFPBQ.default.handler=4,queue=4,port=60020:
> status=Servicing call from :55164: Scan, state=RUNNING,
> startTime=1563305858103, completionTime=-1, queuetimems=1565301002599,
> starttimems=1565301002599, clientaddress=, remoteport=55164,
> packetlength=370, rpcMethod=Scan
> {noformat}
> Notice that the first {{starttimems}} is far in the past. The second
> {{starttimems}} and the {{queuetimems}} are much closer to the log timestamp
> than 180 seconds. I think this is because the warnTime is initialized to the
> time that MonitoredTaskImpl is created, but never updated until we write a
> warn message to the log.
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
[jira] [Commented] (HBASE-22935) TaskMonitor warns MonitoredRPCHandler task may be stuck when it recently started
[
https://issues.apache.org/jira/browse/HBASE-22935?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16917908#comment-16917908
]
Hudson commented on HBASE-22935:
Results for branch master
[build #1369 on
builds.a.o|https://builds.apache.org/job/HBase%20Nightly/job/master/1369/]: (x)
*{color:red}-1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://builds.apache.org/job/HBase%20Nightly/job/master/1369//General_Nightly_Build_Report/]
(/) {color:green}+1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://builds.apache.org/job/HBase%20Nightly/job/master/1363//JDK8_Nightly_Build_Report_(Hadoop2)/]
(x) {color:red}-1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://builds.apache.org/job/HBase%20Nightly/job/master/1363//JDK8_Nightly_Build_Report_(Hadoop3)/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
(/) {color:green}+1 client integration test{color}
> TaskMonitor warns MonitoredRPCHandler task may be stuck when it recently
> started
>
>
> Key: HBASE-22935
> URL: https://issues.apache.org/jira/browse/HBASE-22935
> Project: HBase
> Issue Type: Bug
> Components: logging
>Affects Versions: 3.0.0, 1.4.0, 1.5.0, 1.3.3, 2.0.0
>Reporter: David Manning
>Assignee: David Manning
>Priority: Minor
> Attachments: HBASE-22935.master.001.patch
>
>
> After setting {{hbase.taskmonitor.rpc.warn.time}} to 18, the logs show
> WARN messages such as these
> {noformat}
> 2019-08-08 21:50:02,601 WARN [read for TaskMonitor] monitoring.TaskMonitor -
> Task may be stuck: RpcServer.FifoWFPBQ.default.handler=4,queue=4,port=60020:
> status=Servicing call from :55164: Scan, state=RUNNING,
> startTime=1563305858103, completionTime=-1, queuetimems=1565301002599,
> starttimems=1565301002599, clientaddress=, remoteport=55164,
> packetlength=370, rpcMethod=Scan
> {noformat}
> Notice that the first {{starttimems}} is far in the past. The second
> {{starttimems}} and the {{queuetimems}} are much closer to the log timestamp
> than 180 seconds. I think this is because the warnTime is initialized to the
> time that MonitoredTaskImpl is created, but never updated until we write a
> warn message to the log.
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
[jira] [Commented] (HBASE-21519) Namespace region is never assigned in a HM failover scenario when multiwal is enabled
[
https://issues.apache.org/jira/browse/HBASE-21519?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16917903#comment-16917903
]
stack commented on HBASE-21519:
---
You mean add TestMasterFailoverWithMultiWAL from the patch attached here? Yes,
that sounds like a good idea. If you want me to make a patch w/ it in it, just
say. I'd change name of JIRA too to say test-only. Thanks [~pankaj2461]
> Namespace region is never assigned in a HM failover scenario when multiwal is
> enabled
> -
>
> Key: HBASE-21519
> URL: https://issues.apache.org/jira/browse/HBASE-21519
> Project: HBase
> Issue Type: Bug
> Components: master, wal
>Affects Versions: 2.1.1
>Reporter: Pankaj Kumar
>Assignee: Pankaj Kumar
>Priority: Critical
> Attachments: HBASE-21519.branch-2.patch, providerId_value.png
>
>
> In our test env we found that namespace region is never be assigned on HM
> failover scenario when multiwal feature is enabled,
> {noformat}
> 2018-11-28 01:38:28,085 WARN [master/HM-1:16000:becomeActiveMaster]
> master.HMaster:
> hbase:namespace,,1543339859614.31f6d3383af09e18e1e81ca02a93de15. is NOT
> online; state=\{31f6d3383af09e18e1e81ca02a93de15 state=OPEN,
> ts=1543340156928, server=RS-2,16020,1543339824397};
> ServerCrashProcedures=false. Master startup cannot progress, in
> holding-pattern until region onlined.
> {noformat}
> And finally HM abort with following error,
> {noformat}
> 2018-11-28 01:39:16,858 ERROR
> [ActiveMasterInitializationMonitor-1543338648565] master.HMaster: Master
> failed to complete initialization after 24ms. Please consider submitting
> a bug report including a thread dump of this process.
> 2018-11-28 01:39:18,980 ERROR
> [ActiveMasterInitializationMonitor-1543338648565] master.HMaster: Zombie
> Master exiting. Thread dump to stdout
> {noformat}
> Stack trace:
> {noformat}
> Thread 102 (master/HM-1:16000:becomeActiveMaster):
> State: TIMED_WAITING
> Blocked count: 100
> Waited count: 246
> Stack:
> java.lang.Thread.sleep(Native Method)
> org.apache.hadoop.hbase.util.Threads.sleep(Threads.java:148)
> org.apache.hadoop.hbase.master.HMaster.isRegionOnline(HMaster.java:1166)
>
> org.apache.hadoop.hbase.master.HMaster.waitForNamespaceOnline(HMaster.java:1187)
>
> org.apache.hadoop.hbase.master.HMaster.finishActiveMasterInitialization(HMaster.java:1044)
>
> org.apache.hadoop.hbase.master.HMaster.startActiveMasterManager(HMaster.java:2285)
> org.apache.hadoop.hbase.master.HMaster.lambda$run$0(HMaster.java:590)
> org.apache.hadoop.hbase.master.HMaster$$Lambda$40/1078246575.run(Unknown
> Source)
> java.lang.Thread.run(Thread.java:745)
> {noformat}
>
> Step to reproduce:
> 1) Setup a HBase cluster with 1/2 HM (say HM-1) and 2 RS(say RS-1 & RS-2)
> 2) Enable multiwal feature with following configuration setting and start
> the cluster,
> {noformat}
>
> hbase.wal.provider
> multiwal
>
>
> hbase.wal.regiongrouping.strategy
> identity
>
> {noformat}
> 3) Make sure meta and namespace regions are assigned on different RS, suppose
> RS-1 & RS-2 respectively.
> 4) Create table 't1'
> 5) Flush the meta table explicitly
> 6) Kill the RS-2, so during RS-2 SCP all regions including namespace region
> will be assigned to RS-1.
> 7) Now Kill RS-1 before meta flush happen. Here both RS-2 & RS-1 are
> shutdown now.
> 8) Stop the HM and start RS-1 & RS-2.
> 9) Now start the HM.
> Meta region is assigned successfully but HM is keep waiting for the namespace
> region onlline (Master startup cannot progress, in holding-pattern until
> region onlined) and abort with timeout.
> Observation:
> 1) After step-3 namespace region was assigned to RS-2 and meta entry was as
> follows,
> {noformat}
> hbase:namespace,,1543339859614.31f6d3383af09e18e1e81ca02a93de15.
> column=info:server, timestamp=1543339860920, value=RS-2:16020
> hbase:namespace,,1543339859614.31f6d3383af09e18e1e81ca02a93de15.
> column=info:serverstartcode, timestamp=1543339860920, value=1543339824397
> {noformat}
> 2) After step-6 namespace region was assigned to RS-1 and meta entry was as
> follows,
> {noformat}
> hbase:namespace,,1543339859614.31f6d3383af09e18e1e81ca02a93de15.
> column=info:server, timestamp=1543339880920, value=RS-1:16020
> hbase:namespace,,1543339859614.31f6d3383af09e18e1e81ca02a93de15.
> column=info:serverstartcode, timestamp=1543339880920, value=1543339829288
> {noformat}
> 3) After Step-9, meta entry for namespace region was as follows,
> {noformat}
> hbase:namespace,,1543339859614.31f6d3383af09e18e1e81ca02a93de15.
> column=info:server, timestamp=1543339860920, value=RS-2:16020
> hbase:namespace,,1543339859614.31f6d3383af09e18e1e81ca02a93de15.
> column=info:serverstartcode, timestamp=1543339860920, va
[jira] [Commented] (HBASE-22938) Fold all the system tables to hbase:meta
[
https://issues.apache.org/jira/browse/HBASE-22938?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16917901#comment-16917901
]
stack commented on HBASE-22938:
---
Which system tables? What schema does each system table need?
* Namespace table has been integrated already
* ACL table
* Replication Peers (and history)
What else?
> Fold all the system tables to hbase:meta
>
>
> Key: HBASE-22938
> URL: https://issues.apache.org/jira/browse/HBASE-22938
> Project: HBase
> Issue Type: Brainstorming
>Reporter: Duo Zhang
>Priority: Major
>
> Quote my post on HBASE-15867 here, on how to deal with the dead lock when we
> want to store replication queues to hbase:replication table.
> {quote}
> We could add a special prefix in the row key for different system tables, and
> make a special family for it. For example, for all the records in hbase:acl,
> we could introduce a prefix like ':::acl:::', since we do not allow ':' in
> either namespace or table name, so it will not conflict with the existing
> table related records. And the family could be namd as 'acl'.
> And we could make a special split policy that only splits at these special
> prefixs, so it will not break any assumptions so far, as all the records for
> the 'system table' are in the same region.
> {quote}
> And I think there are also other advantages, for example the start up logic
> can be greatly simplified.
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
[jira] [Commented] (HBASE-22935) TaskMonitor warns MonitoredRPCHandler task may be stuck when it recently started
[
https://issues.apache.org/jira/browse/HBASE-22935?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16917900#comment-16917900
]
Hudson commented on HBASE-22935:
Results for branch branch-2.1
[build #1532 on
builds.a.o|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.1/1532/]:
(x) *{color:red}-1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.1/1532//General_Nightly_Build_Report/]
(/) {color:green}+1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.1/1526//JDK8_Nightly_Build_Report_(Hadoop2)/]
(/) {color:green}+1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.1/1526//JDK8_Nightly_Build_Report_(Hadoop3)/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
(/) {color:green}+1 client integration test{color}
> TaskMonitor warns MonitoredRPCHandler task may be stuck when it recently
> started
>
>
> Key: HBASE-22935
> URL: https://issues.apache.org/jira/browse/HBASE-22935
> Project: HBase
> Issue Type: Bug
> Components: logging
>Affects Versions: 3.0.0, 1.4.0, 1.5.0, 1.3.3, 2.0.0
>Reporter: David Manning
>Assignee: David Manning
>Priority: Minor
> Attachments: HBASE-22935.master.001.patch
>
>
> After setting {{hbase.taskmonitor.rpc.warn.time}} to 18, the logs show
> WARN messages such as these
> {noformat}
> 2019-08-08 21:50:02,601 WARN [read for TaskMonitor] monitoring.TaskMonitor -
> Task may be stuck: RpcServer.FifoWFPBQ.default.handler=4,queue=4,port=60020:
> status=Servicing call from :55164: Scan, state=RUNNING,
> startTime=1563305858103, completionTime=-1, queuetimems=1565301002599,
> starttimems=1565301002599, clientaddress=, remoteport=55164,
> packetlength=370, rpcMethod=Scan
> {noformat}
> Notice that the first {{starttimems}} is far in the past. The second
> {{starttimems}} and the {{queuetimems}} are much closer to the log timestamp
> than 180 seconds. I think this is because the warnTime is initialized to the
> time that MonitoredTaskImpl is created, but never updated until we write a
> warn message to the log.
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
[jira] [Created] (HBASE-22943) Various procedures should not cache log trace level
Sean Busbey created HBASE-22943: --- Summary: Various procedures should not cache log trace level Key: HBASE-22943 URL: https://issues.apache.org/jira/browse/HBASE-22943 Project: HBase Issue Type: Bug Components: proc-v2 Reporter: Sean Busbey Assignee: Sean Busbey several of the procedures have an idiom where they keep a member variable for if the log is at trace level or not, wrapped in a function so that it can be lazily looked up. This gives us an overhead per call of autoboxing and a function call, instead of just the function call from asking the logging system directly. -- This message was sent by Atlassian Jira (v8.3.2#803003)
[jira] [Work started] (HBASE-22943) Various procedures should not cache log trace level
[ https://issues.apache.org/jira/browse/HBASE-22943?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Work on HBASE-22943 started by Sean Busbey. --- > Various procedures should not cache log trace level > --- > > Key: HBASE-22943 > URL: https://issues.apache.org/jira/browse/HBASE-22943 > Project: HBase > Issue Type: Bug > Components: proc-v2 >Reporter: Sean Busbey >Assignee: Sean Busbey >Priority: Minor > > several of the procedures have an idiom where they keep a member variable for > if the log is at trace level or not, wrapped in a function so that it can be > lazily looked up. This gives us an overhead per call of autoboxing and a > function call, instead of just the function call from asking the logging > system directly. -- This message was sent by Atlassian Jira (v8.3.2#803003)
[jira] [Commented] (HBASE-22935) TaskMonitor warns MonitoredRPCHandler task may be stuck when it recently started
[
https://issues.apache.org/jira/browse/HBASE-22935?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16917891#comment-16917891
]
Hudson commented on HBASE-22935:
Results for branch branch-2.2
[build #549 on
builds.a.o|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.2/549/]:
(x) *{color:red}-1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.2/549//General_Nightly_Build_Report/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
(/) {color:green}+1 client integration test{color}
> TaskMonitor warns MonitoredRPCHandler task may be stuck when it recently
> started
>
>
> Key: HBASE-22935
> URL: https://issues.apache.org/jira/browse/HBASE-22935
> Project: HBase
> Issue Type: Bug
> Components: logging
>Affects Versions: 3.0.0, 1.4.0, 1.5.0, 1.3.3, 2.0.0
>Reporter: David Manning
>Assignee: David Manning
>Priority: Minor
> Attachments: HBASE-22935.master.001.patch
>
>
> After setting {{hbase.taskmonitor.rpc.warn.time}} to 18, the logs show
> WARN messages such as these
> {noformat}
> 2019-08-08 21:50:02,601 WARN [read for TaskMonitor] monitoring.TaskMonitor -
> Task may be stuck: RpcServer.FifoWFPBQ.default.handler=4,queue=4,port=60020:
> status=Servicing call from :55164: Scan, state=RUNNING,
> startTime=1563305858103, completionTime=-1, queuetimems=1565301002599,
> starttimems=1565301002599, clientaddress=, remoteport=55164,
> packetlength=370, rpcMethod=Scan
> {noformat}
> Notice that the first {{starttimems}} is far in the past. The second
> {{starttimems}} and the {{queuetimems}} are much closer to the log timestamp
> than 180 seconds. I think this is because the warnTime is initialized to the
> time that MonitoredTaskImpl is created, but never updated until we write a
> warn message to the log.
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
[jira] [Assigned] (HBASE-20688) Refguide has "HBase Backup" section and a chapter named "Backup and Restore"; neither refers to the other
[ https://issues.apache.org/jira/browse/HBASE-20688?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rabi Kumar K C reassigned HBASE-20688: -- Assignee: Rabi Kumar K C > Refguide has "HBase Backup" section and a chapter named "Backup and Restore"; > neither refers to the other > - > > Key: HBASE-20688 > URL: https://issues.apache.org/jira/browse/HBASE-20688 > Project: HBase > Issue Type: Bug >Reporter: stack >Assignee: Rabi Kumar K C >Priority: Major > > The two backup sections are not connected or related. It'd be confusing to > the user. Needs addressing. -- This message was sent by Atlassian Jira (v8.3.2#803003)
[jira] [Commented] (HBASE-22935) TaskMonitor warns MonitoredRPCHandler task may be stuck when it recently started
[
https://issues.apache.org/jira/browse/HBASE-22935?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16917887#comment-16917887
]
Hudson commented on HBASE-22935:
Results for branch branch-2
[build #2202 on
builds.a.o|https://builds.apache.org/job/HBase%20Nightly/job/branch-2/2202/]:
(x) *{color:red}-1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2/2202//General_Nightly_Build_Report/]
(x) {color:red}-1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2/2196//JDK8_Nightly_Build_Report_(Hadoop2)/]
(x) {color:red}-1 jdk8 hadoop3 checks{color}
-- Something went wrong running this stage, please [check relevant console
output|https://builds.apache.org/job/HBase%20Nightly/job/branch-2/2192//console].
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
(/) {color:green}+1 client integration test{color}
> TaskMonitor warns MonitoredRPCHandler task may be stuck when it recently
> started
>
>
> Key: HBASE-22935
> URL: https://issues.apache.org/jira/browse/HBASE-22935
> Project: HBase
> Issue Type: Bug
> Components: logging
>Affects Versions: 3.0.0, 1.4.0, 1.5.0, 1.3.3, 2.0.0
>Reporter: David Manning
>Assignee: David Manning
>Priority: Minor
> Attachments: HBASE-22935.master.001.patch
>
>
> After setting {{hbase.taskmonitor.rpc.warn.time}} to 18, the logs show
> WARN messages such as these
> {noformat}
> 2019-08-08 21:50:02,601 WARN [read for TaskMonitor] monitoring.TaskMonitor -
> Task may be stuck: RpcServer.FifoWFPBQ.default.handler=4,queue=4,port=60020:
> status=Servicing call from :55164: Scan, state=RUNNING,
> startTime=1563305858103, completionTime=-1, queuetimems=1565301002599,
> starttimems=1565301002599, clientaddress=, remoteport=55164,
> packetlength=370, rpcMethod=Scan
> {noformat}
> Notice that the first {{starttimems}} is far in the past. The second
> {{starttimems}} and the {{queuetimems}} are much closer to the log timestamp
> than 180 seconds. I think this is because the warnTime is initialized to the
> time that MonitoredTaskImpl is created, but never updated until we write a
> warn message to the log.
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
[jira] [Created] (HBASE-22942) move snapshot verification during restore / clone into the async master side handling
Sean Busbey created HBASE-22942: --- Summary: move snapshot verification during restore / clone into the async master side handling Key: HBASE-22942 URL: https://issues.apache.org/jira/browse/HBASE-22942 Project: HBase Issue Type: Improvement Components: snapshots Affects Versions: 2.2.0, 2.1.0, 1.4.0, 1.3.0 Reporter: Sean Busbey Right now we do snapshot verification prior to queueing the restore / clone request from the client. That means the initial call from the client has to block until we're done. On large manifests (~single digit millions) this easily takes longer than the default timeout of 20 minutes. Instead we should handle verification as one of the steps in the relevant procedure (hbase 2+) or table handler (hbase 1) so that the master can do it in the background and report status. -- This message was sent by Atlassian Jira (v8.3.2#803003)
[jira] [Commented] (HBASE-20688) Refguide has "HBase Backup" section and a chapter named "Backup and Restore"; neither refers to the other
[ https://issues.apache.org/jira/browse/HBASE-20688?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16917874#comment-16917874 ] stack commented on HBASE-20688: --- No suggestion. At a minimum the two should be linked. Could move stuff around so just the one section? The backup is not in any shipping release currently (and I don't think it ever should be since it seems like an abandoned feature... ) so just keep that in mind making edits. Thanks for giving this a go. > Refguide has "HBase Backup" section and a chapter named "Backup and Restore"; > neither refers to the other > - > > Key: HBASE-20688 > URL: https://issues.apache.org/jira/browse/HBASE-20688 > Project: HBase > Issue Type: Bug >Reporter: stack >Priority: Major > > The two backup sections are not connected or related. It'd be confusing to > the user. Needs addressing. -- This message was sent by Atlassian Jira (v8.3.2#803003)
[jira] [Comment Edited] (HBASE-20688) Refguide has "HBase Backup" section and a chapter named "Backup and Restore"; neither refers to the other
[ https://issues.apache.org/jira/browse/HBASE-20688?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16917874#comment-16917874 ] stack edited comment on HBASE-20688 at 8/28/19 4:02 PM: No suggestion. At a minimum the two should be linked. Could move stuff around so just the one section? Or maybe, given that the backup is not in any shipping release currently (and I don't think it ever should be since it seems like an abandoned feature... ), you'd add link at end of one section to the other for folks looking for 'more'. was (Author: stack): No suggestion. At a minimum the two should be linked. Could move stuff around so just the one section? The backup is not in any shipping release currently (and I don't think it ever should be since it seems like an abandoned feature... ) so just keep that in mind making edits. Thanks for giving this a go. > Refguide has "HBase Backup" section and a chapter named "Backup and Restore"; > neither refers to the other > - > > Key: HBASE-20688 > URL: https://issues.apache.org/jira/browse/HBASE-20688 > Project: HBase > Issue Type: Bug >Reporter: stack >Priority: Major > > The two backup sections are not connected or related. It'd be confusing to > the user. Needs addressing. -- This message was sent by Atlassian Jira (v8.3.2#803003)
[jira] [Commented] (HBASE-22796) [HBCK2] Add fix of overlaps to fixMeta hbck Service
[ https://issues.apache.org/jira/browse/HBASE-22796?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16917872#comment-16917872 ] stack commented on HBASE-22796: --- Oh. Sorry for mis-messaging. I could give it a go but if you want to, go for it. One amendment to my comment above is that we'd not bother doing the 'Next' and 'Thereafter' described above -- i.e. client-side equivalents of what we are adding here; rather, we'd get folks to upgrade rather than try and do some hokey client-side hackup (suggestion by co-worker which I think a good one). > [HBCK2] Add fix of overlaps to fixMeta hbck Service > --- > > Key: HBASE-22796 > URL: https://issues.apache.org/jira/browse/HBASE-22796 > Project: HBase > Issue Type: Sub-task >Reporter: stack >Assignee: Sakthi >Priority: Major > > fixMeta currently does holes in meta only courtesy of HBASE-22771 which added > fixMeta to hbck Service; missing was fix of overlaps too. This JIRA is about > adding fix of overlaps to general fixMeta call. -- This message was sent by Atlassian Jira (v8.3.2#803003)
[GitHub] [hbase] Apache-HBase commented on issue #501: HBASE-22699 refactor isMetaClearingException
Apache-HBase commented on issue #501: HBASE-22699 refactor isMetaClearingException URL: https://github.com/apache/hbase/pull/501#issuecomment-525810212 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | 0 | reexec | 68 | Docker mode activated. | ||| _ Prechecks _ | | +1 | dupname | 0 | No case conflicting files found. | | +1 | hbaseanti | 0 | Patch does not have any anti-patterns. | | +1 | @author | 0 | The patch does not contain any @author tags. | | +1 | test4tests | 0 | The patch appears to include 1 new or modified test files. | ||| _ master Compile Tests _ | | 0 | mvndep | 38 | Maven dependency ordering for branch | | +1 | mvninstall | 425 | master passed | | +1 | compile | 99 | master passed | | +1 | checkstyle | 135 | master passed | | +1 | shadedjars | 346 | branch has no errors when building our shaded downstream artifacts. | | +1 | javadoc | 66 | master passed | | 0 | spotbugs | 280 | Used deprecated FindBugs config; considering switching to SpotBugs. | | +1 | findbugs | 357 | master passed | ||| _ Patch Compile Tests _ | | 0 | mvndep | 16 | Maven dependency ordering for patch | | +1 | mvninstall | 386 | the patch passed | | +1 | compile | 103 | the patch passed | | +1 | javac | 103 | the patch passed | | +1 | checkstyle | 147 | the patch passed | | +1 | whitespace | 0 | The patch has no whitespace issues. | | +1 | shadedjars | 373 | patch has no errors when building our shaded downstream artifacts. | | +1 | hadoopcheck | 1211 | Patch does not cause any errors with Hadoop 2.8.5 2.9.2 or 3.1.2. | | +1 | javadoc | 62 | the patch passed | | +1 | findbugs | 370 | the patch passed | ||| _ Other Tests _ | | +1 | unit | 115 | hbase-client in the patch passed. | | -1 | unit | 10242 | hbase-server in the patch failed. | | +1 | asflicense | 56 | The patch does not generate ASF License warnings. | | | | 15137 | | | Reason | Tests | |---:|:--| | Failed junit tests | hadoop.hbase.client.TestMetaCache | | Subsystem | Report/Notes | |--:|:-| | Docker | Client=19.03.1 Server=19.03.1 base: https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-501/43/artifact/out/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/501 | | Optional Tests | dupname asflicense javac javadoc unit spotbugs findbugs shadedjars hadoopcheck hbaseanti checkstyle compile | | uname | Linux 072890f337b3 4.15.0-58-generic #64-Ubuntu SMP Tue Aug 6 11:12:41 UTC 2019 x86_64 GNU/Linux | | Build tool | maven | | Personality | /home/jenkins/jenkins-slave/workspace/HBase-PreCommit-GitHub-PR_PR-501/out/precommit/personality/provided.sh | | git revision | master / 56980fb097 | | Default Java | 1.8.0_181 | | unit | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-501/43/artifact/out/patch-unit-hbase-server.txt | | Test Results | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-501/43/testReport/ | | Max. process+thread count | 4662 (vs. ulimit of 1) | | modules | C: hbase-client hbase-server U: . | | Console output | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-501/43/console | | versions | git=2.11.0 maven=2018-06-17T18:33:14Z) findbugs=3.1.11 | | Powered by | Apache Yetus 0.10.0 http://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [hbase] saintstack commented on a change in pull request #556: HBASE-22941 merge operation returns parent regions in random order
saintstack commented on a change in pull request #556: HBASE-22941 merge
operation returns parent regions in random order
URL: https://github.com/apache/hbase/pull/556#discussion_r318661230
##
File path:
hbase-server/src/main/java/org/apache/hadoop/hbase/master/assignment/RegionStateStore.java
##
@@ -263,7 +263,7 @@ public void mergeRegions(RegionInfo child, RegionInfo []
parents, ServerName ser
throws IOException {
TableDescriptor htd = getDescriptor(child.getTable());
boolean globalScope = htd.hasGlobalReplicationScope();
-Map parentSeqNums = new HashMap<>(parents.length);
+Map parentSeqNums = new TreeMap<>();
Review comment:
nit... could have been a NavigableMap... but this'll do.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[jira] [Commented] (HBASE-22941) MetaTableAccessor.getMergeRegions() returns parent regions in random order
[ https://issues.apache.org/jira/browse/HBASE-22941?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16917868#comment-16917868 ] István Tóth commented on HBASE-22941: - The attached patch stores the parent regions in META in ascending order. I have also removed a leftover check for merging exactly two regions, and added a unit test. > MetaTableAccessor.getMergeRegions() returns parent regions in random order > -- > > Key: HBASE-22941 > URL: https://issues.apache.org/jira/browse/HBASE-22941 > Project: HBase > Issue Type: Bug >Affects Versions: master >Reporter: István Tóth >Assignee: István Tóth >Priority: Major > > The multi-region merge patch has changed the behaviour of the > MetaTableAccessor.getMergeRegions() method. Earlier, the parent regions were > returned in the same order that they were passed to mergeRegion(). > The patch has changed this to return the regions in random order. To be more > precise, the parent regions are stored in random order in Meta. > I propose storing and returning the parent regions in ascending order. -- This message was sent by Atlassian Jira (v8.3.2#803003)
[GitHub] [hbase] stoty opened a new pull request #556: HBASE-22941 merge operation returns parent regions in random order
stoty opened a new pull request #556: HBASE-22941 merge operation returns parent regions in random order URL: https://github.com/apache/hbase/pull/556 store and return the merge parent regions in ascending order remove left over check for exactly two merged regions add unit test This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
