[jira] [Commented] (HBASE-22695) Store the rsgroup of a table in table configuration
[ https://issues.apache.org/jira/browse/HBASE-22695?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16893330#comment-16893330 ] Duo Zhang commented on HBASE-22695: --- I mean do it asynchronously. Just need to trigger the balancer I think. > Store the rsgroup of a table in table configuration > --- > > Key: HBASE-22695 > URL: https://issues.apache.org/jira/browse/HBASE-22695 > Project: HBase > Issue Type: Sub-task > Components: rsgroup >Reporter: Duo Zhang >Assignee: Duo Zhang >Priority: Major > -- This message was sent by Atlassian JIRA (v7.6.14#76016)
[GitHub] [hbase] casuallc opened a new pull request #411: HBASE-22739 check bound where removeRegion
casuallc opened a new pull request #411: HBASE-22739 check bound where removeRegion URL: https://github.com/apache/hbase/pull/411 https://issues.apache.org/jira/browse/HBASE-22739. this is the last !!! This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Updated] (HBASE-22740) [RSGroup] Forward-port HBASE-22658 to master branch and branch-2.x
[ https://issues.apache.org/jira/browse/HBASE-22740?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reid Chan updated HBASE-22740: -- Component/s: rsgroup > [RSGroup] Forward-port HBASE-22658 to master branch and branch-2.x > -- > > Key: HBASE-22740 > URL: https://issues.apache.org/jira/browse/HBASE-22740 > Project: HBase > Issue Type: Bug > Components: rsgroup >Reporter: Reid Chan >Priority: Major > -- This message was sent by Atlassian JIRA (v7.6.14#76016)
[jira] [Commented] (HBASE-22658) region_mover.rb should choose same rsgroup servers as target servers
[ https://issues.apache.org/jira/browse/HBASE-22658?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16893321#comment-16893321 ] liang.feng commented on HBASE-22658: [~reidchan] Thank you for your help. I learned a lot from you. I've turned off PR. > region_mover.rb should choose same rsgroup servers as target servers > -- > > Key: HBASE-22658 > URL: https://issues.apache.org/jira/browse/HBASE-22658 > Project: HBase > Issue Type: Bug > Components: rsgroup, shell >Affects Versions: 1.4.10 >Reporter: liang.feng >Priority: Major > Labels: gracefulshutdown, region_mover, rsgroup > Fix For: 1.5.0, 1.4.11 > > Attachments: HBASE-22658.branch-1.002.patch, > HBASE-22658.branch-1.patch, with_patch.png, without_patch.png > > > There are many retries when i am using graceful_stop.sh to shutdown region > server after using regroup, because the target server in a different rsgroup. > This makes it slow to graceful shutdown a regionserver. So i think that > region_mover.rb should only choose same rsgroup servers as target servers. > Region mover is implemented by jruby in hbase1.x and is implemented by java > in hbase2.x . I tried to modify the RegionMover.java class to use the same > logic in hbase2.x, but mvn package failed due to hbase-server module and > hbase-rsgroup moudle needed to depend on each other, then maven throw a "The > projects in the reactor contain a cyclic reference". I couldn't solve it.So I > just uploaded patch for hbase1.x . > -- This message was sent by Atlassian JIRA (v7.6.14#76016)
[jira] [Commented] (HBASE-22658) region_mover.rb should choose same rsgroup servers as target servers
[ https://issues.apache.org/jira/browse/HBASE-22658?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16893319#comment-16893319 ] Reid Chan commented on HBASE-22658: --- Filed a forward-port JIRA HBASE-22740. > region_mover.rb should choose same rsgroup servers as target servers > -- > > Key: HBASE-22658 > URL: https://issues.apache.org/jira/browse/HBASE-22658 > Project: HBase > Issue Type: Bug > Components: rsgroup, shell >Affects Versions: 1.4.10 >Reporter: liang.feng >Priority: Major > Labels: gracefulshutdown, region_mover, rsgroup > Fix For: 1.5.0, 1.4.11 > > Attachments: HBASE-22658.branch-1.002.patch, > HBASE-22658.branch-1.patch, with_patch.png, without_patch.png > > > There are many retries when i am using graceful_stop.sh to shutdown region > server after using regroup, because the target server in a different rsgroup. > This makes it slow to graceful shutdown a regionserver. So i think that > region_mover.rb should only choose same rsgroup servers as target servers. > Region mover is implemented by jruby in hbase1.x and is implemented by java > in hbase2.x . I tried to modify the RegionMover.java class to use the same > logic in hbase2.x, but mvn package failed due to hbase-server module and > hbase-rsgroup moudle needed to depend on each other, then maven throw a "The > projects in the reactor contain a cyclic reference". I couldn't solve it.So I > just uploaded patch for hbase1.x . > -- This message was sent by Atlassian JIRA (v7.6.14#76016)
[jira] [Created] (HBASE-22740) [RSGroup] Forward-port HBASE-22658 to master branch and branch-2.x
Reid Chan created HBASE-22740: - Summary: [RSGroup] Forward-port HBASE-22658 to master branch and branch-2.x Key: HBASE-22740 URL: https://issues.apache.org/jira/browse/HBASE-22740 Project: HBase Issue Type: Bug Reporter: Reid Chan -- This message was sent by Atlassian JIRA (v7.6.14#76016)
[GitHub] [hbase] a516072575 closed pull request #367: region_mover.rb should choose same rsgroup servers as target servers
a516072575 closed pull request #367: region_mover.rb should choose same rsgroup servers as target servers URL: https://github.com/apache/hbase/pull/367 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Updated] (HBASE-22658) region_mover.rb should choose same rsgroup servers as target servers
[ https://issues.apache.org/jira/browse/HBASE-22658?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reid Chan updated HBASE-22658: -- Resolution: Fixed Hadoop Flags: Reviewed Status: Resolved (was: Patch Available) > region_mover.rb should choose same rsgroup servers as target servers > -- > > Key: HBASE-22658 > URL: https://issues.apache.org/jira/browse/HBASE-22658 > Project: HBase > Issue Type: Bug > Components: rsgroup, shell >Affects Versions: 1.4.10 >Reporter: liang.feng >Priority: Major > Labels: gracefulshutdown, region_mover, rsgroup > Fix For: 1.5.0, 1.4.11 > > Attachments: HBASE-22658.branch-1.002.patch, > HBASE-22658.branch-1.patch, with_patch.png, without_patch.png > > > There are many retries when i am using graceful_stop.sh to shutdown region > server after using regroup, because the target server in a different rsgroup. > This makes it slow to graceful shutdown a regionserver. So i think that > region_mover.rb should only choose same rsgroup servers as target servers. > Region mover is implemented by jruby in hbase1.x and is implemented by java > in hbase2.x . I tried to modify the RegionMover.java class to use the same > logic in hbase2.x, but mvn package failed due to hbase-server module and > hbase-rsgroup moudle needed to depend on each other, then maven throw a "The > projects in the reactor contain a cyclic reference". I couldn't solve it.So I > just uploaded patch for hbase1.x . > -- This message was sent by Atlassian JIRA (v7.6.14#76016)
[jira] [Updated] (HBASE-22658) region_mover.rb should choose same rsgroup servers as target servers
[ https://issues.apache.org/jira/browse/HBASE-22658?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reid Chan updated HBASE-22658: -- Issue Type: Bug (was: Improvement) > region_mover.rb should choose same rsgroup servers as target servers > -- > > Key: HBASE-22658 > URL: https://issues.apache.org/jira/browse/HBASE-22658 > Project: HBase > Issue Type: Bug > Components: rsgroup, shell >Affects Versions: 1.4.10 >Reporter: liang.feng >Priority: Major > Labels: gracefulshutdown, region_mover, rsgroup > Fix For: 1.5.0, 1.4.11 > > Attachments: HBASE-22658.branch-1.002.patch, > HBASE-22658.branch-1.patch, with_patch.png, without_patch.png > > > There are many retries when i am using graceful_stop.sh to shutdown region > server after using regroup, because the target server in a different rsgroup. > This makes it slow to graceful shutdown a regionserver. So i think that > region_mover.rb should only choose same rsgroup servers as target servers. > Region mover is implemented by jruby in hbase1.x and is implemented by java > in hbase2.x . I tried to modify the RegionMover.java class to use the same > logic in hbase2.x, but mvn package failed due to hbase-server module and > hbase-rsgroup moudle needed to depend on each other, then maven throw a "The > projects in the reactor contain a cyclic reference". I couldn't solve it.So I > just uploaded patch for hbase1.x . > -- This message was sent by Atlassian JIRA (v7.6.14#76016)
[jira] [Updated] (HBASE-22658) region_mover.rb should choose same rsgroup servers as target servers
[ https://issues.apache.org/jira/browse/HBASE-22658?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reid Chan updated HBASE-22658: -- Fix Version/s: 1.4.11 1.5.0 > region_mover.rb should choose same rsgroup servers as target servers > -- > > Key: HBASE-22658 > URL: https://issues.apache.org/jira/browse/HBASE-22658 > Project: HBase > Issue Type: Improvement > Components: rsgroup, shell >Affects Versions: 1.4.10 >Reporter: liang.feng >Priority: Major > Labels: gracefulshutdown, region_mover, rsgroup > Fix For: 1.5.0, 1.4.11 > > Attachments: HBASE-22658.branch-1.002.patch, > HBASE-22658.branch-1.patch, with_patch.png, without_patch.png > > > There are many retries when i am using graceful_stop.sh to shutdown region > server after using regroup, because the target server in a different rsgroup. > This makes it slow to graceful shutdown a regionserver. So i think that > region_mover.rb should only choose same rsgroup servers as target servers. > Region mover is implemented by jruby in hbase1.x and is implemented by java > in hbase2.x . I tried to modify the RegionMover.java class to use the same > logic in hbase2.x, but mvn package failed due to hbase-server module and > hbase-rsgroup moudle needed to depend on each other, then maven throw a "The > projects in the reactor contain a cyclic reference". I couldn't solve it.So I > just uploaded patch for hbase1.x . > -- This message was sent by Atlassian JIRA (v7.6.14#76016)
[jira] [Commented] (HBASE-22658) region_mover.rb should choose same rsgroup servers as target servers
[ https://issues.apache.org/jira/browse/HBASE-22658?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16893316#comment-16893316 ] Reid Chan commented on HBASE-22658: --- Pushed to branch-1 and branch-1.4, thanks for the contribution, [~a516072575], and please close the PR on github. > region_mover.rb should choose same rsgroup servers as target servers > -- > > Key: HBASE-22658 > URL: https://issues.apache.org/jira/browse/HBASE-22658 > Project: HBase > Issue Type: Improvement > Components: rsgroup, shell >Affects Versions: 1.4.10 >Reporter: liang.feng >Priority: Major > Labels: gracefulshutdown, region_mover, rsgroup > Attachments: HBASE-22658.branch-1.002.patch, > HBASE-22658.branch-1.patch, with_patch.png, without_patch.png > > > There are many retries when i am using graceful_stop.sh to shutdown region > server after using regroup, because the target server in a different rsgroup. > This makes it slow to graceful shutdown a regionserver. So i think that > region_mover.rb should only choose same rsgroup servers as target servers. > Region mover is implemented by jruby in hbase1.x and is implemented by java > in hbase2.x . I tried to modify the RegionMover.java class to use the same > logic in hbase2.x, but mvn package failed due to hbase-server module and > hbase-rsgroup moudle needed to depend on each other, then maven throw a "The > projects in the reactor contain a cyclic reference". I couldn't solve it.So I > just uploaded patch for hbase1.x . > -- This message was sent by Atlassian JIRA (v7.6.14#76016)
[jira] [Updated] (HBASE-22709) Add a chore thread in master to do hbck checking and display results in 'HBCK Report' page
[ https://issues.apache.org/jira/browse/HBASE-22709?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack updated HBASE-22709: -- Summary: Add a chore thread in master to do hbck checking and display results in 'HBCK Report' page (was: Add a chore thread in master to do hbck checking) > Add a chore thread in master to do hbck checking and display results in 'HBCK > Report' page > -- > > Key: HBASE-22709 > URL: https://issues.apache.org/jira/browse/HBASE-22709 > Project: HBase > Issue Type: Sub-task >Reporter: Guanghao Zhang >Assignee: Guanghao Zhang >Priority: Major > Attachments: HBASE-22709.master.001.patch > > > See HBASE-21965. There may have two type of failed splited/merged regions. > One is orphan region on filesystem. Another one is unassigned daughter > regions or merged regions. Plan to add a seperate hbck.jsp for them. Will add > more hbck problems to this UI in future. -- This message was sent by Atlassian JIRA (v7.6.14#76016)
[GitHub] [hbase] saintstack commented on issue #404: HBASE-22709 Add a chore thread in master to do hbck checking
saintstack commented on issue #404: HBASE-22709 Add a chore thread in master to do hbck checking URL: https://github.com/apache/hbase/pull/404#issuecomment-515301782 Ok on all of the above. Can do aggregation as a follow-on if opportunity. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Commented] (HBASE-22734) Format error message in create_namespace command to avoid stacktrace
[
https://issues.apache.org/jira/browse/HBASE-22734?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16893308#comment-16893308
]
stack commented on HBASE-22734:
---
+1 on w/o the frightening stack trace.
> Format error message in create_namespace command to avoid stacktrace
>
>
> Key: HBASE-22734
> URL: https://issues.apache.org/jira/browse/HBASE-22734
> Project: HBase
> Issue Type: Improvement
>Reporter: Sakthi
>Priority: Minor
>
> This is what it looks like currently:
>
> {code:java}
> hbase(main):003:0> create_namespace 'ns1:t2'
> ERROR: java.io.IOException: Illegal character <:> at 3. Namespaces may only
> contain 'alphanumeric characters' from any language and digits: ns1:t2
> at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:472)
> at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:132)
> at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:338)
> at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:318)
> Caused by: java.lang.IllegalArgumentException: Illegal character <:> at 3.
> Namespaces may only contain 'alphanumeric characters' from any language and
> digits: ns1:t2
> at org.apache.hadoop.hbase.TableName.isLegalNamespaceName(TableName.java:251)
> at org.apache.hadoop.hbase.TableName.isLegalNamespaceName(TableName.java:225)
> at org.apache.hadoop.hbase.master.HMaster.createNamespace(HMaster.java:3267)
> at
> org.apache.hadoop.hbase.master.MasterRpcServices.createNamespace(MasterRpcServices.java:649)
> at
> org.apache.hadoop.hbase.shaded.protobuf.generated.MasterProtos$MasterService$2.callBlockingMethod(MasterProtos.java)
> at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:413)
> ... 3 more
> For usage try 'help "create_namespace"'
> Took 8.2554 seconds{code}
--
This message was sent by Atlassian JIRA
(v7.6.14#76016)
[jira] [Commented] (HBASE-22610) [BucketCache] Rename "hbase.offheapcache.minblocksize"
[
https://issues.apache.org/jira/browse/HBASE-22610?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16893306#comment-16893306
]
Reid Chan commented on HBASE-22610:
---
Good to raise it, i didn't notice that.
Checked, it is the latter case, both used in FirstLevelBlockCache(L1 in
branch-1) and BucketCache
> [BucketCache] Rename "hbase.offheapcache.minblocksize"
> --
>
> Key: HBASE-22610
> URL: https://issues.apache.org/jira/browse/HBASE-22610
> Project: HBase
> Issue Type: Improvement
>Reporter: Reid Chan
>Assignee: Murtaza Hassan
>Priority: Trivial
> Labels: beginner
> Fix For: 3.0.0, 1.5.0, 2.3.0, 2.2.1, 2.1.6, 1.4.11
>
>
> {code}
> /**
>* The target block size used by blockcache instances. Defaults to
>* {@link HConstants#DEFAULT_BLOCKSIZE}.
>* TODO: this config point is completely wrong, as it's used to determine
> the
>* target block size of BlockCache instances. Rename.
>*/
> public static final String BLOCKCACHE_BLOCKSIZE_KEY =
> "hbase.offheapcache.minblocksize";
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.14#76016)
[jira] [Updated] (HBASE-22739) ArrayIndexOutOfBoundsException when balance
[
https://issues.apache.org/jira/browse/HBASE-22739?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
casuallc updated HBASE-22739:
-
Description:
{code:java}
2019-07-25 15:19:59,828 ERROR [master/nna:16000.Chore.1] hbase.ScheduledChore:
Caught error
java.lang.ArrayIndexOutOfBoundsException: 3171
at
org.apache.hadoop.hbase.master.balancer.BaseLoadBalancer$Cluster.removeRegion(BaseLoadBalancer.java:873)
at
org.apache.hadoop.hbase.master.balancer.BaseLoadBalancer$Cluster.doAction(BaseLoadBalancer.java:716)
at
org.apache.hadoop.hbase.master.balancer.StochasticLoadBalancer.balanceCluster(StochasticLoadBalancer.java:407)
at
org.apache.hadoop.hbase.master.balancer.StochasticLoadBalancer.balanceCluster(StochasticLoadBalancer.java:318)
at org.apache.hadoop.hbase.master.HMaster.balance(HMaster.java:1650)
at org.apache.hadoop.hbase.master.HMaster.balance(HMaster.java:1567)
at
org.apache.hadoop.hbase.master.balancer.BalancerChore.chore(BalancerChore.java:49)
at org.apache.hadoop.hbase.ScheduledChore.run(ScheduledChore.java:186)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308)
at
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180)
at
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294)
at
org.apache.hadoop.hbase.JitterScheduledThreadPoolExecutorImpl$JitteredRunnableScheduledFuture.run(JitterScheduledThreadPoolExecutorImpl.java:111)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)
{code}
should check if the regionIndex is valid when removeRegion,
java:
hbase-server/src/main/java/org/apache/hadoop/hbase/master/balancer/BaseLoadBalancer.java
{code:java}
int[] removeRegion(int[] regions, int regionIndex) {
//TODO: this maybe costly. Consider using linked lists
int[] newRegions = new int[regions.length - 1];
int i = 0;
for (i = 0; i < regions.length; i++) {
if (regions[i] == regionIndex) {
break;
}
if (i == regions.length - 1) {
return Arrays.copyOf(regions, regions.length);
}
newRegions[i] = regions[i];
}
System.arraycopy(regions, i+1, newRegions, i, newRegions.length - i);
return newRegions;
}
{code}
was:
{code:java}
2019-07-25 15:19:59,828 ERROR [master/nna:16000.Chore.1] hbase.ScheduledChore:
Caught error
java.lang.ArrayIndexOutOfBoundsException: 3171
at
org.apache.hadoop.hbase.master.balancer.BaseLoadBalancer$Cluster.removeRegion(BaseLoadBalancer.java:873)
at
org.apache.hadoop.hbase.master.balancer.BaseLoadBalancer$Cluster.doAction(BaseLoadBalancer.java:716)
at
org.apache.hadoop.hbase.master.balancer.StochasticLoadBalancer.balanceCluster(StochasticLoadBalancer.java:407)
at
org.apache.hadoop.hbase.master.balancer.StochasticLoadBalancer.balanceCluster(StochasticLoadBalancer.java:318)
at org.apache.hadoop.hbase.master.HMaster.balance(HMaster.java:1650)
at org.apache.hadoop.hbase.master.HMaster.balance(HMaster.java:1567)
at
org.apache.hadoop.hbase.master.balancer.BalancerChore.chore(BalancerChore.java:49)
at org.apache.hadoop.hbase.ScheduledChore.run(ScheduledChore.java:186)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308)
at
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180)
at
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294)
at
org.apache.hadoop.hbase.JitterScheduledThreadPoolExecutorImpl$JitteredRunnableScheduledFuture.run(JitterScheduledThreadPoolExecutorImpl.java:111)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)
{code}
should check if the regionIndex is valid when removeRegion,
java:
hbase-server/src/main/java/org/apache/hadoop/hbase/master/balancer/BaseLoadBalancer.java
{code:java}
int[] removeRegion(int[] regions, int regionIndex) {
//TODO: this maybe costly. Consider using linked lists
int[] newRegions = new int[regions.length - 1];
int i = 0;
for (i = 0; i < regions.length; i++) {
if (i == regions.length - 1) {
return Arrays.copyOf(regions, regions.length);
}
if (regions[i] == regionIndex) {
break;
}
newRegions[i] = regions[i];
}
System.arraycopy(regions, i+1, newRegions, i, newRegions.length - i);
return newRegions;
}
{code}
> ArrayIndexOutOfBoundsException when balance
>
[GitHub] [hbase] casuallc closed pull request #410: check regionIndex
casuallc closed pull request #410: check regionIndex URL: https://github.com/apache/hbase/pull/410 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Commented] (HBASE-22739) ArrayIndexOutOfBoundsException when balance
[
https://issues.apache.org/jira/browse/HBASE-22739?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16893290#comment-16893290
]
Reid Chan commented on HBASE-22739:
---
You can provide a patch or create a pull request. If you choose latter, please
link that to this jira.
> ArrayIndexOutOfBoundsException when balance
> ---
>
> Key: HBASE-22739
> URL: https://issues.apache.org/jira/browse/HBASE-22739
> Project: HBase
> Issue Type: Bug
> Components: Balancer
>Reporter: casuallc
>Priority: Major
> Fix For: 2.1.1
>
>
>
> {code:java}
> 2019-07-25 15:19:59,828 ERROR [master/nna:16000.Chore.1]
> hbase.ScheduledChore: Caught error
> java.lang.ArrayIndexOutOfBoundsException: 3171
> at
> org.apache.hadoop.hbase.master.balancer.BaseLoadBalancer$Cluster.removeRegion(BaseLoadBalancer.java:873)
> at
> org.apache.hadoop.hbase.master.balancer.BaseLoadBalancer$Cluster.doAction(BaseLoadBalancer.java:716)
> at
> org.apache.hadoop.hbase.master.balancer.StochasticLoadBalancer.balanceCluster(StochasticLoadBalancer.java:407)
> at
> org.apache.hadoop.hbase.master.balancer.StochasticLoadBalancer.balanceCluster(StochasticLoadBalancer.java:318)
> at org.apache.hadoop.hbase.master.HMaster.balance(HMaster.java:1650)
> at org.apache.hadoop.hbase.master.HMaster.balance(HMaster.java:1567)
> at
> org.apache.hadoop.hbase.master.balancer.BalancerChore.chore(BalancerChore.java:49)
> at org.apache.hadoop.hbase.ScheduledChore.run(ScheduledChore.java:186)
> at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
> at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308)
> at
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180)
> at
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294)
> at
> org.apache.hadoop.hbase.JitterScheduledThreadPoolExecutorImpl$JitteredRunnableScheduledFuture.run(JitterScheduledThreadPoolExecutorImpl.java:111)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> at java.lang.Thread.run(Thread.java:745)
> {code}
> should check if the regionIndex is valid when removeRegion,
> java:
> hbase-server/src/main/java/org/apache/hadoop/hbase/master/balancer/BaseLoadBalancer.java
> {code:java}
> int[] removeRegion(int[] regions, int regionIndex) {
> //TODO: this maybe costly. Consider using linked lists
> int[] newRegions = new int[regions.length - 1];
> int i = 0;
> for (i = 0; i < regions.length; i++) {
> if (i == regions.length - 1) {
> return Arrays.copyOf(regions, regions.length);
> }
> if (regions[i] == regionIndex) {
> break;
> }
> newRegions[i] = regions[i];
> }
> System.arraycopy(regions, i+1, newRegions, i, newRegions.length - i);
> return newRegions;
> }
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.14#76016)
[jira] [Commented] (HBASE-22610) [BucketCache] Rename "hbase.offheapcache.minblocksize"
[
https://issues.apache.org/jira/browse/HBASE-22610?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16893293#comment-16893293
]
Anoop Sam John commented on HBASE-22610:
Is this config used only in bucket cache or in both block caches? I believe it
is the former and then it would have been better to call the new config as
"hbase.bucketcache.minblocksize"? Sorry for being late in asking this.
> [BucketCache] Rename "hbase.offheapcache.minblocksize"
> --
>
> Key: HBASE-22610
> URL: https://issues.apache.org/jira/browse/HBASE-22610
> Project: HBase
> Issue Type: Improvement
>Reporter: Reid Chan
>Assignee: Murtaza Hassan
>Priority: Trivial
> Labels: beginner
> Fix For: 3.0.0, 1.5.0, 2.3.0, 2.2.1, 2.1.6, 1.4.11
>
>
> {code}
> /**
>* The target block size used by blockcache instances. Defaults to
>* {@link HConstants#DEFAULT_BLOCKSIZE}.
>* TODO: this config point is completely wrong, as it's used to determine
> the
>* target block size of BlockCache instances. Rename.
>*/
> public static final String BLOCKCACHE_BLOCKSIZE_KEY =
> "hbase.offheapcache.minblocksize";
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.14#76016)
[jira] [Commented] (HBASE-22648) Snapshot TTL
[ https://issues.apache.org/jira/browse/HBASE-22648?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16893295#comment-16893295 ] Anoop Sam John commented on HBASE-22648: Can you pls add a RN [~vjasani]. Will be very helpful for future ref. > Snapshot TTL > > > Key: HBASE-22648 > URL: https://issues.apache.org/jira/browse/HBASE-22648 > Project: HBase > Issue Type: New Feature > Components: snapshots >Affects Versions: 3.0.0 >Reporter: Andrew Purtell >Assignee: Viraj Jasani >Priority: Minor > Fix For: 3.0.0, 1.5.0, 2.3.0 > > Attachments: HBASE-22648-branch-1.patch, HBASE-22648-branch-1.patch, > HBASE-22648-branch-2.patch, HBASE-22648-master-v2.patch, > HBASE-22648-master-v3.patch, HBASE-22648-master-v4.patch, > HBASE-22648-master-v5.patch, HBASE-22648-master-v6.patch, > HBASE-22648-master-v8.patch, HBASE-22648-master.patch, Screen Shot 2019-07-10 > at 8.49.13 PM.png, Screen Shot 2019-07-10 at 8.52.30 PM.png, Screen Shot > 2019-07-10 at 9.06.36 PM.png, Screen Shot 2019-07-16 at 11.06.03 AM.png > > > Snapshots have a lifecycle that is independent from the table from which they > are created. Although data in a table may be stored with TTL the data files > containing them become frozen by the snapshot. Space consumed by expired > cells will not be reclaimed by normal table housekeeping like compaction. > While this is expected it can be inconvenient at scale. When many snapshots > are under management and the data in various tables is expired by TTL some > notion of optional TTL (and optional default TTL) for snapshots could be > useful. It will help prevent the accumulation of junk files by automatically > dropping the snapshot after the assigned TTL, making their data files > eligible for cleaning. More comprehensive snapshot lifecycle management may > be considered in the future but this one case is expected to be immediately > useful given TTls on data are commonly applied for similar convenience. -- This message was sent by Atlassian JIRA (v7.6.14#76016)
[jira] [Updated] (HBASE-22739) ArrayIndexOutOfBoundsException when balance
[
https://issues.apache.org/jira/browse/HBASE-22739?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Reid Chan updated HBASE-22739:
--
Flags: (was: Patch)
> ArrayIndexOutOfBoundsException when balance
> ---
>
> Key: HBASE-22739
> URL: https://issues.apache.org/jira/browse/HBASE-22739
> Project: HBase
> Issue Type: Bug
> Components: Balancer
>Reporter: casuallc
>Priority: Major
> Fix For: 2.1.1
>
>
>
> {code:java}
> 2019-07-25 15:19:59,828 ERROR [master/nna:16000.Chore.1]
> hbase.ScheduledChore: Caught error
> java.lang.ArrayIndexOutOfBoundsException: 3171
> at
> org.apache.hadoop.hbase.master.balancer.BaseLoadBalancer$Cluster.removeRegion(BaseLoadBalancer.java:873)
> at
> org.apache.hadoop.hbase.master.balancer.BaseLoadBalancer$Cluster.doAction(BaseLoadBalancer.java:716)
> at
> org.apache.hadoop.hbase.master.balancer.StochasticLoadBalancer.balanceCluster(StochasticLoadBalancer.java:407)
> at
> org.apache.hadoop.hbase.master.balancer.StochasticLoadBalancer.balanceCluster(StochasticLoadBalancer.java:318)
> at org.apache.hadoop.hbase.master.HMaster.balance(HMaster.java:1650)
> at org.apache.hadoop.hbase.master.HMaster.balance(HMaster.java:1567)
> at
> org.apache.hadoop.hbase.master.balancer.BalancerChore.chore(BalancerChore.java:49)
> at org.apache.hadoop.hbase.ScheduledChore.run(ScheduledChore.java:186)
> at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
> at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308)
> at
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180)
> at
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294)
> at
> org.apache.hadoop.hbase.JitterScheduledThreadPoolExecutorImpl$JitteredRunnableScheduledFuture.run(JitterScheduledThreadPoolExecutorImpl.java:111)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> at java.lang.Thread.run(Thread.java:745)
> {code}
> should check if the regionIndex is valid when removeRegion,
> java:
> hbase-server/src/main/java/org/apache/hadoop/hbase/master/balancer/BaseLoadBalancer.java
> {code:java}
> int[] removeRegion(int[] regions, int regionIndex) {
> //TODO: this maybe costly. Consider using linked lists
> int[] newRegions = new int[regions.length - 1];
> int i = 0;
> for (i = 0; i < regions.length; i++) {
> if (i == regions.length - 1) {
> return Arrays.copyOf(regions, regions.length);
> }
> if (regions[i] == regionIndex) {
> break;
> }
> newRegions[i] = regions[i];
> }
> System.arraycopy(regions, i+1, newRegions, i, newRegions.length - i);
> return newRegions;
> }
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.14#76016)
[jira] [Updated] (HBASE-22739) ArrayIndexOutOfBoundsException when balance
[
https://issues.apache.org/jira/browse/HBASE-22739?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Reid Chan updated HBASE-22739:
--
Component/s: Balancer
> ArrayIndexOutOfBoundsException when balance
> ---
>
> Key: HBASE-22739
> URL: https://issues.apache.org/jira/browse/HBASE-22739
> Project: HBase
> Issue Type: Bug
> Components: Balancer
>Reporter: casuallc
>Priority: Major
> Fix For: 2.1.1
>
>
>
> {code:java}
> 2019-07-25 15:19:59,828 ERROR [master/nna:16000.Chore.1]
> hbase.ScheduledChore: Caught error
> java.lang.ArrayIndexOutOfBoundsException: 3171
> at
> org.apache.hadoop.hbase.master.balancer.BaseLoadBalancer$Cluster.removeRegion(BaseLoadBalancer.java:873)
> at
> org.apache.hadoop.hbase.master.balancer.BaseLoadBalancer$Cluster.doAction(BaseLoadBalancer.java:716)
> at
> org.apache.hadoop.hbase.master.balancer.StochasticLoadBalancer.balanceCluster(StochasticLoadBalancer.java:407)
> at
> org.apache.hadoop.hbase.master.balancer.StochasticLoadBalancer.balanceCluster(StochasticLoadBalancer.java:318)
> at org.apache.hadoop.hbase.master.HMaster.balance(HMaster.java:1650)
> at org.apache.hadoop.hbase.master.HMaster.balance(HMaster.java:1567)
> at
> org.apache.hadoop.hbase.master.balancer.BalancerChore.chore(BalancerChore.java:49)
> at org.apache.hadoop.hbase.ScheduledChore.run(ScheduledChore.java:186)
> at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
> at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308)
> at
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180)
> at
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294)
> at
> org.apache.hadoop.hbase.JitterScheduledThreadPoolExecutorImpl$JitteredRunnableScheduledFuture.run(JitterScheduledThreadPoolExecutorImpl.java:111)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> at java.lang.Thread.run(Thread.java:745)
> {code}
> should check if the regionIndex is valid when removeRegion,
> java:
> hbase-server/src/main/java/org/apache/hadoop/hbase/master/balancer/BaseLoadBalancer.java
> {code:java}
> int[] removeRegion(int[] regions, int regionIndex) {
> //TODO: this maybe costly. Consider using linked lists
> int[] newRegions = new int[regions.length - 1];
> int i = 0;
> for (i = 0; i < regions.length; i++) {
> if (i == regions.length - 1) {
> return Arrays.copyOf(regions, regions.length);
> }
> if (regions[i] == regionIndex) {
> break;
> }
> newRegions[i] = regions[i];
> }
> System.arraycopy(regions, i+1, newRegions, i, newRegions.length - i);
> return newRegions;
> }
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.14#76016)
[jira] [Created] (HBASE-22739) ArrayIndexOutOfBoundsException when balance
casuallc created HBASE-22739:
Summary: ArrayIndexOutOfBoundsException when balance
Key: HBASE-22739
URL: https://issues.apache.org/jira/browse/HBASE-22739
Project: HBase
Issue Type: Bug
Reporter: casuallc
Fix For: 2.1.1
{code:java}
2019-07-25 15:19:59,828 ERROR [master/nna:16000.Chore.1] hbase.ScheduledChore:
Caught error
java.lang.ArrayIndexOutOfBoundsException: 3171
at
org.apache.hadoop.hbase.master.balancer.BaseLoadBalancer$Cluster.removeRegion(BaseLoadBalancer.java:873)
at
org.apache.hadoop.hbase.master.balancer.BaseLoadBalancer$Cluster.doAction(BaseLoadBalancer.java:716)
at
org.apache.hadoop.hbase.master.balancer.StochasticLoadBalancer.balanceCluster(StochasticLoadBalancer.java:407)
at
org.apache.hadoop.hbase.master.balancer.StochasticLoadBalancer.balanceCluster(StochasticLoadBalancer.java:318)
at org.apache.hadoop.hbase.master.HMaster.balance(HMaster.java:1650)
at org.apache.hadoop.hbase.master.HMaster.balance(HMaster.java:1567)
at
org.apache.hadoop.hbase.master.balancer.BalancerChore.chore(BalancerChore.java:49)
at org.apache.hadoop.hbase.ScheduledChore.run(ScheduledChore.java:186)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308)
at
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180)
at
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294)
at
org.apache.hadoop.hbase.JitterScheduledThreadPoolExecutorImpl$JitteredRunnableScheduledFuture.run(JitterScheduledThreadPoolExecutorImpl.java:111)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)
{code}
should check if the regionIndex is valid when removeRegion,
java:
hbase-server/src/main/java/org/apache/hadoop/hbase/master/balancer/BaseLoadBalancer.java
{code:java}
int[] removeRegion(int[] regions, int regionIndex) {
//TODO: this maybe costly. Consider using linked lists
int[] newRegions = new int[regions.length - 1];
int i = 0;
for (i = 0; i < regions.length; i++) {
if (i == regions.length - 1) {
return Arrays.copyOf(regions, regions.length);
}
if (regions[i] == regionIndex) {
break;
}
newRegions[i] = regions[i];
}
System.arraycopy(regions, i+1, newRegions, i, newRegions.length - i);
return newRegions;
}
{code}
--
This message was sent by Atlassian JIRA
(v7.6.14#76016)
[GitHub] [hbase] Reidddddd commented on issue #410: check regionIndex
Reidd commented on issue #410: check regionIndex URL: https://github.com/apache/hbase/pull/410#issuecomment-515293053 Please refer to [Getting Involved](http://hbase.apache.org/book.html#getting.involved) This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Updated] (HBASE-22738) Fallback to default group to choose RS when there are no RS in current group
[ https://issues.apache.org/jira/browse/HBASE-22738?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Guanghao Zhang updated HBASE-22738: --- Summary: Fallback to default group to choose RS when there are no RS in current group (was: Fallback to other group to choose RS when there are no RS in current group) > Fallback to default group to choose RS when there are no RS in current group > > > Key: HBASE-22738 > URL: https://issues.apache.org/jira/browse/HBASE-22738 > Project: HBase > Issue Type: Bug >Reporter: Guanghao Zhang >Priority: Major > > We configure one regionserver for hbase system table. But when rolling > upgrade, you need move the region to other regionservers. But because there > are no other regionservers in this group, you cannot move the region... > > -- This message was sent by Atlassian JIRA (v7.6.14#76016)
[jira] [Created] (HBASE-22738) Fallback to other group to choose RS when there are no RS in current group
Guanghao Zhang created HBASE-22738: -- Summary: Fallback to other group to choose RS when there are no RS in current group Key: HBASE-22738 URL: https://issues.apache.org/jira/browse/HBASE-22738 Project: HBase Issue Type: Bug Reporter: Guanghao Zhang We configure one regionserver for hbase system table. But when rolling upgrade, you need move the region to other regionservers. But because there are no other regionservers in this group, you cannot move the region... -- This message was sent by Atlassian JIRA (v7.6.14#76016)
[GitHub] [hbase] casuallc commented on issue #410: check regionIndex
casuallc commented on issue #410: check regionIndex URL: https://github.com/apache/hbase/pull/410#issuecomment-515290924 > Please file a JIRA first. I did not do this before, what should I do? This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [hbase] casuallc closed pull request #409: check bounds when removeRegion
casuallc closed pull request #409: check bounds when removeRegion URL: https://github.com/apache/hbase/pull/409 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [hbase] Reidddddd commented on issue #410: check regionIndex
Reidd commented on issue #410: check regionIndex URL: https://github.com/apache/hbase/pull/410#issuecomment-515290287 Please file a JIRA first. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [hbase] casuallc opened a new pull request #410: check regionIndex
casuallc opened a new pull request #410: check regionIndex URL: https://github.com/apache/hbase/pull/410 if regionIndex not in regions, there will be ArrayIndexOutOfBoundsException. such as 2019-07-25 15:19:59,828 ERROR [master/nna:16000.Chore.1] hbase.ScheduledChore: Caught error java.lang.ArrayIndexOutOfBoundsException: 3171 at org.apache.hadoop.hbase.master.balancer.BaseLoadBalancer$Cluster.removeRegion(BaseLoadBalancer.java:873) at org.apache.hadoop.hbase.master.balancer.BaseLoadBalancer$Cluster.doAction(BaseLoadBalancer.java:716) at org.apache.hadoop.hbase.master.balancer.StochasticLoadBalancer.balanceCluster(StochasticLoadBalancer.java:407) at org.apache.hadoop.hbase.master.balancer.StochasticLoadBalancer.balanceCluster(StochasticLoadBalancer.java:318) at org.apache.hadoop.hbase.master.HMaster.balance(HMaster.java:1650) at org.apache.hadoop.hbase.master.HMaster.balance(HMaster.java:1567) at org.apache.hadoop.hbase.master.balancer.BalancerChore.chore(BalancerChore.java:49) at org.apache.hadoop.hbase.ScheduledChore.run(ScheduledChore.java:186) at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308) at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180) at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294) at org.apache.hadoop.hbase.JitterScheduledThreadPoolExecutorImpl$JitteredRunnableScheduledFuture.run(JitterScheduledThreadPoolExecutorImpl.java:111) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) at java.lang.Thread.run(Thread.java:745) This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Created] (HBASE-22736) Add text to show how to fix the HBCK checking result
Guanghao Zhang created HBASE-22736: -- Summary: Add text to show how to fix the HBCK checking result Key: HBASE-22736 URL: https://issues.apache.org/jira/browse/HBASE-22736 Project: HBase Issue Type: Sub-task Reporter: Guanghao Zhang A follow-on issue after HBASE-22709. Copied the comment here. bq. In follow-on, we can add text on how to fix including commands. Also may want a json version of this page. Lets see. -- This message was sent by Atlassian JIRA (v7.6.14#76016)
[GitHub] [hbase] Apache-HBase commented on issue #404: HBASE-22709 Add a chore thread in master to do hbck checking
Apache-HBase commented on issue #404: HBASE-22709 Add a chore thread in master to do hbck checking URL: https://github.com/apache/hbase/pull/404#issuecomment-515285584 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | 0 | reexec | 87 | Docker mode activated. | ||| _ Prechecks _ | | +1 | dupname | 0 | No case conflicting files found. | | +1 | hbaseanti | 0 | Patch does not have any anti-patterns. | | +1 | @author | 0 | The patch does not contain any @author tags. | | +1 | test4tests | 0 | The patch appears to include 2 new or modified test files. | ||| _ master Compile Tests _ | | -1 | mvninstall | 361 | root in master failed. | | +1 | compile | 62 | master passed | | +1 | checkstyle | 87 | master passed | | +1 | shadedjars | 318 | branch has no errors when building our shaded downstream artifacts. | | +1 | javadoc | 37 | master passed | | 0 | spotbugs | 255 | Used deprecated FindBugs config; considering switching to SpotBugs. | | +1 | findbugs | 253 | master passed | ||| _ Patch Compile Tests _ | | -1 | mvninstall | 154 | root in the patch failed. | | -1 | compile | 28 | hbase-server in the patch failed. | | -1 | javac | 28 | hbase-server in the patch failed. | | -1 | checkstyle | 75 | hbase-server: The patch generated 1 new + 134 unchanged - 0 fixed = 135 total (was 134) | | +1 | whitespace | 0 | The patch has no whitespace issues. | | -1 | shadedjars | 193 | patch has 16 errors when building our shaded downstream artifacts. | | -1 | hadoopcheck | 99 | The patch causes 16 errors with Hadoop v2.8.5. | | -1 | hadoopcheck | 201 | The patch causes 16 errors with Hadoop v2.9.2. | | -1 | hadoopcheck | 303 | The patch causes 16 errors with Hadoop v3.1.2. | | -1 | javadoc | 17 | hbase-server in the patch failed. | | -1 | findbugs | 29 | hbase-server in the patch failed. | ||| _ Other Tests _ | | -1 | unit | 27 | hbase-server in the patch failed. | | +1 | asflicense | 11 | The patch does not generate ASF License warnings. | | | | 2122 | | | Subsystem | Report/Notes | |--:|:-| | Docker | Client=19.03.1 Server=19.03.1 base: https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-404/14/artifact/out/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/404 | | Optional Tests | dupname asflicense javac javadoc unit spotbugs findbugs shadedjars hadoopcheck hbaseanti checkstyle compile | | uname | Linux 4154eb4c7113 4.4.0-138-generic #164-Ubuntu SMP Tue Oct 2 17:16:02 UTC 2018 x86_64 GNU/Linux | | Build tool | maven | | Personality | /home/jenkins/jenkins-slave/workspace/HBase-PreCommit-GitHub-PR_PR-404/out/precommit/personality/provided.sh | | git revision | master / 747d000808 | | Default Java | 1.8.0_181 | | mvninstall | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-404/14/artifact/out/branch-mvninstall-root.txt | | mvninstall | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-404/14/artifact/out/patch-mvninstall-root.txt | | compile | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-404/14/artifact/out/patch-compile-hbase-server.txt | | javac | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-404/14/artifact/out/patch-compile-hbase-server.txt | | checkstyle | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-404/14/artifact/out/diff-checkstyle-hbase-server.txt | | shadedjars | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-404/14/artifact/out/patch-shadedjars.txt | | hadoopcheck | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-404/14/artifact/out/patch-javac-2.8.5.txt | | hadoopcheck | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-404/14/artifact/out/patch-javac-2.9.2.txt | | hadoopcheck | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-404/14/artifact/out/patch-javac-3.1.2.txt | | javadoc | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-404/14/artifact/out/patch-javadoc-hbase-server.txt | | findbugs | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-404/14/artifact/out/patch-findbugs-hbase-server.txt | | unit | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-404/14/artifact/out/patch-unit-hbase-server.txt | | Test Results | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-404/14/testReport/ | | Max. process+thread count | 96 (vs. ulimit of 1) | | modules | C: hbase-server U: hbase-server | | Console output | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-404/14/console | | versions | git=2.11.0 maven=2018-06-17T18:33:14Z) findbugs=3.1.11 | | Powered by | Apache Yetus 0.10.0 http://yetus.apache.org | This message was automatically
[jira] [Created] (HBASE-22737) Add a new shell cmd to trigger the hbck checker to run
Guanghao Zhang created HBASE-22737:
--
Summary: Add a new shell cmd to trigger the hbck checker to run
Key: HBASE-22737
URL: https://issues.apache.org/jira/browse/HBASE-22737
Project: HBase
Issue Type: Sub-task
Reporter: Guanghao Zhang
A follow-on issue after HBASE-22709.
{quote}Oh, also as a follow-on, should make it so user can trigger this chore
to run... or perhaps a new shell command which is called hbck_report that runs
this chore and the catalogjanitor, etc., to produce all hbck.
{quote}
--
This message was sent by Atlassian JIRA
(v7.6.14#76016)
[GitHub] [hbase] infraio commented on a change in pull request #404: HBASE-22709 Add a chore thread in master to do hbck checking
infraio commented on a change in pull request #404: HBASE-22709 Add a chore
thread in master to do hbck checking
URL: https://github.com/apache/hbase/pull/404#discussion_r307567386
##
File path: hbase-server/src/main/resources/hbase-webapps/master/hbck.jsp
##
@@ -0,0 +1,139 @@
+<%--
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+--%>
+<%@ page contentType="text/html;charset=UTF-8"
+ import="java.util.List"
+ import="java.util.Map"
+ import="java.util.stream.Collectors"
+%>
+<%@ page import="org.apache.hadoop.hbase.master.HbckChecker" %>
+<%@ page import="org.apache.hadoop.hbase.master.HMaster" %>
+<%@ page import="org.apache.hadoop.hbase.ServerName" %>
+<%@ page import="org.apache.hadoop.hbase.util.Pair" %>
+<%
+ HMaster master = (HMaster) getServletContext().getAttribute(HMaster.MASTER);
+ pageContext.setAttribute("pageTitle", "HBase Master HBCK Report: " +
master.getServerName());
+ HbckChecker hbckChecker = master.getHbckChecker();
+ Map>> inconsistentRegions = null;
+ Map orphanRegionsOnRS = null;
+ List orphanRegionsOnFS = null;
+ if (hbckChecker != null) {
+inconsistentRegions = hbckChecker.getInconsistentRegions();
+orphanRegionsOnRS = hbckChecker.getOrphanRegionsOnRS();
+orphanRegionsOnFS = hbckChecker.getOrphanRegionsOnFS();
+ }
+%>
+
+
+
+
+
+
+ <% if (!master.isInitialized()) { %>
+
+
+ Master is not initialized
+
+
+
+ <% } else { %>
+
+
+
+ Inconsistent Regions
+
+
+There are three case: 1. Master thought this region opened, but no
regionserver reported it.
+2. Master thought this region opened on Server1, but regionserver
reported Server2.
+3. More than one regionservers reported opened this region.
+Notice: the reported online regionservers may be not right when there
are regions in transition.
+Please check them in regionserver's web UI.
+
+
+
+
+
+ <% if (inconsistentRegions != null && inconsistentRegions.size() > 0) { %>
+
+
+ Region
+ Location in META
+ Reported Online RegionServers
+
+<% for (Map.Entry>> entry :
inconsistentRegions.entrySet()) { %>
+
+ <%= entry.getKey() %>
+ <%= entry.getValue().getFirst() %>
+ <%=
entry.getValue().getSecond().stream().map(ServerName::getServerName)
+.collect(Collectors.joining(", ")) %>
+
+<% } %>
+
+<%= inconsistentRegions.size() %> region(s) in set.
+
+ <% } %>
+
+
+
+ Orphan Regions on RegionServer
+
+
+
+ <% if (orphanRegionsOnRS != null && orphanRegionsOnRS.size() > 0) { %>
+
+
+ Region
+ Reported Online RegionServer
+
+<% for (Map.Entry entry :
orphanRegionsOnRS.entrySet()) { %>
+
+ <%= entry.getKey() %>
+ <%= entry.getValue() %>
+
+<% } %>
+
+<%= orphanRegionsOnRS.size() %> region(s) in set.
+
+ <% } %>
+
+
+
+ Orphan Regions on FileSystem
+
+
+
+ <% if (orphanRegionsOnFS != null && orphanRegionsOnFS.size() > 0) { %>
+
+
+ Region
+
+<% for (String region : orphanRegionsOnFS) { %>
+
+ <%= region %>
+
+<% } %>
+
+<%= orphanRegionsOnFS.size() %> region(s) in set.
+
+ <% } %>
+
+ <% } %>
+
+
+
Review comment:
> Beautiful.
>
> In follow-on, we can add text on how to fix including commands.
>
> Also may want a json version of this page. Lets see.
Let me open a new issue for this.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [hbase] infraio commented on issue #404: HBASE-22709 Add a chore thread in master to do hbck checking
infraio commented on issue #404: HBASE-22709 Add a chore thread in master to do hbck checking URL: https://github.com/apache/hbase/pull/404#issuecomment-515278562 > clusterstatuspublisher The ClusterStatusPublisher only publish version, master and deadservers. Didn't have overlap with this. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Commented] (HBASE-22728) Upgrade jackson dependencies in branch-1
[ https://issues.apache.org/jira/browse/HBASE-22728?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16893247#comment-16893247 ] Andrew Purtell commented on HBASE-22728: Any thoughts on how best to move forward? > Upgrade jackson dependencies in branch-1 > > > Key: HBASE-22728 > URL: https://issues.apache.org/jira/browse/HBASE-22728 > Project: HBase > Issue Type: Sub-task >Affects Versions: 1.4.10, 1.3.5 >Reporter: Andrew Purtell >Priority: Major > Fix For: 1.5.0, 1.3.6, 1.4.11 > > > Avoid Jackson versions and dependencies with known CVEs -- This message was sent by Atlassian JIRA (v7.6.14#76016)
[jira] [Created] (HBASE-22735) list_regions may throw an error if a region is RIT
Andrew Purtell created HBASE-22735:
--
Summary: list_regions may throw an error if a region is RIT
Key: HBASE-22735
URL: https://issues.apache.org/jira/browse/HBASE-22735
Project: HBase
Issue Type: Bug
Components: shell
Affects Versions: 1.5.0
Reporter: Andrew Purtell
Fix For: 3.0.0, 1.5.0, 2.3.0, 2.2.1, 2.1.6, 1.3.6, 1.4.11
The 'list_regions' shell command gets a list of regions for a given table and
then prints them and some attributes such as the server where they are located,
current request count, data locality, and such.
However if a region is in transition the command might fail with
{{ERROR: undefined method `getDataLocality' for nil:NilClass}}
and there may be other ways this can happen. Protect against use of nil
references and just display what we can.
--
This message was sent by Atlassian JIRA
(v7.6.14#76016)
[jira] [Commented] (HBASE-22733) TestSplitTransactionOnCluster.testMasterRestartAtRegionSplitPendingCatalogJanitor is flakey
[
https://issues.apache.org/jira/browse/HBASE-22733?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16893244#comment-16893244
]
Hudson commented on HBASE-22733:
Results for branch branch-2.2
[build #465 on
builds.a.o|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.2/465/]:
(x) *{color:red}-1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.2/465//General_Nightly_Build_Report/]
(x) {color:red}-1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.2/465//JDK8_Nightly_Build_Report_(Hadoop2)/]
(x) {color:red}-1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.2/465//JDK8_Nightly_Build_Report_(Hadoop3)/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
(/) {color:green}+1 client integration test{color}
> TestSplitTransactionOnCluster.testMasterRestartAtRegionSplitPendingCatalogJanitor
> is flakey
> ---
>
> Key: HBASE-22733
> URL: https://issues.apache.org/jira/browse/HBASE-22733
> Project: HBase
> Issue Type: Bug
>Reporter: Xiaolin Ha
>Assignee: Xiaolin Ha
>Priority: Major
> Fix For: 3.0.0, 2.3.0, 2.0.6, 2.2.1, 2.1.6
>
> Attachments: test -
> testMasterRestartAtRegionSplitPendingCatalogJanitor [Jenkins].txt
>
>
> RITs will lead to CatalogJanitor not running, and then parent region will not
> be cleaned.
> As a result, the last assertion of checking if parent region is null will
> fail.
> {quote}
> h3. Error Message
> expected: but was:
> h3. Stacktrace
> java.lang.AssertionError: expected: but
> was: at
> org.apache.hadoop.hbase.regionserver.TestSplitTransactionOnCluster.testMasterRestartAtRegionSplitPendingCatalogJanitor(TestSplitTransactionOnCluster.java:577){quote}
> Currently, the case is only waiting for master be active while not care about
> RITs completion when master starts up.
> Failed test results is attached. We can see CatalogJanitor chore run failed
> log in it,
> {quote}2019-07-23 18:34:03,241 INFO [Time-limited test]
> regionserver.TestSplitTransactionOnCluster(572): Starting run of
> CatalogJanitor 2019-07-23 18:34:03,241 WARN [Time-limited test]
> master.CatalogJanitor(119): CatalogJanitor is disabled! Enabled=true,
> maintenanceMode=false,
> am=org.apache.hadoop.hbase.master.assignment.AssignmentManager@1ce25bee,
> metaLoaded=true, hasRIT=true clusterShutDown=false
> {quote}
>
>
--
This message was sent by Atlassian JIRA
(v7.6.14#76016)
[GitHub] [hbase] infraio commented on issue #404: HBASE-22709 Add a chore thread in master to do hbck checking
infraio commented on issue #404: HBASE-22709 Add a chore thread in master to do hbck checking URL: https://github.com/apache/hbase/pull/404#issuecomment-515268858 > One other thought, is if there is overlap with existing chores? > > Maybe the clusterstatuspublisher? Let me check this. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [hbase] Apache-HBase commented on issue #336: HBASE-22580 Add a table attribute to make user scan snapshot feature configurable for table
Apache-HBase commented on issue #336: HBASE-22580 Add a table attribute to make user scan snapshot feature configurable for table URL: https://github.com/apache/hbase/pull/336#issuecomment-515268833 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | 0 | reexec | 81 | Docker mode activated. | ||| _ Prechecks _ | | +1 | dupname | 0 | No case conflicting files found. | | +1 | hbaseanti | 0 | Patch does not have any anti-patterns. | | +1 | @author | 0 | The patch does not contain any @author tags. | | +1 | test4tests | 0 | The patch appears to include 1 new or modified test files. | ||| _ master Compile Tests _ | | +1 | mvninstall | 438 | master passed | | +1 | compile | 77 | master passed | | +1 | checkstyle | 106 | master passed | | +1 | shadedjars | 372 | branch has no errors when building our shaded downstream artifacts. | | +1 | javadoc | 46 | master passed | | 0 | spotbugs | 318 | Used deprecated FindBugs config; considering switching to SpotBugs. | | +1 | findbugs | 315 | master passed | ||| _ Patch Compile Tests _ | | +1 | mvninstall | 443 | the patch passed | | +1 | compile | 75 | the patch passed | | +1 | javac | 75 | the patch passed | | +1 | checkstyle | 103 | the patch passed | | +1 | whitespace | 0 | The patch has no whitespace issues. | | +1 | shadedjars | 383 | patch has no errors when building our shaded downstream artifacts. | | +1 | hadoopcheck | 1370 | Patch does not cause any errors with Hadoop 2.8.5 2.9.2 or 3.1.2. | | +1 | javadoc | 43 | the patch passed | | +1 | findbugs | 327 | the patch passed | ||| _ Other Tests _ | | -1 | unit | 17808 | hbase-server in the patch failed. | | +1 | asflicense | 29 | The patch does not generate ASF License warnings. | | | | 22552 | | | Reason | Tests | |---:|:--| | Failed junit tests | hadoop.hbase.security.access.TestSnapshotScannerHDFSAclController | | | hadoop.hbase.tool.TestSecureBulkLoadHFiles | | | hadoop.hbase.client.TestAsyncTableAdminApi | | Subsystem | Report/Notes | |--:|:-| | Docker | Client=19.03.0 Server=19.03.0 base: https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-336/32/artifact/out/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/336 | | Optional Tests | dupname asflicense javac javadoc unit spotbugs findbugs shadedjars hadoopcheck hbaseanti checkstyle compile | | uname | Linux 35ff22d5831c 4.4.0-154-generic #181-Ubuntu SMP Tue Jun 25 05:29:03 UTC 2019 x86_64 GNU/Linux | | Build tool | maven | | Personality | /home/jenkins/jenkins-slave/workspace/HBase-PreCommit-GitHub-PR_PR-336/out/precommit/personality/provided.sh | | git revision | master / 747d000808 | | Default Java | 1.8.0_181 | | unit | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-336/32/artifact/out/patch-unit-hbase-server.txt | | Test Results | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-336/32/testReport/ | | Max. process+thread count | 4688 (vs. ulimit of 1) | | modules | C: hbase-server U: hbase-server | | Console output | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-336/32/console | | versions | git=2.11.0 maven=2018-06-17T18:33:14Z) findbugs=3.1.11 | | Powered by | Apache Yetus 0.10.0 http://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [hbase] infraio commented on issue #404: HBASE-22709 Add a chore thread in master to do hbck checking
infraio commented on issue #404: HBASE-22709 Add a chore thread in master to do hbck checking URL: https://github.com/apache/hbase/pull/404#issuecomment-515268972 > Oh, also as a follow-on, should make it so user can trigger this chore to run... or perhaps a new shell command which is called hbck_report that runs this chore and the catalogjanitor, etc., to produce all hbck. This can be done by a new issue. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [hbase] infraio commented on a change in pull request #404: HBASE-22709 Add a chore thread in master to do hbck checking
infraio commented on a change in pull request #404: HBASE-22709 Add a chore
thread in master to do hbck checking
URL: https://github.com/apache/hbase/pull/404#discussion_r307554624
##
File path: hbase-server/src/main/resources/hbase-webapps/master/hbck.jsp
##
@@ -0,0 +1,139 @@
+<%--
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+--%>
+<%@ page contentType="text/html;charset=UTF-8"
+ import="java.util.List"
+ import="java.util.Map"
+ import="java.util.stream.Collectors"
+%>
+<%@ page import="org.apache.hadoop.hbase.master.HbckChecker" %>
+<%@ page import="org.apache.hadoop.hbase.master.HMaster" %>
+<%@ page import="org.apache.hadoop.hbase.ServerName" %>
+<%@ page import="org.apache.hadoop.hbase.util.Pair" %>
+<%
+ HMaster master = (HMaster) getServletContext().getAttribute(HMaster.MASTER);
+ pageContext.setAttribute("pageTitle", "HBase Master HBCK Report: " +
master.getServerName());
+ HbckChecker hbckChecker = master.getHbckChecker();
+ Map>> inconsistentRegions = null;
+ Map orphanRegionsOnRS = null;
+ List orphanRegionsOnFS = null;
+ if (hbckChecker != null) {
+inconsistentRegions = hbckChecker.getInconsistentRegions();
+orphanRegionsOnRS = hbckChecker.getOrphanRegionsOnRS();
+orphanRegionsOnFS = hbckChecker.getOrphanRegionsOnFS();
Review comment:
Yes.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [hbase] infraio commented on a change in pull request #404: HBASE-22709 Add a chore thread in master to do hbck checking
infraio commented on a change in pull request #404: HBASE-22709 Add a chore
thread in master to do hbck checking
URL: https://github.com/apache/hbase/pull/404#discussion_r307554663
##
File path: hbase-server/src/main/resources/hbase-webapps/master/hbck.jsp
##
@@ -0,0 +1,139 @@
+<%--
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+--%>
+<%@ page contentType="text/html;charset=UTF-8"
+ import="java.util.List"
+ import="java.util.Map"
+ import="java.util.stream.Collectors"
+%>
+<%@ page import="org.apache.hadoop.hbase.master.HbckChecker" %>
+<%@ page import="org.apache.hadoop.hbase.master.HMaster" %>
+<%@ page import="org.apache.hadoop.hbase.ServerName" %>
+<%@ page import="org.apache.hadoop.hbase.util.Pair" %>
+<%
+ HMaster master = (HMaster) getServletContext().getAttribute(HMaster.MASTER);
+ pageContext.setAttribute("pageTitle", "HBase Master HBCK Report: " +
master.getServerName());
+ HbckChecker hbckChecker = master.getHbckChecker();
+ Map>> inconsistentRegions = null;
+ Map orphanRegionsOnRS = null;
+ List orphanRegionsOnFS = null;
+ if (hbckChecker != null) {
+inconsistentRegions = hbckChecker.getInconsistentRegions();
+orphanRegionsOnRS = hbckChecker.getOrphanRegionsOnRS();
+orphanRegionsOnFS = hbckChecker.getOrphanRegionsOnFS();
+ }
+%>
+
+
+
+
+
+
+ <% if (!master.isInitialized()) { %>
+
+
+ Master is not initialized
+
+
+
+ <% } else { %>
+
+
+
+ Inconsistent Regions
+
+
+There are three case: 1. Master thought this region opened, but no
regionserver reported it.
+2. Master thought this region opened on Server1, but regionserver
reported Server2.
+3. More than one regionservers reported opened this region.
+Notice: the reported online regionservers may be not right when there
are regions in transition.
+Please check them in regionserver's web UI.
+
+
+
+
+
+ <% if (inconsistentRegions != null && inconsistentRegions.size() > 0) { %>
+
+
+ Region
+ Location in META
+ Reported Online RegionServers
+
+<% for (Map.Entry>> entry :
inconsistentRegions.entrySet()) { %>
+
+ <%= entry.getKey() %>
+ <%= entry.getValue().getFirst() %>
+ <%=
entry.getValue().getSecond().stream().map(ServerName::getServerName)
+.collect(Collectors.joining(", ")) %>
+
+<% } %>
+
+<%= inconsistentRegions.size() %> region(s) in set.
+
+ <% } %>
+
+
+
+ Orphan Regions on RegionServer
+
+
+
+ <% if (orphanRegionsOnRS != null && orphanRegionsOnRS.size() > 0) { %>
+
+
+ Region
+ Reported Online RegionServer
+
+<% for (Map.Entry entry :
orphanRegionsOnRS.entrySet()) { %>
+
+ <%= entry.getKey() %>
+ <%= entry.getValue() %>
+
+<% } %>
+
+<%= orphanRegionsOnRS.size() %> region(s) in set.
+
+ <% } %>
+
+
+
+ Orphan Regions on FileSystem
+
+
+
+ <% if (orphanRegionsOnFS != null && orphanRegionsOnFS.size() > 0) { %>
+
+
+ Region
+
+<% for (String region : orphanRegionsOnFS) { %>
+
+ <%= region %>
+
+<% } %>
+
+<%= orphanRegionsOnFS.size() %> region(s) in set.
+
+ <% } %>
+
+ <% } %>
+
+
+
Review comment:
Ok.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [hbase] infraio commented on a change in pull request #404: HBASE-22709 Add a chore thread in master to do hbck checking
infraio commented on a change in pull request #404: HBASE-22709 Add a chore
thread in master to do hbck checking
URL: https://github.com/apache/hbase/pull/404#discussion_r307554462
##
File path:
hbase-server/src/main/java/org/apache/hadoop/hbase/master/HbckChecker.java
##
@@ -0,0 +1,220 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.hadoop.hbase.master;
+
+import java.io.IOException;
+import java.util.HashMap;
+import java.util.LinkedList;
+import java.util.List;
+import java.util.Map;
+import java.util.Set;
+
+import org.apache.hadoop.fs.FileStatus;
+import org.apache.hadoop.fs.FileSystem;
+import org.apache.hadoop.fs.Path;
+import org.apache.hadoop.hbase.ScheduledChore;
+import org.apache.hadoop.hbase.ServerName;
+import org.apache.hadoop.hbase.client.RegionInfo;
+import org.apache.hadoop.hbase.util.FSUtils;
+import org.apache.hadoop.hbase.util.HbckRegionInfo;
+import org.apache.hadoop.hbase.util.Pair;
+import org.apache.yetus.audience.InterfaceAudience;
+import org.apache.yetus.audience.InterfaceStability;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+import org.apache.hbase.thirdparty.com.google.common.collect.Lists;
+
+/**
+ * Used to do the hbck checking job at master side.
+ */
+@InterfaceAudience.Private
+@InterfaceStability.Evolving
+public class HbckChecker extends ScheduledChore {
+ private static final Logger LOG =
LoggerFactory.getLogger(HbckChecker.class.getName());
+
+ private static final String HBCK_CHECKER_INTERVAL =
"hbase.master.hbck.checker.interval";
+ private static final int DEFAULT_HBCK_CHECKER_INTERVAL = 60 * 60 * 1000;
+
+ private final MasterServices master;
+
+ /**
+ * This map contains the state of all hbck items. It maps from encoded
region
+ * name to HbckRegionInfo structure. The information contained in
HbckRegionInfo is used
+ * to detect and correct consistency (hdfs/meta/deployment) problems.
+ */
+ private final Map regionInfoMap = new HashMap<>();
+
+ private final Map orphanRegionsOnRS = new HashMap<>();
+ private final List orphanRegionsOnFS = new LinkedList<>();
+ private final Map>>
inconsistentRegions =
+ new HashMap<>();
+
+ private final Map orphanRegionsOnRSSnapshot = new
HashMap<>();
+ private final List orphanRegionsOnFSSnapshot = new LinkedList<>();
+ private final Map>>
inconsistentRegionsSnapshot =
+ new HashMap<>();
+
+ private volatile boolean running = false;
+
+ public HbckChecker(MasterServices master) {
+super("HbckChecker-", master,
+master.getConfiguration().getInt(HBCK_CHECKER_INTERVAL,
DEFAULT_HBCK_CHECKER_INTERVAL));
+this.master = master;
+ }
+
+ @Override
+ protected void chore() {
+running = true;
+regionInfoMap.clear();
+orphanRegionsOnRS.clear();
+orphanRegionsOnFS.clear();
+inconsistentRegions.clear();
+loadRegionsFromInMemoryState();
+loadRegionsFromRSReport();
+try {
+ loadRegionsFromFS();
Review comment:
Yes. This need to time to finish.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [hbase] infraio commented on a change in pull request #404: HBASE-22709 Add a chore thread in master to do hbck checking
infraio commented on a change in pull request #404: HBASE-22709 Add a chore
thread in master to do hbck checking
URL: https://github.com/apache/hbase/pull/404#discussion_r307554462
##
File path:
hbase-server/src/main/java/org/apache/hadoop/hbase/master/HbckChecker.java
##
@@ -0,0 +1,220 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.hadoop.hbase.master;
+
+import java.io.IOException;
+import java.util.HashMap;
+import java.util.LinkedList;
+import java.util.List;
+import java.util.Map;
+import java.util.Set;
+
+import org.apache.hadoop.fs.FileStatus;
+import org.apache.hadoop.fs.FileSystem;
+import org.apache.hadoop.fs.Path;
+import org.apache.hadoop.hbase.ScheduledChore;
+import org.apache.hadoop.hbase.ServerName;
+import org.apache.hadoop.hbase.client.RegionInfo;
+import org.apache.hadoop.hbase.util.FSUtils;
+import org.apache.hadoop.hbase.util.HbckRegionInfo;
+import org.apache.hadoop.hbase.util.Pair;
+import org.apache.yetus.audience.InterfaceAudience;
+import org.apache.yetus.audience.InterfaceStability;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+import org.apache.hbase.thirdparty.com.google.common.collect.Lists;
+
+/**
+ * Used to do the hbck checking job at master side.
+ */
+@InterfaceAudience.Private
+@InterfaceStability.Evolving
+public class HbckChecker extends ScheduledChore {
+ private static final Logger LOG =
LoggerFactory.getLogger(HbckChecker.class.getName());
+
+ private static final String HBCK_CHECKER_INTERVAL =
"hbase.master.hbck.checker.interval";
+ private static final int DEFAULT_HBCK_CHECKER_INTERVAL = 60 * 60 * 1000;
+
+ private final MasterServices master;
+
+ /**
+ * This map contains the state of all hbck items. It maps from encoded
region
+ * name to HbckRegionInfo structure. The information contained in
HbckRegionInfo is used
+ * to detect and correct consistency (hdfs/meta/deployment) problems.
+ */
+ private final Map regionInfoMap = new HashMap<>();
+
+ private final Map orphanRegionsOnRS = new HashMap<>();
+ private final List orphanRegionsOnFS = new LinkedList<>();
+ private final Map>>
inconsistentRegions =
+ new HashMap<>();
+
+ private final Map orphanRegionsOnRSSnapshot = new
HashMap<>();
+ private final List orphanRegionsOnFSSnapshot = new LinkedList<>();
+ private final Map>>
inconsistentRegionsSnapshot =
+ new HashMap<>();
+
+ private volatile boolean running = false;
+
+ public HbckChecker(MasterServices master) {
+super("HbckChecker-", master,
+master.getConfiguration().getInt(HBCK_CHECKER_INTERVAL,
DEFAULT_HBCK_CHECKER_INTERVAL));
+this.master = master;
+ }
+
+ @Override
+ protected void chore() {
+running = true;
+regionInfoMap.clear();
+orphanRegionsOnRS.clear();
+orphanRegionsOnFS.clear();
+inconsistentRegions.clear();
+loadRegionsFromInMemoryState();
+loadRegionsFromRSReport();
+try {
+ loadRegionsFromFS();
Review comment:
Yes. This need time to finish.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[jira] [Comment Edited] (HBASE-22734) Format error message in create_namespace command to avoid stacktrace
[
https://issues.apache.org/jira/browse/HBASE-22734?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16893240#comment-16893240
]
Sakthi edited comment on HBASE-22734 at 7/26/19 12:59 AM:
--
Though this was discovered by mistake but I thought it would be nice to have a
well formatted just the ERROR message?
{code:java}
ERROR: java.io.IOException: Illegal character <:> at 3. Namespaces may only
contain 'alphanumeric characters' from any language and digits: ns1:t2{code}
and not the full stacktrace. So the description can be altered to some other
non-acceptable namespace name.
Basically something along the lines of HBASE-21360
was (Author: jatsakthi):
Though this was discovered by mistake but I thought it would be nice to have a
well formatted just the ERROR message?
{code:java}
ERROR: java.io.IOException: Illegal character <:> at 3. Namespaces may only
contain 'alphanumeric characters' from any language and digits: ns1:t2{code}
and not the full stacktrace. So the description can be altered to some other
non-acceptable namespace name.
> Format error message in create_namespace command to avoid stacktrace
>
>
> Key: HBASE-22734
> URL: https://issues.apache.org/jira/browse/HBASE-22734
> Project: HBase
> Issue Type: Improvement
>Reporter: Sakthi
>Priority: Minor
>
> This is what it looks like currently:
>
> {code:java}
> hbase(main):003:0> create_namespace 'ns1:t2'
> ERROR: java.io.IOException: Illegal character <:> at 3. Namespaces may only
> contain 'alphanumeric characters' from any language and digits: ns1:t2
> at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:472)
> at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:132)
> at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:338)
> at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:318)
> Caused by: java.lang.IllegalArgumentException: Illegal character <:> at 3.
> Namespaces may only contain 'alphanumeric characters' from any language and
> digits: ns1:t2
> at org.apache.hadoop.hbase.TableName.isLegalNamespaceName(TableName.java:251)
> at org.apache.hadoop.hbase.TableName.isLegalNamespaceName(TableName.java:225)
> at org.apache.hadoop.hbase.master.HMaster.createNamespace(HMaster.java:3267)
> at
> org.apache.hadoop.hbase.master.MasterRpcServices.createNamespace(MasterRpcServices.java:649)
> at
> org.apache.hadoop.hbase.shaded.protobuf.generated.MasterProtos$MasterService$2.callBlockingMethod(MasterProtos.java)
> at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:413)
> ... 3 more
> For usage try 'help "create_namespace"'
> Took 8.2554 seconds{code}
--
This message was sent by Atlassian JIRA
(v7.6.14#76016)
[GitHub] [hbase] infraio commented on a change in pull request #404: HBASE-22709 Add a chore thread in master to do hbck checking
infraio commented on a change in pull request #404: HBASE-22709 Add a chore
thread in master to do hbck checking
URL: https://github.com/apache/hbase/pull/404#discussion_r307554427
##
File path:
hbase-server/src/main/java/org/apache/hadoop/hbase/master/HbckChecker.java
##
@@ -0,0 +1,220 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.hadoop.hbase.master;
+
+import java.io.IOException;
+import java.util.HashMap;
+import java.util.LinkedList;
+import java.util.List;
+import java.util.Map;
+import java.util.Set;
+
+import org.apache.hadoop.fs.FileStatus;
+import org.apache.hadoop.fs.FileSystem;
+import org.apache.hadoop.fs.Path;
+import org.apache.hadoop.hbase.ScheduledChore;
+import org.apache.hadoop.hbase.ServerName;
+import org.apache.hadoop.hbase.client.RegionInfo;
+import org.apache.hadoop.hbase.util.FSUtils;
+import org.apache.hadoop.hbase.util.HbckRegionInfo;
+import org.apache.hadoop.hbase.util.Pair;
+import org.apache.yetus.audience.InterfaceAudience;
+import org.apache.yetus.audience.InterfaceStability;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+import org.apache.hbase.thirdparty.com.google.common.collect.Lists;
+
+/**
+ * Used to do the hbck checking job at master side.
+ */
+@InterfaceAudience.Private
+@InterfaceStability.Evolving
+public class HbckChecker extends ScheduledChore {
+ private static final Logger LOG =
LoggerFactory.getLogger(HbckChecker.class.getName());
+
+ private static final String HBCK_CHECKER_INTERVAL =
"hbase.master.hbck.checker.interval";
+ private static final int DEFAULT_HBCK_CHECKER_INTERVAL = 60 * 60 * 1000;
+
+ private final MasterServices master;
+
+ /**
+ * This map contains the state of all hbck items. It maps from encoded
region
+ * name to HbckRegionInfo structure. The information contained in
HbckRegionInfo is used
+ * to detect and correct consistency (hdfs/meta/deployment) problems.
+ */
+ private final Map regionInfoMap = new HashMap<>();
+
+ private final Map orphanRegionsOnRS = new HashMap<>();
+ private final List orphanRegionsOnFS = new LinkedList<>();
+ private final Map>>
inconsistentRegions =
+ new HashMap<>();
+
+ private final Map orphanRegionsOnRSSnapshot = new
HashMap<>();
+ private final List orphanRegionsOnFSSnapshot = new LinkedList<>();
+ private final Map>>
inconsistentRegionsSnapshot =
+ new HashMap<>();
+
+ private volatile boolean running = false;
+
+ public HbckChecker(MasterServices master) {
+super("HbckChecker-", master,
+master.getConfiguration().getInt(HBCK_CHECKER_INTERVAL,
DEFAULT_HBCK_CHECKER_INTERVAL));
+this.master = master;
+ }
+
+ @Override
+ protected void chore() {
+running = true;
+regionInfoMap.clear();
+orphanRegionsOnRS.clear();
+orphanRegionsOnFS.clear();
+inconsistentRegions.clear();
Review comment:
Need to clear first. Then generate this round's HBCK result. Then save it to
"snapshot".
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[jira] [Commented] (HBASE-22734) Format error message in create_namespace command to avoid stacktrace
[
https://issues.apache.org/jira/browse/HBASE-22734?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16893240#comment-16893240
]
Sakthi commented on HBASE-22734:
Though this was discovered by mistake but I thought it would be nice to have a
well formatted just the ERROR message?
{code:java}
ERROR: java.io.IOException: Illegal character <:> at 3. Namespaces may only
contain 'alphanumeric characters' from any language and digits: ns1:t2{code}
and not the full stacktrace. So the description can be altered to some other
non-acceptable namespace name.
> Format error message in create_namespace command to avoid stacktrace
>
>
> Key: HBASE-22734
> URL: https://issues.apache.org/jira/browse/HBASE-22734
> Project: HBase
> Issue Type: Improvement
>Reporter: Sakthi
>Priority: Minor
>
> This is what it looks like currently:
>
> {code:java}
> hbase(main):003:0> create_namespace 'ns1:t2'
> ERROR: java.io.IOException: Illegal character <:> at 3. Namespaces may only
> contain 'alphanumeric characters' from any language and digits: ns1:t2
> at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:472)
> at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:132)
> at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:338)
> at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:318)
> Caused by: java.lang.IllegalArgumentException: Illegal character <:> at 3.
> Namespaces may only contain 'alphanumeric characters' from any language and
> digits: ns1:t2
> at org.apache.hadoop.hbase.TableName.isLegalNamespaceName(TableName.java:251)
> at org.apache.hadoop.hbase.TableName.isLegalNamespaceName(TableName.java:225)
> at org.apache.hadoop.hbase.master.HMaster.createNamespace(HMaster.java:3267)
> at
> org.apache.hadoop.hbase.master.MasterRpcServices.createNamespace(MasterRpcServices.java:649)
> at
> org.apache.hadoop.hbase.shaded.protobuf.generated.MasterProtos$MasterService$2.callBlockingMethod(MasterProtos.java)
> at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:413)
> ... 3 more
> For usage try 'help "create_namespace"'
> Took 8.2554 seconds{code}
--
This message was sent by Atlassian JIRA
(v7.6.14#76016)
[jira] [Commented] (HBASE-22733) TestSplitTransactionOnCluster.testMasterRestartAtRegionSplitPendingCatalogJanitor is flakey
[
https://issues.apache.org/jira/browse/HBASE-22733?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16893239#comment-16893239
]
Hudson commented on HBASE-22733:
Results for branch master
[build #1272 on
builds.a.o|https://builds.apache.org/job/HBase%20Nightly/job/master/1272/]: (x)
*{color:red}-1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://builds.apache.org/job/HBase%20Nightly/job/master/1272//General_Nightly_Build_Report/]
(x) {color:red}-1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://builds.apache.org/job/HBase%20Nightly/job/master/1272//JDK8_Nightly_Build_Report_(Hadoop2)/]
(/) {color:green}+1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://builds.apache.org/job/HBase%20Nightly/job/master/1272//JDK8_Nightly_Build_Report_(Hadoop3)/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
(/) {color:green}+1 client integration test{color}
> TestSplitTransactionOnCluster.testMasterRestartAtRegionSplitPendingCatalogJanitor
> is flakey
> ---
>
> Key: HBASE-22733
> URL: https://issues.apache.org/jira/browse/HBASE-22733
> Project: HBase
> Issue Type: Bug
>Reporter: Xiaolin Ha
>Assignee: Xiaolin Ha
>Priority: Major
> Fix For: 3.0.0, 2.3.0, 2.0.6, 2.2.1, 2.1.6
>
> Attachments: test -
> testMasterRestartAtRegionSplitPendingCatalogJanitor [Jenkins].txt
>
>
> RITs will lead to CatalogJanitor not running, and then parent region will not
> be cleaned.
> As a result, the last assertion of checking if parent region is null will
> fail.
> {quote}
> h3. Error Message
> expected: but was:
> h3. Stacktrace
> java.lang.AssertionError: expected: but
> was: at
> org.apache.hadoop.hbase.regionserver.TestSplitTransactionOnCluster.testMasterRestartAtRegionSplitPendingCatalogJanitor(TestSplitTransactionOnCluster.java:577){quote}
> Currently, the case is only waiting for master be active while not care about
> RITs completion when master starts up.
> Failed test results is attached. We can see CatalogJanitor chore run failed
> log in it,
> {quote}2019-07-23 18:34:03,241 INFO [Time-limited test]
> regionserver.TestSplitTransactionOnCluster(572): Starting run of
> CatalogJanitor 2019-07-23 18:34:03,241 WARN [Time-limited test]
> master.CatalogJanitor(119): CatalogJanitor is disabled! Enabled=true,
> maintenanceMode=false,
> am=org.apache.hadoop.hbase.master.assignment.AssignmentManager@1ce25bee,
> metaLoaded=true, hasRIT=true clusterShutDown=false
> {quote}
>
>
--
This message was sent by Atlassian JIRA
(v7.6.14#76016)
[GitHub] [hbase] infraio commented on a change in pull request #404: HBASE-22709 Add a chore thread in master to do hbck checking
infraio commented on a change in pull request #404: HBASE-22709 Add a chore
thread in master to do hbck checking
URL: https://github.com/apache/hbase/pull/404#discussion_r307553408
##
File path:
hbase-server/src/main/java/org/apache/hadoop/hbase/master/HbckChecker.java
##
@@ -0,0 +1,220 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.hadoop.hbase.master;
+
+import java.io.IOException;
+import java.util.HashMap;
+import java.util.LinkedList;
+import java.util.List;
+import java.util.Map;
+import java.util.Set;
+
+import org.apache.hadoop.fs.FileStatus;
+import org.apache.hadoop.fs.FileSystem;
+import org.apache.hadoop.fs.Path;
+import org.apache.hadoop.hbase.ScheduledChore;
+import org.apache.hadoop.hbase.ServerName;
+import org.apache.hadoop.hbase.client.RegionInfo;
+import org.apache.hadoop.hbase.util.FSUtils;
+import org.apache.hadoop.hbase.util.HbckRegionInfo;
+import org.apache.hadoop.hbase.util.Pair;
+import org.apache.yetus.audience.InterfaceAudience;
+import org.apache.yetus.audience.InterfaceStability;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+import org.apache.hbase.thirdparty.com.google.common.collect.Lists;
+
+/**
+ * Used to do the hbck checking job at master side.
+ */
+@InterfaceAudience.Private
+@InterfaceStability.Evolving
+public class HbckChecker extends ScheduledChore {
+ private static final Logger LOG =
LoggerFactory.getLogger(HbckChecker.class.getName());
+
+ private static final String HBCK_CHECKER_INTERVAL =
"hbase.master.hbck.checker.interval";
+ private static final int DEFAULT_HBCK_CHECKER_INTERVAL = 60 * 60 * 1000;
+
+ private final MasterServices master;
+
+ /**
+ * This map contains the state of all hbck items. It maps from encoded
region
+ * name to HbckRegionInfo structure. The information contained in
HbckRegionInfo is used
+ * to detect and correct consistency (hdfs/meta/deployment) problems.
+ */
+ private final Map regionInfoMap = new HashMap<>();
+
+ private final Map orphanRegionsOnRS = new HashMap<>();
+ private final List orphanRegionsOnFS = new LinkedList<>();
+ private final Map>>
inconsistentRegions =
+ new HashMap<>();
+
+ private final Map orphanRegionsOnRSSnapshot = new
HashMap<>();
Review comment:
Yes. Will add more javadoc/comments later.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[jira] [Commented] (HBASE-22734) Format error message in create_namespace command to avoid stacktrace
[
https://issues.apache.org/jira/browse/HBASE-22734?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16893237#comment-16893237
]
stack commented on HBASE-22734:
---
You are creating a table, not a namespace. The namespace to create would be
'ns1'?
> Format error message in create_namespace command to avoid stacktrace
>
>
> Key: HBASE-22734
> URL: https://issues.apache.org/jira/browse/HBASE-22734
> Project: HBase
> Issue Type: Improvement
>Reporter: Sakthi
>Priority: Minor
>
> This is what it looks like currently:
>
> {code:java}
> hbase(main):003:0> create_namespace 'ns1:t2'
> ERROR: java.io.IOException: Illegal character <:> at 3. Namespaces may only
> contain 'alphanumeric characters' from any language and digits: ns1:t2
> at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:472)
> at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:132)
> at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:338)
> at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:318)
> Caused by: java.lang.IllegalArgumentException: Illegal character <:> at 3.
> Namespaces may only contain 'alphanumeric characters' from any language and
> digits: ns1:t2
> at org.apache.hadoop.hbase.TableName.isLegalNamespaceName(TableName.java:251)
> at org.apache.hadoop.hbase.TableName.isLegalNamespaceName(TableName.java:225)
> at org.apache.hadoop.hbase.master.HMaster.createNamespace(HMaster.java:3267)
> at
> org.apache.hadoop.hbase.master.MasterRpcServices.createNamespace(MasterRpcServices.java:649)
> at
> org.apache.hadoop.hbase.shaded.protobuf.generated.MasterProtos$MasterService$2.callBlockingMethod(MasterProtos.java)
> at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:413)
> ... 3 more
> For usage try 'help "create_namespace"'
> Took 8.2554 seconds{code}
--
This message was sent by Atlassian JIRA
(v7.6.14#76016)
[GitHub] [hbase] infraio commented on a change in pull request #404: HBASE-22709 Add a chore thread in master to do hbck checking
infraio commented on a change in pull request #404: HBASE-22709 Add a chore
thread in master to do hbck checking
URL: https://github.com/apache/hbase/pull/404#discussion_r307553328
##
File path:
hbase-server/src/main/java/org/apache/hadoop/hbase/master/HbckChecker.java
##
@@ -0,0 +1,220 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.hadoop.hbase.master;
+
+import java.io.IOException;
+import java.util.HashMap;
+import java.util.LinkedList;
+import java.util.List;
+import java.util.Map;
+import java.util.Set;
+
+import org.apache.hadoop.fs.FileStatus;
+import org.apache.hadoop.fs.FileSystem;
+import org.apache.hadoop.fs.Path;
+import org.apache.hadoop.hbase.ScheduledChore;
+import org.apache.hadoop.hbase.ServerName;
+import org.apache.hadoop.hbase.client.RegionInfo;
+import org.apache.hadoop.hbase.util.FSUtils;
+import org.apache.hadoop.hbase.util.HbckRegionInfo;
+import org.apache.hadoop.hbase.util.Pair;
+import org.apache.yetus.audience.InterfaceAudience;
+import org.apache.yetus.audience.InterfaceStability;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+import org.apache.hbase.thirdparty.com.google.common.collect.Lists;
+
+/**
+ * Used to do the hbck checking job at master side.
+ */
+@InterfaceAudience.Private
+@InterfaceStability.Evolving
+public class HbckChecker extends ScheduledChore {
+ private static final Logger LOG =
LoggerFactory.getLogger(HbckChecker.class.getName());
+
+ private static final String HBCK_CHECKER_INTERVAL =
"hbase.master.hbck.checker.interval";
+ private static final int DEFAULT_HBCK_CHECKER_INTERVAL = 60 * 60 * 1000;
+
+ private final MasterServices master;
+
+ /**
+ * This map contains the state of all hbck items. It maps from encoded
region
+ * name to HbckRegionInfo structure. The information contained in
HbckRegionInfo is used
+ * to detect and correct consistency (hdfs/meta/deployment) problems.
+ */
+ private final Map regionInfoMap = new HashMap<>();
+
+ private final Map orphanRegionsOnRS = new HashMap<>();
+ private final List orphanRegionsOnFS = new LinkedList<>();
+ private final Map>>
inconsistentRegions =
Review comment:
orphanRegionsOnRS means the region was opened on one RS. But not in meta...
Only in meta case can be handled by inconsistentRegions. Let me add more
comments here.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[jira] [Created] (HBASE-22734) Format error message in create_namespace command to avoid stacktrace
Sakthi created HBASE-22734:
--
Summary: Format error message in create_namespace command to avoid
stacktrace
Key: HBASE-22734
URL: https://issues.apache.org/jira/browse/HBASE-22734
Project: HBase
Issue Type: Improvement
Reporter: Sakthi
This is what it looks like currently:
{code:java}
hbase(main):003:0> create_namespace 'ns1:t2'
ERROR: java.io.IOException: Illegal character <:> at 3. Namespaces may only
contain 'alphanumeric characters' from any language and digits: ns1:t2
at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:472)
at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:132)
at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:338)
at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:318)
Caused by: java.lang.IllegalArgumentException: Illegal character <:> at 3.
Namespaces may only contain 'alphanumeric characters' from any language and
digits: ns1:t2
at org.apache.hadoop.hbase.TableName.isLegalNamespaceName(TableName.java:251)
at org.apache.hadoop.hbase.TableName.isLegalNamespaceName(TableName.java:225)
at org.apache.hadoop.hbase.master.HMaster.createNamespace(HMaster.java:3267)
at
org.apache.hadoop.hbase.master.MasterRpcServices.createNamespace(MasterRpcServices.java:649)
at
org.apache.hadoop.hbase.shaded.protobuf.generated.MasterProtos$MasterService$2.callBlockingMethod(MasterProtos.java)
at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:413)
... 3 more
For usage try 'help "create_namespace"'
Took 8.2554 seconds{code}
--
This message was sent by Atlassian JIRA
(v7.6.14#76016)
[jira] [Commented] (HBASE-22733) TestSplitTransactionOnCluster.testMasterRestartAtRegionSplitPendingCatalogJanitor is flakey
[
https://issues.apache.org/jira/browse/HBASE-22733?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16893235#comment-16893235
]
Hudson commented on HBASE-22733:
Results for branch branch-2
[build #2114 on
builds.a.o|https://builds.apache.org/job/HBase%20Nightly/job/branch-2/2114/]:
(x) *{color:red}-1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2/2114//General_Nightly_Build_Report/]
(x) {color:red}-1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2/2114//JDK8_Nightly_Build_Report_(Hadoop2)/]
(/) {color:green}+1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2/2114//JDK8_Nightly_Build_Report_(Hadoop3)/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
(/) {color:green}+1 client integration test{color}
> TestSplitTransactionOnCluster.testMasterRestartAtRegionSplitPendingCatalogJanitor
> is flakey
> ---
>
> Key: HBASE-22733
> URL: https://issues.apache.org/jira/browse/HBASE-22733
> Project: HBase
> Issue Type: Bug
>Reporter: Xiaolin Ha
>Assignee: Xiaolin Ha
>Priority: Major
> Fix For: 3.0.0, 2.3.0, 2.0.6, 2.2.1, 2.1.6
>
> Attachments: test -
> testMasterRestartAtRegionSplitPendingCatalogJanitor [Jenkins].txt
>
>
> RITs will lead to CatalogJanitor not running, and then parent region will not
> be cleaned.
> As a result, the last assertion of checking if parent region is null will
> fail.
> {quote}
> h3. Error Message
> expected: but was:
> h3. Stacktrace
> java.lang.AssertionError: expected: but
> was: at
> org.apache.hadoop.hbase.regionserver.TestSplitTransactionOnCluster.testMasterRestartAtRegionSplitPendingCatalogJanitor(TestSplitTransactionOnCluster.java:577){quote}
> Currently, the case is only waiting for master be active while not care about
> RITs completion when master starts up.
> Failed test results is attached. We can see CatalogJanitor chore run failed
> log in it,
> {quote}2019-07-23 18:34:03,241 INFO [Time-limited test]
> regionserver.TestSplitTransactionOnCluster(572): Starting run of
> CatalogJanitor 2019-07-23 18:34:03,241 WARN [Time-limited test]
> master.CatalogJanitor(119): CatalogJanitor is disabled! Enabled=true,
> maintenanceMode=false,
> am=org.apache.hadoop.hbase.master.assignment.AssignmentManager@1ce25bee,
> metaLoaded=true, hasRIT=true clusterShutDown=false
> {quote}
>
>
--
This message was sent by Atlassian JIRA
(v7.6.14#76016)
[jira] [Commented] (HBASE-22733) TestSplitTransactionOnCluster.testMasterRestartAtRegionSplitPendingCatalogJanitor is flakey
[
https://issues.apache.org/jira/browse/HBASE-22733?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16893234#comment-16893234
]
Hudson commented on HBASE-22733:
Results for branch branch-2.1
[build #1411 on
builds.a.o|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.1/1411/]:
(x) *{color:red}-1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.1/1411//General_Nightly_Build_Report/]
(x) {color:red}-1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.1/1411//JDK8_Nightly_Build_Report_(Hadoop2)/]
(x) {color:red}-1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.1/1411//JDK8_Nightly_Build_Report_(Hadoop3)/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
(/) {color:green}+1 client integration test{color}
> TestSplitTransactionOnCluster.testMasterRestartAtRegionSplitPendingCatalogJanitor
> is flakey
> ---
>
> Key: HBASE-22733
> URL: https://issues.apache.org/jira/browse/HBASE-22733
> Project: HBase
> Issue Type: Bug
>Reporter: Xiaolin Ha
>Assignee: Xiaolin Ha
>Priority: Major
> Fix For: 3.0.0, 2.3.0, 2.0.6, 2.2.1, 2.1.6
>
> Attachments: test -
> testMasterRestartAtRegionSplitPendingCatalogJanitor [Jenkins].txt
>
>
> RITs will lead to CatalogJanitor not running, and then parent region will not
> be cleaned.
> As a result, the last assertion of checking if parent region is null will
> fail.
> {quote}
> h3. Error Message
> expected: but was:
> h3. Stacktrace
> java.lang.AssertionError: expected: but
> was: at
> org.apache.hadoop.hbase.regionserver.TestSplitTransactionOnCluster.testMasterRestartAtRegionSplitPendingCatalogJanitor(TestSplitTransactionOnCluster.java:577){quote}
> Currently, the case is only waiting for master be active while not care about
> RITs completion when master starts up.
> Failed test results is attached. We can see CatalogJanitor chore run failed
> log in it,
> {quote}2019-07-23 18:34:03,241 INFO [Time-limited test]
> regionserver.TestSplitTransactionOnCluster(572): Starting run of
> CatalogJanitor 2019-07-23 18:34:03,241 WARN [Time-limited test]
> master.CatalogJanitor(119): CatalogJanitor is disabled! Enabled=true,
> maintenanceMode=false,
> am=org.apache.hadoop.hbase.master.assignment.AssignmentManager@1ce25bee,
> metaLoaded=true, hasRIT=true clusterShutDown=false
> {quote}
>
>
--
This message was sent by Atlassian JIRA
(v7.6.14#76016)
[jira] [Commented] (HBASE-22733) TestSplitTransactionOnCluster.testMasterRestartAtRegionSplitPendingCatalogJanitor is flakey
[
https://issues.apache.org/jira/browse/HBASE-22733?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16893230#comment-16893230
]
Hudson commented on HBASE-22733:
Results for branch branch-2.0
[build #1793 on
builds.a.o|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.0/1793/]:
(x) *{color:red}-1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.0/1793//General_Nightly_Build_Report/]
(x) {color:red}-1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.0/1793//JDK8_Nightly_Build_Report_(Hadoop2)/]
(/) {color:green}+1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.0/1793//JDK8_Nightly_Build_Report_(Hadoop3)/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
> TestSplitTransactionOnCluster.testMasterRestartAtRegionSplitPendingCatalogJanitor
> is flakey
> ---
>
> Key: HBASE-22733
> URL: https://issues.apache.org/jira/browse/HBASE-22733
> Project: HBase
> Issue Type: Bug
>Reporter: Xiaolin Ha
>Assignee: Xiaolin Ha
>Priority: Major
> Fix For: 3.0.0, 2.3.0, 2.0.6, 2.2.1, 2.1.6
>
> Attachments: test -
> testMasterRestartAtRegionSplitPendingCatalogJanitor [Jenkins].txt
>
>
> RITs will lead to CatalogJanitor not running, and then parent region will not
> be cleaned.
> As a result, the last assertion of checking if parent region is null will
> fail.
> {quote}
> h3. Error Message
> expected: but was:
> h3. Stacktrace
> java.lang.AssertionError: expected: but
> was: at
> org.apache.hadoop.hbase.regionserver.TestSplitTransactionOnCluster.testMasterRestartAtRegionSplitPendingCatalogJanitor(TestSplitTransactionOnCluster.java:577){quote}
> Currently, the case is only waiting for master be active while not care about
> RITs completion when master starts up.
> Failed test results is attached. We can see CatalogJanitor chore run failed
> log in it,
> {quote}2019-07-23 18:34:03,241 INFO [Time-limited test]
> regionserver.TestSplitTransactionOnCluster(572): Starting run of
> CatalogJanitor 2019-07-23 18:34:03,241 WARN [Time-limited test]
> master.CatalogJanitor(119): CatalogJanitor is disabled! Enabled=true,
> maintenanceMode=false,
> am=org.apache.hadoop.hbase.master.assignment.AssignmentManager@1ce25bee,
> metaLoaded=true, hasRIT=true clusterShutDown=false
> {quote}
>
>
--
This message was sent by Atlassian JIRA
(v7.6.14#76016)
[GitHub] [hbase] Apache-HBase commented on issue #361: HBase-22027: Split non-MR related parts of TokenUtil off into a Clien…
Apache-HBase commented on issue #361: HBase-22027: Split non-MR related parts of TokenUtil off into a Clien… URL: https://github.com/apache/hbase/pull/361#issuecomment-515263251 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | 0 | reexec | 51 | Docker mode activated. | ||| _ Prechecks _ | | +1 | dupname | 0 | No case conflicting files found. | | +1 | hbaseanti | 0 | Patch does not have any anti-patterns. | | +1 | @author | 0 | The patch does not contain any @author tags. | | +1 | test4tests | 0 | The patch appears to include 2 new or modified test files. | ||| _ master Compile Tests _ | | 0 | mvndep | 20 | Maven dependency ordering for branch | | +1 | mvninstall | 336 | master passed | | +1 | compile | 84 | master passed | | +1 | checkstyle | 106 | master passed | | +1 | shadedjars | 293 | branch has no errors when building our shaded downstream artifacts. | | +1 | javadoc | 57 | master passed | | 0 | spotbugs | 225 | Used deprecated FindBugs config; considering switching to SpotBugs. | | +1 | findbugs | 290 | master passed | ||| _ Patch Compile Tests _ | | 0 | mvndep | 16 | Maven dependency ordering for patch | | +1 | mvninstall | 287 | the patch passed | | +1 | compile | 76 | the patch passed | | +1 | javac | 76 | the patch passed | | +1 | checkstyle | 109 | the patch passed | | +1 | whitespace | 0 | The patch has no whitespace issues. | | +1 | shadedjars | 294 | patch has no errors when building our shaded downstream artifacts. | | +1 | hadoopcheck | 1017 | Patch does not cause any errors with Hadoop 2.8.5 2.9.2 or 3.1.2. | | +1 | javadoc | 54 | the patch passed | | +1 | findbugs | 299 | the patch passed | ||| _ Other Tests _ | | +1 | unit | 107 | hbase-client in the patch passed. | | -1 | unit | 16629 | hbase-server in the patch failed. | | +1 | asflicense | 64 | The patch does not generate ASF License warnings. | | | | 20565 | | | Reason | Tests | |---:|:--| | Failed junit tests | hadoop.hbase.client.TestSnapshotTemporaryDirectory | | | hadoop.hbase.client.TestConnection | | | hadoop.hbase.client.TestAsyncTableAdminApi | | | hadoop.hbase.replication.TestReplicationSmallTests | | Subsystem | Report/Notes | |--:|:-| | Docker | Client=19.03.0 Server=19.03.0 base: https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-361/30/artifact/out/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/361 | | Optional Tests | dupname asflicense javac javadoc unit spotbugs findbugs shadedjars hadoopcheck hbaseanti checkstyle compile | | uname | Linux a1c72acdf8f8 4.4.0-139-generic #165-Ubuntu SMP Wed Oct 24 10:58:50 UTC 2018 x86_64 GNU/Linux | | Build tool | maven | | Personality | /home/jenkins/jenkins-slave/workspace/HBase-PreCommit-GitHub-PR_PR-361/out/precommit/personality/provided.sh | | git revision | master / 747d000808 | | Default Java | 1.8.0_181 | | unit | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-361/30/artifact/out/patch-unit-hbase-server.txt | | Test Results | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-361/30/testReport/ | | Max. process+thread count | 5047 (vs. ulimit of 1) | | modules | C: hbase-client hbase-server U: . | | Console output | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-361/30/console | | versions | git=2.11.0 maven=2018-06-17T18:33:14Z) findbugs=3.1.11 | | Powered by | Apache Yetus 0.10.0 http://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [hbase] Apache-HBase commented on issue #409: check bounds when removeRegion
Apache-HBase commented on issue #409: check bounds when removeRegion URL: https://github.com/apache/hbase/pull/409#issuecomment-515259663 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | 0 | reexec | 76 | Docker mode activated. | ||| _ Prechecks _ | | +1 | dupname | 0 | No case conflicting files found. | | +1 | hbaseanti | 0 | Patch does not have any anti-patterns. | | +1 | @author | 0 | The patch does not contain any @author tags. | | -0 | test4tests | 0 | The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. | ||| _ branch-2.0 Compile Tests _ | | +1 | mvninstall | 302 | branch-2.0 passed | | +1 | compile | 73 | branch-2.0 passed | | +1 | checkstyle | 97 | branch-2.0 passed | | +1 | shadedjars | 329 | branch has no errors when building our shaded downstream artifacts. | | +1 | javadoc | 45 | branch-2.0 passed | | 0 | spotbugs | 233 | Used deprecated FindBugs config; considering switching to SpotBugs. | | +1 | findbugs | 228 | branch-2.0 passed | ||| _ Patch Compile Tests _ | | +1 | mvninstall | 253 | the patch passed | | +1 | compile | 71 | the patch passed | | +1 | javac | 71 | the patch passed | | -1 | checkstyle | 99 | hbase-server: The patch generated 1 new + 60 unchanged - 0 fixed = 61 total (was 60) | | +1 | whitespace | 0 | The patch has no whitespace issues. | | +1 | shadedjars | 335 | patch has no errors when building our shaded downstream artifacts. | | +1 | hadoopcheck | 1238 | Patch does not cause any errors with Hadoop 2.6.5 2.7.7 2.8.5 or 3.0.3 3.1.2. | | +1 | javadoc | 41 | the patch passed | | +1 | findbugs | 222 | the patch passed | ||| _ Other Tests _ | | -1 | unit | 13627 | hbase-server in the patch failed. | | +1 | asflicense | 50 | The patch does not generate ASF License warnings. | | | | 17440 | | | Reason | Tests | |---:|:--| | Failed junit tests | hadoop.hbase.master.balancer.TestStochasticLoadBalancerRegionReplicaLargeCluster | | | hadoop.hbase.master.balancer.TestStochasticLoadBalancerMidCluster | | | hadoop.hbase.master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks | | | hadoop.hbase.coprocessor.TestMasterObserver | | | hadoop.hbase.master.balancer.TestStochasticLoadBalancer | | | hadoop.hbase.master.balancer.TestStochasticLoadBalancerSmallCluster | | | hadoop.hbase.master.procedure.TestProcedurePriority | | | hadoop.hbase.master.balancer.TestStochasticLoadBalancerBalanceCluster | | | hadoop.hbase.TestRegionRebalancing | | | hadoop.hbase.master.balancer.TestStochasticLoadBalancerRegionReplica | | | hadoop.hbase.master.balancer.TestStochasticLoadBalancerLargeCluster | | | hadoop.hbase.master.balancer.TestStochasticLoadBalancerRegionReplicaHighReplication | | | hadoop.hbase.master.balancer.TestStochasticLoadBalancerRegionReplicaReplicationGreaterThanNumNodes | | | hadoop.hbase.master.balancer.TestStochasticLoadBalancerRegionReplicaSameHosts | | | hadoop.hbase.master.balancer.TestStochasticLoadBalancerRegionReplicaMidCluster | | Subsystem | Report/Notes | |--:|:-| | Docker | Client=19.03.0 Server=19.03.0 base: https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-409/3/artifact/out/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/409 | | Optional Tests | dupname asflicense javac javadoc unit spotbugs findbugs shadedjars hadoopcheck hbaseanti checkstyle compile | | uname | Linux 1f705f54b7e1 4.4.0-138-generic #164-Ubuntu SMP Tue Oct 2 17:16:02 UTC 2018 x86_64 GNU/Linux | | Build tool | maven | | Personality | /home/jenkins/jenkins-slave/workspace/HBase-PreCommit-GitHub-PR_PR-409/out/precommit/personality/provided.sh | | git revision | branch-2.0 / dc9017aa49 | | Default Java | 1.8.0_181 | | checkstyle | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-409/3/artifact/out/diff-checkstyle-hbase-server.txt | | unit | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-409/3/artifact/out/patch-unit-hbase-server.txt | | Test Results | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-409/3/testReport/ | | Max. process+thread count | 4623 (vs. ulimit of 1) | | modules | C: hbase-server U: hbase-server | | Console output | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-409/3/console | | versions | git=2.11.0 maven=2018-06-17T18:33:14Z) findbugs=3.1.0-RC3 | | Powered by | Apache Yetus 0.10.0 http://yetus.apache.org | This message was automatically generated. This is an automated message
[GitHub] [hbase] Apache-HBase commented on issue #403: HBASE-22707 [HBCK2] MasterRpcServices assigns method should try to reload regions from meta if the passed regions isn't found under AssignmentMan
Apache-HBase commented on issue #403: HBASE-22707 [HBCK2] MasterRpcServices assigns method should try to reload regions from meta if the passed regions isn't found under AssignmentManager RegionsStateStore URL: https://github.com/apache/hbase/pull/403#issuecomment-515258123 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | 0 | reexec | 76 | Docker mode activated. | ||| _ Prechecks _ | | +1 | dupname | 0 | No case conflicting files found. | | +1 | hbaseanti | 0 | Patch does not have any anti-patterns. | | +1 | @author | 0 | The patch does not contain any @author tags. | | -0 | test4tests | 0 | The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. | ||| _ master Compile Tests _ | | 0 | mvndep | 16 | Maven dependency ordering for branch | | +1 | mvninstall | 345 | master passed | | +1 | compile | 84 | master passed | | +1 | checkstyle | 108 | master passed | | +1 | shadedjars | 265 | branch has no errors when building our shaded downstream artifacts. | | +1 | javadoc | 55 | master passed | | 0 | spotbugs | 237 | Used deprecated FindBugs config; considering switching to SpotBugs. | | +1 | findbugs | 299 | master passed | ||| _ Patch Compile Tests _ | | 0 | mvndep | 15 | Maven dependency ordering for patch | | +1 | mvninstall | 337 | the patch passed | | +1 | compile | 75 | the patch passed | | +1 | javac | 75 | the patch passed | | +1 | checkstyle | 101 | the patch passed | | +1 | whitespace | 0 | The patch has no whitespace issues. | | +1 | shadedjars | 270 | patch has no errors when building our shaded downstream artifacts. | | +1 | hadoopcheck | 984 | Patch does not cause any errors with Hadoop 2.8.5 2.9.2 or 3.1.2. | | +1 | javadoc | 56 | the patch passed | | +1 | findbugs | 328 | the patch passed | ||| _ Other Tests _ | | +1 | unit | 106 | hbase-client in the patch passed. | | -1 | unit | 16624 | hbase-server in the patch failed. | | +1 | asflicense | 56 | The patch does not generate ASF License warnings. | | | | 20593 | | | Reason | Tests | |---:|:--| | Failed junit tests | hadoop.hbase.master.procedure.TestSCPWithReplicas | | Subsystem | Report/Notes | |--:|:-| | Docker | Client=19.03.0 Server=19.03.0 base: https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-403/14/artifact/out/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/403 | | Optional Tests | dupname asflicense javac javadoc unit spotbugs findbugs shadedjars hadoopcheck hbaseanti checkstyle compile | | uname | Linux d3f020dd4431 4.4.0-139-generic #165-Ubuntu SMP Wed Oct 24 10:58:50 UTC 2018 x86_64 GNU/Linux | | Build tool | maven | | Personality | /home/jenkins/jenkins-slave/workspace/HBase-PreCommit-GitHub-PR_PR-403/out/precommit/personality/provided.sh | | git revision | master / 747d000808 | | Default Java | 1.8.0_181 | | unit | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-403/14/artifact/out/patch-unit-hbase-server.txt | | Test Results | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-403/14/testReport/ | | Max. process+thread count | 5052 (vs. ulimit of 1) | | modules | C: hbase-client hbase-server U: . | | Console output | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-403/14/console | | versions | git=2.11.0 maven=2018-06-17T18:33:14Z) findbugs=3.1.11 | | Powered by | Apache Yetus 0.10.0 http://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [hbase] Apache-HBase commented on issue #199: HBASE-22313 Add method to FsDelegation token to accept token kind
Apache-HBase commented on issue #199: HBASE-22313 Add method to FsDelegation token to accept token kind URL: https://github.com/apache/hbase/pull/199#issuecomment-515257918 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | 0 | reexec | 88 | Docker mode activated. | ||| _ Prechecks _ | | +1 | dupname | 0 | No case conflicting files found. | | +1 | hbaseanti | 0 | Patch does not have any anti-patterns. | | +1 | @author | 0 | The patch does not contain any @author tags. | | +1 | test4tests | 0 | The patch appears to include 1 new or modified test files. | ||| _ master Compile Tests _ | | +1 | mvninstall | 455 | master passed | | +1 | compile | 54 | master passed | | +1 | checkstyle | 73 | master passed | | +1 | shadedjars | 269 | branch has no errors when building our shaded downstream artifacts. | | +1 | javadoc | 34 | master passed | | 0 | spotbugs | 243 | Used deprecated FindBugs config; considering switching to SpotBugs. | | +1 | findbugs | 241 | master passed | ||| _ Patch Compile Tests _ | | +1 | mvninstall | 286 | the patch passed | | +1 | compile | 50 | the patch passed | | +1 | javac | 50 | the patch passed | | +1 | checkstyle | 71 | hbase-server: The patch generated 0 new + 0 unchanged - 1 fixed = 0 total (was 1) | | +1 | whitespace | 0 | The patch has no whitespace issues. | | +1 | shadedjars | 263 | patch has no errors when building our shaded downstream artifacts. | | +1 | hadoopcheck | 910 | Patch does not cause any errors with Hadoop 2.8.5 2.9.2 or 3.1.2. | | +1 | javadoc | 31 | the patch passed | | +1 | findbugs | 226 | the patch passed | ||| _ Other Tests _ | | -1 | unit | 15509 | hbase-server in the patch failed. | | +1 | asflicense | 35 | The patch does not generate ASF License warnings. | | | | 18966 | | | Subsystem | Report/Notes | |--:|:-| | Docker | Client=19.03.0 Server=19.03.0 base: https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-199/34/artifact/out/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/199 | | Optional Tests | dupname asflicense javac javadoc unit spotbugs findbugs shadedjars hadoopcheck hbaseanti checkstyle compile | | uname | Linux bb0c991edcd5 4.4.0-138-generic #164-Ubuntu SMP Tue Oct 2 17:16:02 UTC 2018 x86_64 GNU/Linux | | Build tool | maven | | Personality | /home/jenkins/jenkins-slave/workspace/HBase-PreCommit-GitHub-PR_PR-199/out/precommit/personality/provided.sh | | git revision | master / 747d000808 | | Default Java | 1.8.0_181 | | unit | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-199/34/artifact/out/patch-unit-hbase-server.txt | | Test Results | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-199/34/testReport/ | | Max. process+thread count | 4950 (vs. ulimit of 1) | | modules | C: hbase-server U: hbase-server | | Console output | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-199/34/console | | versions | git=2.11.0 maven=2018-06-17T18:33:14Z) findbugs=3.1.11 | | Powered by | Apache Yetus 0.10.0 http://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [hbase] Apache-HBase commented on issue #404: HBASE-22709 Add a chore thread in master to do hbck checking
Apache-HBase commented on issue #404: HBASE-22709 Add a chore thread in master to do hbck checking URL: https://github.com/apache/hbase/pull/404#issuecomment-515257728 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | 0 | reexec | 55 | Docker mode activated. | ||| _ Prechecks _ | | +1 | dupname | 0 | No case conflicting files found. | | +1 | hbaseanti | 0 | Patch does not have any anti-patterns. | | +1 | @author | 0 | The patch does not contain any @author tags. | | +1 | test4tests | 0 | The patch appears to include 2 new or modified test files. | ||| _ master Compile Tests _ | | +1 | mvninstall | 316 | master passed | | +1 | compile | 54 | master passed | | +1 | checkstyle | 81 | master passed | | +1 | shadedjars | 270 | branch has no errors when building our shaded downstream artifacts. | | +1 | javadoc | 47 | master passed | | 0 | spotbugs | 251 | Used deprecated FindBugs config; considering switching to SpotBugs. | | +1 | findbugs | 247 | master passed | ||| _ Patch Compile Tests _ | | +1 | mvninstall | 302 | the patch passed | | +1 | compile | 53 | the patch passed | | +1 | javac | 53 | the patch passed | | +1 | checkstyle | 76 | the patch passed | | +1 | whitespace | 0 | The patch has no whitespace issues. | | +1 | shadedjars | 273 | patch has no errors when building our shaded downstream artifacts. | | +1 | hadoopcheck | 927 | Patch does not cause any errors with Hadoop 2.8.5 2.9.2 or 3.1.2. | | +1 | javadoc | 32 | the patch passed | | +1 | findbugs | 242 | the patch passed | ||| _ Other Tests _ | | -1 | unit | 17093 | hbase-server in the patch failed. | | +1 | asflicense | 31 | The patch does not generate ASF License warnings. | | | | 20476 | | | Reason | Tests | |---:|:--| | Failed junit tests | hadoop.hbase.replication.TestReplicationKillSlaveRS | | | hadoop.hbase.replication.TestReplicationSmallTestsSync | | | hadoop.hbase.master.procedure.TestSCPWithReplicasWithoutZKCoordinated | | | hadoop.hbase.replication.TestReplicationSmallTests | | | hadoop.hbase.client.TestSnapshotDFSTemporaryDirectory | | Subsystem | Report/Notes | |--:|:-| | Docker | Client=19.03.0 Server=19.03.0 base: https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-404/13/artifact/out/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/404 | | Optional Tests | dupname asflicense javac javadoc unit spotbugs findbugs shadedjars hadoopcheck hbaseanti checkstyle compile | | uname | Linux e72ab023eb64 4.4.0-138-generic #164-Ubuntu SMP Tue Oct 2 17:16:02 UTC 2018 x86_64 GNU/Linux | | Build tool | maven | | Personality | /home/jenkins/jenkins-slave/workspace/HBase-PreCommit-GitHub-PR_PR-404/out/precommit/personality/provided.sh | | git revision | master / 747d000808 | | Default Java | 1.8.0_181 | | unit | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-404/13/artifact/out/patch-unit-hbase-server.txt | | Test Results | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-404/13/testReport/ | | Max. process+thread count | 5143 (vs. ulimit of 1) | | modules | C: hbase-server U: hbase-server | | Console output | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-404/13/console | | versions | git=2.11.0 maven=2018-06-17T18:33:14Z) findbugs=3.1.11 | | Powered by | Apache Yetus 0.10.0 http://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Updated] (HBASE-22115) HBase RPC aspires to grow an infinite tree of trace scopes; some other places are also unsafe
[ https://issues.apache.org/jira/browse/HBASE-22115?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack updated HBASE-22115: -- Fix Version/s: 2.1.6 2.0.6 Status: Patch Available (was: Reopened) Get a run in... for branch-2.1. > HBase RPC aspires to grow an infinite tree of trace scopes; some other places > are also unsafe > - > > Key: HBASE-22115 > URL: https://issues.apache.org/jira/browse/HBASE-22115 > Project: HBase > Issue Type: Bug >Reporter: Sergey Shelukhin >Assignee: Sergey Shelukhin >Priority: Critical > Fix For: 3.0.0, 2.3.0, 2.0.6, 2.1.6, 2.2.0 > > Attachments: Clipboard01.jpg, HBASE-22115.01.patch, > HBASE-22115.02.patch, HBASE-22115.branch-2.1.001.patch, HBASE-22115.patch > > > !Clipboard01.jpg! > All of those are ClientServices.Multi in this case. -- This message was sent by Atlassian JIRA (v7.6.14#76016)
[jira] [Updated] (HBASE-22115) HBase RPC aspires to grow an infinite tree of trace scopes; some other places are also unsafe
[ https://issues.apache.org/jira/browse/HBASE-22115?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack updated HBASE-22115: -- Attachment: HBASE-22115.branch-2.1.001.patch > HBase RPC aspires to grow an infinite tree of trace scopes; some other places > are also unsafe > - > > Key: HBASE-22115 > URL: https://issues.apache.org/jira/browse/HBASE-22115 > Project: HBase > Issue Type: Bug >Reporter: Sergey Shelukhin >Assignee: Sergey Shelukhin >Priority: Critical > Fix For: 3.0.0, 2.2.0, 2.3.0 > > Attachments: Clipboard01.jpg, HBASE-22115.01.patch, > HBASE-22115.02.patch, HBASE-22115.branch-2.1.001.patch, HBASE-22115.patch > > > !Clipboard01.jpg! > All of those are ClientServices.Multi in this case. -- This message was sent by Atlassian JIRA (v7.6.14#76016)
[jira] [Reopened] (HBASE-22115) HBase RPC aspires to grow an infinite tree of trace scopes; some other places are also unsafe
[ https://issues.apache.org/jira/browse/HBASE-22115?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack reopened HBASE-22115: --- Reopening to backport to branch-2.1, branch-2.0. > HBase RPC aspires to grow an infinite tree of trace scopes; some other places > are also unsafe > - > > Key: HBASE-22115 > URL: https://issues.apache.org/jira/browse/HBASE-22115 > Project: HBase > Issue Type: Bug >Reporter: Sergey Shelukhin >Assignee: Sergey Shelukhin >Priority: Critical > Fix For: 3.0.0, 2.2.0, 2.3.0 > > Attachments: Clipboard01.jpg, HBASE-22115.01.patch, > HBASE-22115.02.patch, HBASE-22115.patch > > > !Clipboard01.jpg! > All of those are ClientServices.Multi in this case. -- This message was sent by Atlassian JIRA (v7.6.14#76016)
[jira] [Commented] (HBASE-21773) rowcounter utility should respond to pleas for help
[
https://issues.apache.org/jira/browse/HBASE-21773?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16893205#comment-16893205
]
stack commented on HBASE-21773:
---
Patch LGTM [~wchevreuil] Perhaps dump in here what usage looks like now? +1 on
commit.
> rowcounter utility should respond to pleas for help
> ---
>
> Key: HBASE-21773
> URL: https://issues.apache.org/jira/browse/HBASE-21773
> Project: HBase
> Issue Type: Bug
> Components: tooling
>Affects Versions: 2.1.0
>Reporter: Sean Busbey
>Assignee: Wellington Chevreuil
>Priority: Critical
> Attachments: HBASE-21773.master.001.patch,
> HBASE-21773.master.002.patch, HBASE-21773.master.003.patch,
> HBASE-21773.master.004.patch
>
>
> {{hbase rowcounter}} does not respond to reasonable requests for help, i.e.
> {{--help}}, {{-h}}, or {{-?}}
> {code}
> [systest@busbey-training-1 root]$ hbase rowcounter -?
> OpenJDK 64-Bit Server VM warning: Using incremental CMS is deprecated and
> will likely be removed in a future release
> 19/01/24 12:30:00 INFO client.RMProxy: Connecting to ResourceManager at
> busbey-training-1.gce.cloudera.com/172.31.116.31:8032
> 19/01/24 12:30:01 INFO hdfs.DFSClient: Created token for systest:
> HDFS_DELEGATION_TOKEN owner=syst...@gce.cloudera.com, renewer=yarn,
> realUser=, issueDate=1548361801519, maxDate=1548966601519, sequenceNumber=3,
> masterKeyId=8 on 172.31.116.31:8020
> 19/01/24 12:30:01 INFO kms.KMSClientProvider: Getting new token from
> https://busbey-training-3.gce.cloudera.com:16000/kms/v1/,
> renewer:yarn/busbey-training-1.gce.cloudera@gce.cloudera.com
> 19/01/24 12:30:02 INFO kms.KMSClientProvider: New token received: (Kind:
> kms-dt, Service: 172.31.116.52:16000, Ident: (kms-dt owner=systest,
> renewer=yarn, realUser=, issueDate=1548361801965, maxDate=1548966601965,
> sequenceNumber=5, masterKeyId=17))
> 19/01/24 12:30:02 INFO security.TokenCache: Got dt for
> hdfs://busbey-training-1.gce.cloudera.com:8020; Kind: HDFS_DELEGATION_TOKEN,
> Service: 172.31.116.31:8020, Ident: (token for systest: HDFS_DELEGATION_TOKEN
> owner=syst...@gce.cloudera.com, renewer=yarn, realUser=,
> issueDate=1548361801519, maxDate=1548966601519, sequenceNumber=3,
> masterKeyId=8)
> 19/01/24 12:30:02 INFO security.TokenCache: Got dt for
> hdfs://busbey-training-1.gce.cloudera.com:8020; Kind: kms-dt, Service:
> 172.31.116.52:16000, Ident: (kms-dt owner=systest, renewer=yarn, realUser=,
> issueDate=1548361801965, maxDate=1548966601965, sequenceNumber=5,
> masterKeyId=17)
> 19/01/24 12:30:02 INFO kms.KMSClientProvider: Getting new token from
> https://busbey-training-4.gce.cloudera.com:16000/kms/v1/,
> renewer:yarn/busbey-training-1.gce.cloudera@gce.cloudera.com
> 19/01/24 12:30:02 INFO kms.KMSClientProvider: New token received: (Kind:
> kms-dt, Service: 172.31.116.50:16000, Ident: (kms-dt owner=systest,
> renewer=yarn, realUser=, issueDate=1548361802363, maxDate=1548966602363,
> sequenceNumber=6, masterKeyId=18))
> 19/01/24 12:30:02 INFO security.TokenCache: Got dt for
> hdfs://busbey-training-1.gce.cloudera.com:8020; Kind: kms-dt, Service:
> 172.31.116.50:16000, Ident: (kms-dt owner=systest, renewer=yarn, realUser=,
> issueDate=1548361802363, maxDate=1548966602363, sequenceNumber=6,
> masterKeyId=18)
> 19/01/24 12:30:02 INFO mapreduce.JobResourceUploader: Disabling Erasure
> Coding for path: /user/systest/.staging/job_1548349234632_0003
> 19/01/24 12:30:03 INFO mapreduce.JobSubmitter: Cleaning up the staging area
> /user/systest/.staging/job_1548349234632_0003
> Exception in thread "main" java.lang.IllegalArgumentException: Illegal first
> character <45> at 0. User-space table qualifiers can only start with
> 'alphanumeric characters' from any language: -?
> at
> org.apache.hadoop.hbase.TableName.isLegalTableQualifierName(TableName.java:193)
> at
> org.apache.hadoop.hbase.TableName.isLegalTableQualifierName(TableName.java:156)
> at org.apache.hadoop.hbase.TableName.(TableName.java:346)
> at
> org.apache.hadoop.hbase.TableName.createTableNameIfNecessary(TableName.java:382)
> at org.apache.hadoop.hbase.TableName.valueOf(TableName.java:469)
> at
> org.apache.hadoop.hbase.mapreduce.TableInputFormat.initialize(TableInputFormat.java:198)
> at
> org.apache.hadoop.hbase.mapreduce.TableInputFormatBase.getSplits(TableInputFormatBase.java:243)
> at
> org.apache.hadoop.hbase.mapreduce.TableInputFormat.getSplits(TableInputFormat.java:254)
> at
> org.apache.hadoop.mapreduce.JobSubmitter.writeNewSplits(JobSubmitter.java:310)
> at
> org.apache.hadoop.mapreduce.JobSubmitter.writeSplits(JobSubmitter.java:327)
> at
> org.apache.hadoop.mapreduce.JobSubmitter.submitJobInternal(JobSubmitter.java:200)
> at
[jira] [Updated] (HBASE-22708) Remove the deprecated methods in Hbck interface
[ https://issues.apache.org/jira/browse/HBASE-22708?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack updated HBASE-22708: -- Issue Type: Bug (was: Sub-task) Parent: (was: HBASE-21745) > Remove the deprecated methods in Hbck interface > --- > > Key: HBASE-22708 > URL: https://issues.apache.org/jira/browse/HBASE-22708 > Project: HBase > Issue Type: Bug >Affects Versions: 3.0.0 >Reporter: Guanghao Zhang >Assignee: Guanghao Zhang >Priority: Major > > Only for 3.0.0. Remove the methods which mark deprecated in HBASE-22673. -- This message was sent by Atlassian JIRA (v7.6.14#76016)
[jira] [Updated] (HBASE-22708) Remove the deprecated methods in Hbck interface
[ https://issues.apache.org/jira/browse/HBASE-22708?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack updated HBASE-22708: -- Fix Version/s: 3.0.0 > Remove the deprecated methods in Hbck interface > --- > > Key: HBASE-22708 > URL: https://issues.apache.org/jira/browse/HBASE-22708 > Project: HBase > Issue Type: Bug >Affects Versions: 3.0.0 >Reporter: Guanghao Zhang >Assignee: Guanghao Zhang >Priority: Major > Fix For: 3.0.0 > > > Only for 3.0.0. Remove the methods which mark deprecated in HBASE-22673. -- This message was sent by Atlassian JIRA (v7.6.14#76016)
[jira] [Commented] (HBASE-21773) rowcounter utility should respond to pleas for help
[
https://issues.apache.org/jira/browse/HBASE-21773?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16893194#comment-16893194
]
HBase QA commented on HBASE-21773:
--
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
54s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s{color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 1 new or modified test
files. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
38s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 6m
41s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
56s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
53s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 5m
57s{color} | {color:green} branch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m
43s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
43s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
20s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 6m
25s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
56s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
56s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m
24s{color} | {color:red} hbase-mapreduce: The patch generated 1 new + 43
unchanged - 1 fixed = 44 total (was 44) {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 6m
4s{color} | {color:green} patch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
20m 5s{color} | {color:green} Patch does not cause any errors with Hadoop
2.8.5 2.9.2 or 3.1.2. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m
59s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
41s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 3m
24s{color} | {color:green} hbase-common in the patch passed. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 22m
38s{color} | {color:green} hbase-mapreduce in the patch passed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
28s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 90m 9s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=19.03.0 Server=19.03.0 base:
https://builds.apache.org/job/PreCommit-HBASE-Build/671/artifact/patchprocess/Dockerfile
|
| JIRA Issue | HBASE-21773 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12957012/HBASE-21773.master.004.patch
|
| Optional Tests | dupname asflicense javac javadoc unit findbugs
shadedjars hadoopcheck hbaseanti checkstyle compile |
| uname | Linux 06239a792cd1 4.4.0-143-generic #169-Ubuntu SMP Thu Feb 7
07:56:38 UTC 2019 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/hbase-personality.sh |
| git revision | master / 747d000808 |
| maven | version: Apache Maven 3.5.4
[jira] [Resolved] (HBASE-21966) Fix region holes, overlaps, and other region related errors
[ https://issues.apache.org/jira/browse/HBASE-21966?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack resolved HBASE-21966. --- Resolution: Incomplete Resolving as 'incomplete'. Subject only and it is open-ended. Ported the Busbey comment to parent issue where it sits better. > Fix region holes, overlaps, and other region related errors > --- > > Key: HBASE-21966 > URL: https://issues.apache.org/jira/browse/HBASE-21966 > Project: HBase > Issue Type: Sub-task >Reporter: Jingyun Tian >Assignee: Jingyun Tian >Priority: Major > -- This message was sent by Atlassian JIRA (v7.6.14#76016)
[jira] [Commented] (HBASE-21745) Make HBCK2 be able to fix issues other than region assignment
[
https://issues.apache.org/jira/browse/HBASE-21745?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16893192#comment-16893192
]
stack commented on HBASE-21745:
---
Porting comments by [~busbey] from HBASE-21966 here because they are general
and will sit better here than in the issue they were pulled from.
Some notes from a discussion [~elserj] and I ended up having after a customer
escalation:
{quote}We liked the idea
of making very focused operations, rather than sweeping "-repair"
options, e.g. "create a row in hbase:meta for this regiondir", "create
an empty regiondir", "create a regiondir from this hbase:meta row".
With tools like this, you could imagine a CLI tool stepping through
the Regions of a table presenting a three-way merge like tool: for
every region in a table, ask the user if they want to keep the state
in meta or the state in HDFS. We like this idea because it keeps HBase
devs out of the role of figuring out what the correct thing to do is.
A couple of related concerns:
Such a tool would not aim to be sufficient for the bar some folks have
expressed for HBck2 to move the "stable" pointer to HBase 2. Essentially some
ability to rebuild meta from external info ala HBASE-21665 or HBASE-18840
Need to make sure these tools are compose-able. It's all
well-and-good to say that human-insight needs to be applied to know
what HBCK2 commands to run, but we also don't want cluster recoveries
to take 10's of hours. Something easily scriptable is important. To continue
the three-way-merge analogy, merge strategies ala ours/theirs in git.{quote}
> Make HBCK2 be able to fix issues other than region assignment
> -
>
> Key: HBASE-21745
> URL: https://issues.apache.org/jira/browse/HBASE-21745
> Project: HBase
> Issue Type: Umbrella
> Components: hbase-operator-tools, hbck2
>Reporter: Duo Zhang
>Assignee: stack
>Priority: Critical
>
> This is what [~apurtell] posted on mailing-list, HBCK2 should support
> * -Rebuild meta from region metadata in the filesystem, aka offline meta
> rebuild.-
> * -Fix assignment errors (undeployed regions, double assignments (yes,
> should not be possible), etc)- (See
> https://issues.apache.org/jira/browse/HBASE-21745?focusedCommentId=16888302=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-16888302)
> * Fix region holes, overlaps, and other errors in the region chain
> * Fix failed split and merge transactions that have failed to roll back due
> to some bug (related to previous)
> * -Enumerate store files to determine file level corruption and sideline
> corrupt files-
> * -Fix hfile link problems (dangling / broken)-
--
This message was sent by Atlassian JIRA
(v7.6.14#76016)
[jira] [Commented] (HBASE-21745) Make HBCK2 be able to fix issues other than region assignment
[ https://issues.apache.org/jira/browse/HBASE-21745?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16893176#comment-16893176 ] stack commented on HBASE-21745: --- WIP doc on hbck2/hbck1 messings around this issue. > Make HBCK2 be able to fix issues other than region assignment > - > > Key: HBASE-21745 > URL: https://issues.apache.org/jira/browse/HBASE-21745 > Project: HBase > Issue Type: Umbrella > Components: hbase-operator-tools, hbck2 >Reporter: Duo Zhang >Assignee: stack >Priority: Critical > > This is what [~apurtell] posted on mailing-list, HBCK2 should support > * -Rebuild meta from region metadata in the filesystem, aka offline meta > rebuild.- > * -Fix assignment errors (undeployed regions, double assignments (yes, > should not be possible), etc)- (See > https://issues.apache.org/jira/browse/HBASE-21745?focusedCommentId=16888302=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-16888302) > * Fix region holes, overlaps, and other errors in the region chain > * Fix failed split and merge transactions that have failed to roll back due > to some bug (related to previous) > * -Enumerate store files to determine file level corruption and sideline > corrupt files- > * -Fix hfile link problems (dangling / broken)- -- This message was sent by Atlassian JIRA (v7.6.14#76016)
[jira] [Commented] (HBASE-19231) Redo TestMetaWithReplicas#testChangingReplicaCount; uses HBCK to validate
[ https://issues.apache.org/jira/browse/HBASE-19231?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16893171#comment-16893171 ] stack commented on HBASE-19231: --- bq. What's still actionable here, if anything? Reenable commented out test. bq. I'd like to move this out of being a subtask... Done > Redo TestMetaWithReplicas#testChangingReplicaCount; uses HBCK to validate > - > > Key: HBASE-19231 > URL: https://issues.apache.org/jira/browse/HBASE-19231 > Project: HBase > Issue Type: Bug > Components: hbck >Reporter: stack >Assignee: stack >Priority: Major > > Also #testHBaseFsckWithFewerMetaReplicas, > #testHBaseFsckWithExcessMetaReplicas, and > #testHBaseFsckWithFewerMetaReplicaZnodes in TestMetaWithReplicas. Both depend > on HBCK for validation. Disabled for now. -- This message was sent by Atlassian JIRA (v7.6.14#76016)
[jira] [Commented] (HBASE-22115) HBase RPC aspires to grow an infinite tree of trace scopes; some other places are also unsafe
[
https://issues.apache.org/jira/browse/HBASE-22115?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16893173#comment-16893173
]
Hudson commented on HBASE-22115:
Results for branch branch-1.4
[build #925 on
builds.a.o|https://builds.apache.org/job/HBase%20Nightly/job/branch-1.4/925/]:
(x) *{color:red}-1 overall{color}*
details (if available):
(x) {color:red}-1 general checks{color}
-- For more information [see general
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-1.4/925//General_Nightly_Build_Report/]
(x) {color:red}-1 jdk7 checks{color}
-- Something went wrong running this stage, please [check relevant console
output|https://builds.apache.org/job/HBase%20Nightly/job/branch-1.4/925//console].
(x) {color:red}-1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-1.4/925//JDK8_Nightly_Build_Report_(Hadoop2)/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
> HBase RPC aspires to grow an infinite tree of trace scopes; some other places
> are also unsafe
> -
>
> Key: HBASE-22115
> URL: https://issues.apache.org/jira/browse/HBASE-22115
> Project: HBase
> Issue Type: Bug
>Reporter: Sergey Shelukhin
>Assignee: Sergey Shelukhin
>Priority: Critical
> Fix For: 3.0.0, 2.2.0, 2.3.0
>
> Attachments: Clipboard01.jpg, HBASE-22115.01.patch,
> HBASE-22115.02.patch, HBASE-22115.patch
>
>
> !Clipboard01.jpg!
> All of those are ClientServices.Multi in this case.
--
This message was sent by Atlassian JIRA
(v7.6.14#76016)
[jira] [Commented] (HBASE-22132) Backport HBASE-22115 intent to branch-1
[
https://issues.apache.org/jira/browse/HBASE-22132?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16893172#comment-16893172
]
Hudson commented on HBASE-22132:
Results for branch branch-1.4
[build #925 on
builds.a.o|https://builds.apache.org/job/HBase%20Nightly/job/branch-1.4/925/]:
(x) *{color:red}-1 overall{color}*
details (if available):
(x) {color:red}-1 general checks{color}
-- For more information [see general
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-1.4/925//General_Nightly_Build_Report/]
(x) {color:red}-1 jdk7 checks{color}
-- Something went wrong running this stage, please [check relevant console
output|https://builds.apache.org/job/HBase%20Nightly/job/branch-1.4/925//console].
(x) {color:red}-1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-1.4/925//JDK8_Nightly_Build_Report_(Hadoop2)/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
> Backport HBASE-22115 intent to branch-1
> ---
>
> Key: HBASE-22132
> URL: https://issues.apache.org/jira/browse/HBASE-22132
> Project: HBase
> Issue Type: Sub-task
>Affects Versions: 1.5.0
>Reporter: Andrew Purtell
>Assignee: Viraj Jasani
>Priority: Major
> Fix For: 1.5.0, 1.3.6, 1.4.11
>
> Attachments: HBASE-22132-branch-1.patch
>
>
> Check the exposure of branch-1 code to the problems described on HBASE-22115
> and apply the fix approach there.
--
This message was sent by Atlassian JIRA
(v7.6.14#76016)
[jira] [Commented] (HBASE-21745) Make HBCK2 be able to fix issues other than region assignment
[ https://issues.apache.org/jira/browse/HBASE-21745?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16893168#comment-16893168 ] stack commented on HBASE-21745: --- The original hbck2 issue. > Make HBCK2 be able to fix issues other than region assignment > - > > Key: HBASE-21745 > URL: https://issues.apache.org/jira/browse/HBASE-21745 > Project: HBase > Issue Type: Umbrella > Components: hbase-operator-tools, hbck2 >Reporter: Duo Zhang >Assignee: stack >Priority: Critical > > This is what [~apurtell] posted on mailing-list, HBCK2 should support > * -Rebuild meta from region metadata in the filesystem, aka offline meta > rebuild.- > * -Fix assignment errors (undeployed regions, double assignments (yes, > should not be possible), etc)- (See > https://issues.apache.org/jira/browse/HBASE-21745?focusedCommentId=16888302=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-16888302) > * Fix region holes, overlaps, and other errors in the region chain > * Fix failed split and merge transactions that have failed to roll back due > to some bug (related to previous) > * -Enumerate store files to determine file level corruption and sideline > corrupt files- > * -Fix hfile link problems (dangling / broken)- -- This message was sent by Atlassian JIRA (v7.6.14#76016)
[jira] [Commented] (HBASE-21073) "Maintenance mode" master
[ https://issues.apache.org/jira/browse/HBASE-21073?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16893167#comment-16893167 ] stack commented on HBASE-21073: --- Some notes after trying this feature: * UI on Master doesn't show. Does NPE. Would be good if it came up so had access to stuff like Procedures and Locks. * Logs show user-space regions as STUCK. Probably just needs doc'ing. * These logs are good 'HMaster: Master is in maintenanceMode mode, don't run balancer.' and 'HMaster: Master is in maintenance mode, don't run region normalizer.' but says this after Master comes up... 'HMaster: Detected repair mode, skipping final initialization steps.'. Should match the other logs of 'maintenance mode'. * Here is how I started Master only with the maintenance mode flag: ./bin/hbase-daemon.sh start master -Dhbase.master.maintenance_mode=true Could add to doc. Not sure if this feature is useful yet. Dump note in here if find it of use. > "Maintenance mode" master > - > > Key: HBASE-21073 > URL: https://issues.apache.org/jira/browse/HBASE-21073 > Project: HBase > Issue Type: Sub-task > Components: amv2, hbck2, master >Reporter: stack >Assignee: Mike Drob >Priority: Major > Fix For: 3.0.0, 2.2.0, 2.1.1, 2.0.3 > > Attachments: HBASE-21073.branch-2.001.patch, > HBASE-21073.branch-2.1.001.patch, HBASE-21073.branch-2.1.002.patch, > HBASE-21073.master.001.patch, HBASE-21073.master.002.patch, > HBASE-21073.master.003.patch, HBASE-21073.master.004.patch, > HBASE-21073.master.005.patch, HBASE-21073.master.006.patch, > HBASE-21073.master.007.patch, HBASE-21073.master.008.patch, > HBASE-21073.master.009.patch, HBASE-21073.master.010.patch, > HBASE-21073.master.011.patch > > > Make it so we can bring up a Master in "maintenance mode". This is parse of > master wal procs but not taking on regionservers. It would be in a state > where "repair" Procedures could run; e.g. a Procedure that could recover meta > by looking for meta WALs, splitting them, dropping recovered.edits, and even > making it so meta is readable. See parent issue for why needed (disaster > recovery). -- This message was sent by Atlassian JIRA (v7.6.14#76016)
[jira] [Commented] (HBASE-21745) Make HBCK2 be able to fix issues other than region assignment
[ https://issues.apache.org/jira/browse/HBASE-21745?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16893163#comment-16893163 ] stack commented on HBASE-21745: --- HBASE-21447 is about adding hole-fixing to hbck2. Related. > Make HBCK2 be able to fix issues other than region assignment > - > > Key: HBASE-21745 > URL: https://issues.apache.org/jira/browse/HBASE-21745 > Project: HBase > Issue Type: Umbrella > Components: hbase-operator-tools, hbck2 >Reporter: Duo Zhang >Assignee: stack >Priority: Critical > > This is what [~apurtell] posted on mailing-list, HBCK2 should support > * -Rebuild meta from region metadata in the filesystem, aka offline meta > rebuild.- > * -Fix assignment errors (undeployed regions, double assignments (yes, > should not be possible), etc)- (See > https://issues.apache.org/jira/browse/HBASE-21745?focusedCommentId=16888302=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-16888302) > * Fix region holes, overlaps, and other errors in the region chain > * Fix failed split and merge transactions that have failed to roll back due > to some bug (related to previous) > * -Enumerate store files to determine file level corruption and sideline > corrupt files- > * -Fix hfile link problems (dangling / broken)- -- This message was sent by Atlassian JIRA (v7.6.14#76016)
[jira] [Updated] (HBASE-19121) HBCK for AMv2 (A.K.A HBCK2)
[ https://issues.apache.org/jira/browse/HBASE-19121?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack updated HBASE-19121: -- Resolution: Fixed Fix Version/s: (was: hbck2-1.0.0) hbase-operator-tools-1.0.0 Status: Resolved (was: Patch Available) Resolving. All subtasks done. hbck2 is in an adjacent repo, hbase-operator-tools. > HBCK for AMv2 (A.K.A HBCK2) > --- > > Key: HBASE-19121 > URL: https://issues.apache.org/jira/browse/HBASE-19121 > Project: HBase > Issue Type: Umbrella > Components: hbck, hbck2 >Reporter: stack >Assignee: Umesh Agashe >Priority: Major > Fix For: hbase-operator-tools-1.0.0 > > Attachments: hbase-19121.master.001.patch > > > We don't have an hbck for the new AM. Old hbck may actually do damage going > against AMv2. > Fix. -- This message was sent by Atlassian JIRA (v7.6.14#76016)
[jira] [Updated] (HBASE-19231) Redo TestMetaWithReplicas#testChangingReplicaCount; uses HBCK to validate
[ https://issues.apache.org/jira/browse/HBASE-19231?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack updated HBASE-19231: -- Issue Type: Bug (was: Sub-task) Parent: (was: HBASE-19121) > Redo TestMetaWithReplicas#testChangingReplicaCount; uses HBCK to validate > - > > Key: HBASE-19231 > URL: https://issues.apache.org/jira/browse/HBASE-19231 > Project: HBase > Issue Type: Bug > Components: hbck >Reporter: stack >Assignee: stack >Priority: Major > > Also #testHBaseFsckWithFewerMetaReplicas, > #testHBaseFsckWithExcessMetaReplicas, and > #testHBaseFsckWithFewerMetaReplicaZnodes in TestMetaWithReplicas. Both depend > on HBCK for validation. Disabled for now. -- This message was sent by Atlassian JIRA (v7.6.14#76016)
[GitHub] [hbase] Apache-HBase commented on issue #199: HBASE-22313 Add method to FsDelegation token to accept token kind
Apache-HBase commented on issue #199: HBASE-22313 Add method to FsDelegation token to accept token kind URL: https://github.com/apache/hbase/pull/199#issuecomment-515229000 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | 0 | reexec | 84 | Docker mode activated. | ||| _ Prechecks _ | | +1 | dupname | 0 | No case conflicting files found. | | +1 | hbaseanti | 0 | Patch does not have any anti-patterns. | | +1 | @author | 0 | The patch does not contain any @author tags. | | +1 | test4tests | 0 | The patch appears to include 1 new or modified test files. | ||| _ master Compile Tests _ | | +1 | mvninstall | 405 | master passed | | +1 | compile | 67 | master passed | | +1 | checkstyle | 93 | master passed | | +1 | shadedjars | 344 | branch has no errors when building our shaded downstream artifacts. | | +1 | javadoc | 41 | master passed | | 0 | spotbugs | 283 | Used deprecated FindBugs config; considering switching to SpotBugs. | | +1 | findbugs | 281 | master passed | ||| _ Patch Compile Tests _ | | +1 | mvninstall | 388 | the patch passed | | +1 | compile | 69 | the patch passed | | +1 | javac | 69 | the patch passed | | +1 | checkstyle | 95 | hbase-server: The patch generated 0 new + 0 unchanged - 1 fixed = 0 total (was 1) | | +1 | whitespace | 0 | The patch has no whitespace issues. | | +1 | shadedjars | 339 | patch has no errors when building our shaded downstream artifacts. | | +1 | hadoopcheck | 1202 | Patch does not cause any errors with Hadoop 2.8.5 2.9.2 or 3.1.2. | | +1 | javadoc | 43 | the patch passed | | +1 | findbugs | 300 | the patch passed | ||| _ Other Tests _ | | -1 | unit | 19234 | hbase-server in the patch failed. | | +1 | asflicense | 86 | The patch does not generate ASF License warnings. | | | | 23563 | | | Reason | Tests | |---:|:--| | Failed junit tests | hadoop.hbase.replication.multiwal.TestReplicationEndpointWithMultipleAsyncWAL | | | hadoop.hbase.client.TestSnapshotTemporaryDirectory | | | hadoop.hbase.replication.TestReplicationSmallTestsSync | | | hadoop.hbase.io.encoding.TestChangingEncoding | | | hadoop.hbase.io.encoding.TestEncodedSeekers | | | hadoop.hbase.replication.regionserver.TestRegionReplicaReplicationEndpoint | | | hadoop.hbase.client.TestAsyncTableAdminApi | | | hadoop.hbase.client.TestConnection | | | hadoop.hbase.regionserver.wal.TestAsyncWALReplayCompressed | | | hadoop.hbase.regionserver.wal.TestAsyncWALReplay | | | hadoop.hbase.util.TestConnectionCache | | | hadoop.hbase.regionserver.TestRowTooBig | | | hadoop.hbase.replication.TestSerialSyncReplication | | | hadoop.hbase.replication.TestReplicationMetricsforUI | | | hadoop.hbase.client.TestCloneSnapshotFromClientNormal | | | hadoop.hbase.regionserver.TestRegionReplicasWithRestartScenarios | | | hadoop.hbase.replication.TestReplicationKillMasterRSCompressed | | | hadoop.hbase.replication.TestAddToSerialReplicationPeer | | | hadoop.hbase.regionserver.wal.TestLogRollPeriod | | | hadoop.hbase.replication.TestSyncReplicationStandbyKillRS | | | hadoop.hbase.replication.TestSyncReplicationActive | | | hadoop.hbase.util.compaction.TestMajorCompactorTTL | | | hadoop.hbase.TestJMXConnectorServer | | | hadoop.hbase.replication.TestSerialReplication | | | hadoop.hbase.regionserver.TestFSErrorsExposed | | | hadoop.hbase.replication.regionserver.TestReplicationSourceManagerZkImpl | | | hadoop.hbase.TestClusterPortAssignment | | | hadoop.hbase.replication.regionserver.TestSerialReplicationEndpoint | | | hadoop.hbase.replication.multiwal.TestReplicationSyncUpToolWithMultipleAsyncWAL | | | hadoop.hbase.replication.TestMasterReplication | | | hadoop.hbase.regionserver.wal.TestAsyncLogRollPeriod | | | hadoop.hbase.regionserver.TestMutateRowsRecovery | | | hadoop.hbase.replication.regionserver.TestReplicationSource | | | hadoop.hbase.replication.TestReplicationKillMasterRSWithSeparateOldWALs | | | hadoop.hbase.replication.regionserver.TestRaceWhenCreatingReplicationSource | | | hadoop.hbase.replication.TestSyncReplicationRemoveRemoteWAL | | | hadoop.hbase.replication.multiwal.TestReplicationSyncUpToolWithMultipleWAL | | | hadoop.hbase.regionserver.TestCompactingMemStore | | | hadoop.hbase.replication.regionserver.TestReplicator | | | hadoop.hbase.replication.TestReplicationStatusAfterLagging | | | hadoop.hbase.replication.TestSyncReplicationMoreLogsInLocalGiveUpSplitting | | | hadoop.hbase.regionserver.TestHRegionServerBulkLoad | | | hadoop.hbase.replication.regionserver.TestSerialReplicationChecker | | | hadoop.hbase.TestMultiVersions | | |
[jira] [Commented] (HBASE-22702) [Log] 'Group not found for table' is chatty
[
https://issues.apache.org/jira/browse/HBASE-22702?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16893151#comment-16893151
]
Hudson commented on HBASE-22702:
Results for branch branch-2.2
[build #464 on
builds.a.o|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.2/464/]:
(/) *{color:green}+1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.2/464//General_Nightly_Build_Report/]
(/) {color:green}+1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.2/464//JDK8_Nightly_Build_Report_(Hadoop2)/]
(/) {color:green}+1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.2/464//JDK8_Nightly_Build_Report_(Hadoop3)/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
(/) {color:green}+1 client integration test{color}
> [Log] 'Group not found for table' is chatty
> ---
>
> Key: HBASE-22702
> URL: https://issues.apache.org/jira/browse/HBASE-22702
> Project: HBase
> Issue Type: Improvement
>Affects Versions: 1.4.6
>Reporter: Reid Chan
>Assignee: Murtaza Hassan
>Priority: Trivial
> Labels: beginner
> Fix For: 3.0.0, 1.5.0, 2.0.6, 2.2.1, 2.1.6, 1.4.11
>
>
> When rsgroup feature is enabled, there are few places when
> 'RSGroupBasedLoadBalancer' can't find group of a table and print a lot of
> {{Group not found for table ***, using default}} which is chatty and useless
> in master log.
> Plan to make it DEBUG level or remove it, since we can always know what
> tables a rsgroup contains by using hbase shell.
--
This message was sent by Atlassian JIRA
(v7.6.14#76016)
[jira] [Commented] (HBASE-22702) [Log] 'Group not found for table' is chatty
[
https://issues.apache.org/jira/browse/HBASE-22702?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16893147#comment-16893147
]
Hudson commented on HBASE-22702:
Results for branch branch-2.0
[build #1792 on
builds.a.o|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.0/1792/]:
(/) *{color:green}+1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.0/1792//General_Nightly_Build_Report/]
(/) {color:green}+1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.0/1792//JDK8_Nightly_Build_Report_(Hadoop2)/]
(/) {color:green}+1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.0/1792//JDK8_Nightly_Build_Report_(Hadoop3)/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
> [Log] 'Group not found for table' is chatty
> ---
>
> Key: HBASE-22702
> URL: https://issues.apache.org/jira/browse/HBASE-22702
> Project: HBase
> Issue Type: Improvement
>Affects Versions: 1.4.6
>Reporter: Reid Chan
>Assignee: Murtaza Hassan
>Priority: Trivial
> Labels: beginner
> Fix For: 3.0.0, 1.5.0, 2.0.6, 2.2.1, 2.1.6, 1.4.11
>
>
> When rsgroup feature is enabled, there are few places when
> 'RSGroupBasedLoadBalancer' can't find group of a table and print a lot of
> {{Group not found for table ***, using default}} which is chatty and useless
> in master log.
> Plan to make it DEBUG level or remove it, since we can always know what
> tables a rsgroup contains by using hbase shell.
--
This message was sent by Atlassian JIRA
(v7.6.14#76016)
[jira] [Commented] (HBASE-22723) Have CatalogJanitor report holes and overlaps; i.e. problems it sees when doing its regular scan of hbase:meta
[ https://issues.apache.org/jira/browse/HBASE-22723?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16893143#comment-16893143 ] stack commented on HBASE-22723: --- .003 fix tests and checkstyle. > Have CatalogJanitor report holes and overlaps; i.e. problems it sees when > doing its regular scan of hbase:meta > -- > > Key: HBASE-22723 > URL: https://issues.apache.org/jira/browse/HBASE-22723 > Project: HBase > Issue Type: Sub-task >Reporter: stack >Assignee: stack >Priority: Major > Attachments: HBASE-22723.branch-2.001.patch, > HBASE-22723.branch-2.002.patch, HBASE-22723.branch-2.003.patch > > > CatalogJanitor scans hbase:meta on a configurable period. Currently it just > looks for splits and merges to gc. Might as well generate a report on stuff > like holds, overlaps, empty regioninfos, and unknown servers while it is at > it. -- This message was sent by Atlassian JIRA (v7.6.14#76016)
[jira] [Updated] (HBASE-22723) Have CatalogJanitor report holes and overlaps; i.e. problems it sees when doing its regular scan of hbase:meta
[ https://issues.apache.org/jira/browse/HBASE-22723?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack updated HBASE-22723: -- Attachment: HBASE-22723.branch-2.003.patch > Have CatalogJanitor report holes and overlaps; i.e. problems it sees when > doing its regular scan of hbase:meta > -- > > Key: HBASE-22723 > URL: https://issues.apache.org/jira/browse/HBASE-22723 > Project: HBase > Issue Type: Sub-task >Reporter: stack >Assignee: stack >Priority: Major > Attachments: HBASE-22723.branch-2.001.patch, > HBASE-22723.branch-2.002.patch, HBASE-22723.branch-2.003.patch > > > CatalogJanitor scans hbase:meta on a configurable period. Currently it just > looks for splits and merges to gc. Might as well generate a report on stuff > like holds, overlaps, empty regioninfos, and unknown servers while it is at > it. -- This message was sent by Atlassian JIRA (v7.6.14#76016)
[jira] [Commented] (HBASE-22594) Clean up for backup examples
[
https://issues.apache.org/jira/browse/HBASE-22594?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16893141#comment-16893141
]
Hudson commented on HBASE-22594:
Results for branch branch-2
[build #2113 on
builds.a.o|https://builds.apache.org/job/HBase%20Nightly/job/branch-2/2113/]:
(x) *{color:red}-1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2/2113//General_Nightly_Build_Report/]
(x) {color:red}-1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2/2113//JDK8_Nightly_Build_Report_(Hadoop2)/]
(/) {color:green}+1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2/2113//JDK8_Nightly_Build_Report_(Hadoop3)/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
(/) {color:green}+1 client integration test{color}
> Clean up for backup examples
>
>
> Key: HBASE-22594
> URL: https://issues.apache.org/jira/browse/HBASE-22594
> Project: HBase
> Issue Type: Improvement
>Reporter: Jan Hentschel
>Assignee: Jan Hentschel
>Priority: Minor
> Fix For: 3.0.0, 2.3.0
>
>
> The [PR|https://github.com/apache/hbase/pull/145] for HBASE-7003 included
> some clean ups. They should be brought in without moving the files.
--
This message was sent by Atlassian JIRA
(v7.6.14#76016)
[jira] [Commented] (HBASE-22702) [Log] 'Group not found for table' is chatty
[
https://issues.apache.org/jira/browse/HBASE-22702?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16893142#comment-16893142
]
Hudson commented on HBASE-22702:
Results for branch branch-2
[build #2113 on
builds.a.o|https://builds.apache.org/job/HBase%20Nightly/job/branch-2/2113/]:
(x) *{color:red}-1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2/2113//General_Nightly_Build_Report/]
(x) {color:red}-1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2/2113//JDK8_Nightly_Build_Report_(Hadoop2)/]
(/) {color:green}+1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2/2113//JDK8_Nightly_Build_Report_(Hadoop3)/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
(/) {color:green}+1 client integration test{color}
> [Log] 'Group not found for table' is chatty
> ---
>
> Key: HBASE-22702
> URL: https://issues.apache.org/jira/browse/HBASE-22702
> Project: HBase
> Issue Type: Improvement
>Affects Versions: 1.4.6
>Reporter: Reid Chan
>Assignee: Murtaza Hassan
>Priority: Trivial
> Labels: beginner
> Fix For: 3.0.0, 1.5.0, 2.0.6, 2.2.1, 2.1.6, 1.4.11
>
>
> When rsgroup feature is enabled, there are few places when
> 'RSGroupBasedLoadBalancer' can't find group of a table and print a lot of
> {{Group not found for table ***, using default}} which is chatty and useless
> in master log.
> Plan to make it DEBUG level or remove it, since we can always know what
> tables a rsgroup contains by using hbase shell.
--
This message was sent by Atlassian JIRA
(v7.6.14#76016)
[jira] [Commented] (HBASE-22132) Backport HBASE-22115 intent to branch-1
[
https://issues.apache.org/jira/browse/HBASE-22132?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16893139#comment-16893139
]
Hudson commented on HBASE-22132:
Results for branch branch-1.3
[build #889 on
builds.a.o|https://builds.apache.org/job/HBase%20Nightly/job/branch-1.3/889/]:
(x) *{color:red}-1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-1.3/889//General_Nightly_Build_Report/]
(x) {color:red}-1 jdk7 checks{color}
-- For more information [see jdk7
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-1.3/889//JDK7_Nightly_Build_Report/]
(/) {color:green}+1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-1.3/889//JDK8_Nightly_Build_Report_(Hadoop2)/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
> Backport HBASE-22115 intent to branch-1
> ---
>
> Key: HBASE-22132
> URL: https://issues.apache.org/jira/browse/HBASE-22132
> Project: HBase
> Issue Type: Sub-task
>Affects Versions: 1.5.0
>Reporter: Andrew Purtell
>Assignee: Viraj Jasani
>Priority: Major
> Fix For: 1.5.0, 1.3.6, 1.4.11
>
> Attachments: HBASE-22132-branch-1.patch
>
>
> Check the exposure of branch-1 code to the problems described on HBASE-22115
> and apply the fix approach there.
--
This message was sent by Atlassian JIRA
(v7.6.14#76016)
[jira] [Commented] (HBASE-22115) HBase RPC aspires to grow an infinite tree of trace scopes; some other places are also unsafe
[
https://issues.apache.org/jira/browse/HBASE-22115?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16893140#comment-16893140
]
Hudson commented on HBASE-22115:
Results for branch branch-1.3
[build #889 on
builds.a.o|https://builds.apache.org/job/HBase%20Nightly/job/branch-1.3/889/]:
(x) *{color:red}-1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-1.3/889//General_Nightly_Build_Report/]
(x) {color:red}-1 jdk7 checks{color}
-- For more information [see jdk7
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-1.3/889//JDK7_Nightly_Build_Report/]
(/) {color:green}+1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-1.3/889//JDK8_Nightly_Build_Report_(Hadoop2)/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
> HBase RPC aspires to grow an infinite tree of trace scopes; some other places
> are also unsafe
> -
>
> Key: HBASE-22115
> URL: https://issues.apache.org/jira/browse/HBASE-22115
> Project: HBase
> Issue Type: Bug
>Reporter: Sergey Shelukhin
>Assignee: Sergey Shelukhin
>Priority: Critical
> Fix For: 3.0.0, 2.2.0, 2.3.0
>
> Attachments: Clipboard01.jpg, HBASE-22115.01.patch,
> HBASE-22115.02.patch, HBASE-22115.patch
>
>
> !Clipboard01.jpg!
> All of those are ClientServices.Multi in this case.
--
This message was sent by Atlassian JIRA
(v7.6.14#76016)
[GitHub] [hbase] Apache-HBase commented on issue #404: HBASE-22709 Add a chore thread in master to do hbck checking
Apache-HBase commented on issue #404: HBASE-22709 Add a chore thread in master to do hbck checking URL: https://github.com/apache/hbase/pull/404#issuecomment-515213800 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | 0 | reexec | 59 | Docker mode activated. | ||| _ Prechecks _ | | +1 | dupname | 0 | No case conflicting files found. | | +1 | hbaseanti | 0 | Patch does not have any anti-patterns. | | +1 | @author | 0 | The patch does not contain any @author tags. | | +1 | test4tests | 0 | The patch appears to include 2 new or modified test files. | ||| _ master Compile Tests _ | | +1 | mvninstall | 385 | master passed | | +1 | compile | 69 | master passed | | +1 | checkstyle | 93 | master passed | | +1 | shadedjars | 340 | branch has no errors when building our shaded downstream artifacts. | | +1 | javadoc | 43 | master passed | | 0 | spotbugs | 319 | Used deprecated FindBugs config; considering switching to SpotBugs. | | +1 | findbugs | 316 | master passed | ||| _ Patch Compile Tests _ | | +1 | mvninstall | 367 | the patch passed | | +1 | compile | 67 | the patch passed | | +1 | javac | 67 | the patch passed | | +1 | checkstyle | 94 | the patch passed | | +1 | whitespace | 1 | The patch has no whitespace issues. | | +1 | shadedjars | 333 | patch has no errors when building our shaded downstream artifacts. | | +1 | hadoopcheck | 1123 | Patch does not cause any errors with Hadoop 2.8.5 2.9.2 or 3.1.2. | | +1 | javadoc | 43 | the patch passed | | +1 | findbugs | 320 | the patch passed | ||| _ Other Tests _ | | -1 | unit | 16509 | hbase-server in the patch failed. | | +1 | asflicense | 27 | The patch does not generate ASF License warnings. | | | | 20627 | | | Subsystem | Report/Notes | |--:|:-| | Docker | Client=19.03.0 Server=19.03.0 base: https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-404/12/artifact/out/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/404 | | Optional Tests | dupname asflicense javac javadoc unit spotbugs findbugs shadedjars hadoopcheck hbaseanti checkstyle compile | | uname | Linux 3e3d4764ed99 4.4.0-138-generic #164-Ubuntu SMP Tue Oct 2 17:16:02 UTC 2018 x86_64 GNU/Linux | | Build tool | maven | | Personality | /home/jenkins/jenkins-slave/workspace/HBase-PreCommit-GitHub-PR_PR-404/out/precommit/personality/provided.sh | | git revision | master / de98fee288 | | Default Java | 1.8.0_181 | | unit | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-404/12/artifact/out/patch-unit-hbase-server.txt | | Test Results | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-404/12/testReport/ | | Max. process+thread count | 5127 (vs. ulimit of 1) | | modules | C: hbase-server U: hbase-server | | Console output | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-404/12/console | | versions | git=2.11.0 maven=2018-06-17T18:33:14Z) findbugs=3.1.11 | | Powered by | Apache Yetus 0.10.0 http://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [hbase] jatsakthi commented on a change in pull request #403: HBASE-22707 [HBCK2] MasterRpcServices assigns method should try to reload regions from meta if the passed regions isn't found und
jatsakthi commented on a change in pull request #403: HBASE-22707 [HBCK2] MasterRpcServices assigns method should try to reload regions from meta if the passed regions isn't found under AssignmentManager RegionsStateStore URL: https://github.com/apache/hbase/pull/403#discussion_r307499479 ## File path: hbase-server/src/main/java/org/apache/hadoop/hbase/master/MasterRpcServices.java ## @@ -2408,7 +2408,9 @@ private RegionInfo getRegionInfo(HBaseProtos.RegionSpecifier rs) throws UnknownR String encodedRegionName = Bytes.toString(rs.getValue().toByteArray()); RegionState regionState = this.master.getAssignmentManager().getRegionStates(). getRegionState(encodedRegionName); -ri = regionState == null? null: regionState.getRegion(); +ri = regionState == null ? Review comment: My bad! loadRegionFromMeta throws the exception. Please ignore. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Commented] (HBASE-22702) [Log] 'Group not found for table' is chatty
[
https://issues.apache.org/jira/browse/HBASE-22702?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16893134#comment-16893134
]
Hudson commented on HBASE-22702:
Results for branch branch-2.1
[build #1410 on
builds.a.o|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.1/1410/]:
(x) *{color:red}-1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.1/1410//General_Nightly_Build_Report/]
(x) {color:red}-1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.1/1410//JDK8_Nightly_Build_Report_(Hadoop2)/]
(x) {color:red}-1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.1/1410//JDK8_Nightly_Build_Report_(Hadoop3)/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
(/) {color:green}+1 client integration test{color}
> [Log] 'Group not found for table' is chatty
> ---
>
> Key: HBASE-22702
> URL: https://issues.apache.org/jira/browse/HBASE-22702
> Project: HBase
> Issue Type: Improvement
>Affects Versions: 1.4.6
>Reporter: Reid Chan
>Assignee: Murtaza Hassan
>Priority: Trivial
> Labels: beginner
> Fix For: 3.0.0, 1.5.0, 2.0.6, 2.2.1, 2.1.6, 1.4.11
>
>
> When rsgroup feature is enabled, there are few places when
> 'RSGroupBasedLoadBalancer' can't find group of a table and print a lot of
> {{Group not found for table ***, using default}} which is chatty and useless
> in master log.
> Plan to make it DEBUG level or remove it, since we can always know what
> tables a rsgroup contains by using hbase shell.
--
This message was sent by Atlassian JIRA
(v7.6.14#76016)
[jira] [Commented] (HBASE-22115) HBase RPC aspires to grow an infinite tree of trace scopes; some other places are also unsafe
[
https://issues.apache.org/jira/browse/HBASE-22115?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16893132#comment-16893132
]
Hudson commented on HBASE-22115:
Results for branch branch-1
[build #975 on
builds.a.o|https://builds.apache.org/job/HBase%20Nightly/job/branch-1/975/]:
(x) *{color:red}-1 overall{color}*
details (if available):
(x) {color:red}-1 general checks{color}
-- For more information [see general
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-1/975//General_Nightly_Build_Report/]
(x) {color:red}-1 jdk7 checks{color}
-- For more information [see jdk7
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-1/975//JDK7_Nightly_Build_Report/]
(x) {color:red}-1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-1/975//JDK8_Nightly_Build_Report_(Hadoop2)/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
> HBase RPC aspires to grow an infinite tree of trace scopes; some other places
> are also unsafe
> -
>
> Key: HBASE-22115
> URL: https://issues.apache.org/jira/browse/HBASE-22115
> Project: HBase
> Issue Type: Bug
>Reporter: Sergey Shelukhin
>Assignee: Sergey Shelukhin
>Priority: Critical
> Fix For: 3.0.0, 2.2.0, 2.3.0
>
> Attachments: Clipboard01.jpg, HBASE-22115.01.patch,
> HBASE-22115.02.patch, HBASE-22115.patch
>
>
> !Clipboard01.jpg!
> All of those are ClientServices.Multi in this case.
--
This message was sent by Atlassian JIRA
(v7.6.14#76016)
[jira] [Commented] (HBASE-22132) Backport HBASE-22115 intent to branch-1
[
https://issues.apache.org/jira/browse/HBASE-22132?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16893130#comment-16893130
]
Hudson commented on HBASE-22132:
Results for branch branch-1
[build #975 on
builds.a.o|https://builds.apache.org/job/HBase%20Nightly/job/branch-1/975/]:
(x) *{color:red}-1 overall{color}*
details (if available):
(x) {color:red}-1 general checks{color}
-- For more information [see general
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-1/975//General_Nightly_Build_Report/]
(x) {color:red}-1 jdk7 checks{color}
-- For more information [see jdk7
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-1/975//JDK7_Nightly_Build_Report/]
(x) {color:red}-1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-1/975//JDK8_Nightly_Build_Report_(Hadoop2)/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
> Backport HBASE-22115 intent to branch-1
> ---
>
> Key: HBASE-22132
> URL: https://issues.apache.org/jira/browse/HBASE-22132
> Project: HBase
> Issue Type: Sub-task
>Affects Versions: 1.5.0
>Reporter: Andrew Purtell
>Assignee: Viraj Jasani
>Priority: Major
> Fix For: 1.5.0, 1.3.6, 1.4.11
>
> Attachments: HBASE-22132-branch-1.patch
>
>
> Check the exposure of branch-1 code to the problems described on HBASE-22115
> and apply the fix approach there.
--
This message was sent by Atlassian JIRA
(v7.6.14#76016)
[GitHub] [hbase] Apache-HBase commented on issue #403: HBASE-22707 [HBCK2] MasterRpcServices assigns method should try to reload regions from meta if the passed regions isn't found under AssignmentMan
Apache-HBase commented on issue #403: HBASE-22707 [HBCK2] MasterRpcServices assigns method should try to reload regions from meta if the passed regions isn't found under AssignmentManager RegionsStateStore URL: https://github.com/apache/hbase/pull/403#issuecomment-515209853 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | 0 | reexec | 65 | Docker mode activated. | ||| _ Prechecks _ | | +1 | dupname | 0 | No case conflicting files found. | | +1 | hbaseanti | 0 | Patch does not have any anti-patterns. | | +1 | @author | 0 | The patch does not contain any @author tags. | | -0 | test4tests | 0 | The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. | ||| _ master Compile Tests _ | | 0 | mvndep | 14 | Maven dependency ordering for branch | | +1 | mvninstall | 339 | master passed | | +1 | compile | 83 | master passed | | +1 | checkstyle | 104 | master passed | | +1 | shadedjars | 281 | branch has no errors when building our shaded downstream artifacts. | | +1 | javadoc | 59 | master passed | | 0 | spotbugs | 274 | Used deprecated FindBugs config; considering switching to SpotBugs. | | +1 | findbugs | 345 | master passed | ||| _ Patch Compile Tests _ | | 0 | mvndep | 14 | Maven dependency ordering for patch | | +1 | mvninstall | 299 | the patch passed | | +1 | compile | 82 | the patch passed | | +1 | javac | 82 | the patch passed | | +1 | checkstyle | 105 | the patch passed | | +1 | whitespace | 0 | The patch has no whitespace issues. | | +1 | shadedjars | 292 | patch has no errors when building our shaded downstream artifacts. | | -1 | hadoopcheck | 491 | The patch causes 10 errors with Hadoop v2.9.2. | | -1 | hadoopcheck | 652 | The patch causes 10 errors with Hadoop v3.1.2. | | +1 | javadoc | 62 | the patch passed | | +1 | findbugs | 351 | the patch passed | ||| _ Other Tests _ | | +1 | unit | 99 | hbase-client in the patch passed. | | -1 | unit | 16453 | hbase-server in the patch failed. | | +1 | asflicense | 49 | The patch does not generate ASF License warnings. | | | | 19903 | | | Reason | Tests | |---:|:--| | Failed junit tests | hadoop.hbase.client.TestSnapshotDFSTemporaryDirectory | | | hadoop.hbase.client.TestAsyncRegionAdminApi | | | hadoop.hbase.namespace.TestNamespaceAuditor | | Subsystem | Report/Notes | |--:|:-| | Docker | Client=19.03.0 Server=19.03.0 base: https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-403/13/artifact/out/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/403 | | Optional Tests | dupname asflicense javac javadoc unit spotbugs findbugs shadedjars hadoopcheck hbaseanti checkstyle compile | | uname | Linux f93bdb58ba88 4.4.0-137-generic #163-Ubuntu SMP Mon Sep 24 13:14:43 UTC 2018 x86_64 GNU/Linux | | Build tool | maven | | Personality | /home/jenkins/jenkins-slave/workspace/HBase-PreCommit-GitHub-PR_PR-403/out/precommit/personality/provided.sh | | git revision | master / de98fee288 | | Default Java | 1.8.0_181 | | hadoopcheck | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-403/13/artifact/out/patch-javac-2.9.2.txt | | hadoopcheck | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-403/13/artifact/out/patch-javac-3.1.2.txt | | unit | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-403/13/artifact/out/patch-unit-hbase-server.txt | | Test Results | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-403/13/testReport/ | | Max. process+thread count | 5301 (vs. ulimit of 1) | | modules | C: hbase-client hbase-server U: . | | Console output | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-403/13/console | | versions | git=2.11.0 maven=2018-06-17T18:33:14Z) findbugs=3.1.11 | | Powered by | Apache Yetus 0.10.0 http://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [hbase] Apache-HBase commented on issue #408: HBASE-22733 TestSplitTransactionOnCluster.testMasterRestartAtRegionSp…
Apache-HBase commented on issue #408: HBASE-22733 TestSplitTransactionOnCluster.testMasterRestartAtRegionSp… URL: https://github.com/apache/hbase/pull/408#issuecomment-515207703 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | 0 | reexec | 59 | Docker mode activated. | ||| _ Prechecks _ | | +1 | dupname | 0 | No case conflicting files found. | | +1 | hbaseanti | 0 | Patch does not have any anti-patterns. | | +1 | @author | 0 | The patch does not contain any @author tags. | | +1 | test4tests | 0 | The patch appears to include 1 new or modified test files. | ||| _ master Compile Tests _ | | +1 | mvninstall | 302 | master passed | | +1 | compile | 58 | master passed | | +1 | checkstyle | 74 | master passed | | +1 | shadedjars | 266 | branch has no errors when building our shaded downstream artifacts. | | +1 | javadoc | 34 | master passed | | 0 | spotbugs | 247 | Used deprecated FindBugs config; considering switching to SpotBugs. | | +1 | findbugs | 244 | master passed | ||| _ Patch Compile Tests _ | | +1 | mvninstall | 293 | the patch passed | | +1 | compile | 55 | the patch passed | | +1 | javac | 55 | the patch passed | | +1 | checkstyle | 73 | the patch passed | | +1 | whitespace | 0 | The patch has no whitespace issues. | | +1 | shadedjars | 293 | patch has no errors when building our shaded downstream artifacts. | | +1 | hadoopcheck | 899 | Patch does not cause any errors with Hadoop 2.8.5 2.9.2 or 3.1.2. | | +1 | javadoc | 34 | the patch passed | | +1 | findbugs | 249 | the patch passed | ||| _ Other Tests _ | | -1 | unit | 16148 | hbase-server in the patch failed. | | +1 | asflicense | 38 | The patch does not generate ASF License warnings. | | | | 19517 | | | Reason | Tests | |---:|:--| | Failed junit tests | hadoop.hbase.master.procedure.TestSCPWithReplicasWithoutZKCoordinated | | Subsystem | Report/Notes | |--:|:-| | Docker | Client=19.03.0 Server=19.03.0 base: https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-408/2/artifact/out/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/408 | | Optional Tests | dupname asflicense javac javadoc unit spotbugs findbugs shadedjars hadoopcheck hbaseanti checkstyle compile | | uname | Linux 5b66f918f213 4.4.0-137-generic #163-Ubuntu SMP Mon Sep 24 13:14:43 UTC 2018 x86_64 GNU/Linux | | Build tool | maven | | Personality | /home/jenkins/jenkins-slave/workspace/HBase-PreCommit-GitHub-PR_PR-408/out/precommit/personality/provided.sh | | git revision | master / de98fee288 | | Default Java | 1.8.0_181 | | unit | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-408/2/artifact/out/patch-unit-hbase-server.txt | | Test Results | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-408/2/testReport/ | | Max. process+thread count | 5329 (vs. ulimit of 1) | | modules | C: hbase-server U: hbase-server | | Console output | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-408/2/console | | versions | git=2.11.0 maven=2018-06-17T18:33:14Z) findbugs=3.1.11 | | Powered by | Apache Yetus 0.10.0 http://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [hbase] Apache-HBase commented on issue #343: HBASE-22634 : Improve performance of BufferedMutator
Apache-HBase commented on issue #343: HBASE-22634 : Improve performance of BufferedMutator URL: https://github.com/apache/hbase/pull/343#issuecomment-515206212 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | 0 | reexec | 67 | Docker mode activated. | ||| _ Prechecks _ | | +1 | dupname | 0 | No case conflicting files found. | | +1 | hbaseanti | 0 | Patch does not have any anti-patterns. | | +1 | @author | 0 | The patch does not contain any @author tags. | | -0 | test4tests | 0 | The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. | ||| _ branch-2.1 Compile Tests _ | | +1 | mvninstall | 375 | branch-2.1 passed | | +1 | compile | 32 | branch-2.1 passed | | +1 | checkstyle | 44 | branch-2.1 passed | | +1 | shadedjars | 298 | branch has no errors when building our shaded downstream artifacts. | | +1 | javadoc | 27 | branch-2.1 passed | | 0 | spotbugs | 88 | Used deprecated FindBugs config; considering switching to SpotBugs. | | +1 | findbugs | 87 | branch-2.1 passed | ||| _ Patch Compile Tests _ | | +1 | mvninstall | 359 | the patch passed | | +1 | compile | 32 | the patch passed | | +1 | javac | 32 | the patch passed | | +1 | checkstyle | 43 | hbase-client: The patch generated 0 new + 58 unchanged - 4 fixed = 58 total (was 62) | | +1 | whitespace | 0 | The patch has no whitespace issues. | | +1 | shadedjars | 299 | patch has no errors when building our shaded downstream artifacts. | | +1 | hadoopcheck | 1502 | Patch does not cause any errors with Hadoop 2.7.7 2.8.5 or 3.0.3 3.1.2. | | +1 | javadoc | 27 | the patch passed | | +1 | findbugs | 100 | the patch passed | ||| _ Other Tests _ | | +1 | unit | 228 | hbase-client in the patch passed. | | +1 | asflicense | 13 | The patch does not generate ASF License warnings. | | | | 3978 | | | Subsystem | Report/Notes | |--:|:-| | Docker | Client=19.03.0 Server=19.03.0 base: https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-343/13/artifact/out/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/343 | | Optional Tests | dupname asflicense javac javadoc unit spotbugs findbugs shadedjars hadoopcheck hbaseanti checkstyle compile | | uname | Linux 79ee8898f3c0 4.4.0-143-generic #169-Ubuntu SMP Thu Feb 7 07:56:38 UTC 2019 x86_64 GNU/Linux | | Build tool | maven | | Personality | /home/jenkins/jenkins-slave/workspace/HBase-PreCommit-GitHub-PR_PR-343/out/precommit/personality/provided.sh | | git revision | branch-2.1 / 100a625324 | | Default Java | 1.8.0_181 | | Test Results | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-343/13/testReport/ | | Max. process+thread count | 301 (vs. ulimit of 1) | | modules | C: hbase-client U: hbase-client | | Console output | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-343/13/console | | versions | git=2.11.0 maven=2018-06-17T18:33:14Z) findbugs=3.1.11 | | Powered by | Apache Yetus 0.10.0 http://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Commented] (HBASE-22702) [Log] 'Group not found for table' is chatty
[
https://issues.apache.org/jira/browse/HBASE-22702?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16893116#comment-16893116
]
Hudson commented on HBASE-22702:
Results for branch branch-1.4
[build #924 on
builds.a.o|https://builds.apache.org/job/HBase%20Nightly/job/branch-1.4/924/]:
(x) *{color:red}-1 overall{color}*
details (if available):
(x) {color:red}-1 general checks{color}
-- For more information [see general
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-1.4/924//General_Nightly_Build_Report/]
(x) {color:red}-1 jdk7 checks{color}
-- For more information [see jdk7
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-1.4/924//JDK7_Nightly_Build_Report/]
(/) {color:green}+1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-1.4/924//JDK8_Nightly_Build_Report_(Hadoop2)/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
> [Log] 'Group not found for table' is chatty
> ---
>
> Key: HBASE-22702
> URL: https://issues.apache.org/jira/browse/HBASE-22702
> Project: HBase
> Issue Type: Improvement
>Affects Versions: 1.4.6
>Reporter: Reid Chan
>Assignee: Murtaza Hassan
>Priority: Trivial
> Labels: beginner
> Fix For: 3.0.0, 1.5.0, 2.0.6, 2.2.1, 2.1.6, 1.4.11
>
>
> When rsgroup feature is enabled, there are few places when
> 'RSGroupBasedLoadBalancer' can't find group of a table and print a lot of
> {{Group not found for table ***, using default}} which is chatty and useless
> in master log.
> Plan to make it DEBUG level or remove it, since we can always know what
> tables a rsgroup contains by using hbase shell.
--
This message was sent by Atlassian JIRA
(v7.6.14#76016)
[GitHub] [hbase] saintstack commented on a change in pull request #403: HBASE-22707 [HBCK2] MasterRpcServices assigns method should try to reload regions from meta if the passed regions isn't found un
saintstack commented on a change in pull request #403: HBASE-22707 [HBCK2]
MasterRpcServices assigns method should try to reload regions from meta if the
passed regions isn't found under AssignmentManager RegionsStateStore
URL: https://github.com/apache/hbase/pull/403#discussion_r307473340
##
File path:
hbase-client/src/main/java/org/apache/hadoop/hbase/MetaTableAccessor.java
##
@@ -372,6 +374,25 @@ public static Result getRegionResult(Connection
connection,
return get(getMetaHTable(connection), get);
}
+ /**
+ * Scans META table for a row whose key contains the specified
regionEncodedName,
+ * returning a single related Result instance if any row is
found, null otherwise.
+ *
+ * @param connection the connection to query META table.
+ * @param regionEncodedName the region encoded name to look for at META.
+ * @return Result instance with the row related info in META,
null otherwise.
+ * @throws IOException if any errors occur while querying META.
+ */
+ public static Result scanByRegionEncodedName(Connection connection,
+String regionEncodedName) throws IOException {
+RowFilter rowFilter = new RowFilter(CompareOperator.EQUAL,
+ new SubstringComparator(regionEncodedName));
+Scan scan = getMetaScan(connection, 1);
+scan.setFilter(rowFilter);
+ResultScanner resultScanner = getMetaHTable(connection).getScanner(scan);
+return resultScanner.next();
+ }
Review comment:
Do we have to scan meta? Could we look at Master's in-memory list of Regions?
Otherwise, this seems like nice utility. Good to add.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [hbase] saintstack commented on a change in pull request #403: HBASE-22707 [HBCK2] MasterRpcServices assigns method should try to reload regions from meta if the passed regions isn't found un
saintstack commented on a change in pull request #403: HBASE-22707 [HBCK2]
MasterRpcServices assigns method should try to reload regions from meta if the
passed regions isn't found under AssignmentManager RegionsStateStore
URL: https://github.com/apache/hbase/pull/403#discussion_r307480480
##
File path:
hbase-server/src/main/java/org/apache/hadoop/hbase/master/assignment/AssignmentManager.java
##
@@ -1383,51 +1383,76 @@ public void processOfflineRegions() {
}
}
- private void loadMeta() throws IOException {
-// TODO: use a thread pool
-regionStateStore.visitMeta(new RegionStateStore.RegionStateVisitor() {
- @Override
- public void visitRegionState(Result result, final RegionInfo regionInfo,
final State state,
- final ServerName regionLocation, final ServerName lastHost, final
long openSeqNum) {
-if (state == null && regionLocation == null && lastHost == null &&
-openSeqNum == SequenceId.NO_SEQUENCE_ID) {
- // This is a row with nothing in it.
- LOG.warn("Skipping empty row={}", result);
- return;
-}
-State localState = state;
-if (localState == null) {
- // No region state column data in hbase:meta table! Are I doing a
rolling upgrade from
- // hbase1 to hbase2? Am I restoring a SNAPSHOT or otherwise adding a
region to hbase:meta?
- // In any of these cases, state is empty. For now, presume OFFLINE
but there are probably
- // cases where we need to probe more to be sure this correct; TODO
informed by experience.
- LOG.info(regionInfo.getEncodedName() + " regionState=null; presuming
" + State.OFFLINE);
- localState = State.OFFLINE;
-}
-RegionStateNode regionNode =
regionStates.getOrCreateRegionStateNode(regionInfo);
-// Do not need to lock on regionNode, as we can make sure that before
we finish loading
-// meta, all the related procedures can not be executed. The only
exception is for meta
-// region related operations, but here we do not load the informations
for meta region.
-regionNode.setState(localState);
-regionNode.setLastHost(lastHost);
-regionNode.setRegionLocation(regionLocation);
-regionNode.setOpenSeqNum(openSeqNum);
-
-// Note: keep consistent with other methods, see
region(Opening|Opened|Closing)
-// RIT/ServerCrash handling should take care of the transiting
regions.
-if (localState.matches(State.OPEN, State.OPENING, State.CLOSING,
State.SPLITTING,
- State.MERGING)) {
- assert regionLocation != null : "found null region location for " +
regionNode;
- regionStates.addRegionToServer(regionNode);
-} else if (localState == State.OFFLINE || regionInfo.isOffline()) {
- regionStates.addToOfflineRegions(regionNode);
-}
-if (regionNode.getProcedure() != null) {
- regionNode.getProcedure().stateLoaded(AssignmentManager.this,
regionNode);
-}
+ private class RegionMetaLoadingVisitor implements
RegionStateStore.RegionStateVisitor {
+
+@Override
+public void visitRegionState(Result result, final RegionInfo regionInfo,
final State state,
+ final ServerName regionLocation, final ServerName lastHost, final long
openSeqNum) {
+ if (state == null && regionLocation == null && lastHost == null &&
+openSeqNum == SequenceId.NO_SEQUENCE_ID) {
+// This is a row with nothing in it.
+LOG.warn("Skipping empty row={}", result);
+return;
}
-});
+ State localState = state;
+ if (localState == null) {
+// No region state column data in hbase:meta table! Are I doing a
rolling upgrade from
+// hbase1 to hbase2? Am I restoring a SNAPSHOT or otherwise adding a
region to hbase:meta?
+// In any of these cases, state is empty. For now, presume OFFLINE but
there are probably
+// cases where we need to probe more to be sure this correct; TODO
informed by experience.
+LOG.info(regionInfo.getEncodedName() + " regionState=null; presuming "
+ State.OFFLINE);
+localState = State.OFFLINE;
+ }
+ RegionStateNode regionNode =
regionStates.getOrCreateRegionStateNode(regionInfo);
+ // Do not need to lock on regionNode, as we can make sure that before we
finish loading
+ // meta, all the related procedures can not be executed. The only
exception is for meta
+ // region related operations, but here we do not load the informations
for meta region.
+ regionNode.setState(localState);
+ regionNode.setLastHost(lastHost);
+ regionNode.setRegionLocation(regionLocation);
+ regionNode.setOpenSeqNum(openSeqNum);
+
+ // Note: keep consistent with other methods, see
region(Opening|Opened|Closing)
+ // RIT/ServerCrash handling should take care of the transiting
regions.
+ if
[GitHub] [hbase] saintstack commented on a change in pull request #403: HBASE-22707 [HBCK2] MasterRpcServices assigns method should try to reload regions from meta if the passed regions isn't found un
saintstack commented on a change in pull request #403: HBASE-22707 [HBCK2]
MasterRpcServices assigns method should try to reload regions from meta if the
passed regions isn't found under AssignmentManager RegionsStateStore
URL: https://github.com/apache/hbase/pull/403#discussion_r307480944
##
File path:
hbase-server/src/main/java/org/apache/hadoop/hbase/master/assignment/AssignmentManager.java
##
@@ -1383,51 +1383,76 @@ public void processOfflineRegions() {
}
}
- private void loadMeta() throws IOException {
-// TODO: use a thread pool
-regionStateStore.visitMeta(new RegionStateStore.RegionStateVisitor() {
- @Override
- public void visitRegionState(Result result, final RegionInfo regionInfo,
final State state,
- final ServerName regionLocation, final ServerName lastHost, final
long openSeqNum) {
-if (state == null && regionLocation == null && lastHost == null &&
-openSeqNum == SequenceId.NO_SEQUENCE_ID) {
- // This is a row with nothing in it.
- LOG.warn("Skipping empty row={}", result);
- return;
-}
-State localState = state;
-if (localState == null) {
- // No region state column data in hbase:meta table! Are I doing a
rolling upgrade from
- // hbase1 to hbase2? Am I restoring a SNAPSHOT or otherwise adding a
region to hbase:meta?
- // In any of these cases, state is empty. For now, presume OFFLINE
but there are probably
- // cases where we need to probe more to be sure this correct; TODO
informed by experience.
- LOG.info(regionInfo.getEncodedName() + " regionState=null; presuming
" + State.OFFLINE);
- localState = State.OFFLINE;
-}
-RegionStateNode regionNode =
regionStates.getOrCreateRegionStateNode(regionInfo);
-// Do not need to lock on regionNode, as we can make sure that before
we finish loading
-// meta, all the related procedures can not be executed. The only
exception is for meta
-// region related operations, but here we do not load the informations
for meta region.
-regionNode.setState(localState);
-regionNode.setLastHost(lastHost);
-regionNode.setRegionLocation(regionLocation);
-regionNode.setOpenSeqNum(openSeqNum);
-
-// Note: keep consistent with other methods, see
region(Opening|Opened|Closing)
-// RIT/ServerCrash handling should take care of the transiting
regions.
-if (localState.matches(State.OPEN, State.OPENING, State.CLOSING,
State.SPLITTING,
- State.MERGING)) {
- assert regionLocation != null : "found null region location for " +
regionNode;
- regionStates.addRegionToServer(regionNode);
-} else if (localState == State.OFFLINE || regionInfo.isOffline()) {
- regionStates.addToOfflineRegions(regionNode);
-}
-if (regionNode.getProcedure() != null) {
- regionNode.getProcedure().stateLoaded(AssignmentManager.this,
regionNode);
-}
+ private class RegionMetaLoadingVisitor implements
RegionStateStore.RegionStateVisitor {
+
+@Override
+public void visitRegionState(Result result, final RegionInfo regionInfo,
final State state,
+ final ServerName regionLocation, final ServerName lastHost, final long
openSeqNum) {
+ if (state == null && regionLocation == null && lastHost == null &&
+openSeqNum == SequenceId.NO_SEQUENCE_ID) {
+// This is a row with nothing in it.
+LOG.warn("Skipping empty row={}", result);
+return;
}
-});
+ State localState = state;
+ if (localState == null) {
+// No region state column data in hbase:meta table! Are I doing a
rolling upgrade from
+// hbase1 to hbase2? Am I restoring a SNAPSHOT or otherwise adding a
region to hbase:meta?
+// In any of these cases, state is empty. For now, presume OFFLINE but
there are probably
+// cases where we need to probe more to be sure this correct; TODO
informed by experience.
+LOG.info(regionInfo.getEncodedName() + " regionState=null; presuming "
+ State.OFFLINE);
+localState = State.OFFLINE;
+ }
+ RegionStateNode regionNode =
regionStates.getOrCreateRegionStateNode(regionInfo);
+ // Do not need to lock on regionNode, as we can make sure that before we
finish loading
+ // meta, all the related procedures can not be executed. The only
exception is for meta
+ // region related operations, but here we do not load the informations
for meta region.
+ regionNode.setState(localState);
+ regionNode.setLastHost(lastHost);
+ regionNode.setRegionLocation(regionLocation);
+ regionNode.setOpenSeqNum(openSeqNum);
+
+ // Note: keep consistent with other methods, see
region(Opening|Opened|Closing)
+ // RIT/ServerCrash handling should take care of the transiting
regions.
+ if
