[
https://issues.apache.org/jira/browse/HBASE-22538?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16860460#comment-16860460
]
HBase QA commented on HBASE-22538:
----------------------------------
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 18m
50s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} branch-1.4 Compile Tests {color} ||
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:red}-1{color} | {color:red} rubocop {color} | {color:red} 0m
5s{color} | {color:red} The patch generated 2 new + 353 unchanged - 8 fixed =
355 total (was 361) {color} |
| {color:green}+1{color} | {color:green} ruby-lint {color} | {color:green} 0m
2s{color} | {color:green} There were no new ruby-lint issues. {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
47s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 20m 42s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=17.05.0-ce Server=17.05.0-ce base:
https://builds.apache.org/job/PreCommit-HBASE-Build/512/artifact/patchprocess/Dockerfile
|
| JIRA Issue | HBASE-22538 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12971397/HBASE-22538.branch-1.4.002.patch
|
| Optional Tests | dupname asflicense rubocop ruby_lint |
| uname | Linux ec865fcb68e6 4.4.0-138-generic #164-Ubuntu SMP Tue Oct 2
17:16:02 UTC 2018 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/hbase-personality.sh |
| git revision | branch-1.4 / e10bfa0 |
| maven | version: Apache Maven 3.0.5 |
| rubocop | v0.71.0 |
| rubocop |
https://builds.apache.org/job/PreCommit-HBASE-Build/512/artifact/patchprocess/diff-patch-rubocop.txt
|
| ruby-lint | v2.3.1 |
| Max. process+thread count | 40 (vs. ulimit of 10000) |
| modules | C: . U: . |
| Console output |
https://builds.apache.org/job/PreCommit-HBASE-Build/512/console |
| Powered by | Apache Yetus 0.9.0 http://yetus.apache.org |
This message was automatically generated.
> Prevent graceful_stop.sh from shutting down RS too early before finishing
> unloading regions
> -------------------------------------------------------------------------------------------
>
> Key: HBASE-22538
> URL: https://issues.apache.org/jira/browse/HBASE-22538
> Project: HBase
> Issue Type: Bug
> Components: shell
> Affects Versions: 1.4.9
> Reporter: Jeongdae Kim
> Assignee: Jeongdae Kim
> Priority: Minor
> Attachments: HBASE-22538.branch-1.4.001.patch,
> HBASE-22538.branch-1.4.002.patch
>
>
> We can stop or restart region servers gracefully using graceful_stop.sh
> command
> This command should guarantee that all regions are moved out before shutting
> down a region server.
> However, sometimes i saw many requests failed while restarting a region
> server with this command in our production clusters(v1.2.5)
> affected clients got many RegionServerStoppedExceptions and exhausted retry
> count.
> I found it took 0.03 sec to move a region, it’s too fast. and,
> moving(unloading) regions in the region server wasn’t finished, even didn’t
> closed yet when region server got shutdown signal.
> Because a region server serving regions (didn't be closed) were stopped,
> clients got many exception (RegionServerStoppedException)
> But, region_mover should wait until a region is served by other region
> server(meta changed)
> https://github.com/apache/hbase/blob/branch-1.2/bin/region_mover.rb#L153
> I figured out why this early shutdown happened.
> a) our clusters use upper case hostname
> b) region server makes ServerName with lowercase hostname, and it will be
> sent to the master
> https://github.com/apache/hbase/blob/branch-1.2/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/HRegionServer.java#L542
> c) when updating meta, server name will keep its own case
> https://github.com/apache/hbase/blob/branch-1.2/hbase-client/src/main/java/org/apache/hadoop/hbase/MetaTableAccessor.java#L1527
> d) region_mover.rb just compare b) and c), so it is always false
> https://github.com/apache/hbase/blob/branch-1.2/bin/region_mover.rb#L91
> https://github.com/apache/hbase/blob/branch-1.2/bin/region_mover.rb#L52
> I think region_mover should compare server name between master and meta with
> the same case(lower)
> With patch, I confirmed region_mover waited until finishing moving all
> regions, then triggered shutting down region sever. (also observed only
> RegionMovedException before shutdown log, and no exception after starting
> shutdown)
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
