[
https://issues.apache.org/jira/browse/HBASE-21505?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16710239#comment-16710239
]
Hadoop QA commented on HBASE-21505:
-----------------------------------
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
11s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s{color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 2 new or modified test
files. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
27s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 4m
42s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 5m
23s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 2m
42s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 4m
19s{color} | {color:green} branch has no errors when building our shaded
downstream artifacts. {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
27s{color} | {color:blue} hbase-hadoop2-compat in master has 18 extant Findbugs
warnings. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
45s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
14s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:red}-1{color} | {color:red} mvninstall {color} | {color:red} 1m
24s{color} | {color:red} root in the patch failed. {color} |
| {color:red}-1{color} | {color:red} compile {color} | {color:red} 0m
18s{color} | {color:red} hbase-client in the patch failed. {color} |
| {color:red}-1{color} | {color:red} compile {color} | {color:red} 1m
42s{color} | {color:red} hbase-server in the patch failed. {color} |
| {color:red}-1{color} | {color:red} cc {color} | {color:red} 0m 18s{color} |
{color:red} hbase-client in the patch failed. {color} |
| {color:red}-1{color} | {color:red} cc {color} | {color:red} 1m 42s{color} |
{color:red} hbase-server in the patch failed. {color} |
| {color:red}-1{color} | {color:red} javac {color} | {color:red} 0m 18s{color}
| {color:red} hbase-client in the patch failed. {color} |
| {color:red}-1{color} | {color:red} javac {color} | {color:red} 1m 42s{color}
| {color:red} hbase-server in the patch failed. {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m
11s{color} | {color:red} hbase-hadoop2-compat: The patch generated 6 new + 0
unchanged - 0 fixed = 6 total (was 0) {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m
34s{color} | {color:red} hbase-client: The patch generated 5 new + 185
unchanged - 0 fixed = 190 total (was 185) {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 1m
15s{color} | {color:red} hbase-server: The patch generated 19 new + 86
unchanged - 2 fixed = 105 total (was 88) {color} |
| {color:red}-1{color} | {color:red} rubocop {color} | {color:red} 0m
8s{color} | {color:red} The patch generated 55 new + 405 unchanged - 9 fixed =
460 total (was 414) {color} |
| {color:orange}-0{color} | {color:orange} ruby-lint {color} | {color:orange}
0m 4s{color} | {color:orange} The patch generated 3 new + 748 unchanged - 1
fixed = 751 total (was 749) {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:red}-1{color} | {color:red} shadedjars {color} | {color:red} 2m
5s{color} | {color:red} patch has 14 errors when building our shaded downstream
artifacts. {color} |
| {color:red}-1{color} | {color:red} hadoopcheck {color} | {color:red} 1m
21s{color} | {color:red} The patch causes 14 errors with Hadoop v2.7.4. {color}
|
| {color:red}-1{color} | {color:red} hadoopcheck {color} | {color:red} 2m
38s{color} | {color:red} The patch causes 14 errors with Hadoop v3.0.0. {color}
|
| {color:red}-1{color} | {color:red} hbaseprotoc {color} | {color:red} 0m
19s{color} | {color:red} hbase-client in the patch failed. {color} |
| {color:red}-1{color} | {color:red} hbaseprotoc {color} | {color:red} 0m
31s{color} | {color:red} hbase-server in the patch failed. {color} |
| {color:red}-1{color} | {color:red} findbugs {color} | {color:red} 0m
18s{color} | {color:red} hbase-client in the patch failed. {color} |
| {color:red}-1{color} | {color:red} findbugs {color} | {color:red} 0m
29s{color} | {color:red} hbase-server in the patch failed. {color} |
| {color:red}-1{color} | {color:red} javadoc {color} | {color:red} 0m
21s{color} | {color:red} hbase-client generated 1 new + 2 unchanged - 0 fixed =
3 total (was 2) {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 0m
34s{color} | {color:green} hbase-protocol-shaded in the patch passed. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 0m
27s{color} | {color:green} hbase-hadoop-compat in the patch passed. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 0m
30s{color} | {color:green} hbase-hadoop2-compat in the patch passed. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 0m
22s{color} | {color:green} hbase-protocol in the patch passed. {color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 0m 20s{color}
| {color:red} hbase-client in the patch failed. {color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 0m 48s{color}
| {color:red} hbase-server in the patch failed. {color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 6m 53s{color}
| {color:red} hbase-shell in the patch failed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
56s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 63m 58s{color} |
{color:black} {color} |
\\
\\
|| Reason || Tests ||
| Failed junit tests | hadoop.hbase.client.TestShell |
| | hadoop.hbase.client.TestAdminShell2 |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hbase:b002b0b |
| JIRA Issue | HBASE-21505 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12950701/HBASE-21505-master.001.patch
|
| Optional Tests | dupname asflicense javac javadoc unit findbugs
shadedjars hadoopcheck hbaseanti checkstyle compile cc hbaseprotoc
rubocop ruby_lint |
| uname | Linux 62d0de0a3c64 4.4.0-139-generic #165~14.04.1-Ubuntu SMP Wed Oct
31 10:55:11 UTC 2018 x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/component/dev-support/hbase-personality.sh
|
| git revision | master / 8bf966c8e9 |
| maven | version: Apache Maven 3.5.4
(1edded0938998edf8bf061f1ceb3cfdeccf443fe; 2018-06-17T18:33:14Z) |
| Default Java | 1.8.0_181 |
| findbugs | v3.1.0-RC3 |
| mvninstall |
https://builds.apache.org/job/PreCommit-HBASE-Build/15198/artifact/patchprocess/patch-mvninstall-root.txt
|
| compile |
https://builds.apache.org/job/PreCommit-HBASE-Build/15198/artifact/patchprocess/patch-compile-hbase-client.txt
|
| compile |
https://builds.apache.org/job/PreCommit-HBASE-Build/15198/artifact/patchprocess/patch-compile-hbase-server.txt
|
| cc |
https://builds.apache.org/job/PreCommit-HBASE-Build/15198/artifact/patchprocess/patch-compile-hbase-client.txt
|
| cc |
https://builds.apache.org/job/PreCommit-HBASE-Build/15198/artifact/patchprocess/patch-compile-hbase-server.txt
|
| javac |
https://builds.apache.org/job/PreCommit-HBASE-Build/15198/artifact/patchprocess/patch-compile-hbase-client.txt
|
| javac |
https://builds.apache.org/job/PreCommit-HBASE-Build/15198/artifact/patchprocess/patch-compile-hbase-server.txt
|
| checkstyle |
https://builds.apache.org/job/PreCommit-HBASE-Build/15198/artifact/patchprocess/diff-checkstyle-hbase-hadoop2-compat.txt
|
| checkstyle |
https://builds.apache.org/job/PreCommit-HBASE-Build/15198/artifact/patchprocess/diff-checkstyle-hbase-client.txt
|
| checkstyle |
https://builds.apache.org/job/PreCommit-HBASE-Build/15198/artifact/patchprocess/diff-checkstyle-hbase-server.txt
|
| rubocop | v0.60.0 |
| rubocop |
https://builds.apache.org/job/PreCommit-HBASE-Build/15198/artifact/patchprocess/diff-patch-rubocop.txt
|
| ruby-lint | v2.3.1 |
| ruby-lint |
https://builds.apache.org/job/PreCommit-HBASE-Build/15198/artifact/patchprocess/diff-patch-ruby-lint.txt
|
| shadedjars |
https://builds.apache.org/job/PreCommit-HBASE-Build/15198/artifact/patchprocess/patch-shadedjars.txt
|
| hadoopcheck |
https://builds.apache.org/job/PreCommit-HBASE-Build/15198/artifact/patchprocess/patch-javac-2.7.4.txt
|
| hadoopcheck |
https://builds.apache.org/job/PreCommit-HBASE-Build/15198/artifact/patchprocess/patch-javac-3.0.0.txt
|
| hbaseprotoc |
https://builds.apache.org/job/PreCommit-HBASE-Build/15198/artifact/patchprocess/patch-hbaseprotoc-hbase-client.txt
|
| hbaseprotoc |
https://builds.apache.org/job/PreCommit-HBASE-Build/15198/artifact/patchprocess/patch-hbaseprotoc-hbase-server.txt
|
| findbugs |
https://builds.apache.org/job/PreCommit-HBASE-Build/15198/artifact/patchprocess/patch-findbugs-hbase-client.txt
|
| findbugs |
https://builds.apache.org/job/PreCommit-HBASE-Build/15198/artifact/patchprocess/patch-findbugs-hbase-server.txt
|
| javadoc |
https://builds.apache.org/job/PreCommit-HBASE-Build/15198/artifact/patchprocess/diff-javadoc-javadoc-hbase-client.txt
|
| unit |
https://builds.apache.org/job/PreCommit-HBASE-Build/15198/artifact/patchprocess/patch-unit-hbase-client.txt
|
| unit |
https://builds.apache.org/job/PreCommit-HBASE-Build/15198/artifact/patchprocess/patch-unit-hbase-server.txt
|
| unit |
https://builds.apache.org/job/PreCommit-HBASE-Build/15198/artifact/patchprocess/patch-unit-hbase-shell.txt
|
| Test Results |
https://builds.apache.org/job/PreCommit-HBASE-Build/15198/testReport/ |
| Max. process+thread count | 2436 (vs. ulimit of 10000) |
| modules | C: hbase-protocol-shaded hbase-hadoop-compat hbase-hadoop2-compat
hbase-protocol hbase-client hbase-server hbase-shell U: . |
| Console output |
https://builds.apache.org/job/PreCommit-HBASE-Build/15198/console |
| Powered by | Apache Yetus 0.8.0 http://yetus.apache.org |
This message was automatically generated.
> Several inconsistencies on information reported for Replication Sources by
> hbase shell status 'replication' command.
> --------------------------------------------------------------------------------------------------------------------
>
> Key: HBASE-21505
> URL: https://issues.apache.org/jira/browse/HBASE-21505
> Project: HBase
> Issue Type: Bug
> Reporter: Wellington Chevreuil
> Assignee: Wellington Chevreuil
> Priority: Major
> Attachments:
> 0001-HBASE-21505-initial-version-for-more-detailed-report.patch,
> HBASE-21505-master.001.patch
>
>
> While reviewing hbase shell status 'replication' command, noticed the
> following issues related to replication source section:
> 1) TimeStampsOfLastShippedOp keeps getting updated and increasing even when
> no new edits were added to source, so nothing was really shipped. Test steps
> performed:
> 1.1) Source cluster with only one table targeted to replication;
> 1.2) Added a new row, confirmed the row appeared in Target cluster;
> 1.3) Issued status 'replication' command in source, TimeStampsOfLastShippedOp
> shows current timestamp T1.
> 1.4) Waited 30 seconds, no new data added to source. Issued status
> 'replication' command, now shows timestamp T2.
> 2) When replication is stuck due some connectivity issues or target
> unavailability, if new edits are added in source, reported AgeOfLastShippedOp
> is wrongly showing same value as "Replication Lag". This is incorrect,
> AgeOfLastShippedOp should not change until there's indeed another edit
> shipped to target. Test steps performed:
> 2.1) Source cluster with only one table targeted to replication;
> 2.2) Stopped target cluster RS;
> 2.3) Put a new row on source. Running status 'replication' command does show
> lag increasing. TimeStampsOfLastShippedOp seems correct also, no further
> updates as described on bullet #1 above.
> 2.4) AgeOfLastShippedOp keeps increasing together with Replication Lag, even
> though there's no new edit shipped to target:
> {noformat}
> ...
> SOURCE: PeerID=1, AgeOfLastShippedOp=5581, SizeOfLogQueue=1,
> TimeStampsOfLastShippedOp=Wed Nov 21 02:50:23 GMT 2018, Replication Lag=5581
> ...
> ...
> SOURCE: PeerID=1, AgeOfLastShippedOp=8586, SizeOfLogQueue=1,
> TimeStampsOfLastShippedOp=Wed Nov 21 02:50:23 GMT 2018, Replication Lag=8586
> ...
> {noformat}
> 3) AgeOfLastShippedOp gets set to 0 even when a given edit had taken some
> time before it got finally shipped to target. Test steps performed:
> 3.1) Source cluster with only one table targeted to replication;
> 3.2) Stopped target cluster RS;
> 3.3) Put a new row on source.
> 3.4) AgeOfLastShippedOp keeps increasing together with Replication Lag, even
> though there's no new edit shipped to target:
> {noformat}
> T1:
> ...
> SOURCE: PeerID=1, AgeOfLastShippedOp=5581, SizeOfLogQueue=1,
> TimeStampsOfLastShippedOp=Wed Nov 21 02:50:23 GMT 2018, Replication Lag=5581
> ...
> T2:
> ...
> SOURCE: PeerID=1, AgeOfLastShippedOp=8586, SizeOfLogQueue=1,
> TimeStampsOfLastShippedOp=Wed Nov 21 02:50:23 GMT 2018, Replication Lag=8586
> ...
> {noformat}
> 3.5) Restart target cluster RS and verified the new row appeared there. No
> new edit added, but status 'replication' command reports AgeOfLastShippedOp
> as 0, while it should be the diff between the time it concluded shipping at
> target and the time it was added in source:
> {noformat}
> SOURCE: PeerID=1, AgeOfLastShippedOp=0, SizeOfLogQueue=1,
> TimeStampsOfLastShippedOp=Wed Nov 21 02:50:23 GMT 2018, Replication Lag=0
> {noformat}
> 4) When replication is stuck due some connectivity issues or target
> unavailability, if RS is restarted, once recovered queue source is started,
> TimeStampsOfLastShippedOp is set to initial java date (Thu Jan 01 01:00:00
> GMT 1970, for example), thus "Replication Lag" also gives a complete
> inaccurate value.
> Tests performed:
> 4.1) Source cluster with only one table targeted to replication;
> 4.2) Stopped target cluster RS;
> 4.3) Put a new row on source, restart RS on source, waited a few seconds for
> recovery queue source to startup, then it gives:
> {noformat}
> SOURCE: PeerID=1, AgeOfLastShippedOp=0, SizeOfLogQueue=1,
> TimeStampsOfLastShippedOp=Thu Jan 01 01:00:00 GMT 1970, Replication
> Lag=9223372036854775807
> {noformat}
> Also, we should report status to all sources running, current output format
> gives the impression there’s only one, even when there are recovery queues,
> for instance.
> Here is a list of ideas on how the command should report under different
> states of replication:
> a) Source started, target stopped, no edits arrived on source yet:
> Status replication should not show any lags, no edits shipped, no edits
> arrived;
> b) Source started, target stopped, add edit on source:
> Status replication should report following info -> lag, time of edit arrival
> on source, additional message saying no edits had been shipped to target;
> c) Source started, target stopped, edit added on source, restart source:
> Status replication should list two sources, one normal, other recovered.
> Normal source should show no lags, no edits shipped, no edits arrived.
> Recovered should show no edits shipped, but should have edits arrived in
> source and lag > 0;
> d) Source started, target stopped, add edit on source, restart source, add
> another edit on source:
> Status replication should list two sources, one normal, other recovered. Both
> sources should show no edits shipped, but should have edits arrived in source
> and lag > 0;
> e) Source started, target stopped, add edit on source, restart source, add
> another edit on source, start target:
> Status replication should list normal source only (after some short period),
> with proper times for last shipped, last arrived in source and no replication
> lag.
> f) Source started, target stopped, add edit on source, restart source,
> restart target:
> Status replication should list normal source only, with no shipped, nor
> arrived edits, and lag should be 0;
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
