[jira] [Commented] (HBASE-23812) [Flakey Test] TestReplicator#testReplicatorWithErrors: AssertionError: We did not replicate enough rows expected:<10> but was:<7>
[
https://issues.apache.org/jira/browse/HBASE-23812?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17034370#comment-17034370
]
Hudson commented on HBASE-23812:
Results for branch master
[build #1626 on
builds.a.o|https://builds.apache.org/job/HBase%20Nightly/job/master/1626/]: (x)
*{color:red}-1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://builds.apache.org/job/HBase%20Nightly/job/master/1626//General_Nightly_Build_Report/]
(x) {color:red}-1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://builds.apache.org/job/HBase%20Nightly/job/master/1626//JDK8_Nightly_Build_Report_(Hadoop2)/]
(/) {color:green}+1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://builds.apache.org/job/HBase%20Nightly/job/master/1626//JDK8_Nightly_Build_Report_(Hadoop3)/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
(/) {color:green}+1 client integration test{color}
> [Flakey Test] TestReplicator#testReplicatorWithErrors: AssertionError: We did
> not replicate enough rows expected:<10> but was:<7>
> -
>
> Key: HBASE-23812
> URL: https://issues.apache.org/jira/browse/HBASE-23812
> Project: HBase
> Issue Type: Bug
> Components: flakies
>Reporter: Michael Stack
>Assignee: Michael Stack
>Priority: Major
> Fix For: 3.0.0, 2.3.0
>
> Attachments:
> 0001-HBASE-23812-Flakey-Test-TestReplicator-testReplicato.patch
>
>
> I've seen this fail. Here is the latest:
> {code}
> [ERROR] Tests run: 2, Failures: 1, Errors: 0, Skipped: 0, Time elapsed:
> 21.044 s <<< FAILURE! - in
> org.apache.hadoop.hbase.replication.regionserver.TestReplicator
> [ERROR]
> org.apache.hadoop.hbase.replication.regionserver.TestReplicator.testReplicatorWithErrors
> Time elapsed: 7.139 s <<< FAILURE!
> java.lang.AssertionError: We did not replicate enough rows expected:<10> but
> was:<7>
>at
> org.apache.hadoop.hbase.replication.regionserver.TestReplicator.testReplicatorWithErrors(TestReplicator.java:158)
> {code}
> In test logs there are a few (in groups of three) of these:
> {code}
> 4247 2020-02-07 13:50:29,070 WARN
> [RS_REFRESH_PEER-regionserver/localhost:0-1.replicationSource,testReplicatorWithErrors.replicationSource.shipperlocalhost%2C49531%2C1581112194639,testReplicatorWithErrors]
> regionserver. ReplicationSourceShipper(223):
> org.apache.hadoop.hbase.replication.regionserver.TestReplicator$FailureInjectingReplicationEndpointForTest
> threw unknown exception:
> 4248 java.lang.ClassCastException:
> org.apache.hbase.thirdparty.com.google.protobuf.ServiceException cannot be
> cast to java.io.IOException
> 4249 at
> org.apache.hadoop.hbase.replication.regionserver.HBaseInterClusterReplicationEndpoint.parallelReplicate(HBaseInterClusterReplicationEndpoint.java:361)
> 4250 at
> org.apache.hadoop.hbase.replication.regionserver.HBaseInterClusterReplicationEndpoint.replicate(HBaseInterClusterReplicationEndpoint.java:404)
> 4251 at
> org.apache.hadoop.hbase.replication.regionserver.TestReplicator$ReplicationEndpointForTest.replicate(TestReplicator.java:228)
> 4252 at
> org.apache.hadoop.hbase.replication.regionserver.ReplicationSourceShipper.shipEdits(ReplicationSourceShipper.java:188)
> 4253 at
> org.apache.hadoop.hbase.replication.regionserver.ReplicationSourceShipper.run(ReplicationSourceShipper.java:118)
> {code}
> Let me fix this and see if it makes TestReplicator stable again.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HBASE-23812) [Flakey Test] TestReplicator#testReplicatorWithErrors: AssertionError: We did not replicate enough rows expected:<10> but was:<7>
[
https://issues.apache.org/jira/browse/HBASE-23812?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17032809#comment-17032809
]
Hudson commented on HBASE-23812:
Results for branch branch-2
[build #2459 on
builds.a.o|https://builds.apache.org/job/HBase%20Nightly/job/branch-2/2459/]:
(x) *{color:red}-1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2/2459//General_Nightly_Build_Report/]
(x) {color:red}-1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2/2459//JDK8_Nightly_Build_Report_(Hadoop2)/]
(x) {color:red}-1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2/2459//JDK8_Nightly_Build_Report_(Hadoop3)/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
(/) {color:green}+1 client integration test{color}
> [Flakey Test] TestReplicator#testReplicatorWithErrors: AssertionError: We did
> not replicate enough rows expected:<10> but was:<7>
> -
>
> Key: HBASE-23812
> URL: https://issues.apache.org/jira/browse/HBASE-23812
> Project: HBase
> Issue Type: Bug
> Components: flakies
>Reporter: Michael Stack
>Assignee: Michael Stack
>Priority: Major
> Fix For: 3.0.0, 2.3.0
>
> Attachments:
> 0001-HBASE-23812-Flakey-Test-TestReplicator-testReplicato.patch
>
>
> I've seen this fail. Here is the latest:
> {code}
> [ERROR] Tests run: 2, Failures: 1, Errors: 0, Skipped: 0, Time elapsed:
> 21.044 s <<< FAILURE! - in
> org.apache.hadoop.hbase.replication.regionserver.TestReplicator
> [ERROR]
> org.apache.hadoop.hbase.replication.regionserver.TestReplicator.testReplicatorWithErrors
> Time elapsed: 7.139 s <<< FAILURE!
> java.lang.AssertionError: We did not replicate enough rows expected:<10> but
> was:<7>
>at
> org.apache.hadoop.hbase.replication.regionserver.TestReplicator.testReplicatorWithErrors(TestReplicator.java:158)
> {code}
> In test logs there are a few (in groups of three) of these:
> {code}
> 4247 2020-02-07 13:50:29,070 WARN
> [RS_REFRESH_PEER-regionserver/localhost:0-1.replicationSource,testReplicatorWithErrors.replicationSource.shipperlocalhost%2C49531%2C1581112194639,testReplicatorWithErrors]
> regionserver. ReplicationSourceShipper(223):
> org.apache.hadoop.hbase.replication.regionserver.TestReplicator$FailureInjectingReplicationEndpointForTest
> threw unknown exception:
> 4248 java.lang.ClassCastException:
> org.apache.hbase.thirdparty.com.google.protobuf.ServiceException cannot be
> cast to java.io.IOException
> 4249 at
> org.apache.hadoop.hbase.replication.regionserver.HBaseInterClusterReplicationEndpoint.parallelReplicate(HBaseInterClusterReplicationEndpoint.java:361)
> 4250 at
> org.apache.hadoop.hbase.replication.regionserver.HBaseInterClusterReplicationEndpoint.replicate(HBaseInterClusterReplicationEndpoint.java:404)
> 4251 at
> org.apache.hadoop.hbase.replication.regionserver.TestReplicator$ReplicationEndpointForTest.replicate(TestReplicator.java:228)
> 4252 at
> org.apache.hadoop.hbase.replication.regionserver.ReplicationSourceShipper.shipEdits(ReplicationSourceShipper.java:188)
> 4253 at
> org.apache.hadoop.hbase.replication.regionserver.ReplicationSourceShipper.run(ReplicationSourceShipper.java:118)
> {code}
> Let me fix this and see if it makes TestReplicator stable again.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HBASE-23812) [Flakey Test] TestReplicator#testReplicatorWithErrors: AssertionError: We did not replicate enough rows expected:<10> but was:<7>
[
https://issues.apache.org/jira/browse/HBASE-23812?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17032789#comment-17032789
]
Michael Stack commented on HBASE-23812:
---
bq. You want to see it pass after repeated builds?
Yeah. Its a rare case. I want it to clear flakies
https://builds.apache.org/view/H-L/view/HBase/job/HBase-Find-Flaky-Tests/job/branch-2/lastSuccessfulBuild/artifact/dashboard.html
or just stop showing up in my local runs. Day or two? Want something more
clean than this?
> [Flakey Test] TestReplicator#testReplicatorWithErrors: AssertionError: We did
> not replicate enough rows expected:<10> but was:<7>
> -
>
> Key: HBASE-23812
> URL: https://issues.apache.org/jira/browse/HBASE-23812
> Project: HBase
> Issue Type: Bug
> Components: flakies
>Reporter: Michael Stack
>Assignee: Michael Stack
>Priority: Major
> Fix For: 3.0.0, 2.3.0
>
> Attachments:
> 0001-HBASE-23812-Flakey-Test-TestReplicator-testReplicato.patch
>
>
> I've seen this fail. Here is the latest:
> {code}
> [ERROR] Tests run: 2, Failures: 1, Errors: 0, Skipped: 0, Time elapsed:
> 21.044 s <<< FAILURE! - in
> org.apache.hadoop.hbase.replication.regionserver.TestReplicator
> [ERROR]
> org.apache.hadoop.hbase.replication.regionserver.TestReplicator.testReplicatorWithErrors
> Time elapsed: 7.139 s <<< FAILURE!
> java.lang.AssertionError: We did not replicate enough rows expected:<10> but
> was:<7>
>at
> org.apache.hadoop.hbase.replication.regionserver.TestReplicator.testReplicatorWithErrors(TestReplicator.java:158)
> {code}
> In test logs there are a few (in groups of three) of these:
> {code}
> 4247 2020-02-07 13:50:29,070 WARN
> [RS_REFRESH_PEER-regionserver/localhost:0-1.replicationSource,testReplicatorWithErrors.replicationSource.shipperlocalhost%2C49531%2C1581112194639,testReplicatorWithErrors]
> regionserver. ReplicationSourceShipper(223):
> org.apache.hadoop.hbase.replication.regionserver.TestReplicator$FailureInjectingReplicationEndpointForTest
> threw unknown exception:
> 4248 java.lang.ClassCastException:
> org.apache.hbase.thirdparty.com.google.protobuf.ServiceException cannot be
> cast to java.io.IOException
> 4249 at
> org.apache.hadoop.hbase.replication.regionserver.HBaseInterClusterReplicationEndpoint.parallelReplicate(HBaseInterClusterReplicationEndpoint.java:361)
> 4250 at
> org.apache.hadoop.hbase.replication.regionserver.HBaseInterClusterReplicationEndpoint.replicate(HBaseInterClusterReplicationEndpoint.java:404)
> 4251 at
> org.apache.hadoop.hbase.replication.regionserver.TestReplicator$ReplicationEndpointForTest.replicate(TestReplicator.java:228)
> 4252 at
> org.apache.hadoop.hbase.replication.regionserver.ReplicationSourceShipper.shipEdits(ReplicationSourceShipper.java:188)
> 4253 at
> org.apache.hadoop.hbase.replication.regionserver.ReplicationSourceShipper.run(ReplicationSourceShipper.java:118)
> {code}
> Let me fix this and see if it makes TestReplicator stable again.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HBASE-23812) [Flakey Test] TestReplicator#testReplicatorWithErrors: AssertionError: We did not replicate enough rows expected:<10> but was:<7>
[
https://issues.apache.org/jira/browse/HBASE-23812?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17032777#comment-17032777
]
Nick Dimiduk commented on HBASE-23812:
--
bq. Let me push what I have and leave this open to see if it helps.
Pre-commit here or on a PR isn't good enough to track the problem? You want to
see it pass after repeated builds?
> [Flakey Test] TestReplicator#testReplicatorWithErrors: AssertionError: We did
> not replicate enough rows expected:<10> but was:<7>
> -
>
> Key: HBASE-23812
> URL: https://issues.apache.org/jira/browse/HBASE-23812
> Project: HBase
> Issue Type: Bug
> Components: flakies
>Reporter: Michael Stack
>Assignee: Michael Stack
>Priority: Major
> Fix For: 3.0.0, 2.3.0
>
> Attachments:
> 0001-HBASE-23812-Flakey-Test-TestReplicator-testReplicato.patch
>
>
> I've seen this fail. Here is the latest:
> {code}
> [ERROR] Tests run: 2, Failures: 1, Errors: 0, Skipped: 0, Time elapsed:
> 21.044 s <<< FAILURE! - in
> org.apache.hadoop.hbase.replication.regionserver.TestReplicator
> [ERROR]
> org.apache.hadoop.hbase.replication.regionserver.TestReplicator.testReplicatorWithErrors
> Time elapsed: 7.139 s <<< FAILURE!
> java.lang.AssertionError: We did not replicate enough rows expected:<10> but
> was:<7>
>at
> org.apache.hadoop.hbase.replication.regionserver.TestReplicator.testReplicatorWithErrors(TestReplicator.java:158)
> {code}
> In test logs there are a few (in groups of three) of these:
> {code}
> 4247 2020-02-07 13:50:29,070 WARN
> [RS_REFRESH_PEER-regionserver/localhost:0-1.replicationSource,testReplicatorWithErrors.replicationSource.shipperlocalhost%2C49531%2C1581112194639,testReplicatorWithErrors]
> regionserver. ReplicationSourceShipper(223):
> org.apache.hadoop.hbase.replication.regionserver.TestReplicator$FailureInjectingReplicationEndpointForTest
> threw unknown exception:
> 4248 java.lang.ClassCastException:
> org.apache.hbase.thirdparty.com.google.protobuf.ServiceException cannot be
> cast to java.io.IOException
> 4249 at
> org.apache.hadoop.hbase.replication.regionserver.HBaseInterClusterReplicationEndpoint.parallelReplicate(HBaseInterClusterReplicationEndpoint.java:361)
> 4250 at
> org.apache.hadoop.hbase.replication.regionserver.HBaseInterClusterReplicationEndpoint.replicate(HBaseInterClusterReplicationEndpoint.java:404)
> 4251 at
> org.apache.hadoop.hbase.replication.regionserver.TestReplicator$ReplicationEndpointForTest.replicate(TestReplicator.java:228)
> 4252 at
> org.apache.hadoop.hbase.replication.regionserver.ReplicationSourceShipper.shipEdits(ReplicationSourceShipper.java:188)
> 4253 at
> org.apache.hadoop.hbase.replication.regionserver.ReplicationSourceShipper.run(ReplicationSourceShipper.java:118)
> {code}
> Let me fix this and see if it makes TestReplicator stable again.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HBASE-23812) [Flakey Test] TestReplicator#testReplicatorWithErrors: AssertionError: We did not replicate enough rows expected:<10> but was:<7>
[
https://issues.apache.org/jira/browse/HBASE-23812?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17032724#comment-17032724
]
Michael Stack commented on HBASE-23812:
---
Attached patch pushed on branch-2. Leaving issue open to see if this is the fix.
> [Flakey Test] TestReplicator#testReplicatorWithErrors: AssertionError: We did
> not replicate enough rows expected:<10> but was:<7>
> -
>
> Key: HBASE-23812
> URL: https://issues.apache.org/jira/browse/HBASE-23812
> Project: HBase
> Issue Type: Bug
> Components: flakies
>Reporter: Michael Stack
>Assignee: Michael Stack
>Priority: Major
> Fix For: 3.0.0, 2.3.0
>
> Attachments:
> 0001-HBASE-23812-Flakey-Test-TestReplicator-testReplicato.patch
>
>
> I've seen this fail. Here is the latest:
> {code}
> [ERROR] Tests run: 2, Failures: 1, Errors: 0, Skipped: 0, Time elapsed:
> 21.044 s <<< FAILURE! - in
> org.apache.hadoop.hbase.replication.regionserver.TestReplicator
> [ERROR]
> org.apache.hadoop.hbase.replication.regionserver.TestReplicator.testReplicatorWithErrors
> Time elapsed: 7.139 s <<< FAILURE!
> java.lang.AssertionError: We did not replicate enough rows expected:<10> but
> was:<7>
>at
> org.apache.hadoop.hbase.replication.regionserver.TestReplicator.testReplicatorWithErrors(TestReplicator.java:158)
> {code}
> In test logs there are a few (in groups of three) of these:
> {code}
> 4247 2020-02-07 13:50:29,070 WARN
> [RS_REFRESH_PEER-regionserver/localhost:0-1.replicationSource,testReplicatorWithErrors.replicationSource.shipperlocalhost%2C49531%2C1581112194639,testReplicatorWithErrors]
> regionserver. ReplicationSourceShipper(223):
> org.apache.hadoop.hbase.replication.regionserver.TestReplicator$FailureInjectingReplicationEndpointForTest
> threw unknown exception:
> 4248 java.lang.ClassCastException:
> org.apache.hbase.thirdparty.com.google.protobuf.ServiceException cannot be
> cast to java.io.IOException
> 4249 at
> org.apache.hadoop.hbase.replication.regionserver.HBaseInterClusterReplicationEndpoint.parallelReplicate(HBaseInterClusterReplicationEndpoint.java:361)
> 4250 at
> org.apache.hadoop.hbase.replication.regionserver.HBaseInterClusterReplicationEndpoint.replicate(HBaseInterClusterReplicationEndpoint.java:404)
> 4251 at
> org.apache.hadoop.hbase.replication.regionserver.TestReplicator$ReplicationEndpointForTest.replicate(TestReplicator.java:228)
> 4252 at
> org.apache.hadoop.hbase.replication.regionserver.ReplicationSourceShipper.shipEdits(ReplicationSourceShipper.java:188)
> 4253 at
> org.apache.hadoop.hbase.replication.regionserver.ReplicationSourceShipper.run(ReplicationSourceShipper.java:118)
> {code}
> Let me fix this and see if it makes TestReplicator stable again.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HBASE-23812) [Flakey Test] TestReplicator#testReplicatorWithErrors: AssertionError: We did not replicate enough rows expected:<10> but was:<7>
[
https://issues.apache.org/jira/browse/HBASE-23812?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17032722#comment-17032722
]
Michael Stack commented on HBASE-23812:
---
Fixing the class cast issue, i see these are artificially injected errors. They
are usually handled? Elsewhere? I see in this flakies run that the
ClassCastExceptions are present:
https://builds.apache.org/job/HBase-Flaky-Tests/job/branch-2/5370/testReport/junit/org.apache.hadoop.hbase.replication.regionserver/TestReplicator/testReplicatorWithErrors/
Let me push what I have and leave this open to see if it helps.
> [Flakey Test] TestReplicator#testReplicatorWithErrors: AssertionError: We did
> not replicate enough rows expected:<10> but was:<7>
> -
>
> Key: HBASE-23812
> URL: https://issues.apache.org/jira/browse/HBASE-23812
> Project: HBase
> Issue Type: Bug
> Components: flakies
>Reporter: Michael Stack
>Assignee: Michael Stack
>Priority: Major
> Fix For: 3.0.0, 2.3.0
>
> Attachments:
> 0001-HBASE-23812-Flakey-Test-TestReplicator-testReplicato.patch
>
>
> I've seen this fail. Here is the latest:
> {code}
> [ERROR] Tests run: 2, Failures: 1, Errors: 0, Skipped: 0, Time elapsed:
> 21.044 s <<< FAILURE! - in
> org.apache.hadoop.hbase.replication.regionserver.TestReplicator
> [ERROR]
> org.apache.hadoop.hbase.replication.regionserver.TestReplicator.testReplicatorWithErrors
> Time elapsed: 7.139 s <<< FAILURE!
> java.lang.AssertionError: We did not replicate enough rows expected:<10> but
> was:<7>
>at
> org.apache.hadoop.hbase.replication.regionserver.TestReplicator.testReplicatorWithErrors(TestReplicator.java:158)
> {code}
> In test logs there are a few (in groups of three) of these:
> {code}
> 4247 2020-02-07 13:50:29,070 WARN
> [RS_REFRESH_PEER-regionserver/localhost:0-1.replicationSource,testReplicatorWithErrors.replicationSource.shipperlocalhost%2C49531%2C1581112194639,testReplicatorWithErrors]
> regionserver. ReplicationSourceShipper(223):
> org.apache.hadoop.hbase.replication.regionserver.TestReplicator$FailureInjectingReplicationEndpointForTest
> threw unknown exception:
> 4248 java.lang.ClassCastException:
> org.apache.hbase.thirdparty.com.google.protobuf.ServiceException cannot be
> cast to java.io.IOException
> 4249 at
> org.apache.hadoop.hbase.replication.regionserver.HBaseInterClusterReplicationEndpoint.parallelReplicate(HBaseInterClusterReplicationEndpoint.java:361)
> 4250 at
> org.apache.hadoop.hbase.replication.regionserver.HBaseInterClusterReplicationEndpoint.replicate(HBaseInterClusterReplicationEndpoint.java:404)
> 4251 at
> org.apache.hadoop.hbase.replication.regionserver.TestReplicator$ReplicationEndpointForTest.replicate(TestReplicator.java:228)
> 4252 at
> org.apache.hadoop.hbase.replication.regionserver.ReplicationSourceShipper.shipEdits(ReplicationSourceShipper.java:188)
> 4253 at
> org.apache.hadoop.hbase.replication.regionserver.ReplicationSourceShipper.run(ReplicationSourceShipper.java:118)
> {code}
> Let me fix this and see if it makes TestReplicator stable again.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
