[jira] [Commented] (HBASE-19876) The exception happening in converting pb mutation to hbase.mutation messes up the CellScanner
[
https://issues.apache.org/jira/browse/HBASE-19876?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16363039#comment-16363039
]
Hudson commented on HBASE-19876:
FAILURE: Integrated in Jenkins build HBase-Trunk_matrix #4579 (See
[https://builds.apache.org/job/HBase-Trunk_matrix/4579/])
HBASE-19876 The exception happening in converting pb mutation to (chia7712: rev
2f48fdbb26ff555485b4aa3393d835b7dd8797a0)
* (edit)
hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/RSRpcServices.java
* (edit)
hbase-server/src/test/java/org/apache/hadoop/hbase/client/TestMalformedCellFromClient.java
* (edit)
hbase-client/src/main/java/org/apache/hadoop/hbase/shaded/protobuf/RequestConverter.java
> The exception happening in converting pb mutation to hbase.mutation messes up
> the CellScanner
> -
>
> Key: HBASE-19876
> URL: https://issues.apache.org/jira/browse/HBASE-19876
> Project: HBase
> Issue Type: Bug
>Reporter: Chia-Ping Tsai
>Assignee: Chia-Ping Tsai
>Priority: Critical
> Fix For: 1.3.2, 1.5.0, 1.2.7, 2.0.0-beta-2, 1.4.2
>
> Attachments: HBASE-19876.branch-1.2.v0.patch,
> HBASE-19876.master.001.patch, HBASE-19876.v0.patch, HBASE-19876.v1.patch,
> HBASE-19876.v2.patch, HBASE-19876.v3.patch, HBASE-19876.v3.patch,
> HBASE-19876.v3.patch, HBASE-19876.v3.patch, HBASE-19876.v4.patch,
> HBASE-19876.v5.patch, HBASE-19876.v6.patch
>
>
> {code:java}
> 2018-01-27 22:51:43,794 INFO [hconnection-0x3291b443-shared-pool11-t6]
> client.AsyncRequestFutureImpl(778): id=5, table=testQuotaStatusFromMaster3,
> attempt=6/16 failed=20ops, last
> exception=org.apache.hadoop.hbase.client.WrongRowIOException:
> org.apache.hadoop.hbase.client.WrongRowIOException: The row in xxx doesn't
> match the original one aaa
> at org.apache.hadoop.hbase.client.Mutation.add(Mutation.java:776)
> at org.apache.hadoop.hbase.client.Put.add(Put.java:282)
> at
> org.apache.hadoop.hbase.shaded.protobuf.ProtobufUtil.toPut(ProtobufUtil.java:642)
> at
> org.apache.hadoop.hbase.regionserver.RSRpcServices.doBatchOp(RSRpcServices.java:952)
> at
> org.apache.hadoop.hbase.regionserver.RSRpcServices.doNonAtomicRegionMutation(RSRpcServices.java:896)
> at
> org.apache.hadoop.hbase.regionserver.RSRpcServices.multi(RSRpcServices.java:2591)
> at
> org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$ClientService$2.callBlockingMethod(ClientProtos.java:41560)
> at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:404)
> at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:130)
> at
> org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:324)
> at
> org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:304){code}
> I noticed this bug when testing the table space quota.
> When rs are converting pb mutation to hbase.mutation, the quota exception or
> cell exception may be thrown.
> {code}
> Unable to find source-code formatter for language:
> rsrpcservices#dobatchop.java. Available languages are: actionscript, ada,
> applescript, bash, c, c#, c++, cpp, css, erlang, go, groovy, haskell, html,
> java, javascript, js, json, lua, none, nyan, objc, perl, php, python, r,
> rainbow, ruby, scala, sh, sql, swift, visualbasic, xml, yaml for
> (ClientProtos.Action action: mutations) {
> MutationProto m = action.getMutation();
> Mutation mutation;
> if (m.getMutateType() == MutationType.PUT) {
> mutation = ProtobufUtil.toPut(m, cells);
> batchContainsPuts = true;
> } else {
> mutation = ProtobufUtil.toDelete(m, cells);
> batchContainsDelete = true;
> }
> mutationActionMap.put(mutation, action);
> mArray[i++] = mutation;
> checkCellSizeLimit(region, mutation);
> // Check if a space quota disallows this mutation
> spaceQuotaEnforcement.getPolicyEnforcement(region).check(mutation);
> quota.addMutation(mutation);
> }
> {code}
> rs has caught the exception but it doesn't have the cellscanner skip the
> failed cells.
> {code:java}
> } catch (IOException ie) {
> if (atomic) {
> throw ie;
> }
> for (Action mutation : mutations) {
> builder.addResultOrException(getResultOrException(ie,
> mutation.getIndex()));
> }
> }
> {code}
> The bug results in the WrongRowIOException to remaining mutations since they
> refer to invalid cells.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-19876) The exception happening in converting pb mutation to hbase.mutation messes up the CellScanner
[
https://issues.apache.org/jira/browse/HBASE-19876?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16362410#comment-16362410
]
Hudson commented on HBASE-19876:
SUCCESS: Integrated in Jenkins build HBase-1.2-IT #1068 (See
[https://builds.apache.org/job/HBase-1.2-IT/1068/])
HBASE-19876 The exception happening in converting pb mutation to (chia7712: rev
b59c3151dfe41926d242c5c581a248e272834aa1)
* (add)
hbase-server/src/test/java/org/apache/hadoop/hbase/client/TestMalformedCellFromClient.java
* (edit)
hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/RSRpcServices.java
* (edit)
hbase-client/src/main/java/org/apache/hadoop/hbase/protobuf/RequestConverter.java
> The exception happening in converting pb mutation to hbase.mutation messes up
> the CellScanner
> -
>
> Key: HBASE-19876
> URL: https://issues.apache.org/jira/browse/HBASE-19876
> Project: HBase
> Issue Type: Bug
>Reporter: Chia-Ping Tsai
>Assignee: Chia-Ping Tsai
>Priority: Critical
> Fix For: 1.3.2, 1.5.0, 1.2.7, 2.0.0-beta-2, 1.4.2
>
> Attachments: HBASE-19876.branch-1.2.v0.patch,
> HBASE-19876.master.001.patch, HBASE-19876.v0.patch, HBASE-19876.v1.patch,
> HBASE-19876.v2.patch, HBASE-19876.v3.patch, HBASE-19876.v3.patch,
> HBASE-19876.v3.patch, HBASE-19876.v3.patch, HBASE-19876.v4.patch,
> HBASE-19876.v5.patch, HBASE-19876.v6.patch
>
>
> {code:java}
> 2018-01-27 22:51:43,794 INFO [hconnection-0x3291b443-shared-pool11-t6]
> client.AsyncRequestFutureImpl(778): id=5, table=testQuotaStatusFromMaster3,
> attempt=6/16 failed=20ops, last
> exception=org.apache.hadoop.hbase.client.WrongRowIOException:
> org.apache.hadoop.hbase.client.WrongRowIOException: The row in xxx doesn't
> match the original one aaa
> at org.apache.hadoop.hbase.client.Mutation.add(Mutation.java:776)
> at org.apache.hadoop.hbase.client.Put.add(Put.java:282)
> at
> org.apache.hadoop.hbase.shaded.protobuf.ProtobufUtil.toPut(ProtobufUtil.java:642)
> at
> org.apache.hadoop.hbase.regionserver.RSRpcServices.doBatchOp(RSRpcServices.java:952)
> at
> org.apache.hadoop.hbase.regionserver.RSRpcServices.doNonAtomicRegionMutation(RSRpcServices.java:896)
> at
> org.apache.hadoop.hbase.regionserver.RSRpcServices.multi(RSRpcServices.java:2591)
> at
> org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$ClientService$2.callBlockingMethod(ClientProtos.java:41560)
> at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:404)
> at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:130)
> at
> org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:324)
> at
> org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:304){code}
> I noticed this bug when testing the table space quota.
> When rs are converting pb mutation to hbase.mutation, the quota exception or
> cell exception may be thrown.
> {code}
> Unable to find source-code formatter for language:
> rsrpcservices#dobatchop.java. Available languages are: actionscript, ada,
> applescript, bash, c, c#, c++, cpp, css, erlang, go, groovy, haskell, html,
> java, javascript, js, json, lua, none, nyan, objc, perl, php, python, r,
> rainbow, ruby, scala, sh, sql, swift, visualbasic, xml, yaml for
> (ClientProtos.Action action: mutations) {
> MutationProto m = action.getMutation();
> Mutation mutation;
> if (m.getMutateType() == MutationType.PUT) {
> mutation = ProtobufUtil.toPut(m, cells);
> batchContainsPuts = true;
> } else {
> mutation = ProtobufUtil.toDelete(m, cells);
> batchContainsDelete = true;
> }
> mutationActionMap.put(mutation, action);
> mArray[i++] = mutation;
> checkCellSizeLimit(region, mutation);
> // Check if a space quota disallows this mutation
> spaceQuotaEnforcement.getPolicyEnforcement(region).check(mutation);
> quota.addMutation(mutation);
> }
> {code}
> rs has caught the exception but it doesn't have the cellscanner skip the
> failed cells.
> {code:java}
> } catch (IOException ie) {
> if (atomic) {
> throw ie;
> }
> for (Action mutation : mutations) {
> builder.addResultOrException(getResultOrException(ie,
> mutation.getIndex()));
> }
> }
> {code}
> The bug results in the WrongRowIOException to remaining mutations since they
> refer to invalid cells.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-19876) The exception happening in converting pb mutation to hbase.mutation messes up the CellScanner
[
https://issues.apache.org/jira/browse/HBASE-19876?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16362371#comment-16362371
]
Hudson commented on HBASE-19876:
SUCCESS: Integrated in Jenkins build HBase-1.3-IT #348 (See
[https://builds.apache.org/job/HBase-1.3-IT/348/])
HBASE-19876 The exception happening in converting pb mutation to (chia7712: rev
e36dca55bcd5ad95cd99a19a3630d6c143acfcab)
* (add)
hbase-server/src/test/java/org/apache/hadoop/hbase/client/TestMalformedCellFromClient.java
* (edit)
hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/RSRpcServices.java
* (edit)
hbase-client/src/main/java/org/apache/hadoop/hbase/protobuf/RequestConverter.java
> The exception happening in converting pb mutation to hbase.mutation messes up
> the CellScanner
> -
>
> Key: HBASE-19876
> URL: https://issues.apache.org/jira/browse/HBASE-19876
> Project: HBase
> Issue Type: Bug
>Reporter: Chia-Ping Tsai
>Assignee: Chia-Ping Tsai
>Priority: Critical
> Fix For: 1.3.2, 1.5.0, 1.2.7, 2.0.0-beta-2, 1.4.2
>
> Attachments: HBASE-19876.branch-1.2.v0.patch,
> HBASE-19876.master.001.patch, HBASE-19876.v0.patch, HBASE-19876.v1.patch,
> HBASE-19876.v2.patch, HBASE-19876.v3.patch, HBASE-19876.v3.patch,
> HBASE-19876.v3.patch, HBASE-19876.v3.patch, HBASE-19876.v4.patch,
> HBASE-19876.v5.patch, HBASE-19876.v6.patch
>
>
> {code:java}
> 2018-01-27 22:51:43,794 INFO [hconnection-0x3291b443-shared-pool11-t6]
> client.AsyncRequestFutureImpl(778): id=5, table=testQuotaStatusFromMaster3,
> attempt=6/16 failed=20ops, last
> exception=org.apache.hadoop.hbase.client.WrongRowIOException:
> org.apache.hadoop.hbase.client.WrongRowIOException: The row in xxx doesn't
> match the original one aaa
> at org.apache.hadoop.hbase.client.Mutation.add(Mutation.java:776)
> at org.apache.hadoop.hbase.client.Put.add(Put.java:282)
> at
> org.apache.hadoop.hbase.shaded.protobuf.ProtobufUtil.toPut(ProtobufUtil.java:642)
> at
> org.apache.hadoop.hbase.regionserver.RSRpcServices.doBatchOp(RSRpcServices.java:952)
> at
> org.apache.hadoop.hbase.regionserver.RSRpcServices.doNonAtomicRegionMutation(RSRpcServices.java:896)
> at
> org.apache.hadoop.hbase.regionserver.RSRpcServices.multi(RSRpcServices.java:2591)
> at
> org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$ClientService$2.callBlockingMethod(ClientProtos.java:41560)
> at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:404)
> at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:130)
> at
> org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:324)
> at
> org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:304){code}
> I noticed this bug when testing the table space quota.
> When rs are converting pb mutation to hbase.mutation, the quota exception or
> cell exception may be thrown.
> {code}
> Unable to find source-code formatter for language:
> rsrpcservices#dobatchop.java. Available languages are: actionscript, ada,
> applescript, bash, c, c#, c++, cpp, css, erlang, go, groovy, haskell, html,
> java, javascript, js, json, lua, none, nyan, objc, perl, php, python, r,
> rainbow, ruby, scala, sh, sql, swift, visualbasic, xml, yaml for
> (ClientProtos.Action action: mutations) {
> MutationProto m = action.getMutation();
> Mutation mutation;
> if (m.getMutateType() == MutationType.PUT) {
> mutation = ProtobufUtil.toPut(m, cells);
> batchContainsPuts = true;
> } else {
> mutation = ProtobufUtil.toDelete(m, cells);
> batchContainsDelete = true;
> }
> mutationActionMap.put(mutation, action);
> mArray[i++] = mutation;
> checkCellSizeLimit(region, mutation);
> // Check if a space quota disallows this mutation
> spaceQuotaEnforcement.getPolicyEnforcement(region).check(mutation);
> quota.addMutation(mutation);
> }
> {code}
> rs has caught the exception but it doesn't have the cellscanner skip the
> failed cells.
> {code:java}
> } catch (IOException ie) {
> if (atomic) {
> throw ie;
> }
> for (Action mutation : mutations) {
> builder.addResultOrException(getResultOrException(ie,
> mutation.getIndex()));
> }
> }
> {code}
> The bug results in the WrongRowIOException to remaining mutations since they
> refer to invalid cells.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-19876) The exception happening in converting pb mutation to hbase.mutation messes up the CellScanner
[
https://issues.apache.org/jira/browse/HBASE-19876?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16359624#comment-16359624
]
Chia-Ping Tsai commented on HBASE-19876:
{quote} On commit add in the above line?
{quote}
roger that
> The exception happening in converting pb mutation to hbase.mutation messes up
> the CellScanner
> -
>
> Key: HBASE-19876
> URL: https://issues.apache.org/jira/browse/HBASE-19876
> Project: HBase
> Issue Type: Bug
>Reporter: Chia-Ping Tsai
>Assignee: Chia-Ping Tsai
>Priority: Critical
> Fix For: 1.3.2, 1.5.0, 1.2.7, 2.0.0-beta-2, 1.4.2
>
> Attachments: HBASE-19876.branch-1.2.v0.patch,
> HBASE-19876.master.001.patch, HBASE-19876.v0.patch, HBASE-19876.v1.patch,
> HBASE-19876.v2.patch, HBASE-19876.v3.patch, HBASE-19876.v3.patch,
> HBASE-19876.v3.patch, HBASE-19876.v3.patch, HBASE-19876.v4.patch,
> HBASE-19876.v5.patch, HBASE-19876.v6.patch
>
>
> {code:java}
> 2018-01-27 22:51:43,794 INFO [hconnection-0x3291b443-shared-pool11-t6]
> client.AsyncRequestFutureImpl(778): id=5, table=testQuotaStatusFromMaster3,
> attempt=6/16 failed=20ops, last
> exception=org.apache.hadoop.hbase.client.WrongRowIOException:
> org.apache.hadoop.hbase.client.WrongRowIOException: The row in xxx doesn't
> match the original one aaa
> at org.apache.hadoop.hbase.client.Mutation.add(Mutation.java:776)
> at org.apache.hadoop.hbase.client.Put.add(Put.java:282)
> at
> org.apache.hadoop.hbase.shaded.protobuf.ProtobufUtil.toPut(ProtobufUtil.java:642)
> at
> org.apache.hadoop.hbase.regionserver.RSRpcServices.doBatchOp(RSRpcServices.java:952)
> at
> org.apache.hadoop.hbase.regionserver.RSRpcServices.doNonAtomicRegionMutation(RSRpcServices.java:896)
> at
> org.apache.hadoop.hbase.regionserver.RSRpcServices.multi(RSRpcServices.java:2591)
> at
> org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$ClientService$2.callBlockingMethod(ClientProtos.java:41560)
> at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:404)
> at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:130)
> at
> org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:324)
> at
> org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:304){code}
> I noticed this bug when testing the table space quota.
> When rs are converting pb mutation to hbase.mutation, the quota exception or
> cell exception may be thrown.
> {code}
> Unable to find source-code formatter for language:
> rsrpcservices#dobatchop.java. Available languages are: actionscript, ada,
> applescript, bash, c, c#, c++, cpp, css, erlang, go, groovy, haskell, html,
> java, javascript, js, json, lua, none, nyan, objc, perl, php, python, r,
> rainbow, ruby, scala, sh, sql, swift, visualbasic, xml, yaml for
> (ClientProtos.Action action: mutations) {
> MutationProto m = action.getMutation();
> Mutation mutation;
> if (m.getMutateType() == MutationType.PUT) {
> mutation = ProtobufUtil.toPut(m, cells);
> batchContainsPuts = true;
> } else {
> mutation = ProtobufUtil.toDelete(m, cells);
> batchContainsDelete = true;
> }
> mutationActionMap.put(mutation, action);
> mArray[i++] = mutation;
> checkCellSizeLimit(region, mutation);
> // Check if a space quota disallows this mutation
> spaceQuotaEnforcement.getPolicyEnforcement(region).check(mutation);
> quota.addMutation(mutation);
> }
> {code}
> rs has caught the exception but it doesn't have the cellscanner skip the
> failed cells.
> {code:java}
> } catch (IOException ie) {
> if (atomic) {
> throw ie;
> }
> for (Action mutation : mutations) {
> builder.addResultOrException(getResultOrException(ie,
> mutation.getIndex()));
> }
> }
> {code}
> The bug results in the WrongRowIOException to remaining mutations since they
> refer to invalid cells.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-19876) The exception happening in converting pb mutation to hbase.mutation messes up the CellScanner
[
https://issues.apache.org/jira/browse/HBASE-19876?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16359622#comment-16359622
]
stack commented on HBASE-19876:
---
bq. The non-atomic doBatchOp won't throw IOEThe catch here is weird but it
won't be used. We can do a minor refactor for it later.
Agree. On commit add in the above line?
Otherwise all good. Makes sense that the test be portable but I should make it
so standing up a RegionServer is easy in test; will cut down test times.
+1 on commit Monday.
> The exception happening in converting pb mutation to hbase.mutation messes up
> the CellScanner
> -
>
> Key: HBASE-19876
> URL: https://issues.apache.org/jira/browse/HBASE-19876
> Project: HBase
> Issue Type: Bug
>Reporter: Chia-Ping Tsai
>Assignee: Chia-Ping Tsai
>Priority: Critical
> Fix For: 1.3.2, 1.5.0, 1.2.7, 2.0.0-beta-2, 1.4.2
>
> Attachments: HBASE-19876.branch-1.2.v0.patch,
> HBASE-19876.master.001.patch, HBASE-19876.v0.patch, HBASE-19876.v1.patch,
> HBASE-19876.v2.patch, HBASE-19876.v3.patch, HBASE-19876.v3.patch,
> HBASE-19876.v3.patch, HBASE-19876.v3.patch, HBASE-19876.v4.patch,
> HBASE-19876.v5.patch, HBASE-19876.v6.patch
>
>
> {code:java}
> 2018-01-27 22:51:43,794 INFO [hconnection-0x3291b443-shared-pool11-t6]
> client.AsyncRequestFutureImpl(778): id=5, table=testQuotaStatusFromMaster3,
> attempt=6/16 failed=20ops, last
> exception=org.apache.hadoop.hbase.client.WrongRowIOException:
> org.apache.hadoop.hbase.client.WrongRowIOException: The row in xxx doesn't
> match the original one aaa
> at org.apache.hadoop.hbase.client.Mutation.add(Mutation.java:776)
> at org.apache.hadoop.hbase.client.Put.add(Put.java:282)
> at
> org.apache.hadoop.hbase.shaded.protobuf.ProtobufUtil.toPut(ProtobufUtil.java:642)
> at
> org.apache.hadoop.hbase.regionserver.RSRpcServices.doBatchOp(RSRpcServices.java:952)
> at
> org.apache.hadoop.hbase.regionserver.RSRpcServices.doNonAtomicRegionMutation(RSRpcServices.java:896)
> at
> org.apache.hadoop.hbase.regionserver.RSRpcServices.multi(RSRpcServices.java:2591)
> at
> org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$ClientService$2.callBlockingMethod(ClientProtos.java:41560)
> at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:404)
> at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:130)
> at
> org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:324)
> at
> org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:304){code}
> I noticed this bug when testing the table space quota.
> When rs are converting pb mutation to hbase.mutation, the quota exception or
> cell exception may be thrown.
> {code}
> Unable to find source-code formatter for language:
> rsrpcservices#dobatchop.java. Available languages are: actionscript, ada,
> applescript, bash, c, c#, c++, cpp, css, erlang, go, groovy, haskell, html,
> java, javascript, js, json, lua, none, nyan, objc, perl, php, python, r,
> rainbow, ruby, scala, sh, sql, swift, visualbasic, xml, yaml for
> (ClientProtos.Action action: mutations) {
> MutationProto m = action.getMutation();
> Mutation mutation;
> if (m.getMutateType() == MutationType.PUT) {
> mutation = ProtobufUtil.toPut(m, cells);
> batchContainsPuts = true;
> } else {
> mutation = ProtobufUtil.toDelete(m, cells);
> batchContainsDelete = true;
> }
> mutationActionMap.put(mutation, action);
> mArray[i++] = mutation;
> checkCellSizeLimit(region, mutation);
> // Check if a space quota disallows this mutation
> spaceQuotaEnforcement.getPolicyEnforcement(region).check(mutation);
> quota.addMutation(mutation);
> }
> {code}
> rs has caught the exception but it doesn't have the cellscanner skip the
> failed cells.
> {code:java}
> } catch (IOException ie) {
> if (atomic) {
> throw ie;
> }
> for (Action mutation : mutations) {
> builder.addResultOrException(getResultOrException(ie,
> mutation.getIndex()));
> }
> }
> {code}
> The bug results in the WrongRowIOException to remaining mutations since they
> refer to invalid cells.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-19876) The exception happening in converting pb mutation to hbase.mutation messes up the CellScanner
[
https://issues.apache.org/jira/browse/HBASE-19876?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16359618#comment-16359618
]
Chia-Ping Tsai commented on HBASE-19876:
{quote}It is in a catch for an IOE so ... should we remove the catch block this
comment is in?
{quote}
The non-atomic doBatchOp won't throw IOEThe catch here is weird but it
won't be used. We can do a minor refactor for it later. :)
{quote}I was trying to see if you could get rid of the cycle looking for Gets
in checkAndRowMutate now it has been added to doBatchOp but couldn't figure it;
an optimization for another time (smile).
{quote}
ya, another refactor:(
{quote}On below, I agree.
1011 // The atomic ops use the global exception although I feel it is ok to add
the exception
1012 // to each action.
{quote}
Oh...I forgot to remove the comment. After HBASE-19900, our client can handle
both of region exception and action exception accurately. Only adding the
region exception can reduce the memory consumption (it is similar to
HBASE-19715).
{quote}We need an issue for the below?
{quote}
Agreed. It would be a nice feature for 2.0.
{quote}We need to make it so it is easy to stand up a RegionServer w/o a Master
so you don't have to do the below.
{quote}
The test is necessary for both of branch-1.3 and branch-1.2 as we won't add the
such client APIs to them I guess. By contrast, if the impl can be introduced to
client APIs for branch-1.4+, we can substitute the real use case for the hacky
test.
{quote}+1 to commit.
{quote}
I'm on my travels so I will commit the patch on Monday. Thanks for the reviews
[~stack]
> The exception happening in converting pb mutation to hbase.mutation messes up
> the CellScanner
> -
>
> Key: HBASE-19876
> URL: https://issues.apache.org/jira/browse/HBASE-19876
> Project: HBase
> Issue Type: Bug
>Reporter: Chia-Ping Tsai
>Assignee: Chia-Ping Tsai
>Priority: Critical
> Fix For: 1.3.2, 1.5.0, 1.2.7, 2.0.0-beta-2, 1.4.2
>
> Attachments: HBASE-19876.branch-1.2.v0.patch,
> HBASE-19876.master.001.patch, HBASE-19876.v0.patch, HBASE-19876.v1.patch,
> HBASE-19876.v2.patch, HBASE-19876.v3.patch, HBASE-19876.v3.patch,
> HBASE-19876.v3.patch, HBASE-19876.v3.patch, HBASE-19876.v4.patch,
> HBASE-19876.v5.patch, HBASE-19876.v6.patch
>
>
> {code:java}
> 2018-01-27 22:51:43,794 INFO [hconnection-0x3291b443-shared-pool11-t6]
> client.AsyncRequestFutureImpl(778): id=5, table=testQuotaStatusFromMaster3,
> attempt=6/16 failed=20ops, last
> exception=org.apache.hadoop.hbase.client.WrongRowIOException:
> org.apache.hadoop.hbase.client.WrongRowIOException: The row in xxx doesn't
> match the original one aaa
> at org.apache.hadoop.hbase.client.Mutation.add(Mutation.java:776)
> at org.apache.hadoop.hbase.client.Put.add(Put.java:282)
> at
> org.apache.hadoop.hbase.shaded.protobuf.ProtobufUtil.toPut(ProtobufUtil.java:642)
> at
> org.apache.hadoop.hbase.regionserver.RSRpcServices.doBatchOp(RSRpcServices.java:952)
> at
> org.apache.hadoop.hbase.regionserver.RSRpcServices.doNonAtomicRegionMutation(RSRpcServices.java:896)
> at
> org.apache.hadoop.hbase.regionserver.RSRpcServices.multi(RSRpcServices.java:2591)
> at
> org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$ClientService$2.callBlockingMethod(ClientProtos.java:41560)
> at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:404)
> at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:130)
> at
> org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:324)
> at
> org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:304){code}
> I noticed this bug when testing the table space quota.
> When rs are converting pb mutation to hbase.mutation, the quota exception or
> cell exception may be thrown.
> {code}
> Unable to find source-code formatter for language:
> rsrpcservices#dobatchop.java. Available languages are: actionscript, ada,
> applescript, bash, c, c#, c++, cpp, css, erlang, go, groovy, haskell, html,
> java, javascript, js, json, lua, none, nyan, objc, perl, php, python, r,
> rainbow, ruby, scala, sh, sql, swift, visualbasic, xml, yaml for
> (ClientProtos.Action action: mutations) {
> MutationProto m = action.getMutation();
> Mutation mutation;
> if (m.getMutateType() == MutationType.PUT) {
> mutation = ProtobufUtil.toPut(m, cells);
> batchContainsPuts = true;
> } else {
> mutation = ProtobufUtil.toDelete(m, cells);
> batchContainsDelete = true;
> }
> mutationActionMap.put(mutation, action);
> mArray[i++] = mutation;
> checkCellSizeLimit
[jira] [Commented] (HBASE-19876) The exception happening in converting pb mutation to hbase.mutation messes up the CellScanner
[
https://issues.apache.org/jira/browse/HBASE-19876?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16359563#comment-16359563
]
stack commented on HBASE-19876:
---
Took a look. This is tricky stuff. Thanks for diving in.
This comment is a little confusing:
890 // TODO doBatchOp has handled the IOE for all non-atomic
operations
891 // Catching IOE here may confuse readers in the future
It is in a catch for an IOE so ... should we remove the catch block this
comment is in?
I was trying to see if you could get rid of the cycle looking for Gets in
checkAndRowMutate now it has been added to doBatchOp but couldn't figure it; an
optimization for another time (smile).
On below, I agree.
1011// The atomic ops use the global exception although I feel it
is ok to add the exception
1012// to each action.
We need an issue for the below?
184* The invalid cells is in rm. The rm should fail but the subsequent
mutations should succeed.
185* Currently, we have no client api to submit the request consisting
of condition-rm and mutation.
186* Hence, this test build the request manually.
We need to make it so it is easy to stand up a RegionServer w/o a Master so you
don't have to do the below. This should be easier now... since HBASE-18846.
Let me work on it
218 ClientProtos.MultiResponse response =
rs.getRSRpcServices().multi(controller, request);
Great tests [~chia7712]
This patch is super important stuff.
+1 to commit.
> The exception happening in converting pb mutation to hbase.mutation messes up
> the CellScanner
> -
>
> Key: HBASE-19876
> URL: https://issues.apache.org/jira/browse/HBASE-19876
> Project: HBase
> Issue Type: Bug
>Reporter: Chia-Ping Tsai
>Assignee: Chia-Ping Tsai
>Priority: Critical
> Fix For: 1.3.2, 1.5.0, 1.2.7, 2.0.0-beta-2, 1.4.2
>
> Attachments: HBASE-19876.branch-1.2.v0.patch,
> HBASE-19876.master.001.patch, HBASE-19876.v0.patch, HBASE-19876.v1.patch,
> HBASE-19876.v2.patch, HBASE-19876.v3.patch, HBASE-19876.v3.patch,
> HBASE-19876.v3.patch, HBASE-19876.v3.patch, HBASE-19876.v4.patch,
> HBASE-19876.v5.patch, HBASE-19876.v6.patch
>
>
> {code:java}
> 2018-01-27 22:51:43,794 INFO [hconnection-0x3291b443-shared-pool11-t6]
> client.AsyncRequestFutureImpl(778): id=5, table=testQuotaStatusFromMaster3,
> attempt=6/16 failed=20ops, last
> exception=org.apache.hadoop.hbase.client.WrongRowIOException:
> org.apache.hadoop.hbase.client.WrongRowIOException: The row in xxx doesn't
> match the original one aaa
> at org.apache.hadoop.hbase.client.Mutation.add(Mutation.java:776)
> at org.apache.hadoop.hbase.client.Put.add(Put.java:282)
> at
> org.apache.hadoop.hbase.shaded.protobuf.ProtobufUtil.toPut(ProtobufUtil.java:642)
> at
> org.apache.hadoop.hbase.regionserver.RSRpcServices.doBatchOp(RSRpcServices.java:952)
> at
> org.apache.hadoop.hbase.regionserver.RSRpcServices.doNonAtomicRegionMutation(RSRpcServices.java:896)
> at
> org.apache.hadoop.hbase.regionserver.RSRpcServices.multi(RSRpcServices.java:2591)
> at
> org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$ClientService$2.callBlockingMethod(ClientProtos.java:41560)
> at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:404)
> at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:130)
> at
> org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:324)
> at
> org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:304){code}
> I noticed this bug when testing the table space quota.
> When rs are converting pb mutation to hbase.mutation, the quota exception or
> cell exception may be thrown.
> {code}
> Unable to find source-code formatter for language:
> rsrpcservices#dobatchop.java. Available languages are: actionscript, ada,
> applescript, bash, c, c#, c++, cpp, css, erlang, go, groovy, haskell, html,
> java, javascript, js, json, lua, none, nyan, objc, perl, php, python, r,
> rainbow, ruby, scala, sh, sql, swift, visualbasic, xml, yaml for
> (ClientProtos.Action action: mutations) {
> MutationProto m = action.getMutation();
> Mutation mutation;
> if (m.getMutateType() == MutationType.PUT) {
> mutation = ProtobufUtil.toPut(m, cells);
> batchContainsPuts = true;
> } else {
> mutation = ProtobufUtil.toDelete(m, cells);
> batchContainsDelete = true;
> }
> mutationActionMap.put(mutation, action);
> mArray[i++] = mutation;
> checkCellSizeLimit(region, mutation);
> // Check if a space quota disallows this mutation
> spaceQuo
[jira] [Commented] (HBASE-19876) The exception happening in converting pb mutation to hbase.mutation messes up the CellScanner
[
https://issues.apache.org/jira/browse/HBASE-19876?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16359276#comment-16359276
]
Hadoop QA commented on HBASE-19876:
---
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
15s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
0s{color} | {color:blue} Findbugs executables are not available. {color} |
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s{color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 1 new or modified test
files. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
21s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 4m
19s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
1s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
36s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 6m
17s{color} | {color:green} branch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
47s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
13s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 4m
22s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
2s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m
2s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
41s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 4m
49s{color} | {color:green} patch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
19m 14s{color} | {color:green} Patch does not cause any errors with Hadoop
2.6.5 2.7.4 or 3.0.0. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
55s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 3m
1s{color} | {color:green} hbase-client in the patch passed. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green}115m
15s{color} | {color:green} hbase-server in the patch passed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
51s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black}160m 1s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hbase:eee3b01 |
| JIRA Issue | HBASE-19876 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12910018/HBASE-19876.v6.patch |
| Optional Tests | asflicense javac javadoc unit findbugs shadedjars
hadoopcheck hbaseanti checkstyle compile |
| uname | Linux 7387a76c9d23 3.13.0-139-generic #188-Ubuntu SMP Tue Jan 9
14:43:09 UTC 2018 x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/component/dev-support/hbase-personality.sh
|
| git revision | master / 32f235abee |
| maven | version: Apache Maven 3.5.2
(138edd61fd100ec658bfa2d307c43b76940a5d7d; 2017-10-18T07:58:13Z) |
| Default Java | 1.8.0_151 |
| Test Results |
https://builds.apache.org/job/PreCommit-HBASE-Build/11473/testReport/ |
| Max. process+thread c
[jira] [Commented] (HBASE-19876) The exception happening in converting pb mutation to hbase.mutation messes up the CellScanner
[
https://issues.apache.org/jira/browse/HBASE-19876?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16359225#comment-16359225
]
Chia-Ping Tsai commented on HBASE-19876:
The failed tests pass locally...Try the patch for master. Ping for reviews also.
> The exception happening in converting pb mutation to hbase.mutation messes up
> the CellScanner
> -
>
> Key: HBASE-19876
> URL: https://issues.apache.org/jira/browse/HBASE-19876
> Project: HBase
> Issue Type: Bug
>Reporter: Chia-Ping Tsai
>Assignee: Chia-Ping Tsai
>Priority: Critical
> Fix For: 1.3.2, 1.5.0, 1.2.7, 2.0.0-beta-2, 1.4.2
>
> Attachments: HBASE-19876.branch-1.2.v0.patch,
> HBASE-19876.master.001.patch, HBASE-19876.v0.patch, HBASE-19876.v1.patch,
> HBASE-19876.v2.patch, HBASE-19876.v3.patch, HBASE-19876.v3.patch,
> HBASE-19876.v3.patch, HBASE-19876.v3.patch, HBASE-19876.v4.patch,
> HBASE-19876.v5.patch
>
>
> {code:java}
> 2018-01-27 22:51:43,794 INFO [hconnection-0x3291b443-shared-pool11-t6]
> client.AsyncRequestFutureImpl(778): id=5, table=testQuotaStatusFromMaster3,
> attempt=6/16 failed=20ops, last
> exception=org.apache.hadoop.hbase.client.WrongRowIOException:
> org.apache.hadoop.hbase.client.WrongRowIOException: The row in xxx doesn't
> match the original one aaa
> at org.apache.hadoop.hbase.client.Mutation.add(Mutation.java:776)
> at org.apache.hadoop.hbase.client.Put.add(Put.java:282)
> at
> org.apache.hadoop.hbase.shaded.protobuf.ProtobufUtil.toPut(ProtobufUtil.java:642)
> at
> org.apache.hadoop.hbase.regionserver.RSRpcServices.doBatchOp(RSRpcServices.java:952)
> at
> org.apache.hadoop.hbase.regionserver.RSRpcServices.doNonAtomicRegionMutation(RSRpcServices.java:896)
> at
> org.apache.hadoop.hbase.regionserver.RSRpcServices.multi(RSRpcServices.java:2591)
> at
> org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$ClientService$2.callBlockingMethod(ClientProtos.java:41560)
> at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:404)
> at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:130)
> at
> org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:324)
> at
> org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:304){code}
> I noticed this bug when testing the table space quota.
> When rs are converting pb mutation to hbase.mutation, the quota exception or
> cell exception may be thrown.
> {code}
> Unable to find source-code formatter for language:
> rsrpcservices#dobatchop.java. Available languages are: actionscript, ada,
> applescript, bash, c, c#, c++, cpp, css, erlang, go, groovy, haskell, html,
> java, javascript, js, json, lua, none, nyan, objc, perl, php, python, r,
> rainbow, ruby, scala, sh, sql, swift, visualbasic, xml, yaml for
> (ClientProtos.Action action: mutations) {
> MutationProto m = action.getMutation();
> Mutation mutation;
> if (m.getMutateType() == MutationType.PUT) {
> mutation = ProtobufUtil.toPut(m, cells);
> batchContainsPuts = true;
> } else {
> mutation = ProtobufUtil.toDelete(m, cells);
> batchContainsDelete = true;
> }
> mutationActionMap.put(mutation, action);
> mArray[i++] = mutation;
> checkCellSizeLimit(region, mutation);
> // Check if a space quota disallows this mutation
> spaceQuotaEnforcement.getPolicyEnforcement(region).check(mutation);
> quota.addMutation(mutation);
> }
> {code}
> rs has caught the exception but it doesn't have the cellscanner skip the
> failed cells.
> {code:java}
> } catch (IOException ie) {
> if (atomic) {
> throw ie;
> }
> for (Action mutation : mutations) {
> builder.addResultOrException(getResultOrException(ie,
> mutation.getIndex()));
> }
> }
> {code}
> The bug results in the WrongRowIOException to remaining mutations since they
> refer to invalid cells.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-19876) The exception happening in converting pb mutation to hbase.mutation messes up the CellScanner
[
https://issues.apache.org/jira/browse/HBASE-19876?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16358840#comment-16358840
]
Hadoop QA commented on HBASE-19876:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 13m
52s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s{color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 1 new or modified test
files. {color} |
|| || || || {color:brown} branch-1.2 Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 1m
23s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 7m
51s{color} | {color:green} branch-1.2 passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
14s{color} | {color:green} branch-1.2 passed with JDK v1.8.0_162 {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
11s{color} | {color:green} branch-1.2 passed with JDK v1.7.0_171 {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 2m
15s{color} | {color:green} branch-1.2 passed {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 5m
18s{color} | {color:green} branch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 3m
46s{color} | {color:green} branch-1.2 passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
11s{color} | {color:green} branch-1.2 passed with JDK v1.8.0_162 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
17s{color} | {color:green} branch-1.2 passed with JDK v1.7.0_171 {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
14s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
45s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
57s{color} | {color:green} the patch passed with JDK v1.8.0_162 {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
57s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
50s{color} | {color:green} the patch passed with JDK v1.7.0_171 {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
50s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
30s{color} | {color:green} The patch hbase-client passed checkstyle {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
15s{color} | {color:green} hbase-server: The patch generated 0 new + 135
unchanged - 1 fixed = 135 total (was 136) {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 2m
27s{color} | {color:green} patch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
8m 12s{color} | {color:green} Patch does not cause any errors with Hadoop 2.4.1
2.5.2 2.6.5 2.7.4. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 3m
16s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
40s{color} | {color:green} the patch passed with JDK v1.8.0_162 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
55s{color} | {color:green} the patch passed with JDK v1.7.0_171 {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 1m
53s{color} | {color:green} hbase-client in the patch passed. {color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 86m 27s{color}
| {color:red} hbase-server in the patch failed. {color} |
| {colo
[jira] [Commented] (HBASE-19876) The exception happening in converting pb mutation to hbase.mutation messes up the CellScanner
[
https://issues.apache.org/jira/browse/HBASE-19876?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16357552#comment-16357552
]
Hadoop QA commented on HBASE-19876:
---
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
18s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
0s{color} | {color:blue} Findbugs executables are not available. {color} |
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s{color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 1 new or modified test
files. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 4m
35s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
42s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
6s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 5m
49s{color} | {color:green} branch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
27s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 4m
24s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
41s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
41s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
6s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 4m
42s{color} | {color:green} patch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
18m 15s{color} | {color:green} Patch does not cause any errors with Hadoop
2.6.5 2.7.4 or 3.0.0. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
26s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} unit {color} | {color:green}115m
6s{color} | {color:green} hbase-server in the patch passed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
19s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black}152m 21s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hbase:eee3b01 |
| JIRA Issue | HBASE-19876 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12909826/HBASE-19876.v5.patch |
| Optional Tests | asflicense javac javadoc unit findbugs shadedjars
hadoopcheck hbaseanti checkstyle compile |
| uname | Linux e3a1d3476825 3.13.0-135-generic #184-Ubuntu SMP Wed Oct 18
11:55:51 UTC 2017 x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/component/dev-support/hbase-personality.sh
|
| git revision | master / 277ce3d8a3 |
| maven | version: Apache Maven 3.5.2
(138edd61fd100ec658bfa2d307c43b76940a5d7d; 2017-10-18T07:58:13Z) |
| Default Java | 1.8.0_151 |
| Test Results |
https://builds.apache.org/job/PreCommit-HBASE-Build/11455/testReport/ |
| Max. process+thread count | 5748 (vs. ulimit of 1) |
| modules | C: hbase-server U: hbase-server |
| Console output |
https://builds.apache.org/job/PreCommit-HBASE-Build/11455/console |
| Powered by | Apache Yetus 0.7.0 http://yetus.apache.org |
This message was automatically generated.
> The exception happening in converting pb mutation to hbase.mutation messes up
> the CellScanner
>
[jira] [Commented] (HBASE-19876) The exception happening in converting pb mutation to hbase.mutation messes up the CellScanner
[
https://issues.apache.org/jira/browse/HBASE-19876?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16356151#comment-16356151
]
Hadoop QA commented on HBASE-19876:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
16s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
0s{color} | {color:blue} Findbugs executables are not available. {color} |
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s{color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 1 new or modified test
files. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 3m
58s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
38s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
56s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 5m
8s{color} | {color:green} branch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
25s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 4m
2s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
38s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
38s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m
58s{color} | {color:red} hbase-server: The patch generated 2 new + 130
unchanged - 0 fixed = 132 total (was 130) {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 4m
10s{color} | {color:green} patch has no errors when building our shaded
downstream artifacts. {color} |
| {color:red}-1{color} | {color:red} hadoopcheck {color} | {color:red} 6m
10s{color} | {color:red} The patch causes 10 errors with Hadoop v2.6.5. {color}
|
| {color:red}-1{color} | {color:red} hadoopcheck {color} | {color:red} 8m
8s{color} | {color:red} The patch causes 10 errors with Hadoop v2.7.4. {color} |
| {color:red}-1{color} | {color:red} hadoopcheck {color} | {color:red} 10m
16s{color} | {color:red} The patch causes 10 errors with Hadoop v3.0.0. {color}
|
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
25s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 93m
3s{color} | {color:green} hbase-server in the patch passed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
15s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black}120m 15s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hbase:eee3b01 |
| JIRA Issue | HBASE-19876 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12909662/HBASE-19876.v4.patch |
| Optional Tests | asflicense javac javadoc unit findbugs shadedjars
hadoopcheck hbaseanti checkstyle compile |
| uname | Linux afc2078b2513 4.4.0-43-generic #63-Ubuntu SMP Wed Oct 12
13:48:03 UTC 2016 x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/component/dev-support/hbase-personality.sh
|
| git revision | master / d95940776e |
| maven | version: Apache Maven 3.5.2
(138edd61fd100ec658bfa2d307c43b76940a5d7d; 2017-10-18T07:58:13Z) |
| Default Java | 1.8.0_151 |
| checkstyle |
https://builds.apache.org/job/PreCommit-HBASE-Build/11438/artifact/patchprocess/diff-checkstyle-hbase-server.txt
|
| Test Results |
https://builds.apache.org/job/PreCommit-HBASE-Buil
[jira] [Commented] (HBASE-19876) The exception happening in converting pb mutation to hbase.mutation messes up the CellScanner
[
https://issues.apache.org/jira/browse/HBASE-19876?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16350908#comment-16350908
]
Hadoop QA commented on HBASE-19876:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 2m
5s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
1s{color} | {color:blue} Findbugs executables are not available. {color} |
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s{color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:red}-1{color} | {color:red} test4tests {color} | {color:red} 0m
0s{color} | {color:red} The patch doesn't appear to include any new or modified
tests. Please justify why no new tests are needed for this patch. Also please
list what manual steps were performed to verify this patch. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 4m
23s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
43s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
7s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 5m
47s{color} | {color:green} branch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
29s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 4m
21s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
41s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
41s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
8s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 4m
44s{color} | {color:green} patch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
18m 14s{color} | {color:green} Patch does not cause any errors with Hadoop
2.6.5 2.7.4 or 3.0.0. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
27s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:red}-1{color} | {color:red} unit {color} | {color:red}102m 32s{color}
| {color:red} hbase-server in the patch failed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
19s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black}141m 26s{color} |
{color:black} {color} |
\\
\\
|| Reason || Tests ||
| Failed junit tests | hadoop.hbase.replication.TestReplicationDroppedTables |
| | hadoop.hbase.TestFullLogReconstruction |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hbase:eee3b01 |
| JIRA Issue | HBASE-19876 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12909023/HBASE-19876.master.001.patch
|
| Optional Tests | asflicense javac javadoc unit findbugs shadedjars
hadoopcheck hbaseanti checkstyle compile |
| uname | Linux edd281aedb18 3.13.0-139-generic #188-Ubuntu SMP Tue Jan 9
14:43:09 UTC 2018 x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build@2/component/dev-support/hbase-personality.sh
|
| git revision | master / 8143d5afa4 |
| maven | version: Apache Maven 3.5.2
(138edd61fd100ec658bfa2d307c43b76940a5d7d; 2017-10-18T07:58:13Z) |
| Default Java | 1.8.0_151 |
| unit |
https://builds.apache.org/job/PreCommit-HBASE-Build/11363/artifact/patchprocess/patch-unit-hbase-server.txt
|
| Test Results |
https://builds.apache.org/job/PreCommit-HBASE-Build/11363/testReport/ |
| Max. process+thread count | 4797 (vs. ulimit of 1)
[jira] [Commented] (HBASE-19876) The exception happening in converting pb mutation to hbase.mutation messes up the CellScanner
[
https://issues.apache.org/jira/browse/HBASE-19876?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16350734#comment-16350734
]
Chia-Ping Tsai commented on HBASE-19876:
Thanks [~stack]. I forget to say this issue is blocked by HBASE-19900. If
HBASE-19900 is not resolved, the tests in the patch are unstable because of the
region-level exception.
> The exception happening in converting pb mutation to hbase.mutation messes up
> the CellScanner
> -
>
> Key: HBASE-19876
> URL: https://issues.apache.org/jira/browse/HBASE-19876
> Project: HBase
> Issue Type: Bug
>Reporter: Chia-Ping Tsai
>Assignee: Chia-Ping Tsai
>Priority: Critical
> Fix For: 1.3.2, 1.5.0, 1.2.7, 2.0.0-beta-2, 1.4.2
>
> Attachments: HBASE-19876.master.001.patch, HBASE-19876.v0.patch,
> HBASE-19876.v1.patch, HBASE-19876.v2.patch, HBASE-19876.v3.patch,
> HBASE-19876.v3.patch, HBASE-19876.v3.patch, HBASE-19876.v3.patch
>
>
> {code:java}
> 2018-01-27 22:51:43,794 INFO [hconnection-0x3291b443-shared-pool11-t6]
> client.AsyncRequestFutureImpl(778): id=5, table=testQuotaStatusFromMaster3,
> attempt=6/16 failed=20ops, last
> exception=org.apache.hadoop.hbase.client.WrongRowIOException:
> org.apache.hadoop.hbase.client.WrongRowIOException: The row in xxx doesn't
> match the original one aaa
> at org.apache.hadoop.hbase.client.Mutation.add(Mutation.java:776)
> at org.apache.hadoop.hbase.client.Put.add(Put.java:282)
> at
> org.apache.hadoop.hbase.shaded.protobuf.ProtobufUtil.toPut(ProtobufUtil.java:642)
> at
> org.apache.hadoop.hbase.regionserver.RSRpcServices.doBatchOp(RSRpcServices.java:952)
> at
> org.apache.hadoop.hbase.regionserver.RSRpcServices.doNonAtomicRegionMutation(RSRpcServices.java:896)
> at
> org.apache.hadoop.hbase.regionserver.RSRpcServices.multi(RSRpcServices.java:2591)
> at
> org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$ClientService$2.callBlockingMethod(ClientProtos.java:41560)
> at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:404)
> at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:130)
> at
> org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:324)
> at
> org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:304){code}
> I noticed this bug when testing the table space quota.
> When rs are converting pb mutation to hbase.mutation, the quota exception or
> cell exception may be thrown.
> {code}
> Unable to find source-code formatter for language:
> rsrpcservices#dobatchop.java. Available languages are: actionscript, ada,
> applescript, bash, c, c#, c++, cpp, css, erlang, go, groovy, haskell, html,
> java, javascript, js, json, lua, none, nyan, objc, perl, php, python, r,
> rainbow, ruby, scala, sh, sql, swift, visualbasic, xml, yaml for
> (ClientProtos.Action action: mutations) {
> MutationProto m = action.getMutation();
> Mutation mutation;
> if (m.getMutateType() == MutationType.PUT) {
> mutation = ProtobufUtil.toPut(m, cells);
> batchContainsPuts = true;
> } else {
> mutation = ProtobufUtil.toDelete(m, cells);
> batchContainsDelete = true;
> }
> mutationActionMap.put(mutation, action);
> mArray[i++] = mutation;
> checkCellSizeLimit(region, mutation);
> // Check if a space quota disallows this mutation
> spaceQuotaEnforcement.getPolicyEnforcement(region).check(mutation);
> quota.addMutation(mutation);
> }
> {code}
> rs has caught the exception but it doesn't have the cellscanner skip the
> failed cells.
> {code:java}
> } catch (IOException ie) {
> if (atomic) {
> throw ie;
> }
> for (Action mutation : mutations) {
> builder.addResultOrException(getResultOrException(ie,
> mutation.getIndex()));
> }
> }
> {code}
> The bug results in the WrongRowIOException to remaining mutations since they
> refer to invalid cells.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-19876) The exception happening in converting pb mutation to hbase.mutation messes up the CellScanner
[
https://issues.apache.org/jira/browse/HBASE-19876?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16350730#comment-16350730
]
stack commented on HBASE-19876:
---
.001 rebase of [~chia7712] patch.
> The exception happening in converting pb mutation to hbase.mutation messes up
> the CellScanner
> -
>
> Key: HBASE-19876
> URL: https://issues.apache.org/jira/browse/HBASE-19876
> Project: HBase
> Issue Type: Bug
>Reporter: Chia-Ping Tsai
>Assignee: Chia-Ping Tsai
>Priority: Critical
> Fix For: 1.3.2, 1.5.0, 1.2.7, 2.0.0-beta-2, 1.4.2
>
> Attachments: HBASE-19876.master.001.patch, HBASE-19876.v0.patch,
> HBASE-19876.v1.patch, HBASE-19876.v2.patch, HBASE-19876.v3.patch,
> HBASE-19876.v3.patch, HBASE-19876.v3.patch, HBASE-19876.v3.patch
>
>
> {code:java}
> 2018-01-27 22:51:43,794 INFO [hconnection-0x3291b443-shared-pool11-t6]
> client.AsyncRequestFutureImpl(778): id=5, table=testQuotaStatusFromMaster3,
> attempt=6/16 failed=20ops, last
> exception=org.apache.hadoop.hbase.client.WrongRowIOException:
> org.apache.hadoop.hbase.client.WrongRowIOException: The row in xxx doesn't
> match the original one aaa
> at org.apache.hadoop.hbase.client.Mutation.add(Mutation.java:776)
> at org.apache.hadoop.hbase.client.Put.add(Put.java:282)
> at
> org.apache.hadoop.hbase.shaded.protobuf.ProtobufUtil.toPut(ProtobufUtil.java:642)
> at
> org.apache.hadoop.hbase.regionserver.RSRpcServices.doBatchOp(RSRpcServices.java:952)
> at
> org.apache.hadoop.hbase.regionserver.RSRpcServices.doNonAtomicRegionMutation(RSRpcServices.java:896)
> at
> org.apache.hadoop.hbase.regionserver.RSRpcServices.multi(RSRpcServices.java:2591)
> at
> org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$ClientService$2.callBlockingMethod(ClientProtos.java:41560)
> at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:404)
> at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:130)
> at
> org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:324)
> at
> org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:304){code}
> I noticed this bug when testing the table space quota.
> When rs are converting pb mutation to hbase.mutation, the quota exception or
> cell exception may be thrown.
> {code}
> Unable to find source-code formatter for language:
> rsrpcservices#dobatchop.java. Available languages are: actionscript, ada,
> applescript, bash, c, c#, c++, cpp, css, erlang, go, groovy, haskell, html,
> java, javascript, js, json, lua, none, nyan, objc, perl, php, python, r,
> rainbow, ruby, scala, sh, sql, swift, visualbasic, xml, yaml for
> (ClientProtos.Action action: mutations) {
> MutationProto m = action.getMutation();
> Mutation mutation;
> if (m.getMutateType() == MutationType.PUT) {
> mutation = ProtobufUtil.toPut(m, cells);
> batchContainsPuts = true;
> } else {
> mutation = ProtobufUtil.toDelete(m, cells);
> batchContainsDelete = true;
> }
> mutationActionMap.put(mutation, action);
> mArray[i++] = mutation;
> checkCellSizeLimit(region, mutation);
> // Check if a space quota disallows this mutation
> spaceQuotaEnforcement.getPolicyEnforcement(region).check(mutation);
> quota.addMutation(mutation);
> }
> {code}
> rs has caught the exception but it doesn't have the cellscanner skip the
> failed cells.
> {code:java}
> } catch (IOException ie) {
> if (atomic) {
> throw ie;
> }
> for (Action mutation : mutations) {
> builder.addResultOrException(getResultOrException(ie,
> mutation.getIndex()));
> }
> }
> {code}
> The bug results in the WrongRowIOException to remaining mutations since they
> refer to invalid cells.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-19876) The exception happening in converting pb mutation to hbase.mutation messes up the CellScanner
[
https://issues.apache.org/jira/browse/HBASE-19876?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16350174#comment-16350174
]
Hadoop QA commented on HBASE-19876:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
0s{color} | {color:blue} Docker mode activated. {color} |
| {color:red}-1{color} | {color:red} patch {color} | {color:red} 0m 4s{color}
| {color:red} HBASE-19876 does not apply to master. Rebase required? Wrong
Branch? See https://yetus.apache.org/documentation/0.7.0/precommit-patchnames
for help. {color} |
\\
\\
|| Subsystem || Report/Notes ||
| JIRA Issue | HBASE-19876 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12908964/HBASE-19876.v3.patch |
| Console output |
https://builds.apache.org/job/PreCommit-HBASE-Build/11360/console |
| Powered by | Apache Yetus 0.7.0 http://yetus.apache.org |
This message was automatically generated.
> The exception happening in converting pb mutation to hbase.mutation messes up
> the CellScanner
> -
>
> Key: HBASE-19876
> URL: https://issues.apache.org/jira/browse/HBASE-19876
> Project: HBase
> Issue Type: Bug
>Reporter: Chia-Ping Tsai
>Assignee: Chia-Ping Tsai
>Priority: Critical
> Fix For: 1.3.2, 1.5.0, 1.2.7, 2.0.0-beta-2, 1.4.2
>
> Attachments: HBASE-19876.v0.patch, HBASE-19876.v1.patch,
> HBASE-19876.v2.patch, HBASE-19876.v3.patch, HBASE-19876.v3.patch,
> HBASE-19876.v3.patch, HBASE-19876.v3.patch
>
>
> {code:java}
> 2018-01-27 22:51:43,794 INFO [hconnection-0x3291b443-shared-pool11-t6]
> client.AsyncRequestFutureImpl(778): id=5, table=testQuotaStatusFromMaster3,
> attempt=6/16 failed=20ops, last
> exception=org.apache.hadoop.hbase.client.WrongRowIOException:
> org.apache.hadoop.hbase.client.WrongRowIOException: The row in xxx doesn't
> match the original one aaa
> at org.apache.hadoop.hbase.client.Mutation.add(Mutation.java:776)
> at org.apache.hadoop.hbase.client.Put.add(Put.java:282)
> at
> org.apache.hadoop.hbase.shaded.protobuf.ProtobufUtil.toPut(ProtobufUtil.java:642)
> at
> org.apache.hadoop.hbase.regionserver.RSRpcServices.doBatchOp(RSRpcServices.java:952)
> at
> org.apache.hadoop.hbase.regionserver.RSRpcServices.doNonAtomicRegionMutation(RSRpcServices.java:896)
> at
> org.apache.hadoop.hbase.regionserver.RSRpcServices.multi(RSRpcServices.java:2591)
> at
> org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$ClientService$2.callBlockingMethod(ClientProtos.java:41560)
> at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:404)
> at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:130)
> at
> org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:324)
> at
> org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:304){code}
> I noticed this bug when testing the table space quota.
> When rs are converting pb mutation to hbase.mutation, the quota exception or
> cell exception may be thrown.
> {code}
> Unable to find source-code formatter for language:
> rsrpcservices#dobatchop.java. Available languages are: actionscript, ada,
> applescript, bash, c, c#, c++, cpp, css, erlang, go, groovy, haskell, html,
> java, javascript, js, json, lua, none, nyan, objc, perl, php, python, r,
> rainbow, ruby, scala, sh, sql, swift, visualbasic, xml, yaml for
> (ClientProtos.Action action: mutations) {
> MutationProto m = action.getMutation();
> Mutation mutation;
> if (m.getMutateType() == MutationType.PUT) {
> mutation = ProtobufUtil.toPut(m, cells);
> batchContainsPuts = true;
> } else {
> mutation = ProtobufUtil.toDelete(m, cells);
> batchContainsDelete = true;
> }
> mutationActionMap.put(mutation, action);
> mArray[i++] = mutation;
> checkCellSizeLimit(region, mutation);
> // Check if a space quota disallows this mutation
> spaceQuotaEnforcement.getPolicyEnforcement(region).check(mutation);
> quota.addMutation(mutation);
> }
> {code}
> rs has caught the exception but it doesn't have the cellscanner skip the
> failed cells.
> {code:java}
> } catch (IOException ie) {
> if (atomic) {
> throw ie;
> }
> for (Action mutation : mutations) {
> builder.addResultOrException(getResultOrException(ie,
> mutation.getIndex()));
> }
> }
> {code}
> The bug results in the WrongRowIOException to remaining mutations since they
> refer to invalid cells.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-19876) The exception happening in converting pb mutation to hbase.mutation messes up the CellScanner
[
https://issues.apache.org/jira/browse/HBASE-19876?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16349332#comment-16349332
]
Hadoop QA commented on HBASE-19876:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
0s{color} | {color:blue} Docker mode activated. {color} |
| {color:red}-1{color} | {color:red} patch {color} | {color:red} 0m 4s{color}
| {color:red} HBASE-19876 does not apply to master. Rebase required? Wrong
Branch? See https://yetus.apache.org/documentation/0.7.0/precommit-patchnames
for help. {color} |
\\
\\
|| Subsystem || Report/Notes ||
| JIRA Issue | HBASE-19876 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12908874/HBASE-19876.v3.patch |
| Console output |
https://builds.apache.org/job/PreCommit-HBASE-Build/11346/console |
| Powered by | Apache Yetus 0.7.0 http://yetus.apache.org |
This message was automatically generated.
> The exception happening in converting pb mutation to hbase.mutation messes up
> the CellScanner
> -
>
> Key: HBASE-19876
> URL: https://issues.apache.org/jira/browse/HBASE-19876
> Project: HBase
> Issue Type: Bug
>Reporter: Chia-Ping Tsai
>Assignee: Chia-Ping Tsai
>Priority: Critical
> Fix For: 1.3.2, 1.5.0, 1.2.7, 2.0.0-beta-2, 1.4.2
>
> Attachments: HBASE-19876.v0.patch, HBASE-19876.v1.patch,
> HBASE-19876.v2.patch, HBASE-19876.v3.patch, HBASE-19876.v3.patch,
> HBASE-19876.v3.patch
>
>
> {code:java}
> 2018-01-27 22:51:43,794 INFO [hconnection-0x3291b443-shared-pool11-t6]
> client.AsyncRequestFutureImpl(778): id=5, table=testQuotaStatusFromMaster3,
> attempt=6/16 failed=20ops, last
> exception=org.apache.hadoop.hbase.client.WrongRowIOException:
> org.apache.hadoop.hbase.client.WrongRowIOException: The row in xxx doesn't
> match the original one aaa
> at org.apache.hadoop.hbase.client.Mutation.add(Mutation.java:776)
> at org.apache.hadoop.hbase.client.Put.add(Put.java:282)
> at
> org.apache.hadoop.hbase.shaded.protobuf.ProtobufUtil.toPut(ProtobufUtil.java:642)
> at
> org.apache.hadoop.hbase.regionserver.RSRpcServices.doBatchOp(RSRpcServices.java:952)
> at
> org.apache.hadoop.hbase.regionserver.RSRpcServices.doNonAtomicRegionMutation(RSRpcServices.java:896)
> at
> org.apache.hadoop.hbase.regionserver.RSRpcServices.multi(RSRpcServices.java:2591)
> at
> org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$ClientService$2.callBlockingMethod(ClientProtos.java:41560)
> at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:404)
> at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:130)
> at
> org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:324)
> at
> org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:304){code}
> I noticed this bug when testing the table space quota.
> When rs are converting pb mutation to hbase.mutation, the quota exception or
> cell exception may be thrown.
> {code}
> Unable to find source-code formatter for language:
> rsrpcservices#dobatchop.java. Available languages are: actionscript, ada,
> applescript, bash, c, c#, c++, cpp, css, erlang, go, groovy, haskell, html,
> java, javascript, js, json, lua, none, nyan, objc, perl, php, python, r,
> rainbow, ruby, scala, sh, sql, swift, visualbasic, xml, yaml for
> (ClientProtos.Action action: mutations) {
> MutationProto m = action.getMutation();
> Mutation mutation;
> if (m.getMutateType() == MutationType.PUT) {
> mutation = ProtobufUtil.toPut(m, cells);
> batchContainsPuts = true;
> } else {
> mutation = ProtobufUtil.toDelete(m, cells);
> batchContainsDelete = true;
> }
> mutationActionMap.put(mutation, action);
> mArray[i++] = mutation;
> checkCellSizeLimit(region, mutation);
> // Check if a space quota disallows this mutation
> spaceQuotaEnforcement.getPolicyEnforcement(region).check(mutation);
> quota.addMutation(mutation);
> }
> {code}
> rs has caught the exception but it doesn't have the cellscanner skip the
> failed cells.
> {code:java}
> } catch (IOException ie) {
> if (atomic) {
> throw ie;
> }
> for (Action mutation : mutations) {
> builder.addResultOrException(getResultOrException(ie,
> mutation.getIndex()));
> }
> }
> {code}
> The bug results in the WrongRowIOException to remaining mutations since they
> refer to invalid cells.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-19876) The exception happening in converting pb mutation to hbase.mutation messes up the CellScanner
[
https://issues.apache.org/jira/browse/HBASE-19876?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16348824#comment-16348824
]
Hadoop QA commented on HBASE-19876:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 2m
29s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
1s{color} | {color:blue} Findbugs executables are not available. {color} |
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s{color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 1 new or modified test
files. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 3m
53s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
38s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
58s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 5m
5s{color} | {color:green} branch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
26s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 3m
54s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
40s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
40s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
54s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 4m
13s{color} | {color:green} patch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
16m 24s{color} | {color:green} Patch does not cause any errors with Hadoop
2.6.5 2.7.4 or 3.0.0. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
27s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 98m 9s{color}
| {color:red} hbase-server in the patch failed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
14s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black}133m 28s{color} |
{color:black} {color} |
\\
\\
|| Reason || Tests ||
| Failed junit tests | hadoop.hbase.TestZooKeeper |
| | hadoop.hbase.TestFullLogReconstruction |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hbase:eee3b01 |
| JIRA Issue | HBASE-19876 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12908813/HBASE-19876.v3.patch |
| Optional Tests | asflicense javac javadoc unit findbugs shadedjars
hadoopcheck hbaseanti checkstyle compile |
| uname | Linux 6faf42f71166 4.4.0-43-generic #63-Ubuntu SMP Wed Oct 12
13:48:03 UTC 2016 x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/component/dev-support/hbase-personality.sh
|
| git revision | master / 9272f40a5c |
| maven | version: Apache Maven 3.5.2
(138edd61fd100ec658bfa2d307c43b76940a5d7d; 2017-10-18T07:58:13Z) |
| Default Java | 1.8.0_151 |
| unit |
https://builds.apache.org/job/PreCommit-HBASE-Build/11331/artifact/patchprocess/patch-unit-hbase-server.txt
|
| Test Results |
https://builds.apache.org/job/PreCommit-HBASE-Build/11331/testReport/ |
| Max. process+thread count | 5828 (vs. ulimit of 2) |
| modules | C: hbase-server U: hbase-server |
| Console output |
https://builds.apache.org/job/PreCommit-HBASE-Build/11331/console |
| Powered by | Apache Yetus 0.7
[jira] [Commented] (HBASE-19876) The exception happening in converting pb mutation to hbase.mutation messes up the CellScanner
[
https://issues.apache.org/jira/browse/HBASE-19876?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16348604#comment-16348604
]
Hadoop QA commented on HBASE-19876:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
0s{color} | {color:blue} Docker mode activated. {color} |
| {color:red}-1{color} | {color:red} docker {color} | {color:red} 0m
2s{color} | {color:red} Docker failed to build yetus/hbase:tp-29582. {color} |
\\
\\
|| Subsystem || Report/Notes ||
| JIRA Issue | HBASE-19876 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12908805/HBASE-19876.v3.patch |
| Console output |
https://builds.apache.org/job/PreCommit-HBASE-Build/11330/console |
| Powered by | Apache Yetus 0.7.0 http://yetus.apache.org |
This message was automatically generated.
> The exception happening in converting pb mutation to hbase.mutation messes up
> the CellScanner
> -
>
> Key: HBASE-19876
> URL: https://issues.apache.org/jira/browse/HBASE-19876
> Project: HBase
> Issue Type: Bug
>Reporter: Chia-Ping Tsai
>Assignee: Chia-Ping Tsai
>Priority: Critical
> Fix For: 1.3.2, 1.5.0, 1.2.7, 2.0.0-beta-2, 1.4.2
>
> Attachments: HBASE-19876.v0.patch, HBASE-19876.v1.patch,
> HBASE-19876.v2.patch, HBASE-19876.v3.patch
>
>
> {code:java}
> 2018-01-27 22:51:43,794 INFO [hconnection-0x3291b443-shared-pool11-t6]
> client.AsyncRequestFutureImpl(778): id=5, table=testQuotaStatusFromMaster3,
> attempt=6/16 failed=20ops, last
> exception=org.apache.hadoop.hbase.client.WrongRowIOException:
> org.apache.hadoop.hbase.client.WrongRowIOException: The row in xxx doesn't
> match the original one aaa
> at org.apache.hadoop.hbase.client.Mutation.add(Mutation.java:776)
> at org.apache.hadoop.hbase.client.Put.add(Put.java:282)
> at
> org.apache.hadoop.hbase.shaded.protobuf.ProtobufUtil.toPut(ProtobufUtil.java:642)
> at
> org.apache.hadoop.hbase.regionserver.RSRpcServices.doBatchOp(RSRpcServices.java:952)
> at
> org.apache.hadoop.hbase.regionserver.RSRpcServices.doNonAtomicRegionMutation(RSRpcServices.java:896)
> at
> org.apache.hadoop.hbase.regionserver.RSRpcServices.multi(RSRpcServices.java:2591)
> at
> org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$ClientService$2.callBlockingMethod(ClientProtos.java:41560)
> at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:404)
> at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:130)
> at
> org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:324)
> at
> org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:304){code}
> I noticed this bug when testing the table space quota.
> When rs are converting pb mutation to hbase.mutation, the quota exception or
> cell exception may be thrown.
> {code}
> Unable to find source-code formatter for language:
> rsrpcservices#dobatchop.java. Available languages are: actionscript, ada,
> applescript, bash, c, c#, c++, cpp, css, erlang, go, groovy, haskell, html,
> java, javascript, js, json, lua, none, nyan, objc, perl, php, python, r,
> rainbow, ruby, scala, sh, sql, swift, visualbasic, xml, yaml for
> (ClientProtos.Action action: mutations) {
> MutationProto m = action.getMutation();
> Mutation mutation;
> if (m.getMutateType() == MutationType.PUT) {
> mutation = ProtobufUtil.toPut(m, cells);
> batchContainsPuts = true;
> } else {
> mutation = ProtobufUtil.toDelete(m, cells);
> batchContainsDelete = true;
> }
> mutationActionMap.put(mutation, action);
> mArray[i++] = mutation;
> checkCellSizeLimit(region, mutation);
> // Check if a space quota disallows this mutation
> spaceQuotaEnforcement.getPolicyEnforcement(region).check(mutation);
> quota.addMutation(mutation);
> }
> {code}
> rs has caught the exception but it doesn't have the cellscanner skip the
> failed cells.
> {code:java}
> } catch (IOException ie) {
> if (atomic) {
> throw ie;
> }
> for (Action mutation : mutations) {
> builder.addResultOrException(getResultOrException(ie,
> mutation.getIndex()));
> }
> }
> {code}
> The bug results in the WrongRowIOException to remaining mutations since they
> refer to invalid cells.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-19876) The exception happening in converting pb mutation to hbase.mutation messes up the CellScanner
[
https://issues.apache.org/jira/browse/HBASE-19876?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16348598#comment-16348598
]
Chia-Ping Tsai commented on HBASE-19876:
v3 fix the failed test...
> The exception happening in converting pb mutation to hbase.mutation messes up
> the CellScanner
> -
>
> Key: HBASE-19876
> URL: https://issues.apache.org/jira/browse/HBASE-19876
> Project: HBase
> Issue Type: Bug
>Reporter: Chia-Ping Tsai
>Assignee: Chia-Ping Tsai
>Priority: Critical
> Fix For: 1.3.2, 1.5.0, 1.2.7, 2.0.0-beta-2, 1.4.2
>
> Attachments: HBASE-19876.v0.patch, HBASE-19876.v1.patch,
> HBASE-19876.v2.patch, HBASE-19876.v3.patch
>
>
> {code:java}
> 2018-01-27 22:51:43,794 INFO [hconnection-0x3291b443-shared-pool11-t6]
> client.AsyncRequestFutureImpl(778): id=5, table=testQuotaStatusFromMaster3,
> attempt=6/16 failed=20ops, last
> exception=org.apache.hadoop.hbase.client.WrongRowIOException:
> org.apache.hadoop.hbase.client.WrongRowIOException: The row in xxx doesn't
> match the original one aaa
> at org.apache.hadoop.hbase.client.Mutation.add(Mutation.java:776)
> at org.apache.hadoop.hbase.client.Put.add(Put.java:282)
> at
> org.apache.hadoop.hbase.shaded.protobuf.ProtobufUtil.toPut(ProtobufUtil.java:642)
> at
> org.apache.hadoop.hbase.regionserver.RSRpcServices.doBatchOp(RSRpcServices.java:952)
> at
> org.apache.hadoop.hbase.regionserver.RSRpcServices.doNonAtomicRegionMutation(RSRpcServices.java:896)
> at
> org.apache.hadoop.hbase.regionserver.RSRpcServices.multi(RSRpcServices.java:2591)
> at
> org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$ClientService$2.callBlockingMethod(ClientProtos.java:41560)
> at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:404)
> at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:130)
> at
> org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:324)
> at
> org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:304){code}
> I noticed this bug when testing the table space quota.
> When rs are converting pb mutation to hbase.mutation, the quota exception or
> cell exception may be thrown.
> {code}
> Unable to find source-code formatter for language:
> rsrpcservices#dobatchop.java. Available languages are: actionscript, ada,
> applescript, bash, c, c#, c++, cpp, css, erlang, go, groovy, haskell, html,
> java, javascript, js, json, lua, none, nyan, objc, perl, php, python, r,
> rainbow, ruby, scala, sh, sql, swift, visualbasic, xml, yaml for
> (ClientProtos.Action action: mutations) {
> MutationProto m = action.getMutation();
> Mutation mutation;
> if (m.getMutateType() == MutationType.PUT) {
> mutation = ProtobufUtil.toPut(m, cells);
> batchContainsPuts = true;
> } else {
> mutation = ProtobufUtil.toDelete(m, cells);
> batchContainsDelete = true;
> }
> mutationActionMap.put(mutation, action);
> mArray[i++] = mutation;
> checkCellSizeLimit(region, mutation);
> // Check if a space quota disallows this mutation
> spaceQuotaEnforcement.getPolicyEnforcement(region).check(mutation);
> quota.addMutation(mutation);
> }
> {code}
> rs has caught the exception but it doesn't have the cellscanner skip the
> failed cells.
> {code:java}
> } catch (IOException ie) {
> if (atomic) {
> throw ie;
> }
> for (Action mutation : mutations) {
> builder.addResultOrException(getResultOrException(ie,
> mutation.getIndex()));
> }
> }
> {code}
> The bug results in the WrongRowIOException to remaining mutations since they
> refer to invalid cells.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-19876) The exception happening in converting pb mutation to hbase.mutation messes up the CellScanner
[
https://issues.apache.org/jira/browse/HBASE-19876?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16348378#comment-16348378
]
Hadoop QA commented on HBASE-19876:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
10s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
0s{color} | {color:blue} Findbugs executables are not available. {color} |
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s{color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 1 new or modified test
files. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 3m
54s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
38s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
55s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 5m
48s{color} | {color:green} branch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
7s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 7m
1s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
19s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m
19s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
41s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 6m
33s{color} | {color:green} patch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
22m 53s{color} | {color:green} Patch does not cause any errors with Hadoop
2.6.5 2.7.4 or 3.0.0. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
26s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 95m 54s{color}
| {color:red} hbase-server in the patch failed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
20s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black}141m 39s{color} |
{color:black} {color} |
\\
\\
|| Reason || Tests ||
| Failed junit tests | hadoop.hbase.TestFullLogReconstruction |
| | hadoop.hbase.client.TestPartialMalformedCells |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hbase:eee3b01 |
| JIRA Issue | HBASE-19876 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12908749/HBASE-19876.v2.patch |
| Optional Tests | asflicense javac javadoc unit findbugs shadedjars
hadoopcheck hbaseanti checkstyle compile |
| uname | Linux 6fae00c93d03 4.4.0-43-generic #63-Ubuntu SMP Wed Oct 12
13:48:03 UTC 2016 x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/component/dev-support/hbase-personality.sh
|
| git revision | master / 9272f40a5c |
| maven | version: Apache Maven 3.5.2
(138edd61fd100ec658bfa2d307c43b76940a5d7d; 2017-10-18T07:58:13Z) |
| Default Java | 1.8.0_151 |
| unit |
https://builds.apache.org/job/PreCommit-HBASE-Build/11322/artifact/patchprocess/patch-unit-hbase-server.txt
|
| Test Results |
https://builds.apache.org/job/PreCommit-HBASE-Build/11322/testReport/ |
| Max. process+thread count | 5422 (vs. ulimit of 6000) |
| modules | C: hbase-server U: hbase-server |
| Console output |
https://builds.apache.org/job/PreCommit-HBASE-Build/11322/console |
| Powered by
[jira] [Commented] (HBASE-19876) The exception happening in converting pb mutation to hbase.mutation messes up the CellScanner
[
https://issues.apache.org/jira/browse/HBASE-19876?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16348187#comment-16348187
]
Chia-Ping Tsai commented on HBASE-19876:
The failed tests are caused by new timeout rule. HBASE-19911 is dealing with
them.
v2 fix the checkstyle warnings.
> The exception happening in converting pb mutation to hbase.mutation messes up
> the CellScanner
> -
>
> Key: HBASE-19876
> URL: https://issues.apache.org/jira/browse/HBASE-19876
> Project: HBase
> Issue Type: Bug
>Reporter: Chia-Ping Tsai
>Assignee: Chia-Ping Tsai
>Priority: Critical
> Fix For: 1.3.2, 1.5.0, 1.2.7, 2.0.0-beta-2, 1.4.2
>
> Attachments: HBASE-19876.v0.patch, HBASE-19876.v1.patch,
> HBASE-19876.v2.patch
>
>
> {code:java}
> 2018-01-27 22:51:43,794 INFO [hconnection-0x3291b443-shared-pool11-t6]
> client.AsyncRequestFutureImpl(778): id=5, table=testQuotaStatusFromMaster3,
> attempt=6/16 failed=20ops, last
> exception=org.apache.hadoop.hbase.client.WrongRowIOException:
> org.apache.hadoop.hbase.client.WrongRowIOException: The row in xxx doesn't
> match the original one aaa
> at org.apache.hadoop.hbase.client.Mutation.add(Mutation.java:776)
> at org.apache.hadoop.hbase.client.Put.add(Put.java:282)
> at
> org.apache.hadoop.hbase.shaded.protobuf.ProtobufUtil.toPut(ProtobufUtil.java:642)
> at
> org.apache.hadoop.hbase.regionserver.RSRpcServices.doBatchOp(RSRpcServices.java:952)
> at
> org.apache.hadoop.hbase.regionserver.RSRpcServices.doNonAtomicRegionMutation(RSRpcServices.java:896)
> at
> org.apache.hadoop.hbase.regionserver.RSRpcServices.multi(RSRpcServices.java:2591)
> at
> org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$ClientService$2.callBlockingMethod(ClientProtos.java:41560)
> at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:404)
> at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:130)
> at
> org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:324)
> at
> org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:304){code}
> I noticed this bug when testing the table space quota.
> When rs are converting pb mutation to hbase.mutation, the quota exception or
> cell exception may be thrown.
> {code}
> Unable to find source-code formatter for language:
> rsrpcservices#dobatchop.java. Available languages are: actionscript, ada,
> applescript, bash, c, c#, c++, cpp, css, erlang, go, groovy, haskell, html,
> java, javascript, js, json, lua, none, nyan, objc, perl, php, python, r,
> rainbow, ruby, scala, sh, sql, swift, visualbasic, xml, yaml for
> (ClientProtos.Action action: mutations) {
> MutationProto m = action.getMutation();
> Mutation mutation;
> if (m.getMutateType() == MutationType.PUT) {
> mutation = ProtobufUtil.toPut(m, cells);
> batchContainsPuts = true;
> } else {
> mutation = ProtobufUtil.toDelete(m, cells);
> batchContainsDelete = true;
> }
> mutationActionMap.put(mutation, action);
> mArray[i++] = mutation;
> checkCellSizeLimit(region, mutation);
> // Check if a space quota disallows this mutation
> spaceQuotaEnforcement.getPolicyEnforcement(region).check(mutation);
> quota.addMutation(mutation);
> }
> {code}
> rs has caught the exception but it doesn't have the cellscanner skip the
> failed cells.
> {code:java}
> } catch (IOException ie) {
> if (atomic) {
> throw ie;
> }
> for (Action mutation : mutations) {
> builder.addResultOrException(getResultOrException(ie,
> mutation.getIndex()));
> }
> }
> {code}
> The bug results in the WrongRowIOException to remaining mutations since they
> refer to invalid cells.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-19876) The exception happening in converting pb mutation to hbase.mutation messes up the CellScanner
[
https://issues.apache.org/jira/browse/HBASE-19876?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16348143#comment-16348143
]
Hadoop QA commented on HBASE-19876:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
15s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
0s{color} | {color:blue} Findbugs executables are not available. {color} |
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s{color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 1 new or modified test
files. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 4m
58s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
45s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
8s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 5m
54s{color} | {color:green} branch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
31s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 4m
32s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
43s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
43s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 1m
8s{color} | {color:red} hbase-server: The patch generated 2 new + 130 unchanged
- 0 fixed = 132 total (was 130) {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 5m
4s{color} | {color:green} patch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
21m 15s{color} | {color:green} Patch does not cause any errors with Hadoop
2.6.5 2.7.4 or 3.0.0. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
58s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 38m 45s{color}
| {color:red} hbase-server in the patch failed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
27s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 80m 23s{color} |
{color:black} {color} |
\\
\\
|| Reason || Tests ||
| Failed junit tests | hadoop.hbase.TestClientClusterMetrics |
| | hadoop.hbase.favored.TestFavoredNodeAssignmentHelper |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hbase:eee3b01 |
| JIRA Issue | HBASE-19876 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12908728/HBASE-19876.v1.patch |
| Optional Tests | asflicense javac javadoc unit findbugs shadedjars
hadoopcheck hbaseanti checkstyle compile |
| uname | Linux e3eea62c867b 3.13.0-135-generic #184-Ubuntu SMP Wed Oct 18
11:55:51 UTC 2017 x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/component/dev-support/hbase-personality.sh
|
| git revision | master / 9272f40a5c |
| maven | version: Apache Maven 3.5.2
(138edd61fd100ec658bfa2d307c43b76940a5d7d; 2017-10-18T07:58:13Z) |
| Default Java | 1.8.0_151 |
| checkstyle |
https://builds.apache.org/job/PreCommit-HBASE-Build/11316/artifact/patchprocess/diff-checkstyle-hbase-server.txt
|
| unit |
https://builds.apache.org/job/PreCommit-HBASE-Build/11316/artifact/patchprocess/patch-unit-hbase-server.txt
|
| Test Results |
https://builds.apache.org/job/PreCommit-HBASE-Build/11316/testReport/ |
[jira] [Commented] (HBASE-19876) The exception happening in converting pb mutation to hbase.mutation messes up the CellScanner
[
https://issues.apache.org/jira/browse/HBASE-19876?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16348062#comment-16348062
]
Chia-Ping Tsai commented on HBASE-19876:
v1 address [~yuzhih...@gmail.com]'s comment
> The exception happening in converting pb mutation to hbase.mutation messes up
> the CellScanner
> -
>
> Key: HBASE-19876
> URL: https://issues.apache.org/jira/browse/HBASE-19876
> Project: HBase
> Issue Type: Bug
>Reporter: Chia-Ping Tsai
>Assignee: Chia-Ping Tsai
>Priority: Critical
> Fix For: 1.3.2, 1.5.0, 1.2.7, 2.0.0-beta-2, 1.4.2
>
> Attachments: HBASE-19876.v0.patch, HBASE-19876.v1.patch
>
>
> {code:java}
> 2018-01-27 22:51:43,794 INFO [hconnection-0x3291b443-shared-pool11-t6]
> client.AsyncRequestFutureImpl(778): id=5, table=testQuotaStatusFromMaster3,
> attempt=6/16 failed=20ops, last
> exception=org.apache.hadoop.hbase.client.WrongRowIOException:
> org.apache.hadoop.hbase.client.WrongRowIOException: The row in xxx doesn't
> match the original one aaa
> at org.apache.hadoop.hbase.client.Mutation.add(Mutation.java:776)
> at org.apache.hadoop.hbase.client.Put.add(Put.java:282)
> at
> org.apache.hadoop.hbase.shaded.protobuf.ProtobufUtil.toPut(ProtobufUtil.java:642)
> at
> org.apache.hadoop.hbase.regionserver.RSRpcServices.doBatchOp(RSRpcServices.java:952)
> at
> org.apache.hadoop.hbase.regionserver.RSRpcServices.doNonAtomicRegionMutation(RSRpcServices.java:896)
> at
> org.apache.hadoop.hbase.regionserver.RSRpcServices.multi(RSRpcServices.java:2591)
> at
> org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$ClientService$2.callBlockingMethod(ClientProtos.java:41560)
> at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:404)
> at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:130)
> at
> org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:324)
> at
> org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:304){code}
> I noticed this bug when testing the table space quota.
> When rs are converting pb mutation to hbase.mutation, the quota exception or
> cell exception may be thrown.
> {code}
> Unable to find source-code formatter for language:
> rsrpcservices#dobatchop.java. Available languages are: actionscript, ada,
> applescript, bash, c, c#, c++, cpp, css, erlang, go, groovy, haskell, html,
> java, javascript, js, json, lua, none, nyan, objc, perl, php, python, r,
> rainbow, ruby, scala, sh, sql, swift, visualbasic, xml, yaml for
> (ClientProtos.Action action: mutations) {
> MutationProto m = action.getMutation();
> Mutation mutation;
> if (m.getMutateType() == MutationType.PUT) {
> mutation = ProtobufUtil.toPut(m, cells);
> batchContainsPuts = true;
> } else {
> mutation = ProtobufUtil.toDelete(m, cells);
> batchContainsDelete = true;
> }
> mutationActionMap.put(mutation, action);
> mArray[i++] = mutation;
> checkCellSizeLimit(region, mutation);
> // Check if a space quota disallows this mutation
> spaceQuotaEnforcement.getPolicyEnforcement(region).check(mutation);
> quota.addMutation(mutation);
> }
> {code}
> rs has caught the exception but it doesn't have the cellscanner skip the
> failed cells.
> {code:java}
> } catch (IOException ie) {
> if (atomic) {
> throw ie;
> }
> for (Action mutation : mutations) {
> builder.addResultOrException(getResultOrException(ie,
> mutation.getIndex()));
> }
> }
> {code}
> The bug results in the WrongRowIOException to remaining mutations since they
> refer to invalid cells.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-19876) The exception happening in converting pb mutation to hbase.mutation messes up the CellScanner
[
https://issues.apache.org/jira/browse/HBASE-19876?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16345937#comment-16345937
]
Ted Yu commented on HBASE-19876:
Looks good overall
{code}
1008// if the mArray[i] != null, it means the cell scanner has be
read.
{code}
Typo: 'be'
Please remove commented out code in the test.
> The exception happening in converting pb mutation to hbase.mutation messes up
> the CellScanner
> -
>
> Key: HBASE-19876
> URL: https://issues.apache.org/jira/browse/HBASE-19876
> Project: HBase
> Issue Type: Bug
>Reporter: Chia-Ping Tsai
>Assignee: Chia-Ping Tsai
>Priority: Critical
> Fix For: 1.3.2, 1.5.0, 1.2.7, 2.0.0-beta-2, 1.4.2
>
> Attachments: HBASE-19876.v0.patch
>
>
> {code:java}
> 2018-01-27 22:51:43,794 INFO [hconnection-0x3291b443-shared-pool11-t6]
> client.AsyncRequestFutureImpl(778): id=5, table=testQuotaStatusFromMaster3,
> attempt=6/16 failed=20ops, last
> exception=org.apache.hadoop.hbase.client.WrongRowIOException:
> org.apache.hadoop.hbase.client.WrongRowIOException: The row in xxx doesn't
> match the original one aaa
> at org.apache.hadoop.hbase.client.Mutation.add(Mutation.java:776)
> at org.apache.hadoop.hbase.client.Put.add(Put.java:282)
> at
> org.apache.hadoop.hbase.shaded.protobuf.ProtobufUtil.toPut(ProtobufUtil.java:642)
> at
> org.apache.hadoop.hbase.regionserver.RSRpcServices.doBatchOp(RSRpcServices.java:952)
> at
> org.apache.hadoop.hbase.regionserver.RSRpcServices.doNonAtomicRegionMutation(RSRpcServices.java:896)
> at
> org.apache.hadoop.hbase.regionserver.RSRpcServices.multi(RSRpcServices.java:2591)
> at
> org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$ClientService$2.callBlockingMethod(ClientProtos.java:41560)
> at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:404)
> at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:130)
> at
> org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:324)
> at
> org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:304){code}
> I noticed this bug when testing the table space quota.
> When rs are converting pb mutation to hbase.mutation, the quota exception or
> cell exception may be thrown.
> {code}
> Unable to find source-code formatter for language:
> rsrpcservices#dobatchop.java. Available languages are: actionscript, ada,
> applescript, bash, c, c#, c++, cpp, css, erlang, go, groovy, haskell, html,
> java, javascript, js, json, lua, none, nyan, objc, perl, php, python, r,
> rainbow, ruby, scala, sh, sql, swift, visualbasic, xml, yaml for
> (ClientProtos.Action action: mutations) {
> MutationProto m = action.getMutation();
> Mutation mutation;
> if (m.getMutateType() == MutationType.PUT) {
> mutation = ProtobufUtil.toPut(m, cells);
> batchContainsPuts = true;
> } else {
> mutation = ProtobufUtil.toDelete(m, cells);
> batchContainsDelete = true;
> }
> mutationActionMap.put(mutation, action);
> mArray[i++] = mutation;
> checkCellSizeLimit(region, mutation);
> // Check if a space quota disallows this mutation
> spaceQuotaEnforcement.getPolicyEnforcement(region).check(mutation);
> quota.addMutation(mutation);
> }
> {code}
> rs has caught the exception but it doesn't have the cellscanner skip the
> failed cells.
> {code:java}
> } catch (IOException ie) {
> if (atomic) {
> throw ie;
> }
> for (Action mutation : mutations) {
> builder.addResultOrException(getResultOrException(ie,
> mutation.getIndex()));
> }
> }
> {code}
> The bug results in the WrongRowIOException to remaining mutations since they
> refer to invalid cells.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-19876) The exception happening in converting pb mutation to hbase.mutation messes up the CellScanner
[
https://issues.apache.org/jira/browse/HBASE-19876?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16344363#comment-16344363
]
Anoop Sam John commented on HBASE-19876:
Append and increment Mutation may have N Cells in that (for same row). Even if
one cell is malformed, we have to fail the op. Ya that make sense. Need to
verify once though. Thanks Chia.
> The exception happening in converting pb mutation to hbase.mutation messes up
> the CellScanner
> -
>
> Key: HBASE-19876
> URL: https://issues.apache.org/jira/browse/HBASE-19876
> Project: HBase
> Issue Type: Bug
>Reporter: Chia-Ping Tsai
>Assignee: Chia-Ping Tsai
>Priority: Critical
> Fix For: 1.3.2, 1.5.0, 1.2.7, 2.0.0-beta-2, 1.4.2
>
>
> {code:java}
> 2018-01-27 22:51:43,794 INFO [hconnection-0x3291b443-shared-pool11-t6]
> client.AsyncRequestFutureImpl(778): id=5, table=testQuotaStatusFromMaster3,
> attempt=6/16 failed=20ops, last
> exception=org.apache.hadoop.hbase.client.WrongRowIOException:
> org.apache.hadoop.hbase.client.WrongRowIOException: The row in xxx doesn't
> match the original one aaa
> at org.apache.hadoop.hbase.client.Mutation.add(Mutation.java:776)
> at org.apache.hadoop.hbase.client.Put.add(Put.java:282)
> at
> org.apache.hadoop.hbase.shaded.protobuf.ProtobufUtil.toPut(ProtobufUtil.java:642)
> at
> org.apache.hadoop.hbase.regionserver.RSRpcServices.doBatchOp(RSRpcServices.java:952)
> at
> org.apache.hadoop.hbase.regionserver.RSRpcServices.doNonAtomicRegionMutation(RSRpcServices.java:896)
> at
> org.apache.hadoop.hbase.regionserver.RSRpcServices.multi(RSRpcServices.java:2591)
> at
> org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$ClientService$2.callBlockingMethod(ClientProtos.java:41560)
> at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:404)
> at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:130)
> at
> org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:324)
> at
> org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:304){code}
> I noticed this bug when testing the table space quota.
> When rs are converting pb mutation to hbase.mutation, the quota exception or
> cell exception may be thrown.
> {code}
> Unable to find source-code formatter for language:
> rsrpcservices#dobatchop.java. Available languages are: actionscript, ada,
> applescript, bash, c, c#, c++, cpp, css, erlang, go, groovy, haskell, html,
> java, javascript, js, json, lua, none, nyan, objc, perl, php, python, r,
> rainbow, ruby, scala, sh, sql, swift, visualbasic, xml, yaml for
> (ClientProtos.Action action: mutations) {
> MutationProto m = action.getMutation();
> Mutation mutation;
> if (m.getMutateType() == MutationType.PUT) {
> mutation = ProtobufUtil.toPut(m, cells);
> batchContainsPuts = true;
> } else {
> mutation = ProtobufUtil.toDelete(m, cells);
> batchContainsDelete = true;
> }
> mutationActionMap.put(mutation, action);
> mArray[i++] = mutation;
> checkCellSizeLimit(region, mutation);
> // Check if a space quota disallows this mutation
> spaceQuotaEnforcement.getPolicyEnforcement(region).check(mutation);
> quota.addMutation(mutation);
> }
> {code}
> rs has caught the exception but it doesn't have the cellscanner skip the
> failed cells.
> {code:java}
> } catch (IOException ie) {
> if (atomic) {
> throw ie;
> }
> for (Action mutation : mutations) {
> builder.addResultOrException(getResultOrException(ie,
> mutation.getIndex()));
> }
> }
> {code}
> The bug results in the WrongRowIOException to remaining mutations since they
> refer to invalid cells.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-19876) The exception happening in converting pb mutation to hbase.mutation messes up the CellScanner
[
https://issues.apache.org/jira/browse/HBASE-19876?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16344001#comment-16344001
]
Chia-Ping Tsai commented on HBASE-19876:
The append/increment wont do the batch so only batch of put/Delete has this
issue. That makes the fix easy. Will come back soon.
> The exception happening in converting pb mutation to hbase.mutation messes up
> the CellScanner
> -
>
> Key: HBASE-19876
> URL: https://issues.apache.org/jira/browse/HBASE-19876
> Project: HBase
> Issue Type: Bug
>Reporter: Chia-Ping Tsai
>Assignee: Chia-Ping Tsai
>Priority: Critical
> Fix For: 1.3.2, 1.5.0, 1.2.7, 2.0.0-beta-2, 1.4.2
>
>
> {code:java}
> 2018-01-27 22:51:43,794 INFO [hconnection-0x3291b443-shared-pool11-t6]
> client.AsyncRequestFutureImpl(778): id=5, table=testQuotaStatusFromMaster3,
> attempt=6/16 failed=20ops, last
> exception=org.apache.hadoop.hbase.client.WrongRowIOException:
> org.apache.hadoop.hbase.client.WrongRowIOException: The row in xxx doesn't
> match the original one aaa
> at org.apache.hadoop.hbase.client.Mutation.add(Mutation.java:776)
> at org.apache.hadoop.hbase.client.Put.add(Put.java:282)
> at
> org.apache.hadoop.hbase.shaded.protobuf.ProtobufUtil.toPut(ProtobufUtil.java:642)
> at
> org.apache.hadoop.hbase.regionserver.RSRpcServices.doBatchOp(RSRpcServices.java:952)
> at
> org.apache.hadoop.hbase.regionserver.RSRpcServices.doNonAtomicRegionMutation(RSRpcServices.java:896)
> at
> org.apache.hadoop.hbase.regionserver.RSRpcServices.multi(RSRpcServices.java:2591)
> at
> org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$ClientService$2.callBlockingMethod(ClientProtos.java:41560)
> at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:404)
> at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:130)
> at
> org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:324)
> at
> org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:304){code}
> I noticed this bug when testing the table space quota.
> When rs are converting pb mutation to hbase.mutation, the quota exception or
> cell exception may be thrown.
> {code}
> Unable to find source-code formatter for language:
> rsrpcservices#dobatchop.java. Available languages are: actionscript, ada,
> applescript, bash, c, c#, c++, cpp, css, erlang, go, groovy, haskell, html,
> java, javascript, js, json, lua, none, nyan, objc, perl, php, python, r,
> rainbow, ruby, scala, sh, sql, swift, visualbasic, xml, yaml for
> (ClientProtos.Action action: mutations) {
> MutationProto m = action.getMutation();
> Mutation mutation;
> if (m.getMutateType() == MutationType.PUT) {
> mutation = ProtobufUtil.toPut(m, cells);
> batchContainsPuts = true;
> } else {
> mutation = ProtobufUtil.toDelete(m, cells);
> batchContainsDelete = true;
> }
> mutationActionMap.put(mutation, action);
> mArray[i++] = mutation;
> checkCellSizeLimit(region, mutation);
> // Check if a space quota disallows this mutation
> spaceQuotaEnforcement.getPolicyEnforcement(region).check(mutation);
> quota.addMutation(mutation);
> }
> {code}
> rs has caught the exception but it doesn't have the cellscanner skip the
> failed cells.
> {code:java}
> } catch (IOException ie) {
> if (atomic) {
> throw ie;
> }
> for (Action mutation : mutations) {
> builder.addResultOrException(getResultOrException(ie,
> mutation.getIndex()));
> }
> }
> {code}
> The bug results in the WrongRowIOException to remaining mutations since they
> refer to invalid cells.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-19876) The exception happening in converting pb mutation to hbase.mutation messes up the CellScanner
[
https://issues.apache.org/jira/browse/HBASE-19876?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16343007#comment-16343007
]
Chia-Ping Tsai commented on HBASE-19876:
{quote}A related issue was fixed some time back.
{quote}
HBASE-15214
{quote} Skipping the cur cell which is problematic. So that might be missing
for some places exception handling !!!
{quote}
yep, all append/increment/doBatchOp have similar issue...
> The exception happening in converting pb mutation to hbase.mutation messes up
> the CellScanner
> -
>
> Key: HBASE-19876
> URL: https://issues.apache.org/jira/browse/HBASE-19876
> Project: HBase
> Issue Type: Bug
>Reporter: Chia-Ping Tsai
>Assignee: Chia-Ping Tsai
>Priority: Critical
> Fix For: 1.3.2, 1.5.0, 1.2.7, 2.0.0-beta-2, 1.4.2
>
>
> {code:java}
> 2018-01-27 22:51:43,794 INFO [hconnection-0x3291b443-shared-pool11-t6]
> client.AsyncRequestFutureImpl(778): id=5, table=testQuotaStatusFromMaster3,
> attempt=6/16 failed=20ops, last
> exception=org.apache.hadoop.hbase.client.WrongRowIOException:
> org.apache.hadoop.hbase.client.WrongRowIOException: The row in xxx doesn't
> match the original one aaa
> at org.apache.hadoop.hbase.client.Mutation.add(Mutation.java:776)
> at org.apache.hadoop.hbase.client.Put.add(Put.java:282)
> at
> org.apache.hadoop.hbase.shaded.protobuf.ProtobufUtil.toPut(ProtobufUtil.java:642)
> at
> org.apache.hadoop.hbase.regionserver.RSRpcServices.doBatchOp(RSRpcServices.java:952)
> at
> org.apache.hadoop.hbase.regionserver.RSRpcServices.doNonAtomicRegionMutation(RSRpcServices.java:896)
> at
> org.apache.hadoop.hbase.regionserver.RSRpcServices.multi(RSRpcServices.java:2591)
> at
> org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$ClientService$2.callBlockingMethod(ClientProtos.java:41560)
> at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:404)
> at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:130)
> at
> org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:324)
> at
> org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:304){code}
> I noticed this bug when testing the table space quota.
> When rs are converting pb mutation to hbase.mutation, the quota exception or
> cell exception may be thrown.
> {code}
> Unable to find source-code formatter for language:
> rsrpcservices#dobatchop.java. Available languages are: actionscript, ada,
> applescript, bash, c, c#, c++, cpp, css, erlang, go, groovy, haskell, html,
> java, javascript, js, json, lua, none, nyan, objc, perl, php, python, r,
> rainbow, ruby, scala, sh, sql, swift, visualbasic, xml, yaml for
> (ClientProtos.Action action: mutations) {
> MutationProto m = action.getMutation();
> Mutation mutation;
> if (m.getMutateType() == MutationType.PUT) {
> mutation = ProtobufUtil.toPut(m, cells);
> batchContainsPuts = true;
> } else {
> mutation = ProtobufUtil.toDelete(m, cells);
> batchContainsDelete = true;
> }
> mutationActionMap.put(mutation, action);
> mArray[i++] = mutation;
> checkCellSizeLimit(region, mutation);
> // Check if a space quota disallows this mutation
> spaceQuotaEnforcement.getPolicyEnforcement(region).check(mutation);
> quota.addMutation(mutation);
> }
> {code}
> rs has caught the exception but it doesn't have the cellscanner skip the
> failed cells.
> {code:java}
> } catch (IOException ie) {
> if (atomic) {
> throw ie;
> }
> for (Action mutation : mutations) {
> builder.addResultOrException(getResultOrException(ie,
> mutation.getIndex()));
> }
> }
> {code}
> The bug results in the WrongRowIOException to remaining mutations since they
> refer to invalid cells.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-19876) The exception happening in converting pb mutation to hbase.mutation messes up the CellScanner
[
https://issues.apache.org/jira/browse/HBASE-19876?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16342938#comment-16342938
]
Anoop Sam John commented on HBASE-19876:
A related issue was fixed some time back.. Skipping the cur cell which is
problematic. So that might be missing for some places exception handling !!!
> The exception happening in converting pb mutation to hbase.mutation messes up
> the CellScanner
> -
>
> Key: HBASE-19876
> URL: https://issues.apache.org/jira/browse/HBASE-19876
> Project: HBase
> Issue Type: Bug
>Reporter: Chia-Ping Tsai
>Assignee: Chia-Ping Tsai
>Priority: Critical
> Fix For: 1.3.2, 1.5.0, 1.2.7, 2.0.0-beta-2, 1.4.2
>
>
> {code:java}
> 2018-01-27 22:51:43,794 INFO [hconnection-0x3291b443-shared-pool11-t6]
> client.AsyncRequestFutureImpl(778): id=5, table=testQuotaStatusFromMaster3,
> attempt=6/16 failed=20ops, last
> exception=org.apache.hadoop.hbase.client.WrongRowIOException:
> org.apache.hadoop.hbase.client.WrongRowIOException: The row in xxx doesn't
> match the original one aaa
> at org.apache.hadoop.hbase.client.Mutation.add(Mutation.java:776)
> at org.apache.hadoop.hbase.client.Put.add(Put.java:282)
> at
> org.apache.hadoop.hbase.shaded.protobuf.ProtobufUtil.toPut(ProtobufUtil.java:642)
> at
> org.apache.hadoop.hbase.regionserver.RSRpcServices.doBatchOp(RSRpcServices.java:952)
> at
> org.apache.hadoop.hbase.regionserver.RSRpcServices.doNonAtomicRegionMutation(RSRpcServices.java:896)
> at
> org.apache.hadoop.hbase.regionserver.RSRpcServices.multi(RSRpcServices.java:2591)
> at
> org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$ClientService$2.callBlockingMethod(ClientProtos.java:41560)
> at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:404)
> at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:130)
> at
> org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:324)
> at
> org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:304){code}
> I noticed this bug when testing the table space quota.
> When rs are converting pb mutation to hbase.mutation, the quota exception or
> cell exception may be thrown.
> {code}
> Unable to find source-code formatter for language:
> rsrpcservices#dobatchop.java. Available languages are: actionscript, ada,
> applescript, bash, c, c#, c++, cpp, css, erlang, go, groovy, haskell, html,
> java, javascript, js, json, lua, none, nyan, objc, perl, php, python, r,
> rainbow, ruby, scala, sh, sql, swift, visualbasic, xml, yaml for
> (ClientProtos.Action action: mutations) {
> MutationProto m = action.getMutation();
> Mutation mutation;
> if (m.getMutateType() == MutationType.PUT) {
> mutation = ProtobufUtil.toPut(m, cells);
> batchContainsPuts = true;
> } else {
> mutation = ProtobufUtil.toDelete(m, cells);
> batchContainsDelete = true;
> }
> mutationActionMap.put(mutation, action);
> mArray[i++] = mutation;
> checkCellSizeLimit(region, mutation);
> // Check if a space quota disallows this mutation
> spaceQuotaEnforcement.getPolicyEnforcement(region).check(mutation);
> quota.addMutation(mutation);
> }
> {code}
> rs has caught the exception but it doesn't have the cellscanner skip the
> failed cells.
> {code:java}
> } catch (IOException ie) {
> if (atomic) {
> throw ie;
> }
> for (Action mutation : mutations) {
> builder.addResultOrException(getResultOrException(ie,
> mutation.getIndex()));
> }
> }
> {code}
> The bug results in the WrongRowIOException to remaining mutations since they
> refer to invalid cells.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
