[jira] [Commented] (HBASE-13415) Procedure V2 - Use nonces for double submits from client
[ https://issues.apache.org/jira/browse/HBASE-13415?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15086492#comment-15086492 ] Hudson commented on HBASE-13415: FAILURE: Integrated in HBase-1.1-JDK8 #1721 (See [https://builds.apache.org/job/HBase-1.1-JDK8/1721/]) HBASE-13415 Procedure V2 - Use nonces for double submits from client (syuanjiangdev: rev 0c900fe7eebf5d5253c4f2bf69f04127dcf0c80a) * hbase-server/src/test/java/org/apache/hadoop/hbase/master/TestProcedureConf.java * hbase-server/src/test/java/org/apache/hadoop/hbase/master/procedure/TestEnableTableProcedure.java * hbase-server/src/test/java/org/apache/hadoop/hbase/master/procedure/TestTruncateTableProcedure.java * hbase-server/src/main/java/org/apache/hadoop/hbase/master/MasterServices.java * hbase-client/src/main/java/org/apache/hadoop/hbase/protobuf/RequestConverter.java * hbase-server/src/test/java/org/apache/hadoop/hbase/client/TestHBaseAdminNoCluster.java * hbase-server/src/main/java/org/apache/hadoop/hbase/security/access/AccessControlLists.java * hbase-common/src/main/java/org/apache/hadoop/hbase/util/NonceKey.java * hbase-server/src/test/java/org/apache/hadoop/hbase/master/procedure/MasterProcedureTestingUtility.java * hbase-server/src/test/java/org/apache/hadoop/hbase/master/procedure/TestCreateTableProcedure.java * hbase-server/src/main/java/org/apache/hadoop/hbase/security/visibility/VisibilityController.java * hbase-server/src/main/java/org/apache/hadoop/hbase/master/MasterRpcServices.java * hbase-server/src/test/java/org/apache/hadoop/hbase/master/procedure/TestDeleteTableProcedure.java * hbase-server/src/test/java/org/apache/hadoop/hbase/master/procedure/TestModifyColumnFamilyProcedure.java * hbase-server/src/main/java/org/apache/hadoop/hbase/master/HMaster.java * hbase-procedure/src/main/java/org/apache/hadoop/hbase/procedure2/Procedure.java * hbase-procedure/src/main/java/org/apache/hadoop/hbase/procedure2/ProcedureResult.java * hbase-server/src/test/java/org/apache/hadoop/hbase/master/procedure/TestModifyTableProcedure.java * hbase-protocol/src/main/java/org/apache/hadoop/hbase/protobuf/generated/MasterProtos.java * hbase-client/src/main/java/org/apache/hadoop/hbase/client/HBaseAdmin.java * hbase-procedure/src/test/java/org/apache/hadoop/hbase/procedure2/TestProcedureRecovery.java * hbase-server/src/test/java/org/apache/hadoop/hbase/master/TestAssignmentManagerOnCluster.java * hbase-protocol/src/main/protobuf/Master.proto * hbase-server/src/test/java/org/apache/hadoop/hbase/master/procedure/TestDisableTableProcedure.java * hbase-procedure/src/test/java/org/apache/hadoop/hbase/procedure2/ProcedureTestingUtility.java * hbase-protocol/src/main/java/org/apache/hadoop/hbase/protobuf/generated/ProcedureProtos.java * hbase-protocol/src/main/protobuf/Procedure.proto * hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/ServerNonceManager.java * hbase-server/src/test/java/org/apache/hadoop/hbase/master/TestCatalogJanitor.java * hbase-server/src/test/java/org/apache/hadoop/hbase/master/procedure/TestAddColumnFamilyProcedure.java * hbase-server/src/test/java/org/apache/hadoop/hbase/master/procedure/TestDeleteColumnFamilyProcedure.java * hbase-procedure/src/main/java/org/apache/hadoop/hbase/procedure2/ProcedureExecutor.java > Procedure V2 - Use nonces for double submits from client > > > Key: HBASE-13415 > URL: https://issues.apache.org/jira/browse/HBASE-13415 > Project: HBase > Issue Type: Sub-task > Components: master >Reporter: Enis Soztutar >Assignee: Stephen Yuan Jiang >Priority: Blocker > Fix For: 2.0.0, 1.2.0, 1.3.0, 1.1.3 > > Attachments: HBASE-13415.v1-branch-1.1.patch, > HBASE-13415.v1-master.patch, HBASE-13415.v2-master.patch, > HBASE-13415.v3-master.patch > > > The client can submit a procedure, but before getting the procId back, the > master might fail. In this case, the client request will fail and the client > will re-submit the request. If 1.1 client or if there is no contention for > the table lock, the time window is pretty small, but still might happen. > If the proc was accepted and stored in the procedure store, a re-submit from > the client will add another procedure, which will execute after the first > one. The first one will likely succeed, and the second one will fail (for > example in the case of create table, the second one will throw > TableExistsException). > One idea is to use client generated nonces (that we already have) to guard > against these cases. The client will submit the request with the nonce and > the nonce will be saved together with the procedure in the store. In case of > a double submit, the nonce-cache is checked and the procId of the original > request is returned. -- This message was sent by
[jira] [Commented] (HBASE-13415) Procedure V2 - Use nonces for double submits from client
[ https://issues.apache.org/jira/browse/HBASE-13415?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15086529#comment-15086529 ] Hudson commented on HBASE-13415: FAILURE: Integrated in HBase-1.1-JDK7 #1634 (See [https://builds.apache.org/job/HBase-1.1-JDK7/1634/]) HBASE-13415 Procedure V2 - Use nonces for double submits from client (syuanjiangdev: rev 0c900fe7eebf5d5253c4f2bf69f04127dcf0c80a) * hbase-procedure/src/main/java/org/apache/hadoop/hbase/procedure2/Procedure.java * hbase-server/src/test/java/org/apache/hadoop/hbase/master/TestCatalogJanitor.java * hbase-server/src/test/java/org/apache/hadoop/hbase/master/TestProcedureConf.java * hbase-server/src/test/java/org/apache/hadoop/hbase/master/TestAssignmentManagerOnCluster.java * hbase-protocol/src/main/protobuf/Master.proto * hbase-procedure/src/test/java/org/apache/hadoop/hbase/procedure2/TestProcedureRecovery.java * hbase-server/src/test/java/org/apache/hadoop/hbase/master/procedure/TestEnableTableProcedure.java * hbase-server/src/test/java/org/apache/hadoop/hbase/master/procedure/TestTruncateTableProcedure.java * hbase-server/src/test/java/org/apache/hadoop/hbase/master/procedure/TestModifyTableProcedure.java * hbase-server/src/test/java/org/apache/hadoop/hbase/master/procedure/TestModifyColumnFamilyProcedure.java * hbase-procedure/src/main/java/org/apache/hadoop/hbase/procedure2/ProcedureExecutor.java * hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/ServerNonceManager.java * hbase-server/src/test/java/org/apache/hadoop/hbase/master/procedure/TestAddColumnFamilyProcedure.java * hbase-server/src/test/java/org/apache/hadoop/hbase/master/procedure/TestDeleteTableProcedure.java * hbase-common/src/main/java/org/apache/hadoop/hbase/util/NonceKey.java * hbase-server/src/main/java/org/apache/hadoop/hbase/master/MasterServices.java * hbase-server/src/test/java/org/apache/hadoop/hbase/master/procedure/TestCreateTableProcedure.java * hbase-server/src/main/java/org/apache/hadoop/hbase/master/HMaster.java * hbase-procedure/src/test/java/org/apache/hadoop/hbase/procedure2/ProcedureTestingUtility.java * hbase-procedure/src/main/java/org/apache/hadoop/hbase/procedure2/ProcedureResult.java * hbase-server/src/main/java/org/apache/hadoop/hbase/security/visibility/VisibilityController.java * hbase-server/src/test/java/org/apache/hadoop/hbase/master/procedure/MasterProcedureTestingUtility.java * hbase-protocol/src/main/java/org/apache/hadoop/hbase/protobuf/generated/MasterProtos.java * hbase-client/src/main/java/org/apache/hadoop/hbase/client/HBaseAdmin.java * hbase-server/src/main/java/org/apache/hadoop/hbase/security/access/AccessControlLists.java * hbase-protocol/src/main/java/org/apache/hadoop/hbase/protobuf/generated/ProcedureProtos.java * hbase-server/src/test/java/org/apache/hadoop/hbase/master/procedure/TestDisableTableProcedure.java * hbase-server/src/test/java/org/apache/hadoop/hbase/client/TestHBaseAdminNoCluster.java * hbase-server/src/main/java/org/apache/hadoop/hbase/master/MasterRpcServices.java * hbase-client/src/main/java/org/apache/hadoop/hbase/protobuf/RequestConverter.java * hbase-server/src/test/java/org/apache/hadoop/hbase/master/procedure/TestDeleteColumnFamilyProcedure.java * hbase-protocol/src/main/protobuf/Procedure.proto > Procedure V2 - Use nonces for double submits from client > > > Key: HBASE-13415 > URL: https://issues.apache.org/jira/browse/HBASE-13415 > Project: HBase > Issue Type: Sub-task > Components: master >Reporter: Enis Soztutar >Assignee: Stephen Yuan Jiang >Priority: Blocker > Fix For: 2.0.0, 1.2.0, 1.3.0, 1.1.3 > > Attachments: HBASE-13415.v1-branch-1.1.patch, > HBASE-13415.v1-master.patch, HBASE-13415.v2-master.patch, > HBASE-13415.v3-master.patch > > > The client can submit a procedure, but before getting the procId back, the > master might fail. In this case, the client request will fail and the client > will re-submit the request. If 1.1 client or if there is no contention for > the table lock, the time window is pretty small, but still might happen. > If the proc was accepted and stored in the procedure store, a re-submit from > the client will add another procedure, which will execute after the first > one. The first one will likely succeed, and the second one will fail (for > example in the case of create table, the second one will throw > TableExistsException). > One idea is to use client generated nonces (that we already have) to guard > against these cases. The client will submit the request with the nonce and > the nonce will be saved together with the procedure in the store. In case of > a double submit, the nonce-cache is checked and the procId of the original > request is returned. -- This message was sent by
[jira] [Commented] (HBASE-13415) Procedure V2 - Use nonces for double submits from client
[ https://issues.apache.org/jira/browse/HBASE-13415?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14625189#comment-14625189 ] Hudson commented on HBASE-13415: FAILURE: Integrated in HBase-TRUNK #6647 (See [https://builds.apache.org/job/HBase-TRUNK/6647/]) HBASE-13415 Procedure v2 - Use nonces for double submits from client (Stephen Yuan Jiang) (busbey: rev 951ec7a0b7ade9ea8aa0b112537a267b5ce693c3) * hbase-procedure/src/main/java/org/apache/hadoop/hbase/procedure2/ProcedureResult.java * hbase-server/src/test/java/org/apache/hadoop/hbase/master/procedure/TestAddColumnFamilyProcedure.java * hbase-server/src/test/java/org/apache/hadoop/hbase/master/procedure/TestModifyColumnFamilyProcedure.java * hbase-protocol/src/main/protobuf/Master.proto * hbase-server/src/test/java/org/apache/hadoop/hbase/master/procedure/MasterProcedureTestingUtility.java * hbase-protocol/src/main/java/org/apache/hadoop/hbase/protobuf/generated/ProcedureProtos.java * hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/ServerNonceManager.java * hbase-server/src/main/java/org/apache/hadoop/hbase/security/visibility/VisibilityController.java * hbase-procedure/src/test/java/org/apache/hadoop/hbase/procedure2/TestProcedureRecovery.java * hbase-server/src/main/java/org/apache/hadoop/hbase/master/MasterRpcServices.java * hbase-server/src/test/java/org/apache/hadoop/hbase/master/procedure/TestEnableTableProcedure.java * hbase-server/src/test/java/org/apache/hadoop/hbase/master/procedure/TestTruncateTableProcedure.java * hbase-client/src/main/java/org/apache/hadoop/hbase/protobuf/RequestConverter.java * hbase-server/src/test/java/org/apache/hadoop/hbase/client/TestHBaseAdminNoCluster.java * hbase-procedure/src/main/java/org/apache/hadoop/hbase/procedure2/Procedure.java * hbase-protocol/src/main/protobuf/Procedure.proto * hbase-common/src/main/java/org/apache/hadoop/hbase/util/NonceKey.java * hbase-protocol/src/main/java/org/apache/hadoop/hbase/protobuf/generated/MasterProtos.java * hbase-server/src/test/java/org/apache/hadoop/hbase/master/procedure/TestDisableTableProcedure.java * hbase-server/src/test/java/org/apache/hadoop/hbase/master/TestCatalogJanitor.java * hbase-server/src/main/java/org/apache/hadoop/hbase/master/MasterServices.java * hbase-procedure/src/main/java/org/apache/hadoop/hbase/procedure2/ProcedureExecutor.java * hbase-server/src/test/java/org/apache/hadoop/hbase/master/procedure/TestCreateTableProcedure.java * hbase-client/src/main/java/org/apache/hadoop/hbase/client/HBaseAdmin.java * hbase-server/src/test/java/org/apache/hadoop/hbase/master/procedure/TestModifyTableProcedure.java * hbase-server/src/test/java/org/apache/hadoop/hbase/master/procedure/TestDeleteTableProcedure.java * hbase-server/src/test/java/org/apache/hadoop/hbase/master/procedure/TestDeleteColumnFamilyProcedure.java * hbase-server/src/main/java/org/apache/hadoop/hbase/master/HMaster.java * hbase-server/src/test/java/org/apache/hadoop/hbase/master/TestAssignmentManagerOnCluster.java * hbase-server/src/main/java/org/apache/hadoop/hbase/security/access/AccessControlLists.java * hbase-procedure/src/test/java/org/apache/hadoop/hbase/procedure2/ProcedureTestingUtility.java Procedure V2 - Use nonces for double submits from client Key: HBASE-13415 URL: https://issues.apache.org/jira/browse/HBASE-13415 Project: HBase Issue Type: Sub-task Components: master Reporter: Enis Soztutar Assignee: Stephen Yuan Jiang Priority: Blocker Fix For: 2.0.0, 1.2.0, 1.3.0 Attachments: HBASE-13415.v1-master.patch, HBASE-13415.v2-master.patch, HBASE-13415.v3-master.patch The client can submit a procedure, but before getting the procId back, the master might fail. In this case, the client request will fail and the client will re-submit the request. If 1.1 client or if there is no contention for the table lock, the time window is pretty small, but still might happen. If the proc was accepted and stored in the procedure store, a re-submit from the client will add another procedure, which will execute after the first one. The first one will likely succeed, and the second one will fail (for example in the case of create table, the second one will throw TableExistsException). One idea is to use client generated nonces (that we already have) to guard against these cases. The client will submit the request with the nonce and the nonce will be saved together with the procedure in the store. In case of a double submit, the nonce-cache is checked and the procId of the original request is returned. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-13415) Procedure V2 - Use nonces for double submits from client
[ https://issues.apache.org/jira/browse/HBASE-13415?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14625194#comment-14625194 ] Hudson commented on HBASE-13415: FAILURE: Integrated in HBase-1.3 #51 (See [https://builds.apache.org/job/HBase-1.3/51/]) HBASE-13415 Procedure v2 - Use nonces for double submits from client (Stephen Yuan Jiang) (busbey: rev b62f1345450741d823ee82e5cfa8a279837e3b97) * hbase-server/src/test/java/org/apache/hadoop/hbase/master/procedure/TestModifyTableProcedure.java * hbase-client/src/main/java/org/apache/hadoop/hbase/protobuf/RequestConverter.java * hbase-procedure/src/main/java/org/apache/hadoop/hbase/procedure2/ProcedureExecutor.java * hbase-server/src/main/java/org/apache/hadoop/hbase/master/MasterRpcServices.java * hbase-server/src/test/java/org/apache/hadoop/hbase/master/procedure/TestTruncateTableProcedure.java * hbase-procedure/src/main/java/org/apache/hadoop/hbase/procedure2/ProcedureResult.java * hbase-server/src/main/java/org/apache/hadoop/hbase/security/access/AccessControlLists.java * hbase-server/src/test/java/org/apache/hadoop/hbase/master/procedure/TestModifyColumnFamilyProcedure.java * hbase-server/src/test/java/org/apache/hadoop/hbase/master/procedure/TestDisableTableProcedure.java * hbase-protocol/src/main/protobuf/Procedure.proto * hbase-protocol/src/main/java/org/apache/hadoop/hbase/protobuf/generated/MasterProtos.java * hbase-server/src/test/java/org/apache/hadoop/hbase/master/procedure/TestAddColumnFamilyProcedure.java * hbase-server/src/main/java/org/apache/hadoop/hbase/master/MasterServices.java * hbase-server/src/test/java/org/apache/hadoop/hbase/master/procedure/MasterProcedureTestingUtility.java * hbase-server/src/test/java/org/apache/hadoop/hbase/client/TestHBaseAdminNoCluster.java * hbase-protocol/src/main/protobuf/Master.proto * hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/ServerNonceManager.java * hbase-procedure/src/main/java/org/apache/hadoop/hbase/procedure2/Procedure.java * hbase-protocol/src/main/java/org/apache/hadoop/hbase/protobuf/generated/ProcedureProtos.java * hbase-server/src/test/java/org/apache/hadoop/hbase/master/procedure/TestDeleteColumnFamilyProcedure.java * hbase-server/src/test/java/org/apache/hadoop/hbase/master/TestCatalogJanitor.java * hbase-client/src/main/java/org/apache/hadoop/hbase/client/HBaseAdmin.java * hbase-server/src/test/java/org/apache/hadoop/hbase/master/procedure/TestEnableTableProcedure.java * hbase-common/src/main/java/org/apache/hadoop/hbase/util/NonceKey.java * hbase-server/src/main/java/org/apache/hadoop/hbase/security/visibility/VisibilityController.java * hbase-server/src/test/java/org/apache/hadoop/hbase/master/procedure/TestDeleteTableProcedure.java * hbase-server/src/main/java/org/apache/hadoop/hbase/master/HMaster.java * hbase-server/src/test/java/org/apache/hadoop/hbase/master/TestAssignmentManagerOnCluster.java * hbase-server/src/test/java/org/apache/hadoop/hbase/master/procedure/TestCreateTableProcedure.java * hbase-procedure/src/test/java/org/apache/hadoop/hbase/procedure2/TestProcedureRecovery.java * hbase-procedure/src/test/java/org/apache/hadoop/hbase/procedure2/ProcedureTestingUtility.java Procedure V2 - Use nonces for double submits from client Key: HBASE-13415 URL: https://issues.apache.org/jira/browse/HBASE-13415 Project: HBase Issue Type: Sub-task Components: master Reporter: Enis Soztutar Assignee: Stephen Yuan Jiang Priority: Blocker Fix For: 2.0.0, 1.2.0, 1.3.0 Attachments: HBASE-13415.v1-master.patch, HBASE-13415.v2-master.patch, HBASE-13415.v3-master.patch The client can submit a procedure, but before getting the procId back, the master might fail. In this case, the client request will fail and the client will re-submit the request. If 1.1 client or if there is no contention for the table lock, the time window is pretty small, but still might happen. If the proc was accepted and stored in the procedure store, a re-submit from the client will add another procedure, which will execute after the first one. The first one will likely succeed, and the second one will fail (for example in the case of create table, the second one will throw TableExistsException). One idea is to use client generated nonces (that we already have) to guard against these cases. The client will submit the request with the nonce and the nonce will be saved together with the procedure in the store. In case of a double submit, the nonce-cache is checked and the procId of the original request is returned. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-13415) Procedure V2 - Use nonces for double submits from client
[ https://issues.apache.org/jira/browse/HBASE-13415?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14625217#comment-14625217 ] Hudson commented on HBASE-13415: FAILURE: Integrated in HBase-1.2 #64 (See [https://builds.apache.org/job/HBase-1.2/64/]) HBASE-13415 Procedure v2 - Use nonces for double submits from client (Stephen Yuan Jiang) (busbey: rev d360200bdb0ffdb94fef68c6337b9de7012fe525) * hbase-protocol/src/main/java/org/apache/hadoop/hbase/protobuf/generated/MasterProtos.java * hbase-server/src/main/java/org/apache/hadoop/hbase/security/visibility/VisibilityController.java * hbase-procedure/src/main/java/org/apache/hadoop/hbase/procedure2/Procedure.java * hbase-client/src/main/java/org/apache/hadoop/hbase/client/HBaseAdmin.java * hbase-procedure/src/test/java/org/apache/hadoop/hbase/procedure2/ProcedureTestingUtility.java * hbase-server/src/test/java/org/apache/hadoop/hbase/client/TestHBaseAdminNoCluster.java * hbase-server/src/test/java/org/apache/hadoop/hbase/master/TestAssignmentManagerOnCluster.java * hbase-server/src/test/java/org/apache/hadoop/hbase/master/procedure/TestModifyTableProcedure.java * hbase-client/src/main/java/org/apache/hadoop/hbase/protobuf/RequestConverter.java * hbase-server/src/test/java/org/apache/hadoop/hbase/master/procedure/MasterProcedureTestingUtility.java * hbase-procedure/src/main/java/org/apache/hadoop/hbase/procedure2/ProcedureResult.java * hbase-server/src/test/java/org/apache/hadoop/hbase/master/procedure/TestDeleteColumnFamilyProcedure.java * hbase-server/src/test/java/org/apache/hadoop/hbase/master/procedure/TestTruncateTableProcedure.java * hbase-procedure/src/test/java/org/apache/hadoop/hbase/procedure2/TestProcedureRecovery.java * hbase-server/src/test/java/org/apache/hadoop/hbase/master/procedure/TestModifyColumnFamilyProcedure.java * hbase-common/src/main/java/org/apache/hadoop/hbase/util/NonceKey.java * hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/ServerNonceManager.java * hbase-server/src/test/java/org/apache/hadoop/hbase/master/TestCatalogJanitor.java * hbase-server/src/main/java/org/apache/hadoop/hbase/master/HMaster.java * hbase-protocol/src/main/java/org/apache/hadoop/hbase/protobuf/generated/ProcedureProtos.java * hbase-protocol/src/main/protobuf/Procedure.proto * hbase-server/src/test/java/org/apache/hadoop/hbase/master/procedure/TestCreateTableProcedure.java * hbase-server/src/main/java/org/apache/hadoop/hbase/master/MasterRpcServices.java * hbase-procedure/src/main/java/org/apache/hadoop/hbase/procedure2/ProcedureExecutor.java * hbase-server/src/test/java/org/apache/hadoop/hbase/master/procedure/TestAddColumnFamilyProcedure.java * hbase-server/src/main/java/org/apache/hadoop/hbase/master/MasterServices.java * hbase-server/src/main/java/org/apache/hadoop/hbase/security/access/AccessControlLists.java * hbase-server/src/test/java/org/apache/hadoop/hbase/master/procedure/TestEnableTableProcedure.java * hbase-server/src/test/java/org/apache/hadoop/hbase/master/procedure/TestDisableTableProcedure.java * hbase-server/src/test/java/org/apache/hadoop/hbase/master/procedure/TestDeleteTableProcedure.java * hbase-protocol/src/main/protobuf/Master.proto Procedure V2 - Use nonces for double submits from client Key: HBASE-13415 URL: https://issues.apache.org/jira/browse/HBASE-13415 Project: HBase Issue Type: Sub-task Components: master Reporter: Enis Soztutar Assignee: Stephen Yuan Jiang Priority: Blocker Fix For: 2.0.0, 1.2.0, 1.3.0 Attachments: HBASE-13415.v1-master.patch, HBASE-13415.v2-master.patch, HBASE-13415.v3-master.patch The client can submit a procedure, but before getting the procId back, the master might fail. In this case, the client request will fail and the client will re-submit the request. If 1.1 client or if there is no contention for the table lock, the time window is pretty small, but still might happen. If the proc was accepted and stored in the procedure store, a re-submit from the client will add another procedure, which will execute after the first one. The first one will likely succeed, and the second one will fail (for example in the case of create table, the second one will throw TableExistsException). One idea is to use client generated nonces (that we already have) to guard against these cases. The client will submit the request with the nonce and the nonce will be saved together with the procedure in the store. In case of a double submit, the nonce-cache is checked and the procId of the original request is returned. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-13415) Procedure V2 - Use nonces for double submits from client
[ https://issues.apache.org/jira/browse/HBASE-13415?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14625362#comment-14625362 ] Hudson commented on HBASE-13415: SUCCESS: Integrated in HBase-1.2-IT #50 (See [https://builds.apache.org/job/HBase-1.2-IT/50/]) HBASE-13415 Procedure v2 - Use nonces for double submits from client (Stephen Yuan Jiang) (busbey: rev d360200bdb0ffdb94fef68c6337b9de7012fe525) * hbase-client/src/main/java/org/apache/hadoop/hbase/protobuf/RequestConverter.java * hbase-server/src/test/java/org/apache/hadoop/hbase/master/procedure/TestDisableTableProcedure.java * hbase-protocol/src/main/java/org/apache/hadoop/hbase/protobuf/generated/MasterProtos.java * hbase-server/src/test/java/org/apache/hadoop/hbase/master/procedure/TestAddColumnFamilyProcedure.java * hbase-server/src/test/java/org/apache/hadoop/hbase/master/procedure/TestDeleteColumnFamilyProcedure.java * hbase-procedure/src/test/java/org/apache/hadoop/hbase/procedure2/TestProcedureRecovery.java * hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/ServerNonceManager.java * hbase-server/src/test/java/org/apache/hadoop/hbase/master/procedure/TestCreateTableProcedure.java * hbase-server/src/test/java/org/apache/hadoop/hbase/master/TestCatalogJanitor.java * hbase-procedure/src/main/java/org/apache/hadoop/hbase/procedure2/ProcedureResult.java * hbase-common/src/main/java/org/apache/hadoop/hbase/util/NonceKey.java * hbase-client/src/main/java/org/apache/hadoop/hbase/client/HBaseAdmin.java * hbase-server/src/test/java/org/apache/hadoop/hbase/master/procedure/TestEnableTableProcedure.java * hbase-protocol/src/main/protobuf/Master.proto * hbase-procedure/src/main/java/org/apache/hadoop/hbase/procedure2/ProcedureExecutor.java * hbase-server/src/main/java/org/apache/hadoop/hbase/master/HMaster.java * hbase-server/src/main/java/org/apache/hadoop/hbase/security/visibility/VisibilityController.java * hbase-server/src/test/java/org/apache/hadoop/hbase/master/procedure/MasterProcedureTestingUtility.java * hbase-server/src/test/java/org/apache/hadoop/hbase/master/procedure/TestDeleteTableProcedure.java * hbase-procedure/src/test/java/org/apache/hadoop/hbase/procedure2/ProcedureTestingUtility.java * hbase-server/src/main/java/org/apache/hadoop/hbase/master/MasterServices.java * hbase-server/src/test/java/org/apache/hadoop/hbase/master/procedure/TestTruncateTableProcedure.java * hbase-server/src/test/java/org/apache/hadoop/hbase/master/procedure/TestModifyColumnFamilyProcedure.java * hbase-server/src/main/java/org/apache/hadoop/hbase/security/access/AccessControlLists.java * hbase-protocol/src/main/protobuf/Procedure.proto * hbase-server/src/test/java/org/apache/hadoop/hbase/master/procedure/TestModifyTableProcedure.java * hbase-protocol/src/main/java/org/apache/hadoop/hbase/protobuf/generated/ProcedureProtos.java * hbase-procedure/src/main/java/org/apache/hadoop/hbase/procedure2/Procedure.java * hbase-server/src/test/java/org/apache/hadoop/hbase/master/TestAssignmentManagerOnCluster.java * hbase-server/src/main/java/org/apache/hadoop/hbase/master/MasterRpcServices.java * hbase-server/src/test/java/org/apache/hadoop/hbase/client/TestHBaseAdminNoCluster.java Procedure V2 - Use nonces for double submits from client Key: HBASE-13415 URL: https://issues.apache.org/jira/browse/HBASE-13415 Project: HBase Issue Type: Sub-task Components: master Reporter: Enis Soztutar Assignee: Stephen Yuan Jiang Priority: Blocker Fix For: 2.0.0, 1.2.0, 1.3.0 Attachments: HBASE-13415.v1-master.patch, HBASE-13415.v2-master.patch, HBASE-13415.v3-master.patch The client can submit a procedure, but before getting the procId back, the master might fail. In this case, the client request will fail and the client will re-submit the request. If 1.1 client or if there is no contention for the table lock, the time window is pretty small, but still might happen. If the proc was accepted and stored in the procedure store, a re-submit from the client will add another procedure, which will execute after the first one. The first one will likely succeed, and the second one will fail (for example in the case of create table, the second one will throw TableExistsException). One idea is to use client generated nonces (that we already have) to guard against these cases. The client will submit the request with the nonce and the nonce will be saved together with the procedure in the store. In case of a double submit, the nonce-cache is checked and the procId of the original request is returned. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-13415) Procedure V2 - Use nonces for double submits from client
[ https://issues.apache.org/jira/browse/HBASE-13415?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14625399#comment-14625399 ] Hudson commented on HBASE-13415: SUCCESS: Integrated in HBase-1.3-IT #37 (See [https://builds.apache.org/job/HBase-1.3-IT/37/]) HBASE-13415 Procedure v2 - Use nonces for double submits from client (Stephen Yuan Jiang) (busbey: rev b62f1345450741d823ee82e5cfa8a279837e3b97) * hbase-server/src/test/java/org/apache/hadoop/hbase/master/procedure/TestCreateTableProcedure.java * hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/ServerNonceManager.java * hbase-server/src/test/java/org/apache/hadoop/hbase/master/TestCatalogJanitor.java * hbase-procedure/src/test/java/org/apache/hadoop/hbase/procedure2/TestProcedureRecovery.java * hbase-common/src/main/java/org/apache/hadoop/hbase/util/NonceKey.java * hbase-client/src/main/java/org/apache/hadoop/hbase/client/HBaseAdmin.java * hbase-server/src/test/java/org/apache/hadoop/hbase/master/procedure/TestAddColumnFamilyProcedure.java * hbase-server/src/main/java/org/apache/hadoop/hbase/master/MasterServices.java * hbase-server/src/test/java/org/apache/hadoop/hbase/master/procedure/TestTruncateTableProcedure.java * hbase-server/src/test/java/org/apache/hadoop/hbase/master/procedure/TestModifyTableProcedure.java * hbase-server/src/main/java/org/apache/hadoop/hbase/security/visibility/VisibilityController.java * hbase-server/src/main/java/org/apache/hadoop/hbase/master/HMaster.java * hbase-server/src/test/java/org/apache/hadoop/hbase/master/procedure/TestModifyColumnFamilyProcedure.java * hbase-procedure/src/main/java/org/apache/hadoop/hbase/procedure2/Procedure.java * hbase-server/src/test/java/org/apache/hadoop/hbase/master/procedure/TestDeleteTableProcedure.java * hbase-procedure/src/main/java/org/apache/hadoop/hbase/procedure2/ProcedureExecutor.java * hbase-procedure/src/test/java/org/apache/hadoop/hbase/procedure2/ProcedureTestingUtility.java * hbase-server/src/test/java/org/apache/hadoop/hbase/master/procedure/MasterProcedureTestingUtility.java * hbase-procedure/src/main/java/org/apache/hadoop/hbase/procedure2/ProcedureResult.java * hbase-protocol/src/main/java/org/apache/hadoop/hbase/protobuf/generated/MasterProtos.java * hbase-server/src/test/java/org/apache/hadoop/hbase/client/TestHBaseAdminNoCluster.java * hbase-server/src/test/java/org/apache/hadoop/hbase/master/TestAssignmentManagerOnCluster.java * hbase-server/src/test/java/org/apache/hadoop/hbase/master/procedure/TestEnableTableProcedure.java * hbase-protocol/src/main/java/org/apache/hadoop/hbase/protobuf/generated/ProcedureProtos.java * hbase-server/src/test/java/org/apache/hadoop/hbase/master/procedure/TestDisableTableProcedure.java * hbase-client/src/main/java/org/apache/hadoop/hbase/protobuf/RequestConverter.java * hbase-server/src/test/java/org/apache/hadoop/hbase/master/procedure/TestDeleteColumnFamilyProcedure.java * hbase-protocol/src/main/protobuf/Master.proto * hbase-protocol/src/main/protobuf/Procedure.proto * hbase-server/src/main/java/org/apache/hadoop/hbase/master/MasterRpcServices.java * hbase-server/src/main/java/org/apache/hadoop/hbase/security/access/AccessControlLists.java Procedure V2 - Use nonces for double submits from client Key: HBASE-13415 URL: https://issues.apache.org/jira/browse/HBASE-13415 Project: HBase Issue Type: Sub-task Components: master Reporter: Enis Soztutar Assignee: Stephen Yuan Jiang Priority: Blocker Fix For: 2.0.0, 1.2.0, 1.3.0 Attachments: HBASE-13415.v1-master.patch, HBASE-13415.v2-master.patch, HBASE-13415.v3-master.patch The client can submit a procedure, but before getting the procId back, the master might fail. In this case, the client request will fail and the client will re-submit the request. If 1.1 client or if there is no contention for the table lock, the time window is pretty small, but still might happen. If the proc was accepted and stored in the procedure store, a re-submit from the client will add another procedure, which will execute after the first one. The first one will likely succeed, and the second one will fail (for example in the case of create table, the second one will throw TableExistsException). One idea is to use client generated nonces (that we already have) to guard against these cases. The client will submit the request with the nonce and the nonce will be saved together with the procedure in the store. In case of a double submit, the nonce-cache is checked and the procId of the original request is returned. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-13415) Procedure V2 - Use nonces for double submits from client
[
https://issues.apache.org/jira/browse/HBASE-13415?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14620005#comment-14620005
]
Hadoop QA commented on HBASE-13415:
---
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12744389/HBASE-13415.v3-master.patch
against master branch at commit 41c8ec7aeae859808a217bd7a561e81be7e3c7ac.
ATTACHMENT ID: 12744389
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 45 new
or modified tests.
{color:green}+1 hadoop versions{color}. The patch compiles with all
supported hadoop versions (2.4.0 2.4.1 2.5.0 2.5.1 2.5.2 2.6.0 2.7.0)
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 protoc{color}. The applied patch does not increase the
total number of protoc compiler warnings.
{color:green}+1 javadoc{color}. The javadoc tool did not generate any
warning messages.
{color:green}+1 checkstyle{color}. The applied patch does not increase the
total number of checkstyle errors
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 2.0.3) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:red}-1 lineLengths{color}. The patch introduces the following lines
longer than 100:
+ (\r\022\014\n\004type\030\002 \002(\r\022\016\n\006log_id\030\003
\002(\004\022\023\n\013min_ +
+ \030\003
\001(\004\J\n\004Type\022\007\n\003EOF\020\001\022\010\n\004INIT\020\002\022\n\n\006INS
+
+ new java.lang.String[] { ClassName, ParentId, ProcId,
StartTime, Owner, State, StackId, LastUpdate, Timeout, Exception,
Result, StateData, NonceGroup, Nonce, });
{color:green}+1 site{color}. The mvn post-site goal succeeds with this patch.
{color:red}-1 core tests{color}. The patch failed these unit tests:
{color:red}-1 core zombie tests{color}. There are 2 zombie test(s):
at
org.apache.hadoop.hbase.snapshot.TestExportSnapshot.testExportFileSystemState(TestExportSnapshot.java:288)
at
org.apache.hadoop.hbase.snapshot.TestExportSnapshot.testExportFileSystemState(TestExportSnapshot.java:262)
at
org.apache.hadoop.hbase.snapshot.TestExportSnapshot.testExportFileSystemStateWithSkipTmp(TestExportSnapshot.java:201)
at
org.apache.hadoop.hbase.snapshot.TestExportSnapshot.testExportFailure(TestExportSnapshot.java:317)
Test results:
https://builds.apache.org/job/PreCommit-HBASE-Build/14712//testReport/
Release Findbugs (version 2.0.3)warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/14712//artifact/patchprocess/newFindbugsWarnings.html
Checkstyle Errors:
https://builds.apache.org/job/PreCommit-HBASE-Build/14712//artifact/patchprocess/checkstyle-aggregate.html
Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/14712//console
This message is automatically generated.
Procedure V2 - Use nonces for double submits from client
Key: HBASE-13415
URL: https://issues.apache.org/jira/browse/HBASE-13415
Project: HBase
Issue Type: Sub-task
Components: master
Reporter: Enis Soztutar
Assignee: Stephen Yuan Jiang
Priority: Blocker
Fix For: 2.0.0, 1.2.0, 1.3.0
Attachments: HBASE-13415.v1-master.patch,
HBASE-13415.v2-master.patch, HBASE-13415.v3-master.patch
The client can submit a procedure, but before getting the procId back, the
master might fail. In this case, the client request will fail and the client
will re-submit the request. If 1.1 client or if there is no contention for
the table lock, the time window is pretty small, but still might happen.
If the proc was accepted and stored in the procedure store, a re-submit from
the client will add another procedure, which will execute after the first
one. The first one will likely succeed, and the second one will fail (for
example in the case of create table, the second one will throw
TableExistsException).
One idea is to use client generated nonces (that we already have) to guard
against these cases. The client will submit the request with the nonce and
the nonce will be saved together with the procedure in the store. In case of
a double submit, the nonce-cache is checked and the procId of the original
request is returned.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-13415) Procedure V2 - Use nonces for double submits from client
[
https://issues.apache.org/jira/browse/HBASE-13415?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14619107#comment-14619107
]
Hadoop QA commented on HBASE-13415:
---
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12744269/HBASE-13415.v2-master.patch
against master branch at commit f5ad736282c8c9c27b14131919d60b72834ec9e4.
ATTACHMENT ID: 12744269
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 45 new
or modified tests.
{color:green}+1 hadoop versions{color}. The patch compiles with all
supported hadoop versions (2.4.0 2.4.1 2.5.0 2.5.1 2.5.2 2.6.0 2.7.0)
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 protoc{color}. The applied patch does not increase the
total number of protoc compiler warnings.
{color:green}+1 javadoc{color}. The javadoc tool did not generate any
warning messages.
{color:green}+1 checkstyle{color}. The applied patch does not increase the
total number of checkstyle errors
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 2.0.3) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:red}-1 lineLengths{color}. The patch introduces the following lines
longer than 100:
+ (\r\022\014\n\004type\030\002 \002(\r\022\016\n\006log_id\030\003
\002(\004\022\023\n\013min_ +
+ \030\003
\001(\004\J\n\004Type\022\007\n\003EOF\020\001\022\010\n\004INIT\020\002\022\n\n\006INS
+
+ new java.lang.String[] { ClassName, ParentId, ProcId,
StartTime, Owner, State, StackId, LastUpdate, Timeout, Exception,
Result, StateData, NonceGroup, Nonce, });
{color:green}+1 site{color}. The mvn post-site goal succeeds with this patch.
{color:red}-1 core tests{color}. The patch failed these unit tests:
Test results:
https://builds.apache.org/job/PreCommit-HBASE-Build/14704//testReport/
Release Findbugs (version 2.0.3)warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/14704//artifact/patchprocess/newFindbugsWarnings.html
Checkstyle Errors:
https://builds.apache.org/job/PreCommit-HBASE-Build/14704//artifact/patchprocess/checkstyle-aggregate.html
Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/14704//console
This message is automatically generated.
Procedure V2 - Use nonces for double submits from client
Key: HBASE-13415
URL: https://issues.apache.org/jira/browse/HBASE-13415
Project: HBase
Issue Type: Sub-task
Components: master
Reporter: Enis Soztutar
Assignee: Stephen Yuan Jiang
Priority: Blocker
Fix For: 2.0.0, 1.2.0, 1.3.0
Attachments: HBASE-13415.v1-master.patch, HBASE-13415.v2-master.patch
The client can submit a procedure, but before getting the procId back, the
master might fail. In this case, the client request will fail and the client
will re-submit the request. If 1.1 client or if there is no contention for
the table lock, the time window is pretty small, but still might happen.
If the proc was accepted and stored in the procedure store, a re-submit from
the client will add another procedure, which will execute after the first
one. The first one will likely succeed, and the second one will fail (for
example in the case of create table, the second one will throw
TableExistsException).
One idea is to use client generated nonces (that we already have) to guard
against these cases. The client will submit the request with the nonce and
the nonce will be saved together with the procedure in the store. In case of
a double submit, the nonce-cache is checked and the procId of the original
request is returned.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-13415) Procedure V2 - Use nonces for double submits from client
[ https://issues.apache.org/jira/browse/HBASE-13415?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14619138#comment-14619138 ] Stephen Yuan Jiang commented on HBASE-13415: The lineLenths from generated files and hence I have no control. I could not find any test failure from testReport (https://builds.apache.org/job/PreCommit-HBASE-Build/14704//testReport/), but the medium and large class test not run. Procedure V2 - Use nonces for double submits from client Key: HBASE-13415 URL: https://issues.apache.org/jira/browse/HBASE-13415 Project: HBase Issue Type: Sub-task Components: master Reporter: Enis Soztutar Assignee: Stephen Yuan Jiang Priority: Blocker Fix For: 2.0.0, 1.2.0, 1.3.0 Attachments: HBASE-13415.v1-master.patch, HBASE-13415.v2-master.patch The client can submit a procedure, but before getting the procId back, the master might fail. In this case, the client request will fail and the client will re-submit the request. If 1.1 client or if there is no contention for the table lock, the time window is pretty small, but still might happen. If the proc was accepted and stored in the procedure store, a re-submit from the client will add another procedure, which will execute after the first one. The first one will likely succeed, and the second one will fail (for example in the case of create table, the second one will throw TableExistsException). One idea is to use client generated nonces (that we already have) to guard against these cases. The client will submit the request with the nonce and the nonce will be saved together with the procedure in the store. In case of a double submit, the nonce-cache is checked and the procId of the original request is returned. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-13415) Procedure V2 - Use nonces for double submits from client
[ https://issues.apache.org/jira/browse/HBASE-13415?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14619153#comment-14619153 ] Matteo Bertozzi commented on HBASE-13415: - looks good to me, let me commit it. I'll re-run the procedure related tests to confirm that everything is ok. Procedure V2 - Use nonces for double submits from client Key: HBASE-13415 URL: https://issues.apache.org/jira/browse/HBASE-13415 Project: HBase Issue Type: Sub-task Components: master Reporter: Enis Soztutar Assignee: Stephen Yuan Jiang Priority: Blocker Fix For: 2.0.0, 1.2.0, 1.3.0 Attachments: HBASE-13415.v1-master.patch, HBASE-13415.v2-master.patch The client can submit a procedure, but before getting the procId back, the master might fail. In this case, the client request will fail and the client will re-submit the request. If 1.1 client or if there is no contention for the table lock, the time window is pretty small, but still might happen. If the proc was accepted and stored in the procedure store, a re-submit from the client will add another procedure, which will execute after the first one. The first one will likely succeed, and the second one will fail (for example in the case of create table, the second one will throw TableExistsException). One idea is to use client generated nonces (that we already have) to guard against these cases. The client will submit the request with the nonce and the nonce will be saved together with the procedure in the store. In case of a double submit, the nonce-cache is checked and the procId of the original request is returned. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-13415) Procedure V2 - Use nonces for double submits from client
[ https://issues.apache.org/jira/browse/HBASE-13415?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14618134#comment-14618134 ] Stephen Yuan Jiang commented on HBASE-13415: The checkstyle and test failure are related to the patch. Will post a new patch soon. Procedure V2 - Use nonces for double submits from client Key: HBASE-13415 URL: https://issues.apache.org/jira/browse/HBASE-13415 Project: HBase Issue Type: Sub-task Components: master Reporter: Enis Soztutar Assignee: Stephen Yuan Jiang Priority: Blocker Fix For: 2.0.0, 1.2.0, 1.3.0 Attachments: HBASE-13415.v1-master.patch The client can submit a procedure, but before getting the procId back, the master might fail. In this case, the client request will fail and the client will re-submit the request. If 1.1 client or if there is no contention for the table lock, the time window is pretty small, but still might happen. If the proc was accepted and stored in the procedure store, a re-submit from the client will add another procedure, which will execute after the first one. The first one will likely succeed, and the second one will fail (for example in the case of create table, the second one will throw TableExistsException). One idea is to use client generated nonces (that we already have) to guard against these cases. The client will submit the request with the nonce and the nonce will be saved together with the procedure in the store. In case of a double submit, the nonce-cache is checked and the procId of the original request is returned. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-13415) Procedure V2 - Use nonces for double submits from client
[
https://issues.apache.org/jira/browse/HBASE-13415?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14618060#comment-14618060
]
Hadoop QA commented on HBASE-13415:
---
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12744134/HBASE-13415.v1-master.patch
against master branch at commit f5ad736282c8c9c27b14131919d60b72834ec9e4.
ATTACHMENT ID: 12744134
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 42 new
or modified tests.
{color:green}+1 hadoop versions{color}. The patch compiles with all
supported hadoop versions (2.4.0 2.4.1 2.5.0 2.5.1 2.5.2 2.6.0 2.7.0)
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 protoc{color}. The applied patch does not increase the
total number of protoc compiler warnings.
{color:green}+1 javadoc{color}. The javadoc tool did not generate any
warning messages.
{color:red}-1 checkstyle{color}. The applied patch generated
1906 checkstyle errors (more than the master's current 1898 errors).
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 2.0.3) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:red}-1 lineLengths{color}. The patch introduces the following lines
longer than 100:
+ final TableName tableName, final HColumnDescriptor column, final
long nonceGroup, final long nonce) {
+ final TableName tableName, final byte [] columnName, final long
nonceGroup, final long nonce) {
+ final TableName tableName, final HColumnDescriptor column, final long
nonceGroup, final long nonce) {
+ public static DeleteTableRequest buildDeleteTableRequest(final TableName
tableName, final long nonceGroup, final long nonce) {
+ public static EnableTableRequest buildEnableTableRequest(final TableName
tableName, final long nonceGroup, final long nonce) {
+ public static DisableTableRequest buildDisableTableRequest(final TableName
tableName, final long nonceGroup, final long nonce) {
+ final HTableDescriptor hTableDesc, final byte [][] splitKeys, final long
nonceGroup, final long nonce) {
+ final TableName tableName, final HTableDescriptor hTableDesc, final long
nonceGroup, final long nonce) {
+ (\r\022\014\n\004type\030\002 \002(\r\022\016\n\006log_id\030\003
\002(\004\022\023\n\013min_ +
+ \030\003
\001(\004\J\n\004Type\022\007\n\003EOF\020\001\022\010\n\004INIT\020\002\022\n\n\006INS
+
{color:green}+1 site{color}. The mvn post-site goal succeeds with this patch.
{color:red}-1 core tests{color}. The patch failed these unit tests:
org.apache.hadoop.hbase.client.TestHBaseAdminNoCluster
Test results:
https://builds.apache.org/job/PreCommit-HBASE-Build/14699//testReport/
Release Findbugs (version 2.0.3)warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/14699//artifact/patchprocess/newFindbugsWarnings.html
Checkstyle Errors:
https://builds.apache.org/job/PreCommit-HBASE-Build/14699//artifact/patchprocess/checkstyle-aggregate.html
Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/14699//console
This message is automatically generated.
Procedure V2 - Use nonces for double submits from client
Key: HBASE-13415
URL: https://issues.apache.org/jira/browse/HBASE-13415
Project: HBase
Issue Type: Sub-task
Components: master
Reporter: Enis Soztutar
Assignee: Stephen Yuan Jiang
Priority: Blocker
Fix For: 2.0.0, 1.2.0, 1.3.0
Attachments: HBASE-13415.v1-master.patch
The client can submit a procedure, but before getting the procId back, the
master might fail. In this case, the client request will fail and the client
will re-submit the request. If 1.1 client or if there is no contention for
the table lock, the time window is pretty small, but still might happen.
If the proc was accepted and stored in the procedure store, a re-submit from
the client will add another procedure, which will execute after the first
one. The first one will likely succeed, and the second one will fail (for
example in the case of create table, the second one will throw
TableExistsException).
One idea is to use client generated nonces (that we already have) to guard
against these cases. The client will submit the request with the nonce and
the nonce will be saved together with the procedure in the store. In case of
a double submit, the nonce-cache is
[jira] [Commented] (HBASE-13415) Procedure V2 - Use nonces for double submits from client
[ https://issues.apache.org/jira/browse/HBASE-13415?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14619754#comment-14619754 ] Enis Soztutar commented on HBASE-13415: --- v2 patch looks good. I have only one comment in RB. Procedure V2 - Use nonces for double submits from client Key: HBASE-13415 URL: https://issues.apache.org/jira/browse/HBASE-13415 Project: HBase Issue Type: Sub-task Components: master Reporter: Enis Soztutar Assignee: Stephen Yuan Jiang Priority: Blocker Fix For: 2.0.0, 1.2.0, 1.3.0 Attachments: HBASE-13415.v1-master.patch, HBASE-13415.v2-master.patch The client can submit a procedure, but before getting the procId back, the master might fail. In this case, the client request will fail and the client will re-submit the request. If 1.1 client or if there is no contention for the table lock, the time window is pretty small, but still might happen. If the proc was accepted and stored in the procedure store, a re-submit from the client will add another procedure, which will execute after the first one. The first one will likely succeed, and the second one will fail (for example in the case of create table, the second one will throw TableExistsException). One idea is to use client generated nonces (that we already have) to guard against these cases. The client will submit the request with the nonce and the nonce will be saved together with the procedure in the store. In case of a double submit, the nonce-cache is checked and the procId of the original request is returned. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-13415) Procedure V2 - Use nonces for double submits from client
[ https://issues.apache.org/jira/browse/HBASE-13415?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14619790#comment-14619790 ] Stephen Yuan Jiang commented on HBASE-13415: The V3 patch addressed the comment in RB. Procedure V2 - Use nonces for double submits from client Key: HBASE-13415 URL: https://issues.apache.org/jira/browse/HBASE-13415 Project: HBase Issue Type: Sub-task Components: master Reporter: Enis Soztutar Assignee: Stephen Yuan Jiang Priority: Blocker Fix For: 2.0.0, 1.2.0, 1.3.0 Attachments: HBASE-13415.v1-master.patch, HBASE-13415.v2-master.patch, HBASE-13415.v3-master.patch The client can submit a procedure, but before getting the procId back, the master might fail. In this case, the client request will fail and the client will re-submit the request. If 1.1 client or if there is no contention for the table lock, the time window is pretty small, but still might happen. If the proc was accepted and stored in the procedure store, a re-submit from the client will add another procedure, which will execute after the first one. The first one will likely succeed, and the second one will fail (for example in the case of create table, the second one will throw TableExistsException). One idea is to use client generated nonces (that we already have) to guard against these cases. The client will submit the request with the nonce and the nonce will be saved together with the procedure in the store. In case of a double submit, the nonce-cache is checked and the procId of the original request is returned. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-13415) Procedure V2 - Use nonces for double submits from client
[ https://issues.apache.org/jira/browse/HBASE-13415?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14616904#comment-14616904 ] Matteo Bertozzi commented on HBASE-13415: - I think it just need another round with the fixes pointed out on reviewboard, nothing too big. if [~syuanjiang] does not have time I may be able to complete it. Procedure V2 - Use nonces for double submits from client Key: HBASE-13415 URL: https://issues.apache.org/jira/browse/HBASE-13415 Project: HBase Issue Type: Sub-task Components: master Reporter: Enis Soztutar Assignee: Stephen Yuan Jiang Priority: Blocker Fix For: 2.0.0, 1.2.0, 1.3.0 The client can submit a procedure, but before getting the procId back, the master might fail. In this case, the client request will fail and the client will re-submit the request. If 1.1 client or if there is no contention for the table lock, the time window is pretty small, but still might happen. If the proc was accepted and stored in the procedure store, a re-submit from the client will add another procedure, which will execute after the first one. The first one will likely succeed, and the second one will fail (for example in the case of create table, the second one will throw TableExistsException). One idea is to use client generated nonces (that we already have) to guard against these cases. The client will submit the request with the nonce and the nonce will be saved together with the procedure in the store. In case of a double submit, the nonce-cache is checked and the procId of the original request is returned. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-13415) Procedure V2 - Use nonces for double submits from client
[ https://issues.apache.org/jira/browse/HBASE-13415?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14616972#comment-14616972 ] Stephen Yuan Jiang commented on HBASE-13415: [~busbey] I should be be able to complete soon. Procedure V2 - Use nonces for double submits from client Key: HBASE-13415 URL: https://issues.apache.org/jira/browse/HBASE-13415 Project: HBase Issue Type: Sub-task Components: master Reporter: Enis Soztutar Assignee: Stephen Yuan Jiang Priority: Blocker Fix For: 2.0.0, 1.2.0, 1.3.0 The client can submit a procedure, but before getting the procId back, the master might fail. In this case, the client request will fail and the client will re-submit the request. If 1.1 client or if there is no contention for the table lock, the time window is pretty small, but still might happen. If the proc was accepted and stored in the procedure store, a re-submit from the client will add another procedure, which will execute after the first one. The first one will likely succeed, and the second one will fail (for example in the case of create table, the second one will throw TableExistsException). One idea is to use client generated nonces (that we already have) to guard against these cases. The client will submit the request with the nonce and the nonce will be saved together with the procedure in the store. In case of a double submit, the nonce-cache is checked and the procId of the original request is returned. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-13415) Procedure V2 - Use nonces for double submits from client
[ https://issues.apache.org/jira/browse/HBASE-13415?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14616736#comment-14616736 ] Sean Busbey commented on HBASE-13415: - bumping priority to Blocker for 1.2 per request from [~enis]. How's this coming along? Procedure V2 - Use nonces for double submits from client Key: HBASE-13415 URL: https://issues.apache.org/jira/browse/HBASE-13415 Project: HBase Issue Type: Sub-task Components: master Reporter: Enis Soztutar Assignee: Stephen Yuan Jiang Priority: Blocker Fix For: 2.0.0, 1.2.0, 1.3.0 The client can submit a procedure, but before getting the procId back, the master might fail. In this case, the client request will fail and the client will re-submit the request. If 1.1 client or if there is no contention for the table lock, the time window is pretty small, but still might happen. If the proc was accepted and stored in the procedure store, a re-submit from the client will add another procedure, which will execute after the first one. The first one will likely succeed, and the second one will fail (for example in the case of create table, the second one will throw TableExistsException). One idea is to use client generated nonces (that we already have) to guard against these cases. The client will submit the request with the nonce and the nonce will be saved together with the procedure in the store. In case of a double submit, the nonce-cache is checked and the procId of the original request is returned. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-13415) Procedure V2 - Use nonces for double submits from client
[ https://issues.apache.org/jira/browse/HBASE-13415?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14617994#comment-14617994 ] Stephen Yuan Jiang commented on HBASE-13415: I run the TestModifyTableProcedure test suite under both maven (mvn) and eclipse - everything is normal. Procedure V2 - Use nonces for double submits from client Key: HBASE-13415 URL: https://issues.apache.org/jira/browse/HBASE-13415 Project: HBase Issue Type: Sub-task Components: master Reporter: Enis Soztutar Assignee: Stephen Yuan Jiang Priority: Blocker Fix For: 2.0.0, 1.2.0, 1.3.0 Attachments: HBASE-13415.v1-master.patch The client can submit a procedure, but before getting the procId back, the master might fail. In this case, the client request will fail and the client will re-submit the request. If 1.1 client or if there is no contention for the table lock, the time window is pretty small, but still might happen. If the proc was accepted and stored in the procedure store, a re-submit from the client will add another procedure, which will execute after the first one. The first one will likely succeed, and the second one will fail (for example in the case of create table, the second one will throw TableExistsException). One idea is to use client generated nonces (that we already have) to guard against these cases. The client will submit the request with the nonce and the nonce will be saved together with the procedure in the store. In case of a double submit, the nonce-cache is checked and the procId of the original request is returned. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-13415) Procedure V2 - Use nonces for double submits from client
[ https://issues.apache.org/jira/browse/HBASE-13415?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14603855#comment-14603855 ] Stephen Yuan Jiang commented on HBASE-13415: [~mbertozzi] [~enis] [~tedyu] The change is in https://reviews.apache.org/r/35957/ for review. Procedure V2 - Use nonces for double submits from client Key: HBASE-13415 URL: https://issues.apache.org/jira/browse/HBASE-13415 Project: HBase Issue Type: Sub-task Components: master Reporter: Enis Soztutar Assignee: Stephen Yuan Jiang The client can submit a procedure, but before getting the procId back, the master might fail. In this case, the client request will fail and the client will re-submit the request. If 1.1 client or if there is no contention for the table lock, the time window is pretty small, but still might happen. If the proc was accepted and stored in the procedure store, a re-submit from the client will add another procedure, which will execute after the first one. The first one will likely succeed, and the second one will fail (for example in the case of create table, the second one will throw TableExistsException). One idea is to use client generated nonces (that we already have) to guard against these cases. The client will submit the request with the nonce and the nonce will be saved together with the procedure in the store. In case of a double submit, the nonce-cache is checked and the procId of the original request is returned. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-13415) Procedure V2 - Use nonces for double submits from client
[ https://issues.apache.org/jira/browse/HBASE-13415?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14603953#comment-14603953 ] Ted Yu commented on HBASE-13415: I ran TestModifyTableProcedure twice - it hangs. Please check. Procedure V2 - Use nonces for double submits from client Key: HBASE-13415 URL: https://issues.apache.org/jira/browse/HBASE-13415 Project: HBase Issue Type: Sub-task Components: master Reporter: Enis Soztutar Assignee: Stephen Yuan Jiang The client can submit a procedure, but before getting the procId back, the master might fail. In this case, the client request will fail and the client will re-submit the request. If 1.1 client or if there is no contention for the table lock, the time window is pretty small, but still might happen. If the proc was accepted and stored in the procedure store, a re-submit from the client will add another procedure, which will execute after the first one. The first one will likely succeed, and the second one will fail (for example in the case of create table, the second one will throw TableExistsException). One idea is to use client generated nonces (that we already have) to guard against these cases. The client will submit the request with the nonce and the nonce will be saved together with the procedure in the store. In case of a double submit, the nonce-cache is checked and the procId of the original request is returned. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-13415) Procedure V2 - Use nonces for double submits from client
[ https://issues.apache.org/jira/browse/HBASE-13415?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14486110#comment-14486110 ] Keith Turner commented on HBASE-13415: -- Accumulo's FATE handles this by having the four thrift calls for submitting FATE operations. Its safe for a client to make each thrift call multiple times until it succeeds. The main difference with what proposed here is that the master generates a transaction ID instead of a client generating a nonce. * A thrift call to the Accumulo master get a new FATE transaction ID. This creates a new FATE transaction in zookeeper with no work to do. * A thrift call to seed a fate transaction with work. This call takes the transaction ID and an operation (like create table). * A thrift call to wait on a FATE transaction to complete. This call takes a transaction ID and returns any failure info. * A thrift call to delete the status information about a completed FATE transaction. Procedure V2 - Use nonces for double submits from client Key: HBASE-13415 URL: https://issues.apache.org/jira/browse/HBASE-13415 Project: HBase Issue Type: Sub-task Components: master Reporter: Enis Soztutar The client can submit a procedure, but before getting the procId back, the master might fail. In this case, the client request will fail and the client will re-submit the request. If 1.1 client or if there is no contention for the table lock, the time window is pretty small, but still might happen. If the proc was accepted and stored in the procedure store, a re-submit from the client will add another procedure, which will execute after the first one. The first one will likely succeed, and the second one will fail (for example in the case of create table, the second one will throw TableExistsException). One idea is to use client generated nonces (that we already have) to guard against these cases. The client will submit the request with the nonce and the nonce will be saved together with the procedure in the store. In case of a double submit, the nonce-cache is checked and the procId of the original request is returned. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-13415) Procedure V2 - Use nonces for double submits from client
[ https://issues.apache.org/jira/browse/HBASE-13415?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14482821#comment-14482821 ] Matteo Bertozzi commented on HBASE-13415: - since this will be on each operation we have on the client, should the nonce be sent as part of the RequestHeader? for example HBASE-3787 does that, but adds the nonce on the request specific proto. Procedure V2 - Use nonces for double submits from client Key: HBASE-13415 URL: https://issues.apache.org/jira/browse/HBASE-13415 Project: HBase Issue Type: Sub-task Components: master Reporter: Enis Soztutar The client can submit a procedure, but before getting the procId back, the master might fail. In this case, the client request will fail and the client will re-submit the request. If 1.1 client or if there is no contention for the table lock, the time window is pretty small, but still might happen. If the proc was accepted and stored in the procedure store, a re-submit from the client will add another procedure, which will execute after the first one. The first one will likely succeed, and the second one will fail (for example in the case of create table, the second one will throw TableExistsException). One idea is to use client generated nonces (that we already have) to guard against these cases. The client will submit the request with the nonce and the nonce will be saved together with the procedure in the store. In case of a double submit, the nonce-cache is checked and the procId of the original request is returned. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-13415) Procedure V2 - Use nonces for double submits from client
[ https://issues.apache.org/jira/browse/HBASE-13415?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14483740#comment-14483740 ] Enis Soztutar commented on HBASE-13415: --- Yes, I was thinking of using the same nonce mechanics introduced in HBASE-3787. However, in HBASE-3787, only Increment and Append does nonces, since Put / Delete are idempotent. I think all master operations need the nonce if possible, but not all client operations (Put). Procedure V2 - Use nonces for double submits from client Key: HBASE-13415 URL: https://issues.apache.org/jira/browse/HBASE-13415 Project: HBase Issue Type: Sub-task Components: master Reporter: Enis Soztutar The client can submit a procedure, but before getting the procId back, the master might fail. In this case, the client request will fail and the client will re-submit the request. If 1.1 client or if there is no contention for the table lock, the time window is pretty small, but still might happen. If the proc was accepted and stored in the procedure store, a re-submit from the client will add another procedure, which will execute after the first one. The first one will likely succeed, and the second one will fail (for example in the case of create table, the second one will throw TableExistsException). One idea is to use client generated nonces (that we already have) to guard against these cases. The client will submit the request with the nonce and the nonce will be saved together with the procedure in the store. In case of a double submit, the nonce-cache is checked and the procId of the original request is returned. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
