[jira] [Updated] (SENTRY-1855) PERM/PATH transactions can fail to commit to the sentry database under load

2017-08-09 Thread Na Li (JIRA)

 [ 
https://issues.apache.org/jira/browse/SENTRY-1855?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Na Li updated SENTRY-1855:
--
Attachment: SENTRY-1855.001.patch

v1 on master. Get perm delta changes using timestamp, not changeID

> PERM/PATH transactions can fail to commit to the sentry database under load
> ---
>
> Key: SENTRY-1855
> URL: https://issues.apache.org/jira/browse/SENTRY-1855
> Project: Sentry
>  Issue Type: Sub-task
>  Components: Sentry
>Affects Versions: 2.0.0
>Reporter: Alexander Kolbasov
>Assignee: Na Li
> Fix For: 2.0.0
>
> Attachments: SENTRY-1855.001.patch, 
> SENTRY-1855.002-sentry-ha-redesign.patch, SENTRY-1855.003-master.patch, 
> SENTRY-1855.01-sentry-ha-redesign.patch
>
>
> Looking at the latest stress runs, we noticed that some of the transactions 
> could fail to commit to the database (with Duplicate key exception) after 
> exhausting all the retries.
> This problem has become more evident if we have more number of clients 
> connecting to Sentry to issue the permission updates. Was able to reproduce 
> consistently with 15 clients doing 100 operations each.
> In the past we introduced exponential backoff (SENTRY-1821) so as part of 
> test run increased the defaults to 750ms sleep and 20 retries. But even after 
> this, the cluster still shows up the transaction failures. This change also 
> increases the latency of every transaction in sentry.
> We need to revisit this and come up with a better way to solve this problem.
> {code}
> 2017-07-13 13:18:14,449 ERROR 
> org.apache.sentry.provider.db.service.persistent.TransactionManager: The 
> transaction has reached max retry number, Exception thrown when executing 
> query
> javax.jdo.JDOException: Exception thrown when executing query
>   at 
> org.datanucleus.api.jdo.NucleusJDOHelper.getJDOExceptionForNucleusException(NucleusJDOHelper.java:596)
>   at org.datanucleus.api.jdo.JDOQuery.execute(JDOQuery.java:252)
>   at 
> org.apache.sentry.provider.db.service.persistent.SentryStore.getRole(SentryStore.java:294)
>   at 
> org.apache.sentry.provider.db.service.persistent.SentryStore.alterSentryRoleGrantPrivilegeCore(SentryStore.java:645)
>   at 
> org.apache.sentry.provider.db.service.persistent.SentryStore.access$500(SentryStore.java:101)
>   at 
> org.apache.sentry.provider.db.service.persistent.SentryStore$11.execute(SentryStore.java:601)
>   at 
> org.apache.sentry.provider.db.service.persistent.TransactionManager.executeTransaction(TransactionManager.java:159)
>   at 
> org.apache.sentry.provider.db.service.persistent.TransactionManager.access$100(TransactionManager.java:63)
>   at 
> org.apache.sentry.provider.db.service.persistent.TransactionManager$2.call(TransactionManager.java:213)
> --
>   at com.mysql.jdbc.SQLError.createSQLException(SQLError.java:971)
>   at com.mysql.jdbc.MysqlIO.checkErrorPacket(MysqlIO.java:3887)
>   at com.mysql.jdbc.MysqlIO.checkErrorPacket(MysqlIO.java:3823)
>   at com.mysql.jdbc.MysqlIO.sendCommand(MysqlIO.java:2435)
>   at com.mysql.jdbc.MysqlIO.sqlQueryDirect(MysqlIO.java:2582)
>   at com.mysql.jdbc.ConnectionImpl.execSQL(ConnectionImpl.java:2530)
>   at 
> com.mysql.jdbc.PreparedStatement.executeInternal(PreparedStatement.java:1907)
>   at 
> com.mysql.jdbc.PreparedStatement.executeUpdate(PreparedStatement.java:2141)
>   at 
> com.mysql.jdbc.PreparedStatement.executeBatchSerially(PreparedStatement.java:1773)
>   ... 33 more
> 2017-07-13 13:18:14,450 ERROR 
> org.apache.sentry.provider.db.service.thrift.SentryPolicyStoreProcessor: 
> Unknown error for request: 
> TAlterSentryRoleGrantPrivilegeRequest(protocol_version:2, 
> requestorUserName:hive, roleName:2017_07_12_15_06_38_1_2_805, 
> privileges:[TSentryPrivilege(privilegeScope:DATABASE, serverName:server1, 
> dbName:2017_07_12_15_06_38_1_2, tableName:, URI:, action:*, 
> createTime:1499904401222, grantOption:FALSE, columnName:)]), message: The 
> transaction has reached max retry number, Exception thrown when executing 
> query
> java.lang.Exception: The transaction has reached max retry number, Exception 
> thrown when executing query
>   at 
> org.apache.sentry.provider.db.service.persistent.TransactionManager$ExponentialBackoff.execute(TransactionManager.java:255)
>   at 
> org.apache.sentry.provider.db.service.persistent.TransactionManager.executeTransactionBlocksWithRetry(TransactionManager.java:209)
>   at 
> org.apache.sentry.provider.db.service.persistent.SentryStore.execute(SentryStore.java:3330)
>   at 
> org.apache.sentry.provider.db.service.persistent.SentryStore.alterSentryRoleGrantPrivilege(SentryStore.java:593)
>   at 
> 

[jira] [Updated] (SENTRY-1855) PERM/PATH transactions can fail to commit to the sentry database under load

2017-07-28 Thread Na Li (JIRA)

 [ 
https://issues.apache.org/jira/browse/SENTRY-1855?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Na Li updated SENTRY-1855:
--
Attachment: SENTRY-1855.003-master.patch

v3. approach 3. for hole at front of list, return one entry to indicate there 
is a hole to NN. NN will retry. Once retry exceeds max, skip the missing 
change, move to next one.

> PERM/PATH transactions can fail to commit to the sentry database under load
> ---
>
> Key: SENTRY-1855
> URL: https://issues.apache.org/jira/browse/SENTRY-1855
> Project: Sentry
>  Issue Type: Sub-task
>  Components: Sentry
>Affects Versions: 2.0.0
>Reporter: Alexander Kolbasov
>Assignee: Na Li
> Fix For: 2.0.0
>
> Attachments: SENTRY-1855.002-sentry-ha-redesign.patch, 
> SENTRY-1855.003-master.patch, SENTRY-1855.01-sentry-ha-redesign.patch
>
>
> Looking at the latest stress runs, we noticed that some of the transactions 
> could fail to commit to the database (with Duplicate key exception) after 
> exhausting all the retries.
> This problem has become more evident if we have more number of clients 
> connecting to Sentry to issue the permission updates. Was able to reproduce 
> consistently with 15 clients doing 100 operations each.
> In the past we introduced exponential backoff (SENTRY-1821) so as part of 
> test run increased the defaults to 750ms sleep and 20 retries. But even after 
> this, the cluster still shows up the transaction failures. This change also 
> increases the latency of every transaction in sentry.
> We need to revisit this and come up with a better way to solve this problem.
> {code}
> 2017-07-13 13:18:14,449 ERROR 
> org.apache.sentry.provider.db.service.persistent.TransactionManager: The 
> transaction has reached max retry number, Exception thrown when executing 
> query
> javax.jdo.JDOException: Exception thrown when executing query
>   at 
> org.datanucleus.api.jdo.NucleusJDOHelper.getJDOExceptionForNucleusException(NucleusJDOHelper.java:596)
>   at org.datanucleus.api.jdo.JDOQuery.execute(JDOQuery.java:252)
>   at 
> org.apache.sentry.provider.db.service.persistent.SentryStore.getRole(SentryStore.java:294)
>   at 
> org.apache.sentry.provider.db.service.persistent.SentryStore.alterSentryRoleGrantPrivilegeCore(SentryStore.java:645)
>   at 
> org.apache.sentry.provider.db.service.persistent.SentryStore.access$500(SentryStore.java:101)
>   at 
> org.apache.sentry.provider.db.service.persistent.SentryStore$11.execute(SentryStore.java:601)
>   at 
> org.apache.sentry.provider.db.service.persistent.TransactionManager.executeTransaction(TransactionManager.java:159)
>   at 
> org.apache.sentry.provider.db.service.persistent.TransactionManager.access$100(TransactionManager.java:63)
>   at 
> org.apache.sentry.provider.db.service.persistent.TransactionManager$2.call(TransactionManager.java:213)
> --
>   at com.mysql.jdbc.SQLError.createSQLException(SQLError.java:971)
>   at com.mysql.jdbc.MysqlIO.checkErrorPacket(MysqlIO.java:3887)
>   at com.mysql.jdbc.MysqlIO.checkErrorPacket(MysqlIO.java:3823)
>   at com.mysql.jdbc.MysqlIO.sendCommand(MysqlIO.java:2435)
>   at com.mysql.jdbc.MysqlIO.sqlQueryDirect(MysqlIO.java:2582)
>   at com.mysql.jdbc.ConnectionImpl.execSQL(ConnectionImpl.java:2530)
>   at 
> com.mysql.jdbc.PreparedStatement.executeInternal(PreparedStatement.java:1907)
>   at 
> com.mysql.jdbc.PreparedStatement.executeUpdate(PreparedStatement.java:2141)
>   at 
> com.mysql.jdbc.PreparedStatement.executeBatchSerially(PreparedStatement.java:1773)
>   ... 33 more
> 2017-07-13 13:18:14,450 ERROR 
> org.apache.sentry.provider.db.service.thrift.SentryPolicyStoreProcessor: 
> Unknown error for request: 
> TAlterSentryRoleGrantPrivilegeRequest(protocol_version:2, 
> requestorUserName:hive, roleName:2017_07_12_15_06_38_1_2_805, 
> privileges:[TSentryPrivilege(privilegeScope:DATABASE, serverName:server1, 
> dbName:2017_07_12_15_06_38_1_2, tableName:, URI:, action:*, 
> createTime:1499904401222, grantOption:FALSE, columnName:)]), message: The 
> transaction has reached max retry number, Exception thrown when executing 
> query
> java.lang.Exception: The transaction has reached max retry number, Exception 
> thrown when executing query
>   at 
> org.apache.sentry.provider.db.service.persistent.TransactionManager$ExponentialBackoff.execute(TransactionManager.java:255)
>   at 
> org.apache.sentry.provider.db.service.persistent.TransactionManager.executeTransactionBlocksWithRetry(TransactionManager.java:209)
>   at 
> org.apache.sentry.provider.db.service.persistent.SentryStore.execute(SentryStore.java:3330)
>   at 
> 

[jira] [Updated] (SENTRY-1855) PERM/PATH transactions can fail to commit to the sentry database under load

2017-07-27 Thread Na Li (JIRA)

 [ 
https://issues.apache.org/jira/browse/SENTRY-1855?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Na Li updated SENTRY-1855:
--
Attachment: SENTRY-1855.002-sentry-ha-redesign.patch

v2. 
1. Make perm changeID to be auto-increased for performance improvement
2. revert change in sentry-1824 to synchronize the perm changes in SentryStore
3. Return continuous delta list starting from requested changeID from 
SentryStore even when the delta list has hole (for example, the list is 
1,2,3,5,6, return 1,2,3). If there is a hole at the front of the list, return 
full snapshot. 

Note: this only applies to perm changes. path changeID is manually increased, 
so has no hole. 
The changeID sent to NN in the full snapshot is in the same transaction of 
getting the full snapshot. 

> PERM/PATH transactions can fail to commit to the sentry database under load
> ---
>
> Key: SENTRY-1855
> URL: https://issues.apache.org/jira/browse/SENTRY-1855
> Project: Sentry
>  Issue Type: Sub-task
>  Components: Sentry
>Affects Versions: 2.0.0
>Reporter: Alexander Kolbasov
>Assignee: Na Li
> Fix For: 2.0.0
>
> Attachments: SENTRY-1855.002-sentry-ha-redesign.patch, 
> SENTRY-1855.01-sentry-ha-redesign.patch
>
>
> Looking at the latest stress runs, we noticed that some of the transactions 
> could fail to commit to the database (with Duplicate key exception) after 
> exhausting all the retries.
> This problem has become more evident if we have more number of clients 
> connecting to Sentry to issue the permission updates. Was able to reproduce 
> consistently with 15 clients doing 100 operations each.
> In the past we introduced exponential backoff (SENTRY-1821) so as part of 
> test run increased the defaults to 750ms sleep and 20 retries. But even after 
> this, the cluster still shows up the transaction failures. This change also 
> increases the latency of every transaction in sentry.
> We need to revisit this and come up with a better way to solve this problem.
> {code}
> 2017-07-13 13:18:14,449 ERROR 
> org.apache.sentry.provider.db.service.persistent.TransactionManager: The 
> transaction has reached max retry number, Exception thrown when executing 
> query
> javax.jdo.JDOException: Exception thrown when executing query
>   at 
> org.datanucleus.api.jdo.NucleusJDOHelper.getJDOExceptionForNucleusException(NucleusJDOHelper.java:596)
>   at org.datanucleus.api.jdo.JDOQuery.execute(JDOQuery.java:252)
>   at 
> org.apache.sentry.provider.db.service.persistent.SentryStore.getRole(SentryStore.java:294)
>   at 
> org.apache.sentry.provider.db.service.persistent.SentryStore.alterSentryRoleGrantPrivilegeCore(SentryStore.java:645)
>   at 
> org.apache.sentry.provider.db.service.persistent.SentryStore.access$500(SentryStore.java:101)
>   at 
> org.apache.sentry.provider.db.service.persistent.SentryStore$11.execute(SentryStore.java:601)
>   at 
> org.apache.sentry.provider.db.service.persistent.TransactionManager.executeTransaction(TransactionManager.java:159)
>   at 
> org.apache.sentry.provider.db.service.persistent.TransactionManager.access$100(TransactionManager.java:63)
>   at 
> org.apache.sentry.provider.db.service.persistent.TransactionManager$2.call(TransactionManager.java:213)
> --
>   at com.mysql.jdbc.SQLError.createSQLException(SQLError.java:971)
>   at com.mysql.jdbc.MysqlIO.checkErrorPacket(MysqlIO.java:3887)
>   at com.mysql.jdbc.MysqlIO.checkErrorPacket(MysqlIO.java:3823)
>   at com.mysql.jdbc.MysqlIO.sendCommand(MysqlIO.java:2435)
>   at com.mysql.jdbc.MysqlIO.sqlQueryDirect(MysqlIO.java:2582)
>   at com.mysql.jdbc.ConnectionImpl.execSQL(ConnectionImpl.java:2530)
>   at 
> com.mysql.jdbc.PreparedStatement.executeInternal(PreparedStatement.java:1907)
>   at 
> com.mysql.jdbc.PreparedStatement.executeUpdate(PreparedStatement.java:2141)
>   at 
> com.mysql.jdbc.PreparedStatement.executeBatchSerially(PreparedStatement.java:1773)
>   ... 33 more
> 2017-07-13 13:18:14,450 ERROR 
> org.apache.sentry.provider.db.service.thrift.SentryPolicyStoreProcessor: 
> Unknown error for request: 
> TAlterSentryRoleGrantPrivilegeRequest(protocol_version:2, 
> requestorUserName:hive, roleName:2017_07_12_15_06_38_1_2_805, 
> privileges:[TSentryPrivilege(privilegeScope:DATABASE, serverName:server1, 
> dbName:2017_07_12_15_06_38_1_2, tableName:, URI:, action:*, 
> createTime:1499904401222, grantOption:FALSE, columnName:)]), message: The 
> transaction has reached max retry number, Exception thrown when executing 
> query
> java.lang.Exception: The transaction has reached max retry number, Exception 
> thrown when executing query
>   at 
> 

[jira] [Updated] (SENTRY-1855) PERM/PATH transactions can fail to commit to the sentry database under load

2017-07-21 Thread Alexander Kolbasov (JIRA)

 [ 
https://issues.apache.org/jira/browse/SENTRY-1855?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Alexander Kolbasov updated SENTRY-1855:
---
Attachment: SENTRY-1855.01-sentry-ha-redesign.patch

> PERM/PATH transactions can fail to commit to the sentry database under load
> ---
>
> Key: SENTRY-1855
> URL: https://issues.apache.org/jira/browse/SENTRY-1855
> Project: Sentry
>  Issue Type: Sub-task
>  Components: Sentry
>Affects Versions: sentry-ha-redesign
>Reporter: Alexander Kolbasov
>Assignee: Alexander Kolbasov
> Fix For: sentry-ha-redesign
>
> Attachments: SENTRY-1855.01-sentry-ha-redesign.patch
>
>
> Looking at the latest stress runs, we noticed that some of the transactions 
> could fail to commit to the database (with Duplicate key exception) after 
> exhausting all the retries.
> This problem has become more evident if we have more number of clients 
> connecting to Sentry to issue the permission updates. Was able to reproduce 
> consistently with 15 clients doing 100 operations each.
> In the past we introduced exponential backoff (SENTRY-1821) so as part of 
> test run increased the defaults to 750ms sleep and 20 retries. But even after 
> this, the cluster still shows up the transaction failures. This change also 
> increases the latency of every transaction in sentry.
> We need to revisit this and come up with a better way to solve this problem.
> {code}
> 2017-07-13 13:18:14,449 ERROR 
> org.apache.sentry.provider.db.service.persistent.TransactionManager: The 
> transaction has reached max retry number, Exception thrown when executing 
> query
> javax.jdo.JDOException: Exception thrown when executing query
>   at 
> org.datanucleus.api.jdo.NucleusJDOHelper.getJDOExceptionForNucleusException(NucleusJDOHelper.java:596)
>   at org.datanucleus.api.jdo.JDOQuery.execute(JDOQuery.java:252)
>   at 
> org.apache.sentry.provider.db.service.persistent.SentryStore.getRole(SentryStore.java:294)
>   at 
> org.apache.sentry.provider.db.service.persistent.SentryStore.alterSentryRoleGrantPrivilegeCore(SentryStore.java:645)
>   at 
> org.apache.sentry.provider.db.service.persistent.SentryStore.access$500(SentryStore.java:101)
>   at 
> org.apache.sentry.provider.db.service.persistent.SentryStore$11.execute(SentryStore.java:601)
>   at 
> org.apache.sentry.provider.db.service.persistent.TransactionManager.executeTransaction(TransactionManager.java:159)
>   at 
> org.apache.sentry.provider.db.service.persistent.TransactionManager.access$100(TransactionManager.java:63)
>   at 
> org.apache.sentry.provider.db.service.persistent.TransactionManager$2.call(TransactionManager.java:213)
> --
>   at com.mysql.jdbc.SQLError.createSQLException(SQLError.java:971)
>   at com.mysql.jdbc.MysqlIO.checkErrorPacket(MysqlIO.java:3887)
>   at com.mysql.jdbc.MysqlIO.checkErrorPacket(MysqlIO.java:3823)
>   at com.mysql.jdbc.MysqlIO.sendCommand(MysqlIO.java:2435)
>   at com.mysql.jdbc.MysqlIO.sqlQueryDirect(MysqlIO.java:2582)
>   at com.mysql.jdbc.ConnectionImpl.execSQL(ConnectionImpl.java:2530)
>   at 
> com.mysql.jdbc.PreparedStatement.executeInternal(PreparedStatement.java:1907)
>   at 
> com.mysql.jdbc.PreparedStatement.executeUpdate(PreparedStatement.java:2141)
>   at 
> com.mysql.jdbc.PreparedStatement.executeBatchSerially(PreparedStatement.java:1773)
>   ... 33 more
> 2017-07-13 13:18:14,450 ERROR 
> org.apache.sentry.provider.db.service.thrift.SentryPolicyStoreProcessor: 
> Unknown error for request: 
> TAlterSentryRoleGrantPrivilegeRequest(protocol_version:2, 
> requestorUserName:hive, roleName:2017_07_12_15_06_38_1_2_805, 
> privileges:[TSentryPrivilege(privilegeScope:DATABASE, serverName:server1, 
> dbName:2017_07_12_15_06_38_1_2, tableName:, URI:, action:*, 
> createTime:1499904401222, grantOption:FALSE, columnName:)]), message: The 
> transaction has reached max retry number, Exception thrown when executing 
> query
> java.lang.Exception: The transaction has reached max retry number, Exception 
> thrown when executing query
>   at 
> org.apache.sentry.provider.db.service.persistent.TransactionManager$ExponentialBackoff.execute(TransactionManager.java:255)
>   at 
> org.apache.sentry.provider.db.service.persistent.TransactionManager.executeTransactionBlocksWithRetry(TransactionManager.java:209)
>   at 
> org.apache.sentry.provider.db.service.persistent.SentryStore.execute(SentryStore.java:3330)
>   at 
> org.apache.sentry.provider.db.service.persistent.SentryStore.alterSentryRoleGrantPrivilege(SentryStore.java:593)
>   at 
> org.apache.sentry.provider.db.service.persistent.SentryStore.alterSentryRoleGrantPrivileges(SentryStore.java:633)
>   at 
> 

[jira] [Updated] (SENTRY-1855) PERM/PATH transactions can fail to commit to the sentry database under load

2017-07-21 Thread Alexander Kolbasov (JIRA)

 [ 
https://issues.apache.org/jira/browse/SENTRY-1855?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Alexander Kolbasov updated SENTRY-1855:
---
Status: Patch Available  (was: Open)

> PERM/PATH transactions can fail to commit to the sentry database under load
> ---
>
> Key: SENTRY-1855
> URL: https://issues.apache.org/jira/browse/SENTRY-1855
> Project: Sentry
>  Issue Type: Sub-task
>  Components: Sentry
>Affects Versions: sentry-ha-redesign
>Reporter: Alexander Kolbasov
>Assignee: Alexander Kolbasov
> Fix For: sentry-ha-redesign
>
> Attachments: SENTRY-1855.01-sentry-ha-redesign.patch
>
>
> Looking at the latest stress runs, we noticed that some of the transactions 
> could fail to commit to the database (with Duplicate key exception) after 
> exhausting all the retries.
> This problem has become more evident if we have more number of clients 
> connecting to Sentry to issue the permission updates. Was able to reproduce 
> consistently with 15 clients doing 100 operations each.
> In the past we introduced exponential backoff (SENTRY-1821) so as part of 
> test run increased the defaults to 750ms sleep and 20 retries. But even after 
> this, the cluster still shows up the transaction failures. This change also 
> increases the latency of every transaction in sentry.
> We need to revisit this and come up with a better way to solve this problem.
> {code}
> 2017-07-13 13:18:14,449 ERROR 
> org.apache.sentry.provider.db.service.persistent.TransactionManager: The 
> transaction has reached max retry number, Exception thrown when executing 
> query
> javax.jdo.JDOException: Exception thrown when executing query
>   at 
> org.datanucleus.api.jdo.NucleusJDOHelper.getJDOExceptionForNucleusException(NucleusJDOHelper.java:596)
>   at org.datanucleus.api.jdo.JDOQuery.execute(JDOQuery.java:252)
>   at 
> org.apache.sentry.provider.db.service.persistent.SentryStore.getRole(SentryStore.java:294)
>   at 
> org.apache.sentry.provider.db.service.persistent.SentryStore.alterSentryRoleGrantPrivilegeCore(SentryStore.java:645)
>   at 
> org.apache.sentry.provider.db.service.persistent.SentryStore.access$500(SentryStore.java:101)
>   at 
> org.apache.sentry.provider.db.service.persistent.SentryStore$11.execute(SentryStore.java:601)
>   at 
> org.apache.sentry.provider.db.service.persistent.TransactionManager.executeTransaction(TransactionManager.java:159)
>   at 
> org.apache.sentry.provider.db.service.persistent.TransactionManager.access$100(TransactionManager.java:63)
>   at 
> org.apache.sentry.provider.db.service.persistent.TransactionManager$2.call(TransactionManager.java:213)
> --
>   at com.mysql.jdbc.SQLError.createSQLException(SQLError.java:971)
>   at com.mysql.jdbc.MysqlIO.checkErrorPacket(MysqlIO.java:3887)
>   at com.mysql.jdbc.MysqlIO.checkErrorPacket(MysqlIO.java:3823)
>   at com.mysql.jdbc.MysqlIO.sendCommand(MysqlIO.java:2435)
>   at com.mysql.jdbc.MysqlIO.sqlQueryDirect(MysqlIO.java:2582)
>   at com.mysql.jdbc.ConnectionImpl.execSQL(ConnectionImpl.java:2530)
>   at 
> com.mysql.jdbc.PreparedStatement.executeInternal(PreparedStatement.java:1907)
>   at 
> com.mysql.jdbc.PreparedStatement.executeUpdate(PreparedStatement.java:2141)
>   at 
> com.mysql.jdbc.PreparedStatement.executeBatchSerially(PreparedStatement.java:1773)
>   ... 33 more
> 2017-07-13 13:18:14,450 ERROR 
> org.apache.sentry.provider.db.service.thrift.SentryPolicyStoreProcessor: 
> Unknown error for request: 
> TAlterSentryRoleGrantPrivilegeRequest(protocol_version:2, 
> requestorUserName:hive, roleName:2017_07_12_15_06_38_1_2_805, 
> privileges:[TSentryPrivilege(privilegeScope:DATABASE, serverName:server1, 
> dbName:2017_07_12_15_06_38_1_2, tableName:, URI:, action:*, 
> createTime:1499904401222, grantOption:FALSE, columnName:)]), message: The 
> transaction has reached max retry number, Exception thrown when executing 
> query
> java.lang.Exception: The transaction has reached max retry number, Exception 
> thrown when executing query
>   at 
> org.apache.sentry.provider.db.service.persistent.TransactionManager$ExponentialBackoff.execute(TransactionManager.java:255)
>   at 
> org.apache.sentry.provider.db.service.persistent.TransactionManager.executeTransactionBlocksWithRetry(TransactionManager.java:209)
>   at 
> org.apache.sentry.provider.db.service.persistent.SentryStore.execute(SentryStore.java:3330)
>   at 
> org.apache.sentry.provider.db.service.persistent.SentryStore.alterSentryRoleGrantPrivilege(SentryStore.java:593)
>   at 
> org.apache.sentry.provider.db.service.persistent.SentryStore.alterSentryRoleGrantPrivileges(SentryStore.java:633)
>   at 
>