[jira] [Updated] (SENTRY-1422) JDO deadlocks while processing grant while a background thread processes Notificationlogs
[
https://issues.apache.org/jira/browse/SENTRY-1422?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Colin Ma updated SENTRY-1422:
-
Attachment: SENTRY-1422.019-sentry-ha-redesign.patch
> JDO deadlocks while processing grant while a background thread processes
> Notificationlogs
> -
>
> Key: SENTRY-1422
> URL: https://issues.apache.org/jira/browse/SENTRY-1422
> Project: Sentry
> Issue Type: Sub-task
> Components: Hdfs Plugin
>Reporter: Sravya Tirukkovalur
>Assignee: Colin Ma
> Fix For: sentry-ha-redesign
>
> Attachments: SENTRY-1422.013-sentry-ha-redesign.patch,
> SENTRY-1422.014-sentry-ha-redesign.patch,
> SENTRY-1422.015-sentry-ha-redesign.patch,
> SENTRY-1422.016-sentry-ha-redesign.patch,
> SENTRY-1422.017-sentry-ha-redesign.patch,
> SENTRY-1422.018-sentry-ha-redesign.patch,
> SENTRY-1422.019-sentry-ha-redesign.patch, sentry_deadlock.png,
> testDeadLock.patch
>
>
> As I was working on Sentry-1321. I see that test case
> TestDbPrivilegeCleanupOnDrop#testDropObjects fails while "Granting select on
> table" with following deadlock.
> {code}
> java.sql.SQLException: Error while processing statement: FAILED: Execution
> Error, return code 1 from
> org.apache.hadoop.hive.ql.exec.SentryGrantRevokeTask. Unknown error for
> request: TAlterSentryRoleGrantPrivilegeRequest(protocol_version:2,
> requestorUserName:admin1, roleName:select_tbl1,
> privileges:[TSentryPrivilege(privilegeScope:TABLE, serverName:server1,
> dbName:db_2, tableName:tb_1, URI:, action:select, createTime:1469562185573,
> grantOption:FALSE, columnName:)]), message: Iteration request failed : SELECT
> 'org.apache.sentry.provider.db.service.model.MSentryPrivilege' AS
> NUCLEUS_TYPE,A1.URI,A1."ACTION",A1."COLUMN_NAME",A1.CREATE_TIME,A1.DB_NAME,A1.WITH_GRANT_OPTION,A1.PRIVILEGE_SCOPE,A1."SERVER_NAME",A1."TABLE_NAME",A1.DB_PRIVILEGE_ID
> FROM SENTRY_ROLE_DB_PRIVILEGE_MAP A0 INNER JOIN SENTRY_DB_PRIVILEGE A1 ON
> A0.DB_PRIVILEGE_ID = A1.DB_PRIVILEGE_ID WHERE A0.ROLE_ID = ?. Server
> Stacktrace: javax.jdo.JDODataStoreException: Iteration request failed :
> SELECT 'org.apache.sentry.provider.db.service.model.MSentryPrivilege' AS
> NUCLEUS_TYPE,A1.URI,A1."ACTION",A1."COLUMN_NAME",A1.CREATE_TIME,A1.DB_NAME,A1.WITH_GRANT_OPTION,A1.PRIVILEGE_SCOPE,A1."SERVER_NAME",A1."TABLE_NAME",A1.DB_PRIVILEGE_ID
> FROM SENTRY_ROLE_DB_PRIVILEGE_MAP A0 INNER JOIN SENTRY_DB_PRIVILEGE A1 ON
> A0.DB_PRIVILEGE_ID = A1.DB_PRIVILEGE_ID WHERE A0.ROLE_ID = ?
> at
> org.datanucleus.api.jdo.NucleusJDOHelper.getJDOExceptionForNucleusException(NucleusJDOHelper.java:451)
> at
> org.datanucleus.api.jdo.JDOTransaction.commit(JDOTransaction.java:165)
> at
> org.apache.sentry.provider.db.service.persistent.SentryStore.commitTransaction(SentryStore.java:279)
> at
> org.apache.sentry.provider.db.service.persistent.SentryStore.commitUpdateTransaction(SentryStore.java:261)
> at
> org.apache.sentry.provider.db.service.persistent.SentryStore.alterSentryRoleGrantPrivileges(SentryStore.java:458)
> at
> org.apache.sentry.provider.db.service.thrift.SentryPolicyStoreProcessor.alter_sentry_role_grant_privilege(SentryPolicyStoreProcessor.java:261)
> at
> org.apache.sentry.provider.db.service.thrift.SentryPolicyService$Processor$alter_sentry_role_grant_privilege.getResult(SentryPolicyService.java:1237)
> at
> org.apache.sentry.provider.db.service.thrift.SentryPolicyService$Processor$alter_sentry_role_grant_privilege.getResult(SentryPolicyService.java:1222)
> at org.apache.thrift.ProcessFunction.process(ProcessFunction.java:39)
> at org.apache.thrift.TBaseProcessor.process(TBaseProcessor.java:39)
> at
> org.apache.sentry.provider.db.service.thrift.SentryProcessorWrapper.process(SentryProcessorWrapper.java:35)
> at
> org.apache.thrift.TMultiplexedProcessor.process(TMultiplexedProcessor.java:123)
> at
> org.apache.thrift.server.TThreadPoolServer$WorkerProcess.run(TThreadPoolServer.java:286)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
> at java.lang.Thread.run(Thread.java:745)
> NestedThrowablesStackTrace:
> java.sql.SQLTransactionRollbackException: A lock could not be obtained due to
> a deadlock, cycle of locks and waiters is:
> Lock : ROW, SENTRY_ROLE_DB_PRIVILEGE_MAP, (1,10)
> Waiting XID : {712, S} , SENTRY, SELECT
> 'org.apache.sentry.provider.db.service.model.MSentryPrivilege' AS
> NUCLEUS_TYPE,A1.URI,A1."ACTION",A1."COLUMN_NAME",A1.CREATE_TIME,A1.DB_NAME,A1.WITH_GRANT_OPTION,A1.PRIVILEGE_SCOPE,A1."SERVER_NAME",A1."TABLE_NAME",A1.DB_PRIVILEGE_ID
> FROM SENTRY_ROLE_DB_PRIVILEGE_MAP A0
[jira] [Commented] (SENTRY-1422) JDO deadlocks while processing grant while a background thread processes Notificationlogs
[
https://issues.apache.org/jira/browse/SENTRY-1422?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15604251#comment-15604251
]
Hadoop QA commented on SENTRY-1422:
---
Here are the results of testing the latest attachment
https://issues.apache.org/jira/secure/attachment/12835053/SENTRY-1422.018-sentry-ha-redesign.patch
against sentry-ha-redesign.
{color:red}Overall:{color} -1 due to an error
{color:red}ERROR:{color} failed to apply patch (exit code 1):
The patch does not appear to apply with p0, p1, or p2
Console output:
https://builds.apache.org/job/PreCommit-SENTRY-Build/2090/console
This message is automatically generated.
> JDO deadlocks while processing grant while a background thread processes
> Notificationlogs
> -
>
> Key: SENTRY-1422
> URL: https://issues.apache.org/jira/browse/SENTRY-1422
> Project: Sentry
> Issue Type: Sub-task
> Components: Hdfs Plugin
>Reporter: Sravya Tirukkovalur
>Assignee: Colin Ma
> Fix For: sentry-ha-redesign
>
> Attachments: SENTRY-1422.013-sentry-ha-redesign.patch,
> SENTRY-1422.014-sentry-ha-redesign.patch,
> SENTRY-1422.015-sentry-ha-redesign.patch,
> SENTRY-1422.016-sentry-ha-redesign.patch,
> SENTRY-1422.017-sentry-ha-redesign.patch,
> SENTRY-1422.018-sentry-ha-redesign.patch, sentry_deadlock.png,
> testDeadLock.patch
>
>
> As I was working on Sentry-1321. I see that test case
> TestDbPrivilegeCleanupOnDrop#testDropObjects fails while "Granting select on
> table" with following deadlock.
> {code}
> java.sql.SQLException: Error while processing statement: FAILED: Execution
> Error, return code 1 from
> org.apache.hadoop.hive.ql.exec.SentryGrantRevokeTask. Unknown error for
> request: TAlterSentryRoleGrantPrivilegeRequest(protocol_version:2,
> requestorUserName:admin1, roleName:select_tbl1,
> privileges:[TSentryPrivilege(privilegeScope:TABLE, serverName:server1,
> dbName:db_2, tableName:tb_1, URI:, action:select, createTime:1469562185573,
> grantOption:FALSE, columnName:)]), message: Iteration request failed : SELECT
> 'org.apache.sentry.provider.db.service.model.MSentryPrivilege' AS
> NUCLEUS_TYPE,A1.URI,A1."ACTION",A1."COLUMN_NAME",A1.CREATE_TIME,A1.DB_NAME,A1.WITH_GRANT_OPTION,A1.PRIVILEGE_SCOPE,A1."SERVER_NAME",A1."TABLE_NAME",A1.DB_PRIVILEGE_ID
> FROM SENTRY_ROLE_DB_PRIVILEGE_MAP A0 INNER JOIN SENTRY_DB_PRIVILEGE A1 ON
> A0.DB_PRIVILEGE_ID = A1.DB_PRIVILEGE_ID WHERE A0.ROLE_ID = ?. Server
> Stacktrace: javax.jdo.JDODataStoreException: Iteration request failed :
> SELECT 'org.apache.sentry.provider.db.service.model.MSentryPrivilege' AS
> NUCLEUS_TYPE,A1.URI,A1."ACTION",A1."COLUMN_NAME",A1.CREATE_TIME,A1.DB_NAME,A1.WITH_GRANT_OPTION,A1.PRIVILEGE_SCOPE,A1."SERVER_NAME",A1."TABLE_NAME",A1.DB_PRIVILEGE_ID
> FROM SENTRY_ROLE_DB_PRIVILEGE_MAP A0 INNER JOIN SENTRY_DB_PRIVILEGE A1 ON
> A0.DB_PRIVILEGE_ID = A1.DB_PRIVILEGE_ID WHERE A0.ROLE_ID = ?
> at
> org.datanucleus.api.jdo.NucleusJDOHelper.getJDOExceptionForNucleusException(NucleusJDOHelper.java:451)
> at
> org.datanucleus.api.jdo.JDOTransaction.commit(JDOTransaction.java:165)
> at
> org.apache.sentry.provider.db.service.persistent.SentryStore.commitTransaction(SentryStore.java:279)
> at
> org.apache.sentry.provider.db.service.persistent.SentryStore.commitUpdateTransaction(SentryStore.java:261)
> at
> org.apache.sentry.provider.db.service.persistent.SentryStore.alterSentryRoleGrantPrivileges(SentryStore.java:458)
> at
> org.apache.sentry.provider.db.service.thrift.SentryPolicyStoreProcessor.alter_sentry_role_grant_privilege(SentryPolicyStoreProcessor.java:261)
> at
> org.apache.sentry.provider.db.service.thrift.SentryPolicyService$Processor$alter_sentry_role_grant_privilege.getResult(SentryPolicyService.java:1237)
> at
> org.apache.sentry.provider.db.service.thrift.SentryPolicyService$Processor$alter_sentry_role_grant_privilege.getResult(SentryPolicyService.java:1222)
> at org.apache.thrift.ProcessFunction.process(ProcessFunction.java:39)
> at org.apache.thrift.TBaseProcessor.process(TBaseProcessor.java:39)
> at
> org.apache.sentry.provider.db.service.thrift.SentryProcessorWrapper.process(SentryProcessorWrapper.java:35)
> at
> org.apache.thrift.TMultiplexedProcessor.process(TMultiplexedProcessor.java:123)
> at
> org.apache.thrift.server.TThreadPoolServer$WorkerProcess.run(TThreadPoolServer.java:286)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
> at java.lang.Thread.run(Thread.java:745)
> NestedThrowablesStackTrace:
> java.sql.SQLTransactionRollbackException: A lock could not be obta
[jira] [Updated] (SENTRY-1463) Ensure HMS point-in-time snapshot consistency
[ https://issues.apache.org/jira/browse/SENTRY-1463?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hao Hao updated SENTRY-1463: Resolution: Fixed Status: Resolved (was: Patch Available) > Ensure HMS point-in-time snapshot consistency > - > > Key: SENTRY-1463 > URL: https://issues.apache.org/jira/browse/SENTRY-1463 > Project: Sentry > Issue Type: Sub-task > Components: Sentry >Affects Versions: sentry-ha-redesign >Reporter: Hao Hao >Assignee: Hao Hao > Fix For: sentry-ha-redesign > > Attachments: SENTRY-1463.001-sentry-ha-redesign.patch, > SENTRY-1463.002-sentry-ha-redesign.patch, > SENTRY-1463.003-sentry-ha-redesign.patch, > SENTRY-1463.004-sentry-ha-redesign.patch, > SENTRY-1463.005-sentry-ha-redesign.patch, > SENTRY-1463.006-sentry-ha-redesign.patch, > SENTRY-1463.007-sentry-ha-redesign.patch > > > During HMSFollower start up, when retrieving HMS full snapshot, also need to > ensure point-in-time snapshot consistency. > It should be: > * Read current HMS notification ID_initial > * Read HMS metadata state > * Read current notification ID_new > * If ID_initial != ID_new then discard the current state and goto 1. > Use configurable property: sentry.hms.snapshot.retries.max.count for max > number of retry. > Ideally, during fetching full snapshot, there should be no write activity on > the HMS. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (SENTRY-1422) JDO deadlocks while processing grant while a background thread processes Notificationlogs
[
https://issues.apache.org/jira/browse/SENTRY-1422?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15604091#comment-15604091
]
Hadoop QA commented on SENTRY-1422:
---
Here are the results of testing the latest attachment
https://issues.apache.org/jira/secure/attachment/12835053/SENTRY-1422.018-sentry-ha-redesign.patch
against sentry-ha-redesign.
{color:red}Overall:{color} -1 due to 2 errors
{color:red}ERROR:{color} mvn test exited 1
{color:red}ERROR:{color} Failed:
org.apache.sentry.tests.e2e.dbprovider.TestDbPrivilegesAtFunctionScope
Console output:
https://builds.apache.org/job/PreCommit-SENTRY-Build/2089/console
This message is automatically generated.
> JDO deadlocks while processing grant while a background thread processes
> Notificationlogs
> -
>
> Key: SENTRY-1422
> URL: https://issues.apache.org/jira/browse/SENTRY-1422
> Project: Sentry
> Issue Type: Sub-task
> Components: Hdfs Plugin
>Reporter: Sravya Tirukkovalur
>Assignee: Colin Ma
> Fix For: sentry-ha-redesign
>
> Attachments: SENTRY-1422.013-sentry-ha-redesign.patch,
> SENTRY-1422.014-sentry-ha-redesign.patch,
> SENTRY-1422.015-sentry-ha-redesign.patch,
> SENTRY-1422.016-sentry-ha-redesign.patch,
> SENTRY-1422.017-sentry-ha-redesign.patch,
> SENTRY-1422.018-sentry-ha-redesign.patch, sentry_deadlock.png,
> testDeadLock.patch
>
>
> As I was working on Sentry-1321. I see that test case
> TestDbPrivilegeCleanupOnDrop#testDropObjects fails while "Granting select on
> table" with following deadlock.
> {code}
> java.sql.SQLException: Error while processing statement: FAILED: Execution
> Error, return code 1 from
> org.apache.hadoop.hive.ql.exec.SentryGrantRevokeTask. Unknown error for
> request: TAlterSentryRoleGrantPrivilegeRequest(protocol_version:2,
> requestorUserName:admin1, roleName:select_tbl1,
> privileges:[TSentryPrivilege(privilegeScope:TABLE, serverName:server1,
> dbName:db_2, tableName:tb_1, URI:, action:select, createTime:1469562185573,
> grantOption:FALSE, columnName:)]), message: Iteration request failed : SELECT
> 'org.apache.sentry.provider.db.service.model.MSentryPrivilege' AS
> NUCLEUS_TYPE,A1.URI,A1."ACTION",A1."COLUMN_NAME",A1.CREATE_TIME,A1.DB_NAME,A1.WITH_GRANT_OPTION,A1.PRIVILEGE_SCOPE,A1."SERVER_NAME",A1."TABLE_NAME",A1.DB_PRIVILEGE_ID
> FROM SENTRY_ROLE_DB_PRIVILEGE_MAP A0 INNER JOIN SENTRY_DB_PRIVILEGE A1 ON
> A0.DB_PRIVILEGE_ID = A1.DB_PRIVILEGE_ID WHERE A0.ROLE_ID = ?. Server
> Stacktrace: javax.jdo.JDODataStoreException: Iteration request failed :
> SELECT 'org.apache.sentry.provider.db.service.model.MSentryPrivilege' AS
> NUCLEUS_TYPE,A1.URI,A1."ACTION",A1."COLUMN_NAME",A1.CREATE_TIME,A1.DB_NAME,A1.WITH_GRANT_OPTION,A1.PRIVILEGE_SCOPE,A1."SERVER_NAME",A1."TABLE_NAME",A1.DB_PRIVILEGE_ID
> FROM SENTRY_ROLE_DB_PRIVILEGE_MAP A0 INNER JOIN SENTRY_DB_PRIVILEGE A1 ON
> A0.DB_PRIVILEGE_ID = A1.DB_PRIVILEGE_ID WHERE A0.ROLE_ID = ?
> at
> org.datanucleus.api.jdo.NucleusJDOHelper.getJDOExceptionForNucleusException(NucleusJDOHelper.java:451)
> at
> org.datanucleus.api.jdo.JDOTransaction.commit(JDOTransaction.java:165)
> at
> org.apache.sentry.provider.db.service.persistent.SentryStore.commitTransaction(SentryStore.java:279)
> at
> org.apache.sentry.provider.db.service.persistent.SentryStore.commitUpdateTransaction(SentryStore.java:261)
> at
> org.apache.sentry.provider.db.service.persistent.SentryStore.alterSentryRoleGrantPrivileges(SentryStore.java:458)
> at
> org.apache.sentry.provider.db.service.thrift.SentryPolicyStoreProcessor.alter_sentry_role_grant_privilege(SentryPolicyStoreProcessor.java:261)
> at
> org.apache.sentry.provider.db.service.thrift.SentryPolicyService$Processor$alter_sentry_role_grant_privilege.getResult(SentryPolicyService.java:1237)
> at
> org.apache.sentry.provider.db.service.thrift.SentryPolicyService$Processor$alter_sentry_role_grant_privilege.getResult(SentryPolicyService.java:1222)
> at org.apache.thrift.ProcessFunction.process(ProcessFunction.java:39)
> at org.apache.thrift.TBaseProcessor.process(TBaseProcessor.java:39)
> at
> org.apache.sentry.provider.db.service.thrift.SentryProcessorWrapper.process(SentryProcessorWrapper.java:35)
> at
> org.apache.thrift.TMultiplexedProcessor.process(TMultiplexedProcessor.java:123)
> at
> org.apache.thrift.server.TThreadPoolServer$WorkerProcess.run(TThreadPoolServer.java:286)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
> at java.lang.Thread.run(Thread.java:745)
> NestedThrowablesStackTrace:
> java.sql.SQLTransactionRollbackExcep
[jira] [Updated] (SENTRY-1422) JDO deadlocks while processing grant while a background thread processes Notificationlogs
[
https://issues.apache.org/jira/browse/SENTRY-1422?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Colin Ma updated SENTRY-1422:
-
Attachment: (was: SENTRY-1422.012-sentry-ha-redesign.patch)
> JDO deadlocks while processing grant while a background thread processes
> Notificationlogs
> -
>
> Key: SENTRY-1422
> URL: https://issues.apache.org/jira/browse/SENTRY-1422
> Project: Sentry
> Issue Type: Sub-task
> Components: Hdfs Plugin
>Reporter: Sravya Tirukkovalur
>Assignee: Colin Ma
> Fix For: sentry-ha-redesign
>
> Attachments: SENTRY-1422.013-sentry-ha-redesign.patch,
> SENTRY-1422.014-sentry-ha-redesign.patch,
> SENTRY-1422.015-sentry-ha-redesign.patch,
> SENTRY-1422.016-sentry-ha-redesign.patch,
> SENTRY-1422.017-sentry-ha-redesign.patch,
> SENTRY-1422.018-sentry-ha-redesign.patch, sentry_deadlock.png,
> testDeadLock.patch
>
>
> As I was working on Sentry-1321. I see that test case
> TestDbPrivilegeCleanupOnDrop#testDropObjects fails while "Granting select on
> table" with following deadlock.
> {code}
> java.sql.SQLException: Error while processing statement: FAILED: Execution
> Error, return code 1 from
> org.apache.hadoop.hive.ql.exec.SentryGrantRevokeTask. Unknown error for
> request: TAlterSentryRoleGrantPrivilegeRequest(protocol_version:2,
> requestorUserName:admin1, roleName:select_tbl1,
> privileges:[TSentryPrivilege(privilegeScope:TABLE, serverName:server1,
> dbName:db_2, tableName:tb_1, URI:, action:select, createTime:1469562185573,
> grantOption:FALSE, columnName:)]), message: Iteration request failed : SELECT
> 'org.apache.sentry.provider.db.service.model.MSentryPrivilege' AS
> NUCLEUS_TYPE,A1.URI,A1."ACTION",A1."COLUMN_NAME",A1.CREATE_TIME,A1.DB_NAME,A1.WITH_GRANT_OPTION,A1.PRIVILEGE_SCOPE,A1."SERVER_NAME",A1."TABLE_NAME",A1.DB_PRIVILEGE_ID
> FROM SENTRY_ROLE_DB_PRIVILEGE_MAP A0 INNER JOIN SENTRY_DB_PRIVILEGE A1 ON
> A0.DB_PRIVILEGE_ID = A1.DB_PRIVILEGE_ID WHERE A0.ROLE_ID = ?. Server
> Stacktrace: javax.jdo.JDODataStoreException: Iteration request failed :
> SELECT 'org.apache.sentry.provider.db.service.model.MSentryPrivilege' AS
> NUCLEUS_TYPE,A1.URI,A1."ACTION",A1."COLUMN_NAME",A1.CREATE_TIME,A1.DB_NAME,A1.WITH_GRANT_OPTION,A1.PRIVILEGE_SCOPE,A1."SERVER_NAME",A1."TABLE_NAME",A1.DB_PRIVILEGE_ID
> FROM SENTRY_ROLE_DB_PRIVILEGE_MAP A0 INNER JOIN SENTRY_DB_PRIVILEGE A1 ON
> A0.DB_PRIVILEGE_ID = A1.DB_PRIVILEGE_ID WHERE A0.ROLE_ID = ?
> at
> org.datanucleus.api.jdo.NucleusJDOHelper.getJDOExceptionForNucleusException(NucleusJDOHelper.java:451)
> at
> org.datanucleus.api.jdo.JDOTransaction.commit(JDOTransaction.java:165)
> at
> org.apache.sentry.provider.db.service.persistent.SentryStore.commitTransaction(SentryStore.java:279)
> at
> org.apache.sentry.provider.db.service.persistent.SentryStore.commitUpdateTransaction(SentryStore.java:261)
> at
> org.apache.sentry.provider.db.service.persistent.SentryStore.alterSentryRoleGrantPrivileges(SentryStore.java:458)
> at
> org.apache.sentry.provider.db.service.thrift.SentryPolicyStoreProcessor.alter_sentry_role_grant_privilege(SentryPolicyStoreProcessor.java:261)
> at
> org.apache.sentry.provider.db.service.thrift.SentryPolicyService$Processor$alter_sentry_role_grant_privilege.getResult(SentryPolicyService.java:1237)
> at
> org.apache.sentry.provider.db.service.thrift.SentryPolicyService$Processor$alter_sentry_role_grant_privilege.getResult(SentryPolicyService.java:1222)
> at org.apache.thrift.ProcessFunction.process(ProcessFunction.java:39)
> at org.apache.thrift.TBaseProcessor.process(TBaseProcessor.java:39)
> at
> org.apache.sentry.provider.db.service.thrift.SentryProcessorWrapper.process(SentryProcessorWrapper.java:35)
> at
> org.apache.thrift.TMultiplexedProcessor.process(TMultiplexedProcessor.java:123)
> at
> org.apache.thrift.server.TThreadPoolServer$WorkerProcess.run(TThreadPoolServer.java:286)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
> at java.lang.Thread.run(Thread.java:745)
> NestedThrowablesStackTrace:
> java.sql.SQLTransactionRollbackException: A lock could not be obtained due to
> a deadlock, cycle of locks and waiters is:
> Lock : ROW, SENTRY_ROLE_DB_PRIVILEGE_MAP, (1,10)
> Waiting XID : {712, S} , SENTRY, SELECT
> 'org.apache.sentry.provider.db.service.model.MSentryPrivilege' AS
> NUCLEUS_TYPE,A1.URI,A1."ACTION",A1."COLUMN_NAME",A1.CREATE_TIME,A1.DB_NAME,A1.WITH_GRANT_OPTION,A1.PRIVILEGE_SCOPE,A1."SERVER_NAME",A1."TABLE_NAME",A1.DB_PRIVILEGE_ID
> FROM SENTRY_ROLE_DB_PRIVILEGE_MAP A0 INNER JOIN SENTRY_DB_PRIVILEGE A1
[jira] [Updated] (SENTRY-1422) JDO deadlocks while processing grant while a background thread processes Notificationlogs
[
https://issues.apache.org/jira/browse/SENTRY-1422?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Colin Ma updated SENTRY-1422:
-
Attachment: (was: SENTRY-1422.011-sentry-ha-redesign.patch)
> JDO deadlocks while processing grant while a background thread processes
> Notificationlogs
> -
>
> Key: SENTRY-1422
> URL: https://issues.apache.org/jira/browse/SENTRY-1422
> Project: Sentry
> Issue Type: Sub-task
> Components: Hdfs Plugin
>Reporter: Sravya Tirukkovalur
>Assignee: Colin Ma
> Fix For: sentry-ha-redesign
>
> Attachments: SENTRY-1422.013-sentry-ha-redesign.patch,
> SENTRY-1422.014-sentry-ha-redesign.patch,
> SENTRY-1422.015-sentry-ha-redesign.patch,
> SENTRY-1422.016-sentry-ha-redesign.patch,
> SENTRY-1422.017-sentry-ha-redesign.patch,
> SENTRY-1422.018-sentry-ha-redesign.patch, sentry_deadlock.png,
> testDeadLock.patch
>
>
> As I was working on Sentry-1321. I see that test case
> TestDbPrivilegeCleanupOnDrop#testDropObjects fails while "Granting select on
> table" with following deadlock.
> {code}
> java.sql.SQLException: Error while processing statement: FAILED: Execution
> Error, return code 1 from
> org.apache.hadoop.hive.ql.exec.SentryGrantRevokeTask. Unknown error for
> request: TAlterSentryRoleGrantPrivilegeRequest(protocol_version:2,
> requestorUserName:admin1, roleName:select_tbl1,
> privileges:[TSentryPrivilege(privilegeScope:TABLE, serverName:server1,
> dbName:db_2, tableName:tb_1, URI:, action:select, createTime:1469562185573,
> grantOption:FALSE, columnName:)]), message: Iteration request failed : SELECT
> 'org.apache.sentry.provider.db.service.model.MSentryPrivilege' AS
> NUCLEUS_TYPE,A1.URI,A1."ACTION",A1."COLUMN_NAME",A1.CREATE_TIME,A1.DB_NAME,A1.WITH_GRANT_OPTION,A1.PRIVILEGE_SCOPE,A1."SERVER_NAME",A1."TABLE_NAME",A1.DB_PRIVILEGE_ID
> FROM SENTRY_ROLE_DB_PRIVILEGE_MAP A0 INNER JOIN SENTRY_DB_PRIVILEGE A1 ON
> A0.DB_PRIVILEGE_ID = A1.DB_PRIVILEGE_ID WHERE A0.ROLE_ID = ?. Server
> Stacktrace: javax.jdo.JDODataStoreException: Iteration request failed :
> SELECT 'org.apache.sentry.provider.db.service.model.MSentryPrivilege' AS
> NUCLEUS_TYPE,A1.URI,A1."ACTION",A1."COLUMN_NAME",A1.CREATE_TIME,A1.DB_NAME,A1.WITH_GRANT_OPTION,A1.PRIVILEGE_SCOPE,A1."SERVER_NAME",A1."TABLE_NAME",A1.DB_PRIVILEGE_ID
> FROM SENTRY_ROLE_DB_PRIVILEGE_MAP A0 INNER JOIN SENTRY_DB_PRIVILEGE A1 ON
> A0.DB_PRIVILEGE_ID = A1.DB_PRIVILEGE_ID WHERE A0.ROLE_ID = ?
> at
> org.datanucleus.api.jdo.NucleusJDOHelper.getJDOExceptionForNucleusException(NucleusJDOHelper.java:451)
> at
> org.datanucleus.api.jdo.JDOTransaction.commit(JDOTransaction.java:165)
> at
> org.apache.sentry.provider.db.service.persistent.SentryStore.commitTransaction(SentryStore.java:279)
> at
> org.apache.sentry.provider.db.service.persistent.SentryStore.commitUpdateTransaction(SentryStore.java:261)
> at
> org.apache.sentry.provider.db.service.persistent.SentryStore.alterSentryRoleGrantPrivileges(SentryStore.java:458)
> at
> org.apache.sentry.provider.db.service.thrift.SentryPolicyStoreProcessor.alter_sentry_role_grant_privilege(SentryPolicyStoreProcessor.java:261)
> at
> org.apache.sentry.provider.db.service.thrift.SentryPolicyService$Processor$alter_sentry_role_grant_privilege.getResult(SentryPolicyService.java:1237)
> at
> org.apache.sentry.provider.db.service.thrift.SentryPolicyService$Processor$alter_sentry_role_grant_privilege.getResult(SentryPolicyService.java:1222)
> at org.apache.thrift.ProcessFunction.process(ProcessFunction.java:39)
> at org.apache.thrift.TBaseProcessor.process(TBaseProcessor.java:39)
> at
> org.apache.sentry.provider.db.service.thrift.SentryProcessorWrapper.process(SentryProcessorWrapper.java:35)
> at
> org.apache.thrift.TMultiplexedProcessor.process(TMultiplexedProcessor.java:123)
> at
> org.apache.thrift.server.TThreadPoolServer$WorkerProcess.run(TThreadPoolServer.java:286)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
> at java.lang.Thread.run(Thread.java:745)
> NestedThrowablesStackTrace:
> java.sql.SQLTransactionRollbackException: A lock could not be obtained due to
> a deadlock, cycle of locks and waiters is:
> Lock : ROW, SENTRY_ROLE_DB_PRIVILEGE_MAP, (1,10)
> Waiting XID : {712, S} , SENTRY, SELECT
> 'org.apache.sentry.provider.db.service.model.MSentryPrivilege' AS
> NUCLEUS_TYPE,A1.URI,A1."ACTION",A1."COLUMN_NAME",A1.CREATE_TIME,A1.DB_NAME,A1.WITH_GRANT_OPTION,A1.PRIVILEGE_SCOPE,A1."SERVER_NAME",A1."TABLE_NAME",A1.DB_PRIVILEGE_ID
> FROM SENTRY_ROLE_DB_PRIVILEGE_MAP A0 INNER JOIN SENTRY_DB_PRIVILEGE A1
[jira] [Updated] (SENTRY-1422) JDO deadlocks while processing grant while a background thread processes Notificationlogs
[
https://issues.apache.org/jira/browse/SENTRY-1422?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Colin Ma updated SENTRY-1422:
-
Attachment: (was: SENTRY-1422.010-sentry-ha-redesign.patch)
> JDO deadlocks while processing grant while a background thread processes
> Notificationlogs
> -
>
> Key: SENTRY-1422
> URL: https://issues.apache.org/jira/browse/SENTRY-1422
> Project: Sentry
> Issue Type: Sub-task
> Components: Hdfs Plugin
>Reporter: Sravya Tirukkovalur
>Assignee: Colin Ma
> Fix For: sentry-ha-redesign
>
> Attachments: SENTRY-1422.013-sentry-ha-redesign.patch,
> SENTRY-1422.014-sentry-ha-redesign.patch,
> SENTRY-1422.015-sentry-ha-redesign.patch,
> SENTRY-1422.016-sentry-ha-redesign.patch,
> SENTRY-1422.017-sentry-ha-redesign.patch,
> SENTRY-1422.018-sentry-ha-redesign.patch, sentry_deadlock.png,
> testDeadLock.patch
>
>
> As I was working on Sentry-1321. I see that test case
> TestDbPrivilegeCleanupOnDrop#testDropObjects fails while "Granting select on
> table" with following deadlock.
> {code}
> java.sql.SQLException: Error while processing statement: FAILED: Execution
> Error, return code 1 from
> org.apache.hadoop.hive.ql.exec.SentryGrantRevokeTask. Unknown error for
> request: TAlterSentryRoleGrantPrivilegeRequest(protocol_version:2,
> requestorUserName:admin1, roleName:select_tbl1,
> privileges:[TSentryPrivilege(privilegeScope:TABLE, serverName:server1,
> dbName:db_2, tableName:tb_1, URI:, action:select, createTime:1469562185573,
> grantOption:FALSE, columnName:)]), message: Iteration request failed : SELECT
> 'org.apache.sentry.provider.db.service.model.MSentryPrivilege' AS
> NUCLEUS_TYPE,A1.URI,A1."ACTION",A1."COLUMN_NAME",A1.CREATE_TIME,A1.DB_NAME,A1.WITH_GRANT_OPTION,A1.PRIVILEGE_SCOPE,A1."SERVER_NAME",A1."TABLE_NAME",A1.DB_PRIVILEGE_ID
> FROM SENTRY_ROLE_DB_PRIVILEGE_MAP A0 INNER JOIN SENTRY_DB_PRIVILEGE A1 ON
> A0.DB_PRIVILEGE_ID = A1.DB_PRIVILEGE_ID WHERE A0.ROLE_ID = ?. Server
> Stacktrace: javax.jdo.JDODataStoreException: Iteration request failed :
> SELECT 'org.apache.sentry.provider.db.service.model.MSentryPrivilege' AS
> NUCLEUS_TYPE,A1.URI,A1."ACTION",A1."COLUMN_NAME",A1.CREATE_TIME,A1.DB_NAME,A1.WITH_GRANT_OPTION,A1.PRIVILEGE_SCOPE,A1."SERVER_NAME",A1."TABLE_NAME",A1.DB_PRIVILEGE_ID
> FROM SENTRY_ROLE_DB_PRIVILEGE_MAP A0 INNER JOIN SENTRY_DB_PRIVILEGE A1 ON
> A0.DB_PRIVILEGE_ID = A1.DB_PRIVILEGE_ID WHERE A0.ROLE_ID = ?
> at
> org.datanucleus.api.jdo.NucleusJDOHelper.getJDOExceptionForNucleusException(NucleusJDOHelper.java:451)
> at
> org.datanucleus.api.jdo.JDOTransaction.commit(JDOTransaction.java:165)
> at
> org.apache.sentry.provider.db.service.persistent.SentryStore.commitTransaction(SentryStore.java:279)
> at
> org.apache.sentry.provider.db.service.persistent.SentryStore.commitUpdateTransaction(SentryStore.java:261)
> at
> org.apache.sentry.provider.db.service.persistent.SentryStore.alterSentryRoleGrantPrivileges(SentryStore.java:458)
> at
> org.apache.sentry.provider.db.service.thrift.SentryPolicyStoreProcessor.alter_sentry_role_grant_privilege(SentryPolicyStoreProcessor.java:261)
> at
> org.apache.sentry.provider.db.service.thrift.SentryPolicyService$Processor$alter_sentry_role_grant_privilege.getResult(SentryPolicyService.java:1237)
> at
> org.apache.sentry.provider.db.service.thrift.SentryPolicyService$Processor$alter_sentry_role_grant_privilege.getResult(SentryPolicyService.java:1222)
> at org.apache.thrift.ProcessFunction.process(ProcessFunction.java:39)
> at org.apache.thrift.TBaseProcessor.process(TBaseProcessor.java:39)
> at
> org.apache.sentry.provider.db.service.thrift.SentryProcessorWrapper.process(SentryProcessorWrapper.java:35)
> at
> org.apache.thrift.TMultiplexedProcessor.process(TMultiplexedProcessor.java:123)
> at
> org.apache.thrift.server.TThreadPoolServer$WorkerProcess.run(TThreadPoolServer.java:286)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
> at java.lang.Thread.run(Thread.java:745)
> NestedThrowablesStackTrace:
> java.sql.SQLTransactionRollbackException: A lock could not be obtained due to
> a deadlock, cycle of locks and waiters is:
> Lock : ROW, SENTRY_ROLE_DB_PRIVILEGE_MAP, (1,10)
> Waiting XID : {712, S} , SENTRY, SELECT
> 'org.apache.sentry.provider.db.service.model.MSentryPrivilege' AS
> NUCLEUS_TYPE,A1.URI,A1."ACTION",A1."COLUMN_NAME",A1.CREATE_TIME,A1.DB_NAME,A1.WITH_GRANT_OPTION,A1.PRIVILEGE_SCOPE,A1."SERVER_NAME",A1."TABLE_NAME",A1.DB_PRIVILEGE_ID
> FROM SENTRY_ROLE_DB_PRIVILEGE_MAP A0 INNER JOIN SENTRY_DB_PRIVILEGE A1
[jira] [Updated] (SENTRY-1422) JDO deadlocks while processing grant while a background thread processes Notificationlogs
[
https://issues.apache.org/jira/browse/SENTRY-1422?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Colin Ma updated SENTRY-1422:
-
Attachment: (was: SENTRY-1422.008-sentry-ha-redesign.patch)
> JDO deadlocks while processing grant while a background thread processes
> Notificationlogs
> -
>
> Key: SENTRY-1422
> URL: https://issues.apache.org/jira/browse/SENTRY-1422
> Project: Sentry
> Issue Type: Sub-task
> Components: Hdfs Plugin
>Reporter: Sravya Tirukkovalur
>Assignee: Colin Ma
> Fix For: sentry-ha-redesign
>
> Attachments: SENTRY-1422.010-sentry-ha-redesign.patch,
> SENTRY-1422.011-sentry-ha-redesign.patch,
> SENTRY-1422.012-sentry-ha-redesign.patch,
> SENTRY-1422.013-sentry-ha-redesign.patch,
> SENTRY-1422.014-sentry-ha-redesign.patch,
> SENTRY-1422.015-sentry-ha-redesign.patch,
> SENTRY-1422.016-sentry-ha-redesign.patch,
> SENTRY-1422.017-sentry-ha-redesign.patch,
> SENTRY-1422.018-sentry-ha-redesign.patch, sentry_deadlock.png,
> testDeadLock.patch
>
>
> As I was working on Sentry-1321. I see that test case
> TestDbPrivilegeCleanupOnDrop#testDropObjects fails while "Granting select on
> table" with following deadlock.
> {code}
> java.sql.SQLException: Error while processing statement: FAILED: Execution
> Error, return code 1 from
> org.apache.hadoop.hive.ql.exec.SentryGrantRevokeTask. Unknown error for
> request: TAlterSentryRoleGrantPrivilegeRequest(protocol_version:2,
> requestorUserName:admin1, roleName:select_tbl1,
> privileges:[TSentryPrivilege(privilegeScope:TABLE, serverName:server1,
> dbName:db_2, tableName:tb_1, URI:, action:select, createTime:1469562185573,
> grantOption:FALSE, columnName:)]), message: Iteration request failed : SELECT
> 'org.apache.sentry.provider.db.service.model.MSentryPrivilege' AS
> NUCLEUS_TYPE,A1.URI,A1."ACTION",A1."COLUMN_NAME",A1.CREATE_TIME,A1.DB_NAME,A1.WITH_GRANT_OPTION,A1.PRIVILEGE_SCOPE,A1."SERVER_NAME",A1."TABLE_NAME",A1.DB_PRIVILEGE_ID
> FROM SENTRY_ROLE_DB_PRIVILEGE_MAP A0 INNER JOIN SENTRY_DB_PRIVILEGE A1 ON
> A0.DB_PRIVILEGE_ID = A1.DB_PRIVILEGE_ID WHERE A0.ROLE_ID = ?. Server
> Stacktrace: javax.jdo.JDODataStoreException: Iteration request failed :
> SELECT 'org.apache.sentry.provider.db.service.model.MSentryPrivilege' AS
> NUCLEUS_TYPE,A1.URI,A1."ACTION",A1."COLUMN_NAME",A1.CREATE_TIME,A1.DB_NAME,A1.WITH_GRANT_OPTION,A1.PRIVILEGE_SCOPE,A1."SERVER_NAME",A1."TABLE_NAME",A1.DB_PRIVILEGE_ID
> FROM SENTRY_ROLE_DB_PRIVILEGE_MAP A0 INNER JOIN SENTRY_DB_PRIVILEGE A1 ON
> A0.DB_PRIVILEGE_ID = A1.DB_PRIVILEGE_ID WHERE A0.ROLE_ID = ?
> at
> org.datanucleus.api.jdo.NucleusJDOHelper.getJDOExceptionForNucleusException(NucleusJDOHelper.java:451)
> at
> org.datanucleus.api.jdo.JDOTransaction.commit(JDOTransaction.java:165)
> at
> org.apache.sentry.provider.db.service.persistent.SentryStore.commitTransaction(SentryStore.java:279)
> at
> org.apache.sentry.provider.db.service.persistent.SentryStore.commitUpdateTransaction(SentryStore.java:261)
> at
> org.apache.sentry.provider.db.service.persistent.SentryStore.alterSentryRoleGrantPrivileges(SentryStore.java:458)
> at
> org.apache.sentry.provider.db.service.thrift.SentryPolicyStoreProcessor.alter_sentry_role_grant_privilege(SentryPolicyStoreProcessor.java:261)
> at
> org.apache.sentry.provider.db.service.thrift.SentryPolicyService$Processor$alter_sentry_role_grant_privilege.getResult(SentryPolicyService.java:1237)
> at
> org.apache.sentry.provider.db.service.thrift.SentryPolicyService$Processor$alter_sentry_role_grant_privilege.getResult(SentryPolicyService.java:1222)
> at org.apache.thrift.ProcessFunction.process(ProcessFunction.java:39)
> at org.apache.thrift.TBaseProcessor.process(TBaseProcessor.java:39)
> at
> org.apache.sentry.provider.db.service.thrift.SentryProcessorWrapper.process(SentryProcessorWrapper.java:35)
> at
> org.apache.thrift.TMultiplexedProcessor.process(TMultiplexedProcessor.java:123)
> at
> org.apache.thrift.server.TThreadPoolServer$WorkerProcess.run(TThreadPoolServer.java:286)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
> at java.lang.Thread.run(Thread.java:745)
> NestedThrowablesStackTrace:
> java.sql.SQLTransactionRollbackException: A lock could not be obtained due to
> a deadlock, cycle of locks and waiters is:
> Lock : ROW, SENTRY_ROLE_DB_PRIVILEGE_MAP, (1,10)
> Waiting XID : {712, S} , SENTRY, SELECT
> 'org.apache.sentry.provider.db.service.model.MSentryPrivilege' AS
> NUCLEUS_TYPE,A1.URI,A1."ACTION",A1."COLUMN_NAME",A1.CREATE_TIME,A1.DB_NAME,A1.WITH_GRANT_OPTION,A1.PRIVILE
[jira] [Updated] (SENTRY-1422) JDO deadlocks while processing grant while a background thread processes Notificationlogs
[
https://issues.apache.org/jira/browse/SENTRY-1422?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Colin Ma updated SENTRY-1422:
-
Attachment: SENTRY-1422.018-sentry-ha-redesign.patch
> JDO deadlocks while processing grant while a background thread processes
> Notificationlogs
> -
>
> Key: SENTRY-1422
> URL: https://issues.apache.org/jira/browse/SENTRY-1422
> Project: Sentry
> Issue Type: Sub-task
> Components: Hdfs Plugin
>Reporter: Sravya Tirukkovalur
>Assignee: Colin Ma
> Fix For: sentry-ha-redesign
>
> Attachments: SENTRY-1422.010-sentry-ha-redesign.patch,
> SENTRY-1422.011-sentry-ha-redesign.patch,
> SENTRY-1422.012-sentry-ha-redesign.patch,
> SENTRY-1422.013-sentry-ha-redesign.patch,
> SENTRY-1422.014-sentry-ha-redesign.patch,
> SENTRY-1422.015-sentry-ha-redesign.patch,
> SENTRY-1422.016-sentry-ha-redesign.patch,
> SENTRY-1422.017-sentry-ha-redesign.patch,
> SENTRY-1422.018-sentry-ha-redesign.patch, sentry_deadlock.png,
> testDeadLock.patch
>
>
> As I was working on Sentry-1321. I see that test case
> TestDbPrivilegeCleanupOnDrop#testDropObjects fails while "Granting select on
> table" with following deadlock.
> {code}
> java.sql.SQLException: Error while processing statement: FAILED: Execution
> Error, return code 1 from
> org.apache.hadoop.hive.ql.exec.SentryGrantRevokeTask. Unknown error for
> request: TAlterSentryRoleGrantPrivilegeRequest(protocol_version:2,
> requestorUserName:admin1, roleName:select_tbl1,
> privileges:[TSentryPrivilege(privilegeScope:TABLE, serverName:server1,
> dbName:db_2, tableName:tb_1, URI:, action:select, createTime:1469562185573,
> grantOption:FALSE, columnName:)]), message: Iteration request failed : SELECT
> 'org.apache.sentry.provider.db.service.model.MSentryPrivilege' AS
> NUCLEUS_TYPE,A1.URI,A1."ACTION",A1."COLUMN_NAME",A1.CREATE_TIME,A1.DB_NAME,A1.WITH_GRANT_OPTION,A1.PRIVILEGE_SCOPE,A1."SERVER_NAME",A1."TABLE_NAME",A1.DB_PRIVILEGE_ID
> FROM SENTRY_ROLE_DB_PRIVILEGE_MAP A0 INNER JOIN SENTRY_DB_PRIVILEGE A1 ON
> A0.DB_PRIVILEGE_ID = A1.DB_PRIVILEGE_ID WHERE A0.ROLE_ID = ?. Server
> Stacktrace: javax.jdo.JDODataStoreException: Iteration request failed :
> SELECT 'org.apache.sentry.provider.db.service.model.MSentryPrivilege' AS
> NUCLEUS_TYPE,A1.URI,A1."ACTION",A1."COLUMN_NAME",A1.CREATE_TIME,A1.DB_NAME,A1.WITH_GRANT_OPTION,A1.PRIVILEGE_SCOPE,A1."SERVER_NAME",A1."TABLE_NAME",A1.DB_PRIVILEGE_ID
> FROM SENTRY_ROLE_DB_PRIVILEGE_MAP A0 INNER JOIN SENTRY_DB_PRIVILEGE A1 ON
> A0.DB_PRIVILEGE_ID = A1.DB_PRIVILEGE_ID WHERE A0.ROLE_ID = ?
> at
> org.datanucleus.api.jdo.NucleusJDOHelper.getJDOExceptionForNucleusException(NucleusJDOHelper.java:451)
> at
> org.datanucleus.api.jdo.JDOTransaction.commit(JDOTransaction.java:165)
> at
> org.apache.sentry.provider.db.service.persistent.SentryStore.commitTransaction(SentryStore.java:279)
> at
> org.apache.sentry.provider.db.service.persistent.SentryStore.commitUpdateTransaction(SentryStore.java:261)
> at
> org.apache.sentry.provider.db.service.persistent.SentryStore.alterSentryRoleGrantPrivileges(SentryStore.java:458)
> at
> org.apache.sentry.provider.db.service.thrift.SentryPolicyStoreProcessor.alter_sentry_role_grant_privilege(SentryPolicyStoreProcessor.java:261)
> at
> org.apache.sentry.provider.db.service.thrift.SentryPolicyService$Processor$alter_sentry_role_grant_privilege.getResult(SentryPolicyService.java:1237)
> at
> org.apache.sentry.provider.db.service.thrift.SentryPolicyService$Processor$alter_sentry_role_grant_privilege.getResult(SentryPolicyService.java:1222)
> at org.apache.thrift.ProcessFunction.process(ProcessFunction.java:39)
> at org.apache.thrift.TBaseProcessor.process(TBaseProcessor.java:39)
> at
> org.apache.sentry.provider.db.service.thrift.SentryProcessorWrapper.process(SentryProcessorWrapper.java:35)
> at
> org.apache.thrift.TMultiplexedProcessor.process(TMultiplexedProcessor.java:123)
> at
> org.apache.thrift.server.TThreadPoolServer$WorkerProcess.run(TThreadPoolServer.java:286)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
> at java.lang.Thread.run(Thread.java:745)
> NestedThrowablesStackTrace:
> java.sql.SQLTransactionRollbackException: A lock could not be obtained due to
> a deadlock, cycle of locks and waiters is:
> Lock : ROW, SENTRY_ROLE_DB_PRIVILEGE_MAP, (1,10)
> Waiting XID : {712, S} , SENTRY, SELECT
> 'org.apache.sentry.provider.db.service.model.MSentryPrivilege' AS
> NUCLEUS_TYPE,A1.URI,A1."ACTION",A1."COLUMN_NAME",A1.CREATE_TIME,A1.DB_NAME,A1.WITH_GRANT_OPTION,A1.PRIVILEGE_SCOPE,A1
[jira] [Updated] (SENTRY-1422) JDO deadlocks while processing grant while a background thread processes Notificationlogs
[
https://issues.apache.org/jira/browse/SENTRY-1422?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Colin Ma updated SENTRY-1422:
-
Attachment: (was: SENTRY-1422.009-sentry-ha-redesign.patch)
> JDO deadlocks while processing grant while a background thread processes
> Notificationlogs
> -
>
> Key: SENTRY-1422
> URL: https://issues.apache.org/jira/browse/SENTRY-1422
> Project: Sentry
> Issue Type: Sub-task
> Components: Hdfs Plugin
>Reporter: Sravya Tirukkovalur
>Assignee: Colin Ma
> Fix For: sentry-ha-redesign
>
> Attachments: SENTRY-1422.010-sentry-ha-redesign.patch,
> SENTRY-1422.011-sentry-ha-redesign.patch,
> SENTRY-1422.012-sentry-ha-redesign.patch,
> SENTRY-1422.013-sentry-ha-redesign.patch,
> SENTRY-1422.014-sentry-ha-redesign.patch,
> SENTRY-1422.015-sentry-ha-redesign.patch,
> SENTRY-1422.016-sentry-ha-redesign.patch,
> SENTRY-1422.017-sentry-ha-redesign.patch,
> SENTRY-1422.018-sentry-ha-redesign.patch, sentry_deadlock.png,
> testDeadLock.patch
>
>
> As I was working on Sentry-1321. I see that test case
> TestDbPrivilegeCleanupOnDrop#testDropObjects fails while "Granting select on
> table" with following deadlock.
> {code}
> java.sql.SQLException: Error while processing statement: FAILED: Execution
> Error, return code 1 from
> org.apache.hadoop.hive.ql.exec.SentryGrantRevokeTask. Unknown error for
> request: TAlterSentryRoleGrantPrivilegeRequest(protocol_version:2,
> requestorUserName:admin1, roleName:select_tbl1,
> privileges:[TSentryPrivilege(privilegeScope:TABLE, serverName:server1,
> dbName:db_2, tableName:tb_1, URI:, action:select, createTime:1469562185573,
> grantOption:FALSE, columnName:)]), message: Iteration request failed : SELECT
> 'org.apache.sentry.provider.db.service.model.MSentryPrivilege' AS
> NUCLEUS_TYPE,A1.URI,A1."ACTION",A1."COLUMN_NAME",A1.CREATE_TIME,A1.DB_NAME,A1.WITH_GRANT_OPTION,A1.PRIVILEGE_SCOPE,A1."SERVER_NAME",A1."TABLE_NAME",A1.DB_PRIVILEGE_ID
> FROM SENTRY_ROLE_DB_PRIVILEGE_MAP A0 INNER JOIN SENTRY_DB_PRIVILEGE A1 ON
> A0.DB_PRIVILEGE_ID = A1.DB_PRIVILEGE_ID WHERE A0.ROLE_ID = ?. Server
> Stacktrace: javax.jdo.JDODataStoreException: Iteration request failed :
> SELECT 'org.apache.sentry.provider.db.service.model.MSentryPrivilege' AS
> NUCLEUS_TYPE,A1.URI,A1."ACTION",A1."COLUMN_NAME",A1.CREATE_TIME,A1.DB_NAME,A1.WITH_GRANT_OPTION,A1.PRIVILEGE_SCOPE,A1."SERVER_NAME",A1."TABLE_NAME",A1.DB_PRIVILEGE_ID
> FROM SENTRY_ROLE_DB_PRIVILEGE_MAP A0 INNER JOIN SENTRY_DB_PRIVILEGE A1 ON
> A0.DB_PRIVILEGE_ID = A1.DB_PRIVILEGE_ID WHERE A0.ROLE_ID = ?
> at
> org.datanucleus.api.jdo.NucleusJDOHelper.getJDOExceptionForNucleusException(NucleusJDOHelper.java:451)
> at
> org.datanucleus.api.jdo.JDOTransaction.commit(JDOTransaction.java:165)
> at
> org.apache.sentry.provider.db.service.persistent.SentryStore.commitTransaction(SentryStore.java:279)
> at
> org.apache.sentry.provider.db.service.persistent.SentryStore.commitUpdateTransaction(SentryStore.java:261)
> at
> org.apache.sentry.provider.db.service.persistent.SentryStore.alterSentryRoleGrantPrivileges(SentryStore.java:458)
> at
> org.apache.sentry.provider.db.service.thrift.SentryPolicyStoreProcessor.alter_sentry_role_grant_privilege(SentryPolicyStoreProcessor.java:261)
> at
> org.apache.sentry.provider.db.service.thrift.SentryPolicyService$Processor$alter_sentry_role_grant_privilege.getResult(SentryPolicyService.java:1237)
> at
> org.apache.sentry.provider.db.service.thrift.SentryPolicyService$Processor$alter_sentry_role_grant_privilege.getResult(SentryPolicyService.java:1222)
> at org.apache.thrift.ProcessFunction.process(ProcessFunction.java:39)
> at org.apache.thrift.TBaseProcessor.process(TBaseProcessor.java:39)
> at
> org.apache.sentry.provider.db.service.thrift.SentryProcessorWrapper.process(SentryProcessorWrapper.java:35)
> at
> org.apache.thrift.TMultiplexedProcessor.process(TMultiplexedProcessor.java:123)
> at
> org.apache.thrift.server.TThreadPoolServer$WorkerProcess.run(TThreadPoolServer.java:286)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
> at java.lang.Thread.run(Thread.java:745)
> NestedThrowablesStackTrace:
> java.sql.SQLTransactionRollbackException: A lock could not be obtained due to
> a deadlock, cycle of locks and waiters is:
> Lock : ROW, SENTRY_ROLE_DB_PRIVILEGE_MAP, (1,10)
> Waiting XID : {712, S} , SENTRY, SELECT
> 'org.apache.sentry.provider.db.service.model.MSentryPrivilege' AS
> NUCLEUS_TYPE,A1.URI,A1."ACTION",A1."COLUMN_NAME",A1.CREATE_TIME,A1.DB_NAME,A1.WITH_GRANT_OPTION,A1.PRIVILE
[jira] [Commented] (SENTRY-1489) Categorize e2e tests into slow and regular tests, so that can adapt the timeout and etc.
[
https://issues.apache.org/jira/browse/SENTRY-1489?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15603839#comment-15603839
]
Hadoop QA commented on SENTRY-1489:
---
Here are the results of testing the latest attachment
https://issues.apache.org/jira/secure/attachment/12835034/SENTRY-1489.1.patch
against master.
{color:green}Overall:{color} +1 all checks pass
{color:green}SUCCESS:{color} all tests passed
Console output:
https://builds.apache.org/job/PreCommit-SENTRY-Build/2087/console
This message is automatically generated.
> Categorize e2e tests into slow and regular tests, so that can adapt the
> timeout and etc.
>
>
> Key: SENTRY-1489
> URL: https://issues.apache.org/jira/browse/SENTRY-1489
> Project: Sentry
> Issue Type: Test
> Components: Sentry
>Affects Versions: 1.8.0, sentry-ha-redesign
>Reporter: Anne Yu
>Assignee: Anne Yu
> Fix For: 1.8.0, sentry-ha-redesign
>
> Attachments: SENTRY-1489.0.patch, SENTRY-1489.1.patch
>
>
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (SENTRY-1463) Ensure HMS point-in-time snapshot consistency
[
https://issues.apache.org/jira/browse/SENTRY-1463?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15603835#comment-15603835
]
Hadoop QA commented on SENTRY-1463:
---
Here are the results of testing the latest attachment
https://issues.apache.org/jira/secure/attachment/12835035/SENTRY-1463.007-sentry-ha-redesign.patch
against sentry-ha-redesign.
{color:red}Overall:{color} -1 due to 2 errors
{color:red}ERROR:{color} mvn test exited 1
{color:red}ERROR:{color} Failed:
org.apache.sentry.tests.e2e.dbprovider.TestConcurrentClients
Console output:
https://builds.apache.org/job/PreCommit-SENTRY-Build/2086/console
This message is automatically generated.
> Ensure HMS point-in-time snapshot consistency
> -
>
> Key: SENTRY-1463
> URL: https://issues.apache.org/jira/browse/SENTRY-1463
> Project: Sentry
> Issue Type: Sub-task
> Components: Sentry
>Affects Versions: sentry-ha-redesign
>Reporter: Hao Hao
>Assignee: Hao Hao
> Fix For: sentry-ha-redesign
>
> Attachments: SENTRY-1463.001-sentry-ha-redesign.patch,
> SENTRY-1463.002-sentry-ha-redesign.patch,
> SENTRY-1463.003-sentry-ha-redesign.patch,
> SENTRY-1463.004-sentry-ha-redesign.patch,
> SENTRY-1463.005-sentry-ha-redesign.patch,
> SENTRY-1463.006-sentry-ha-redesign.patch,
> SENTRY-1463.007-sentry-ha-redesign.patch
>
>
> During HMSFollower start up, when retrieving HMS full snapshot, also need to
> ensure point-in-time snapshot consistency.
> It should be:
> * Read current HMS notification ID_initial
> * Read HMS metadata state
> * Read current notification ID_new
> * If ID_initial != ID_new then discard the current state and goto 1.
> Use configurable property: sentry.hms.snapshot.retries.max.count for max
> number of retry.
> Ideally, during fetching full snapshot, there should be no write activity on
> the HMS.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (SENTRY-1313) Database prefix is not honoured when executing grant statement
[
https://issues.apache.org/jira/browse/SENTRY-1313?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15603788#comment-15603788
]
Hadoop QA commented on SENTRY-1313:
---
Here are the results of testing the latest attachment
https://issues.apache.org/jira/secure/attachment/12835037/SENTRY-1313.1.patch
against master.
{color:red}Overall:{color} -1 due to an error
{color:red}ERROR:{color} mvn test exited 1
Console output:
https://builds.apache.org/job/PreCommit-SENTRY-Build/2088/console
This message is automatically generated.
> Database prefix is not honoured when executing grant statement
> --
>
> Key: SENTRY-1313
> URL: https://issues.apache.org/jira/browse/SENTRY-1313
> Project: Sentry
> Issue Type: Bug
>Reporter: Sravya Tirukkovalur
>Assignee: Sravya Tirukkovalur
>Priority: Critical
> Attachments: SENTRY-1313.0.patch, SENTRY-1313.0.patch,
> SENTRY-1313.1.patch
>
>
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (SENTRY-1508) MetastorePlugin.java does not handle properly initialization failure
[
https://issues.apache.org/jira/browse/SENTRY-1508?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15603751#comment-15603751
]
Alexander Kolbasov commented on SENTRY-1508:
What is the expected semantics of callers to various MetastorePlugin functions
when
* MetastorePlugin initialization isn't complete - should they wait or go ahead?
* MetastorePlugin initialization failed - should they also fail or try to
recover?
> MetastorePlugin.java does not handle properly initialization failure
>
>
> Key: SENTRY-1508
> URL: https://issues.apache.org/jira/browse/SENTRY-1508
> Project: Sentry
> Issue Type: Bug
>Reporter: Vadim Spector
>Assignee: Vadim Spector
>Priority: Critical
> Attachments: SENTRY-1508.001.patch
>
>
> MetastorePlugin.java has several implementation issues that need to be fixed
> by this JIRA:
> a) The state of MetastorePlugin is not communicated cleanly. The constructor
> initializes successfully regardless of whether HMS cache initialization was
> successful or not. The initThread.run() can easily fail when, say,
> cacheInitializer.createInitialUpdate() fails due to misconfigured HMS, so it
> is quite possible. The state is communicated through 3 variables: boolean
> initComplete, Throwable initError, and UpdateableAuthzPaths authzPaths.
> initComplete is always set to true from finally {} block, so it really
> indicates the end of an _attempt_ to initialize the cache. It, thus, should
> only be used to detect whether the cache initialization is still in-progress,
> not whether initialization has been successful. To determine whether cache
> initialization was successful, initComplete should be used together with
> initError, which is set only when initialization fails. This is not how the
> code implements it in more than one place, which should be clear from
> analyzing a patch.
> b) There are several synchronization issues that may lead to the failure of
> synchronizing with the Sentry. They usually have to do with inconsistent use
> of monitor objects to guard access to thread-unsafe objects. E.g.
> authzPaths.updatePartial() uses a lock created just for the scope of a single
> call, which makes it useless for synchronization. A single read-write lock
> should be used instead, as there is one read and one write operation
> performed on authzPaths within MetastorePlugin.
> c) Failure to create authzPaths is not communicated clearly to a caller. When
> it is dereferenced from the public callback APIs, it results in
> NullPointerException. The suggested fix of this issue in SENTRY-1270 avoids
> NullPointerException by detecting the null authzPaths, printing error
> message, and returning. This effectively leads to consistent failure to
> update local cache without notifying the caller. The suggested solution would
> be not only to throw IllegalArgumentException to the caller instead, but to
> also specify why exactly authzPaths == null - due to initialization still
> being in progress or due to its failure.
> d) The housekeeping thread running SyncTask to synchronize with Sentry state
> fails when authzPaths == null. However, it keeps printing misleading "
> Metastore Plugin cache has not finished initialization." message even in the
> case of a permanent cache initialization failure. It needs to print the real
> cause of this condition, by analyzing authzPaths together with initComplete
> and initError values.
> e) getClient() may return null, in which case dereferencing it causes not so
> helpful NullPointerException. Instead, while getClient() may still print
> error message, it should also re-throw an original exception, which would
> then be much easier to debug.
> f) Each code fork deserves log message: e.g. addPath() retrurns immediately
> if PathsUpdate.parsePath() returns null - w/o printing any log.
> g) in SyncTask.run() - if notificationLock.tryLock() succeeds, yet the next
> line's expression "MetastorePlugin.this.authzPaths == null" is evaluated to
> false, run() exits w/o a chance to call notificationLock.unlock(). All the
> code following lock.tryLock() should be inside try-catch-finally, with
> finally's first line being lock.unlock(), as a general safe pattern.
> h) some additional misc synchronization issues may also be addressed by the
> patch.
> What's not in the scope:
> a) the initial patch honors the original design supporting asynchronous
> synchronization. It means that MetastorePlugin is always constructed, even if
> it's in a broken state. It is up to the public callback APIs to properly
> inform the caller of the broken state.
> b) however, if the reviewers decide that support of asynchronous
> initialization is not necessary, it may be preferable to drop
[jira] [Created] (SENTRY-1510) Add option to use non pool model for sentry client
Li Li created SENTRY-1510: - Summary: Add option to use non pool model for sentry client Key: SENTRY-1510 URL: https://issues.apache.org/jira/browse/SENTRY-1510 Project: Sentry Issue Type: Improvement Affects Versions: sentry-ha-redesign Reporter: Li Li Assignee: Li Li Currently sentry clients e.g. impala uses non pool model (SentryPolicyServiceClientDefaultImpl), thus it's better to keep the non pool model for those clients to avoid unnecessary incompatible issues. Also the current pool model (PoolClientInvocationHandler) does not manage the opening connections well. e.g. Opening connections with failed servers should be closed promptly. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (SENTRY-1510) Add option to use non pool model for sentry client
[ https://issues.apache.org/jira/browse/SENTRY-1510?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Li Li updated SENTRY-1510: -- Issue Type: Sub-task (was: Improvement) Parent: SENTRY-872 > Add option to use non pool model for sentry client > -- > > Key: SENTRY-1510 > URL: https://issues.apache.org/jira/browse/SENTRY-1510 > Project: Sentry > Issue Type: Sub-task >Affects Versions: sentry-ha-redesign >Reporter: Li Li >Assignee: Li Li > > Currently sentry clients e.g. impala uses non pool model > (SentryPolicyServiceClientDefaultImpl), thus it's better to keep the non pool > model for those clients to avoid unnecessary incompatible issues. > Also the current pool model (PoolClientInvocationHandler) does not manage the > opening connections well. e.g. Opening connections with failed servers should > be closed promptly. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (SENTRY-1463) Ensure HMS point-in-time snapshot consistency
[ https://issues.apache.org/jira/browse/SENTRY-1463?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15603619#comment-15603619 ] Hao Hao commented on SENTRY-1463: - Thanks Sravya. For the test failure, as commented in SENTRY-1411, TestConcurrentClients is related to SENTRY-1422. SENTRY-1422 should resolve the failure. > Ensure HMS point-in-time snapshot consistency > - > > Key: SENTRY-1463 > URL: https://issues.apache.org/jira/browse/SENTRY-1463 > Project: Sentry > Issue Type: Sub-task > Components: Sentry >Affects Versions: sentry-ha-redesign >Reporter: Hao Hao >Assignee: Hao Hao > Fix For: sentry-ha-redesign > > Attachments: SENTRY-1463.001-sentry-ha-redesign.patch, > SENTRY-1463.002-sentry-ha-redesign.patch, > SENTRY-1463.003-sentry-ha-redesign.patch, > SENTRY-1463.004-sentry-ha-redesign.patch, > SENTRY-1463.005-sentry-ha-redesign.patch, > SENTRY-1463.006-sentry-ha-redesign.patch, > SENTRY-1463.007-sentry-ha-redesign.patch > > > During HMSFollower start up, when retrieving HMS full snapshot, also need to > ensure point-in-time snapshot consistency. > It should be: > * Read current HMS notification ID_initial > * Read HMS metadata state > * Read current notification ID_new > * If ID_initial != ID_new then discard the current state and goto 1. > Use configurable property: sentry.hms.snapshot.retries.max.count for max > number of retry. > Ideally, during fetching full snapshot, there should be no write activity on > the HMS. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (SENTRY-1463) Ensure HMS point-in-time snapshot consistency
[ https://issues.apache.org/jira/browse/SENTRY-1463?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15603577#comment-15603577 ] Sravya Tirukkovalur commented on SENTRY-1463: - With respect to test failure, I briefly looked at the failure and nothing obvious that I could find. But, I remember seeing this failure else where. Is this one of the flaky tests? > Ensure HMS point-in-time snapshot consistency > - > > Key: SENTRY-1463 > URL: https://issues.apache.org/jira/browse/SENTRY-1463 > Project: Sentry > Issue Type: Sub-task > Components: Sentry >Affects Versions: sentry-ha-redesign >Reporter: Hao Hao >Assignee: Hao Hao > Fix For: sentry-ha-redesign > > Attachments: SENTRY-1463.001-sentry-ha-redesign.patch, > SENTRY-1463.002-sentry-ha-redesign.patch, > SENTRY-1463.003-sentry-ha-redesign.patch, > SENTRY-1463.004-sentry-ha-redesign.patch, > SENTRY-1463.005-sentry-ha-redesign.patch, > SENTRY-1463.006-sentry-ha-redesign.patch, > SENTRY-1463.007-sentry-ha-redesign.patch > > > During HMSFollower start up, when retrieving HMS full snapshot, also need to > ensure point-in-time snapshot consistency. > It should be: > * Read current HMS notification ID_initial > * Read HMS metadata state > * Read current notification ID_new > * If ID_initial != ID_new then discard the current state and goto 1. > Use configurable property: sentry.hms.snapshot.retries.max.count for max > number of retry. > Ideally, during fetching full snapshot, there should be no write activity on > the HMS. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (SENTRY-1463) Ensure HMS point-in-time snapshot consistency
[ https://issues.apache.org/jira/browse/SENTRY-1463?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15603572#comment-15603572 ] Sravya Tirukkovalur commented on SENTRY-1463: - Code changes look good to me. +1 > Ensure HMS point-in-time snapshot consistency > - > > Key: SENTRY-1463 > URL: https://issues.apache.org/jira/browse/SENTRY-1463 > Project: Sentry > Issue Type: Sub-task > Components: Sentry >Affects Versions: sentry-ha-redesign >Reporter: Hao Hao >Assignee: Hao Hao > Fix For: sentry-ha-redesign > > Attachments: SENTRY-1463.001-sentry-ha-redesign.patch, > SENTRY-1463.002-sentry-ha-redesign.patch, > SENTRY-1463.003-sentry-ha-redesign.patch, > SENTRY-1463.004-sentry-ha-redesign.patch, > SENTRY-1463.005-sentry-ha-redesign.patch, > SENTRY-1463.006-sentry-ha-redesign.patch, > SENTRY-1463.007-sentry-ha-redesign.patch > > > During HMSFollower start up, when retrieving HMS full snapshot, also need to > ensure point-in-time snapshot consistency. > It should be: > * Read current HMS notification ID_initial > * Read HMS metadata state > * Read current notification ID_new > * If ID_initial != ID_new then discard the current state and goto 1. > Use configurable property: sentry.hms.snapshot.retries.max.count for max > number of retry. > Ideally, during fetching full snapshot, there should be no write activity on > the HMS. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (SENTRY-1313) Database prefix is not honoured when executing grant statement
[ https://issues.apache.org/jira/browse/SENTRY-1313?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sravya Tirukkovalur updated SENTRY-1313: Attachment: SENTRY-1313.1.patch Apparently there is an e2e test for this contributed through SENTRY-858, updating it in this patch after the fix. > Database prefix is not honoured when executing grant statement > -- > > Key: SENTRY-1313 > URL: https://issues.apache.org/jira/browse/SENTRY-1313 > Project: Sentry > Issue Type: Bug >Reporter: Sravya Tirukkovalur >Assignee: Sravya Tirukkovalur >Priority: Critical > Attachments: SENTRY-1313.0.patch, SENTRY-1313.0.patch, > SENTRY-1313.1.patch > > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (SENTRY-1463) Ensure HMS point-in-time snapshot consistency
[ https://issues.apache.org/jira/browse/SENTRY-1463?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hao Hao updated SENTRY-1463: Attachment: SENTRY-1463.007-sentry-ha-redesign.patch > Ensure HMS point-in-time snapshot consistency > - > > Key: SENTRY-1463 > URL: https://issues.apache.org/jira/browse/SENTRY-1463 > Project: Sentry > Issue Type: Sub-task > Components: Sentry >Affects Versions: sentry-ha-redesign >Reporter: Hao Hao >Assignee: Hao Hao > Fix For: sentry-ha-redesign > > Attachments: SENTRY-1463.001-sentry-ha-redesign.patch, > SENTRY-1463.002-sentry-ha-redesign.patch, > SENTRY-1463.003-sentry-ha-redesign.patch, > SENTRY-1463.004-sentry-ha-redesign.patch, > SENTRY-1463.005-sentry-ha-redesign.patch, > SENTRY-1463.006-sentry-ha-redesign.patch, > SENTRY-1463.007-sentry-ha-redesign.patch > > > During HMSFollower start up, when retrieving HMS full snapshot, also need to > ensure point-in-time snapshot consistency. > It should be: > * Read current HMS notification ID_initial > * Read HMS metadata state > * Read current notification ID_new > * If ID_initial != ID_new then discard the current state and goto 1. > Use configurable property: sentry.hms.snapshot.retries.max.count for max > number of retry. > Ideally, during fetching full snapshot, there should be no write activity on > the HMS. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (SENTRY-1489) Categorize e2e tests into slow and regular tests, so that can adapt the timeout and etc.
[ https://issues.apache.org/jira/browse/SENTRY-1489?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Anne Yu updated SENTRY-1489: Attachment: SENTRY-1489.1.patch > Categorize e2e tests into slow and regular tests, so that can adapt the > timeout and etc. > > > Key: SENTRY-1489 > URL: https://issues.apache.org/jira/browse/SENTRY-1489 > Project: Sentry > Issue Type: Test > Components: Sentry >Affects Versions: 1.8.0, sentry-ha-redesign >Reporter: Anne Yu >Assignee: Anne Yu > Fix For: 1.8.0, sentry-ha-redesign > > Attachments: SENTRY-1489.0.patch, SENTRY-1489.1.patch > > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (SENTRY-1313) Database prefix is not honoured when executing grant statement
[ https://issues.apache.org/jira/browse/SENTRY-1313?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15603519#comment-15603519 ] Hao Hao commented on SENTRY-1313: - +1 if all tests pass. Thanks [~sravya]! > Database prefix is not honoured when executing grant statement > -- > > Key: SENTRY-1313 > URL: https://issues.apache.org/jira/browse/SENTRY-1313 > Project: Sentry > Issue Type: Bug >Reporter: Sravya Tirukkovalur >Assignee: Sravya Tirukkovalur >Priority: Critical > Attachments: SENTRY-1313.0.patch, SENTRY-1313.0.patch > > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (SENTRY-1509) Disable solr unit tests from e2e runs.are becoming flaky
[
https://issues.apache.org/jira/browse/SENTRY-1509?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15603520#comment-15603520
]
Anne Yu commented on SENTRY-1509:
-
More outputs:
{noformat}
Error Message
No context information for thread: Thread[id=1, name=main, state=RUNNABLE,
group=main]. Is this thread running under a class
com.carrotsearch.randomizedtesting.RandomizedRunner runner context? Add
@RunWith(class com.carrotsearch.randomizedtesting.RandomizedRunner.class) to
your test class. Make sure your code accesses random contexts within
@BeforeClass and @AfterClass boundary (for example, static test class
initializers are not permitted to access random contexts).
Stacktrace
java.lang.IllegalStateException: No context information for thread:
Thread[id=1, name=main, state=RUNNABLE, group=main]. Is this thread running
under a class com.carrotsearch.randomizedtesting.RandomizedRunner runner
context? Add @RunWith(class
com.carrotsearch.randomizedtesting.RandomizedRunner.class) to your test class.
Make sure your code accesses random contexts within @BeforeClass and
@AfterClass boundary (for example, static test class initializers are not
permitted to access random contexts).
at
com.carrotsearch.randomizedtesting.RandomizedContext.context(RandomizedContext.java:206)
at
com.carrotsearch.randomizedtesting.RandomizedContext.current(RandomizedContext.java:144)
at
org.apache.lucene.util.LuceneTestCase.getBaseTempDirForTestClass(LuceneTestCase.java:2525)
at
org.apache.lucene.util.LuceneTestCase.createTempDir(LuceneTestCase.java:2569)
at
org.apache.lucene.util.LuceneTestCase.createTempDir(LuceneTestCase.java:2557)
at
org.apache.solr.cloud.MiniSolrCloudCluster.(MiniSolrCloudCluster.java:104)
at
org.apache.sentry.tests.e2e.solr.AbstractSolrSentryTestBase.beforeTestSimpleSolrEndToEnd(AbstractSolrSentryTestBase.java:148)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)
at
org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:45)
at
org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:15)
at
org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:42)
at
org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:27)
at
org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:30)
at org.junit.runners.ParentRunner.run(ParentRunner.java:300)
at
org.apache.maven.surefire.junit4.JUnit4Provider.execute(JUnit4Provider.java:283)
at
org.apache.maven.surefire.junit4.JUnit4Provider.executeWithRerun(JUnit4Provider.java:173)
at
org.apache.maven.surefire.junit4.JUnit4Provider.executeTestSet(JUnit4Provider.java:153)
at
org.apache.maven.surefire.junit4.JUnit4Provider.invoke(JUnit4Provider.java:128)
at
org.apache.maven.surefire.booter.ForkedBooter.invokeProviderInSameClassLoader(ForkedBooter.java:203)
at
org.apache.maven.surefire.booter.ForkedBooter.runSuitesInProcess(ForkedBooter.java:155)
at
org.apache.maven.surefire.booter.ForkedBooter.main(ForkedBooter.java:103)
{noformat}
> Disable solr unit tests from e2e runs.are becoming flaky
>
>
> Key: SENTRY-1509
> URL: https://issues.apache.org/jira/browse/SENTRY-1509
> Project: Sentry
> Issue Type: Bug
> Components: Sentry, Solr Plugin
>Affects Versions: 1.8.0, sentry-ha-redesign
>Reporter: Anne Yu
>Assignee: Anne Yu
>
> Occasionally solr e2e tests fail without a known root cause. Disable them
> from mvn test for now. But once root cause is identified, should enable these
> tests.
> {noformat}
> No context information for thread: Thread[id=1, name=main, state=RUNNABLE,
> group=main]. Is this thread running under a class
> com.carrotsearch.randomizedtesting.RandomizedRunner runner context? Add
> @RunWith(class com.carrotsearch.randomizedtesting.RandomizedRunner.class) to
> your test class. Make sure your code accesses random contexts within
> @BeforeClass and @AfterClass boundary (for example, static test class
> initializers are not permitted to access random contexts).
> {noformat}
> >>> org.apache.sentry.tests.e2e.dbprovider.TestDbCrossOperations.testCrossDbTableOperations
> >>>3 min 0 sec 2
> >>> org.apache.sentry.tests.e2e.dbprovider.TestDbCrossOperations.org.apache.sentry.tests.e2e.dbprovider.TestDbCrossOperations
> >>> 3 min 14 sec2
> >>> org.apache.sentry.tests.e2e.solr.TestCollAdm
[jira] [Created] (SENTRY-1509) Disable solr unit tests from e2e runs.are becoming flaky
Anne Yu created SENTRY-1509:
---
Summary: Disable solr unit tests from e2e runs.are becoming flaky
Key: SENTRY-1509
URL: https://issues.apache.org/jira/browse/SENTRY-1509
Project: Sentry
Issue Type: Bug
Components: Sentry, Solr Plugin
Affects Versions: 1.8.0, sentry-ha-redesign
Reporter: Anne Yu
Assignee: Anne Yu
Occasionally solr e2e tests fail without a known root cause. Disable them from
mvn test for now. But once root cause is identified, should enable these tests.
{noformat}
No context information for thread: Thread[id=1, name=main, state=RUNNABLE,
group=main]. Is this thread running under a class
com.carrotsearch.randomizedtesting.RandomizedRunner runner context? Add
@RunWith(class com.carrotsearch.randomizedtesting.RandomizedRunner.class) to
your test class. Make sure your code accesses random contexts within
@BeforeClass and @AfterClass boundary (for example, static test class
initializers are not permitted to access random contexts).
{noformat}
>>> org.apache.sentry.tests.e2e.dbprovider.TestDbCrossOperations.testCrossDbTableOperations
>>> 3 min 0 sec 2
>>> org.apache.sentry.tests.e2e.dbprovider.TestDbCrossOperations.org.apache.sentry.tests.e2e.dbprovider.TestDbCrossOperations
>>>3 min 14 sec2
>>> org.apache.sentry.tests.e2e.solr.TestCollAdminCoreOperations.org.apache.sentry.tests.e2e.solr.TestCollAdminCoreOperations
>>>7.1 sec 3
>>> org.apache.sentry.tests.e2e.solr.TestDocLevelOperations.org.apache.sentry.tests.e2e.solr.TestDocLevelOperations
>>> 7.7 sec 3
>>> org.apache.sentry.tests.e2e.solr.TestQueryOperations.org.apache.sentry.tests.e2e.solr.TestQueryOperations
>>>7.1 sec 3
>>> org.apache.sentry.tests.e2e.solr.TestRealTimeGet.org.apache.sentry.tests.e2e.solr.TestRealTimeGet
>>>7.2 sec 3
>>> org.apache.sentry.tests.e2e.solr.TestUpdateOperations.org.apache.sentry.tests.e2e.solr.TestUpdateOperations
>>> 7.2 sec 3
>>> org.apache.sentry.tests.e2e.solr.db.integration.TestSolrAdminOperations.org.apache.sentry.tests.e2e.solr.db.integration.TestSolrAdminOperations
>>> 6.6 sec 3
>>> org.apache.sentry.tests.e2e.solr.db.integration.TestSolrDocLevelOperations.org.apache.sentry.tests.e2e.solr.db.integration.TestSolrDocLevelOperations
>>>6.6 sec 3
>>> org.apache.sentry.tests.e2e.solr.db.integration.TestSolrQueryOperations.org.apache.sentry.tests.e2e.solr.db.integration.TestSolrQueryOperations
>>> 6.5 sec 3
>>> org.apache.sentry.tests.e2e.solr.db.integration.TestSolrUpdateOperations.org.apache.sentry.tests.e2e.solr.db.integration.TestSolrUpdateOperations
>>>6.5 sec 3
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (SENTRY-1313) Database prefix is not honoured when executing grant statement
[ https://issues.apache.org/jira/browse/SENTRY-1313?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15603486#comment-15603486 ] Alexander Kolbasov commented on SENTRY-1313: [~sravya] Thank you for the fix, looks good to me. > Database prefix is not honoured when executing grant statement > -- > > Key: SENTRY-1313 > URL: https://issues.apache.org/jira/browse/SENTRY-1313 > Project: Sentry > Issue Type: Bug >Reporter: Sravya Tirukkovalur >Assignee: Sravya Tirukkovalur >Priority: Critical > Attachments: SENTRY-1313.0.patch, SENTRY-1313.0.patch > > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (SENTRY-1313) Database prefix is not honoured when executing grant statement
[
https://issues.apache.org/jira/browse/SENTRY-1313?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15603342#comment-15603342
]
Hadoop QA commented on SENTRY-1313:
---
Here are the results of testing the latest attachment
https://issues.apache.org/jira/secure/attachment/12835005/SENTRY-1313.0.patch
against master.
{color:red}Overall:{color} -1 due to 2 errors
{color:red}ERROR:{color} mvn test exited 1
{color:red}ERROR:{color} Failed:
org.apache.sentry.tests.e2e.dbprovider.TestDatabaseProvider
Console output:
https://builds.apache.org/job/PreCommit-SENTRY-Build/2085/console
This message is automatically generated.
> Database prefix is not honoured when executing grant statement
> --
>
> Key: SENTRY-1313
> URL: https://issues.apache.org/jira/browse/SENTRY-1313
> Project: Sentry
> Issue Type: Bug
>Reporter: Sravya Tirukkovalur
>Assignee: Sravya Tirukkovalur
>Priority: Critical
> Attachments: SENTRY-1313.0.patch, SENTRY-1313.0.patch
>
>
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (SENTRY-1508) MetastorePlugin.java does not handle properly initialization failure
[
https://issues.apache.org/jira/browse/SENTRY-1508?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Vadim Spector updated SENTRY-1508:
--
Description:
MetastorePlugin.java has several implementation issues that need to be fixed by
this JIRA:
a) The state of MetastorePlugin is not communicated cleanly. The constructor
initializes successfully regardless of whether HMS cache initialization was
successful or not. The initThread.run() can easily fail when, say,
cacheInitializer.createInitialUpdate() fails due to misconfigured HMS, so it is
quite possible. The state is communicated through 3 variables: boolean
initComplete, Throwable initError, and UpdateableAuthzPaths authzPaths.
initComplete is always set to true from finally {} block, so it really
indicates the end of an _attempt_ to initialize the cache. It, thus, should
only be used to detect whether the cache initialization is still in-progress,
not whether initialization has been successful. To determine whether cache
initialization was successful, initComplete should be used together with
initError, which is set only when initialization fails. This is not how the
code implements it in more than one place, which should be clear from analyzing
a patch.
b) There are several synchronization issues that may lead to the failure of
synchronizing with the Sentry. They usually have to do with inconsistent use of
monitor objects to guard access to thread-unsafe objects. E.g.
authzPaths.updatePartial() uses a lock created just for the scope of a single
call, which makes it useless for synchronization. A single read-write lock
should be used instead, as there is one read and one write operation performed
on authzPaths within MetastorePlugin.
c) Failure to create authzPaths is not communicated clearly to a caller. When
it is dereferenced from the public callback APIs, it results in
NullPointerException. The suggested fix of this issue in SENTRY-1270 avoids
NullPointerException by detecting the null authzPaths, printing error message,
and returning. This effectively leads to consistent failure to update local
cache without notifying the caller. The suggested solution would be not only to
throw IllegalArgumentException to the caller instead, but to also specify why
exactly authzPaths == null - due to initialization still being in progress or
due to its failure.
d) The housekeeping thread running SyncTask to synchronize with Sentry state
fails when authzPaths == null. However, it keeps printing misleading "
Metastore Plugin cache has not finished initialization." message even in the
case of a permanent cache initialization failure. It needs to print the real
cause of this condition, by analyzing authzPaths together with initComplete and
initError values.
e) getClient() may return null, in which case dereferencing it causes not so
helpful NullPointerException. Instead, while getClient() may still print error
message, it should also re-throw an original exception, which would then be
much easier to debug.
f) Each code fork deserves log message: e.g. addPath() retrurns immediately if
PathsUpdate.parsePath() returns null - w/o printing any log.
g) in SyncTask.run() - if notificationLock.tryLock() succeeds, yet the next
line's expression "MetastorePlugin.this.authzPaths == null" is evaluated to
false, run() exits w/o a chance to call notificationLock.unlock(). All the code
following lock.tryLock() should be inside try-catch-finally, with finally's
first line being lock.unlock(), as a general safe pattern.
h) some additional misc synchronization issues may also be addressed by the
patch.
What's not in the scope:
a) the initial patch honors the original design supporting asynchronous
synchronization. It means that MetastorePlugin is always constructed, even if
it's in a broken state. It is up to the public callback APIs to properly inform
the caller of the broken state.
b) however, if the reviewers decide that support of asynchronous initialization
is not necessary, it may be preferable to drop such support in the same JIRA.
In this case, second patch can be generated to only support synchronous
initialization. This would lead to a much simpler and robust implementation.
was:
MetastorePlugin.java has several implementation issues that need to be fixed by
this JIRA:
a) The state of MetastorePlugin is not communicated cleanly. The constructor
initializes successfully regardless of whether HMS cache initialization was
successful or not. The initThread.run() can easily fail when, say,
cacheInitializer.createInitialUpdate() fails due to misconfigured HMS, so it is
quite possible. The state is communicated through 3 variables: boolean
initComplete, Throwable initError, and UpdateableAuthzPaths authzPaths.
initComplete is always set to true from finally {} block, so it really
indicates the end of an _attempt_ to initialize the cache. It, thus,
[jira] [Commented] (SENTRY-1508) MetastorePlugin.java does not handle properly initialization failure
[
https://issues.apache.org/jira/browse/SENTRY-1508?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15603332#comment-15603332
]
Vadim Spector commented on SENTRY-1508:
---
I edited the description to make it more clear what this JIRA is/ is not going
to address.
To re-iterate: this JIRA honors the original design, which is supporting
asynchronous initialization of MetastorePlugin; it just tries to fix the
implementation bugs, and those bugs should all be covered in the Description.
However, should we decide to drop support of asynchronous initialization
altogether (perhaps, as the result of code review and thinking of all the
implementation complexities?), I'd be strongly in favor of it. In this case,
I'd rather submit another patch within the same JIRA, so no, I do not propose
another JIRA.
> MetastorePlugin.java does not handle properly initialization failure
>
>
> Key: SENTRY-1508
> URL: https://issues.apache.org/jira/browse/SENTRY-1508
> Project: Sentry
> Issue Type: Bug
>Reporter: Vadim Spector
>Assignee: Vadim Spector
>Priority: Critical
> Attachments: SENTRY-1508.001.patch
>
>
> MetastorePlugin.java has several implementation issues that need to be fixed
> by this JIRA:
> a) The state of MetastorePlugin is not communicated cleanly. The constructor
> initializes successfully regardless of whether HMS cache initialization was
> successful or not. The initThread.run() can easily fail when, say,
> cacheInitializer.createInitialUpdate() fails due to misconfigured HMS, so it
> is quite possible. The state is communicated through 3 variables: boolean
> initComplete, Throwable initError, and UpdateableAuthzPaths authzPaths.
> initComplete is always set to true from finally {} block, so it really
> indicates the end of an _attempt_ to initialize the cache. It, thus, should
> only be used to detect whether the cache initialization is still in-progress,
> not whether initialization has been successful. To determine whether cache
> initialization was successful, initComplete should be used together with
> initError, which is set only when initialization fails. This is not how the
> code implements it in more than one place, which should be clear from
> analyzing a patch.
> b) There are several synchronization issues that may lead to the failure of
> synchronizing with the Sentry. They usually have to do with inconsistent use
> of monitor objects to guard access to thread-unsafe objects. E.g.
> authzPaths.updatePartial() uses a lock created just for the scope of a single
> call, which makes it useless for synchronization. A single read-write lock
> should be used instead, as there is one read and one write operation
> performed on authzPaths within MetastorePlugin.
> c) Failure to create authzPaths is not communicated clearly to a caller. When
> it is dereferenced from the public callback APIs, it results in
> NullPointerException. The suggested fix of this issue in SENTRY-1270 avoids
> NullPointerException by detecting the null authzPaths, printing error
> message, and returning. This effectively leads to consistent failure to
> update local cache without notifying the caller. The suggested solution would
> be not only to throw IllegalArgumentException to the caller instead, but to
> also specify why exactly authzPaths == null - due to initialization still
> being in progress or due to its failure.
> d) The housekeeping thread running SyncTask to synchronize with Sentry state
> fails when authzPaths == null. However, it keeps printing misleading "
> Metastore Plugin cache has not finished initialization." message even in the
> case of a permanent cache initialization failure. It needs to print the real
> cause of this condition, by analyzing authzPaths together with initComplete
> and initError values.
> e) getClient() may return null, in which case dereferencing it causes not so
> helpful NullPointerException. Instead, while getClient() may still print
> error message, it should also re-throw an original exception, which would
> then be much easier to debug.
> f) Each code fork deserves log message: e.g. addPath() retrurns immediately
> if PathsUpdate.parsePath() returns null - w/o printing any log.
> g) in SyncTask.run() - if notificationLock.tryLock() succeeds, yet the next
> line's expression "MetastorePlugin.this.authzPaths == null" is evaluated to
> false, run() exits w/o a chance to call notificationLock.unlock(). All the
> code following lock.tryLock() should be inside try-catch-finally, with
> finally's first line being lock.unlock(), as a general safe pattern.
> h) some additional misc synchronization issues may also be addressed by the
> patch.
> What's not in the scope:
> a) the initial patch hono
[jira] [Updated] (SENTRY-1508) MetastorePlugin.java does not handle properly initialization failure
[
https://issues.apache.org/jira/browse/SENTRY-1508?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Vadim Spector updated SENTRY-1508:
--
Description:
MetastorePlugin.java has several implementation issues that need to be fixed by
this JIRA:
a) The state of MetastorePlugin is not communicated cleanly. The constructor
initializes successfully regardless of whether HMS cache initialization was
successful or not. The initThread.run() can easily fail when, say,
cacheInitializer.createInitialUpdate() fails due to misconfigured HMS, so it is
quite possible. The state is communicated through 3 variables: boolean
initComplete, Throwable initError, and UpdateableAuthzPaths authzPaths.
initComplete is always set to true from finally {} block, so it really
indicates the end of an _attempt_ to initialize the cache. It, thus, should
only be used to detect whether the cache initialization is still in-progress,
not whether initialization has been successful. To determine whether cache
initialization was successful, initComplete should be used together with
initError, which is set only when initialization fails. This is not how the
code implements it in more than one place, which should be clear from analyzing
a patch.
b) There are several synchronization issues that may lead to the failure of
synchronizing with the Sentry. They usually have to do with inconsistent use of
monitor objects to guard access to thread-unsafe objects. E.g.
authzPaths.updatePartial() uses a lock created just for the scope of a single
call, which makes it useless for synchronization. A single read-write lock
should be used instead, as there is one read and one write operation performed
on authzPaths within MetastorePlugin.
c) Failure to create authzPaths is not communicated clearly to a caller. When
it is dereferenced from the public callback APIs, it results in
NullPointerException. The suggested fix of this issue in SENTRY-1270 avoids
NullPointerException by detecting the null authzPaths, printing error message,
and returning. This effectively leads to consistent failure to update local
cache without notifying the caller. The suggested solution would be not only to
throw IllegalArgumentException to the caller instead, but to also specify why
exactly authzPaths == null - due to initialization still being in progress or
due to its failure.
d) The housekeeping thread running SyncTask to synchronize with Sentry state
fails when authzPaths == null. However, it keeps printing misleading "
Metastore Plugin cache has not finished initialization." message even in the
case of a permanent cache initialization failure. It needs to print the real
cause of this condition, by analyzing authzPaths together with initComplete and
initError values.
e) getClient() may return null, in which case dereferencing it causes not so
helpful NullPointerException. Instead, while getClient() may still print error
message, it should also re-throw an original exception, which would then be
much easier to debug.
f) Each code fork deserves log message: e.g. addPath() retrurns immediately if
PathsUpdate.parsePath() returns null - w/o printing any log.
g) in SyncTask.run() - if notificationLock.tryLock() succeeds, yet the next
line's expression "MetastorePlugin.this.authzPaths == null" is evaluated to
false, run() exits w/o a chance to call notificationLock.unlock(). All the code
following lock.tryLock() should be inside try-catch-finally, with finally's
first line being lock.unlock(), as a general safe pattern.
h) some additional misc synchronization issues may also be addressed by the
patch.
What's not in the scope:
a) the initial patch honors the original design supporting asynchronous
synchronization. It means that MetastorePlugin is always constructed, even if
it's in a broken state. It is up to the public callback APIs to properly inform
the caller of the broken state.
b) however, if the reviewers decide that support of asynchronous initialization
is not necessary, it may be preferable to drop such support in the same JIRA.
In this case, second patch can be generated to only support synchronous
communication. This would lead to a much simpler and robust implementation.
was:
MetastorePlugin.java has several implementation issues that need to be fixed by
this JIRA:
a) The state of MetastorePlugin is not communicated cleanly. The constructor
initializes successfully regardless of whether HMS cache initialization was
successful or not. The initThread.run() can easily fail when, say,
cacheInitializer.createInitialUpdate() fails due to misconfigured HMS, so it is
quite possible. The state is communicated through 3 variables: boolean
initComplete, Throwable initError, and UpdateableAuthzPaths authzPaths.
initComplete is always set to true from finally {} block, so it really
indicates the end of an _attempt_ to initialize the cache. It, thus,
[jira] [Updated] (SENTRY-1508) MetastorePlugin.java does not handle properly initialization failure
[
https://issues.apache.org/jira/browse/SENTRY-1508?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Vadim Spector updated SENTRY-1508:
--
Description:
MetastorePlugin.java has several implementation issues that need to be fixed by
this JIRA:
a) The state of MetastorePlugin is not communicated cleanly. The constructor
initializes successfully regardless of whether HMS cache initialization was
successful or not. The initThread.run() can easily fail when, say,
cacheInitializer.createInitialUpdate() fails due to misconfigured HMS, so it is
quite possible. The state is communicated through 3 variables: boolean
initComplete, Throwable initError, and UpdateableAuthzPaths authzPaths.
initComplete is always set to true from finally {} block, so it really
indicates the end of an _attempt_ to initialize the cache. It, thus, should
only be used to detect whether the cache initialization is still in-progress,
not whether initialization has been successful. To determine whether cache
initialization was successful, initComplete should be used together with
initError, which is set only when initialization fails. This is not how the
code implements it in more than one place, which should be clear from analyzing
a patch.
b) There are several synchronization issues that may lead to the failure of
synchronizing with the Sentry. They usually have to do with inconsistent use of
monitor objects to guard access to thread-unsafe objects. E.g.
authzPaths.updatePartial() uses a lock created just for the scope of a single
call, which makes it useless for synchronization. A single read-write lock
should be used instead, as there is one read and one write operation performed
on authzPaths within MetastorePlugin.
c) Failure to create authzPaths is not communicated clearly to a caller. When
it is dereferenced from the public callback APIs, it results in
NullPointerException. The suggested fix of this issue in SENTRY-1270 avoids
NullPointerException by detecting the null authzPaths, printing error message,
and returning. This effectively leads to consistent failure to update local
cache without notifying the caller. The suggested solution would be not only to
throw IllegalArgumentException to the caller instead, but to also specify why
exactly authzPaths == null - due to initialization still being in progress or
due to its failure.
d) The housekeeping thread running SyncTask to synchronize with Sentry state
fails when authzPaths == null. However, it keeps printing misleading "
Metastore Plugin cache has not finished initialization." message even in the
case of a permanent cache initialization failure. It needs to print the real
cause of this condition, by analyzing authzPaths together with initComplete and
initError values.
e) getClient() may return null, in which case dereferencing it causes not so
helpful NullPointerException. Instead, while getClient() may still print error
message, it should also re-throw an original exception, which would then be
much easier to debug.
f) Each code fork deserves log message: e.g. addPath() retrurns immediately if
PathsUpdate.parsePath() returns null - w/o printing any log.
g) in SyncTask.run() - if notificationLock.tryLock() succeeds, yet the next
line's expression "MetastorePlugin.this.authzPaths == null" is evaluated to
false, run() exits w/o a chance to call notificationLock.unlock(). All the code
following lock.tryLock() should be inside try-catch-finally, with finally's
first line being lock.unlock(), as a general safe pattern.
h) some additional misc synchronization issues may also be addressed by the
patch.
What's not in the scope: the initial patch honors the original design
supporting asynchronous synchronization. It means that MetastorePlugin is
always constructed, even if it's in a broken state. It is up to the public
callback APIs to properly inform the caller of the broken state.
was:
MetastorePlugin.java constructor initializes successfully regardless of whether
initThread.run() was successful. E.g. if cacheInitializer.createInitialUpdate()
cal inside the constructor fails (in one case in the field it was due to an
invalid table uri in HMS database, whose parsing produced
ArrayIndexOutOfBoundsException), initComplete is still assigned to true and
MetastorePlugin gets constructed. While initError is assigned to non-null value
in case of failure, it is effectively ignored as will be described later.
MetastorePlugin implements several callbacks to be called on the Hive side, to
update Sentry state. It is critical functionality for enforcing access control.
Its callbacks (addPath(), removeAllPaths(), etc) all end up calling
notifySentryAndApplyLocal() which starts with the following code:
if (initComplete) {
processUpdate()
}...
which means that processUpdate() is called even when MetastorePlugin
initialization has failed (i.e. initError != null). The
[jira] [Commented] (SENTRY-1508) MetastorePlugin.java does not handle properly initialization failure
[
https://issues.apache.org/jira/browse/SENTRY-1508?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15603240#comment-15603240
]
Hadoop QA commented on SENTRY-1508:
---
Here are the results of testing the latest attachment
https://issues.apache.org/jira/secure/attachment/12834768/SENTRY-1508.001.patch
against master.
{color:green}Overall:{color} +1 all checks pass
{color:green}SUCCESS:{color} all tests passed
Console output:
https://builds.apache.org/job/PreCommit-SENTRY-Build/2084/console
This message is automatically generated.
> MetastorePlugin.java does not handle properly initialization failure
>
>
> Key: SENTRY-1508
> URL: https://issues.apache.org/jira/browse/SENTRY-1508
> Project: Sentry
> Issue Type: Bug
>Reporter: Vadim Spector
>Assignee: Vadim Spector
>Priority: Critical
> Attachments: SENTRY-1508.001.patch
>
>
> MetastorePlugin.java constructor initializes successfully regardless of
> whether initThread.run() was successful. E.g. if
> cacheInitializer.createInitialUpdate() cal inside the constructor fails (in
> one case in the field it was due to an invalid table uri in HMS database,
> whose parsing produced ArrayIndexOutOfBoundsException), initComplete is still
> assigned to true and MetastorePlugin gets constructed. While initError is
> assigned to non-null value in case of failure, it is effectively ignored as
> will be described later.
> MetastorePlugin implements several callbacks to be called on the Hive side,
> to update Sentry state. It is critical functionality for enforcing access
> control. Its callbacks (addPath(), removeAllPaths(), etc) all end up calling
> notifySentryAndApplyLocal() which starts with the following code:
> if (initComplete) {
> processUpdate()
> }...
> which means that processUpdate() is called even when MetastorePlugin
> initialization has failed (i.e. initError != null). There is an option of
> asynchronous initialization, when MetastorePlugin is constructed and returned
> to the caller before initialization is complete, but the code makes no
> distinction between the two cases, as will be shown below.
> processUpdate() called regardless of whether MetastorePlugin a) has been
> initialized (sync mode, success), or b) still being initialized (async mode),
> or c) failed to initialized (either mode). It calls (unconditionally)
> applyLocal() and notifySentry(). aplyLocal() calls authzPaths.updatePartial()
> which immediately fails with NullPointerException if authzPaths == null which
> happens if initThread within the constructor fails.
> SENTRY-1270 avoids NullPointerException by execuitng this before
> dereferencing authzPaths inside applyLocal():
> if(authzPaths == null) {
> LOGGER.error(initializationFailureMsg);
> return;
> }
> However, this leads to hard-to-diagnose behavior, because a) local updates
> fail forever, and b) housekeeping thread running SyncTask to synchronize with
> Sentry-side state fails as well, printing endlessly misleading "
> Metastore Plugin cache has not finished initialization." message suggesting
> its temporary condition as opposed to a permanent failure. Failure to sync up
> with Sentry state can lead to very subtle, often intermittent, problems with
> acls. MetastorePlugin and its caller never get fully aware of the permanent
> nature of init failure.
> Suggestion:
> a) define 3 states in MetastorePlugin: initializing, initialized, init_failed.
> 2) communicate those states to the caller clearly, by _immediately_ throwing
> some kind of exception from public callback APIs in the cases preventing
> those callback from completing successfully:
> a) for state == initializing, throw some kind of
> InitializationInProgressException. This only applies for asynchronous
> initialization configuration. It is unknown whether it is actually deployed
> anywhere, but the code exists and there is no reason why not to support it.
> Async init makes sense as an optimization, as long as
> initialization-in-progress condition is clearly communicated.
>b) for state == init_failed throw InitializationFailedException of some
> sort. For usability, it should contain the cause - the original exception
> originating either from the constructor or from the initThread.run() started
> from the constructor (in sync or async modes alike).
> Although in the sync mode initialization failure could be detected easily by
> constructor failure, we may want to preserve the current design, in which
> constructor always succeeds, so the client code does not need to be aware of
> whether init mode is configured as sync vs async.
> Additional cleanup:
> 1. getClient() is always dereferenced, while it can actually return null
> (exception is swallowed and error is l
[jira] [Commented] (SENTRY-1508) MetastorePlugin.java does not handle properly initialization failure
[
https://issues.apache.org/jira/browse/SENTRY-1508?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15603218#comment-15603218
]
Alexander Kolbasov commented on SENTRY-1508:
Can you clarify what aspects are you fixing with the suggested patch and what
aspects you are leaving alone? Do you plan to open another JIRA for things that
are not addressed here?
> MetastorePlugin.java does not handle properly initialization failure
>
>
> Key: SENTRY-1508
> URL: https://issues.apache.org/jira/browse/SENTRY-1508
> Project: Sentry
> Issue Type: Bug
>Reporter: Vadim Spector
>Assignee: Vadim Spector
>Priority: Critical
> Attachments: SENTRY-1508.001.patch
>
>
> MetastorePlugin.java constructor initializes successfully regardless of
> whether initThread.run() was successful. E.g. if
> cacheInitializer.createInitialUpdate() cal inside the constructor fails (in
> one case in the field it was due to an invalid table uri in HMS database,
> whose parsing produced ArrayIndexOutOfBoundsException), initComplete is still
> assigned to true and MetastorePlugin gets constructed. While initError is
> assigned to non-null value in case of failure, it is effectively ignored as
> will be described later.
> MetastorePlugin implements several callbacks to be called on the Hive side,
> to update Sentry state. It is critical functionality for enforcing access
> control. Its callbacks (addPath(), removeAllPaths(), etc) all end up calling
> notifySentryAndApplyLocal() which starts with the following code:
> if (initComplete) {
> processUpdate()
> }...
> which means that processUpdate() is called even when MetastorePlugin
> initialization has failed (i.e. initError != null). There is an option of
> asynchronous initialization, when MetastorePlugin is constructed and returned
> to the caller before initialization is complete, but the code makes no
> distinction between the two cases, as will be shown below.
> processUpdate() called regardless of whether MetastorePlugin a) has been
> initialized (sync mode, success), or b) still being initialized (async mode),
> or c) failed to initialized (either mode). It calls (unconditionally)
> applyLocal() and notifySentry(). aplyLocal() calls authzPaths.updatePartial()
> which immediately fails with NullPointerException if authzPaths == null which
> happens if initThread within the constructor fails.
> SENTRY-1270 avoids NullPointerException by execuitng this before
> dereferencing authzPaths inside applyLocal():
> if(authzPaths == null) {
> LOGGER.error(initializationFailureMsg);
> return;
> }
> However, this leads to hard-to-diagnose behavior, because a) local updates
> fail forever, and b) housekeeping thread running SyncTask to synchronize with
> Sentry-side state fails as well, printing endlessly misleading "
> Metastore Plugin cache has not finished initialization." message suggesting
> its temporary condition as opposed to a permanent failure. Failure to sync up
> with Sentry state can lead to very subtle, often intermittent, problems with
> acls. MetastorePlugin and its caller never get fully aware of the permanent
> nature of init failure.
> Suggestion:
> a) define 3 states in MetastorePlugin: initializing, initialized, init_failed.
> 2) communicate those states to the caller clearly, by _immediately_ throwing
> some kind of exception from public callback APIs in the cases preventing
> those callback from completing successfully:
> a) for state == initializing, throw some kind of
> InitializationInProgressException. This only applies for asynchronous
> initialization configuration. It is unknown whether it is actually deployed
> anywhere, but the code exists and there is no reason why not to support it.
> Async init makes sense as an optimization, as long as
> initialization-in-progress condition is clearly communicated.
>b) for state == init_failed throw InitializationFailedException of some
> sort. For usability, it should contain the cause - the original exception
> originating either from the constructor or from the initThread.run() started
> from the constructor (in sync or async modes alike).
> Although in the sync mode initialization failure could be detected easily by
> constructor failure, we may want to preserve the current design, in which
> constructor always succeeds, so the client code does not need to be aware of
> whether init mode is configured as sync vs async.
> Additional cleanup:
> 1. getClient() is always dereferenced, while it can actually return null
> (exception is swallowed and error is logged). Suggested fix: getClient() may
> still print error message, but it should also re-throw an original exception
> - it is much easier to debug than dealing with in
[jira] [Commented] (SENTRY-1497) create a sentry scale test tool to add various objects and privileges into Sentry and HMS
[ https://issues.apache.org/jira/browse/SENTRY-1497?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15603233#comment-15603233 ] Anne Yu commented on SENTRY-1497: - wiki: https://cwiki.apache.org/confluence/display/SENTRY/Test+Tools. > create a sentry scale test tool to add various objects and privileges into > Sentry and HMS > - > > Key: SENTRY-1497 > URL: https://issues.apache.org/jira/browse/SENTRY-1497 > Project: Sentry > Issue Type: Test > Components: Sentry >Affects Versions: 1.8.0, sentry-ha-redesign >Reporter: Anne Yu >Assignee: Anne Yu > Fix For: 1.8.0, sentry-ha-redesign > > Attachments: SENTRY-1497.0.patch, SENTRY-1497.1.patch, > SENTRY-1497.2.patch, SENTRY-1497.3.patch, SENTRY-1497.4.patch > > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (SENTRY-1497) create a sentry scale test tool to add various objects and privileges into Sentry and HMS
[ https://issues.apache.org/jira/browse/SENTRY-1497?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Anne Yu updated SENTRY-1497: Resolution: Fixed Status: Resolved (was: Patch Available) Thanks [~haohao]. commit a43f7dff0ea999c57e8135f581ee0735bef7996f Author: Anne Yu Date: Sun Oct 9 23:41:44 2016 -0700 SENTRY-1497: create a sentry scale test tool to add various objects and privileges into Sentry and HMS. (Anne Yu, reviewed by Haohao, Lili and Sravya Tirukkovalur) > create a sentry scale test tool to add various objects and privileges into > Sentry and HMS > - > > Key: SENTRY-1497 > URL: https://issues.apache.org/jira/browse/SENTRY-1497 > Project: Sentry > Issue Type: Test > Components: Sentry >Affects Versions: 1.8.0, sentry-ha-redesign >Reporter: Anne Yu >Assignee: Anne Yu > Fix For: 1.8.0, sentry-ha-redesign > > Attachments: SENTRY-1497.0.patch, SENTRY-1497.1.patch, > SENTRY-1497.2.patch, SENTRY-1497.3.patch, SENTRY-1497.4.patch > > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (SENTRY-1313) Database prefix is not honoured when executing grant statement
[ https://issues.apache.org/jira/browse/SENTRY-1313?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sravya Tirukkovalur updated SENTRY-1313: Attachment: SENTRY-1313.0.patch Attaching the fix along with a unit test case. [~akolb] Will you be able to take a quick look? > Database prefix is not honoured when executing grant statement > -- > > Key: SENTRY-1313 > URL: https://issues.apache.org/jira/browse/SENTRY-1313 > Project: Sentry > Issue Type: Bug >Reporter: Sravya Tirukkovalur >Assignee: Sravya Tirukkovalur >Priority: Critical > Attachments: SENTRY-1313.0.patch, SENTRY-1313.0.patch > > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Comment Edited] (SENTRY-1508) MetastorePlugin.java does not handle properly initialization failure
[
https://issues.apache.org/jira/browse/SENTRY-1508?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15600543#comment-15600543
]
Vadim Spector edited comment on SENTRY-1508 at 10/24/16 7:52 PM:
-
Code review: https://reviews.apache.org/r/53125/
Just to re-iterate, I'd rather see the patch that eliminates asynchronous
initialization altogether, which would make the code much cleaner; but I think
the reviewers should weigh in on this. The initial patch tries to preserve the
original intent, only fixing some aspects of the implementation.
was (Author: vspec...@gmail.com):
Code review: https://reviews.apache.org/r/53125/
> MetastorePlugin.java does not handle properly initialization failure
>
>
> Key: SENTRY-1508
> URL: https://issues.apache.org/jira/browse/SENTRY-1508
> Project: Sentry
> Issue Type: Bug
>Reporter: Vadim Spector
>Assignee: Vadim Spector
>Priority: Critical
> Attachments: SENTRY-1508.001.patch
>
>
> MetastorePlugin.java constructor initializes successfully regardless of
> whether initThread.run() was successful. E.g. if
> cacheInitializer.createInitialUpdate() cal inside the constructor fails (in
> one case in the field it was due to an invalid table uri in HMS database,
> whose parsing produced ArrayIndexOutOfBoundsException), initComplete is still
> assigned to true and MetastorePlugin gets constructed. While initError is
> assigned to non-null value in case of failure, it is effectively ignored as
> will be described later.
> MetastorePlugin implements several callbacks to be called on the Hive side,
> to update Sentry state. It is critical functionality for enforcing access
> control. Its callbacks (addPath(), removeAllPaths(), etc) all end up calling
> notifySentryAndApplyLocal() which starts with the following code:
> if (initComplete) {
> processUpdate()
> }...
> which means that processUpdate() is called even when MetastorePlugin
> initialization has failed (i.e. initError != null). There is an option of
> asynchronous initialization, when MetastorePlugin is constructed and returned
> to the caller before initialization is complete, but the code makes no
> distinction between the two cases, as will be shown below.
> processUpdate() called regardless of whether MetastorePlugin a) has been
> initialized (sync mode, success), or b) still being initialized (async mode),
> or c) failed to initialized (either mode). It calls (unconditionally)
> applyLocal() and notifySentry(). aplyLocal() calls authzPaths.updatePartial()
> which immediately fails with NullPointerException if authzPaths == null which
> happens if initThread within the constructor fails.
> SENTRY-1270 avoids NullPointerException by execuitng this before
> dereferencing authzPaths inside applyLocal():
> if(authzPaths == null) {
> LOGGER.error(initializationFailureMsg);
> return;
> }
> However, this leads to hard-to-diagnose behavior, because a) local updates
> fail forever, and b) housekeeping thread running SyncTask to synchronize with
> Sentry-side state fails as well, printing endlessly misleading "
> Metastore Plugin cache has not finished initialization." message suggesting
> its temporary condition as opposed to a permanent failure. Failure to sync up
> with Sentry state can lead to very subtle, often intermittent, problems with
> acls. MetastorePlugin and its caller never get fully aware of the permanent
> nature of init failure.
> Suggestion:
> a) define 3 states in MetastorePlugin: initializing, initialized, init_failed.
> 2) communicate those states to the caller clearly, by _immediately_ throwing
> some kind of exception from public callback APIs in the cases preventing
> those callback from completing successfully:
> a) for state == initializing, throw some kind of
> InitializationInProgressException. This only applies for asynchronous
> initialization configuration. It is unknown whether it is actually deployed
> anywhere, but the code exists and there is no reason why not to support it.
> Async init makes sense as an optimization, as long as
> initialization-in-progress condition is clearly communicated.
>b) for state == init_failed throw InitializationFailedException of some
> sort. For usability, it should contain the cause - the original exception
> originating either from the constructor or from the initThread.run() started
> from the constructor (in sync or async modes alike).
> Although in the sync mode initialization failure could be detected easily by
> constructor failure, we may want to preserve the current design, in which
> constructor always succeeds, so the client code does not need to be aware of
> whether init mode is configured as sync vs async.
> Additional clea
[jira] [Updated] (SENTRY-1508) MetastorePlugin.java does not handle properly initialization failure
[
https://issues.apache.org/jira/browse/SENTRY-1508?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Vadim Spector updated SENTRY-1508:
--
Assignee: Vadim Spector
Status: Patch Available (was: Open)
> MetastorePlugin.java does not handle properly initialization failure
>
>
> Key: SENTRY-1508
> URL: https://issues.apache.org/jira/browse/SENTRY-1508
> Project: Sentry
> Issue Type: Bug
>Reporter: Vadim Spector
>Assignee: Vadim Spector
>Priority: Critical
> Attachments: SENTRY-1508.001.patch
>
>
> MetastorePlugin.java constructor initializes successfully regardless of
> whether initThread.run() was successful. E.g. if
> cacheInitializer.createInitialUpdate() cal inside the constructor fails (in
> one case in the field it was due to an invalid table uri in HMS database,
> whose parsing produced ArrayIndexOutOfBoundsException), initComplete is still
> assigned to true and MetastorePlugin gets constructed. While initError is
> assigned to non-null value in case of failure, it is effectively ignored as
> will be described later.
> MetastorePlugin implements several callbacks to be called on the Hive side,
> to update Sentry state. It is critical functionality for enforcing access
> control. Its callbacks (addPath(), removeAllPaths(), etc) all end up calling
> notifySentryAndApplyLocal() which starts with the following code:
> if (initComplete) {
> processUpdate()
> }...
> which means that processUpdate() is called even when MetastorePlugin
> initialization has failed (i.e. initError != null). There is an option of
> asynchronous initialization, when MetastorePlugin is constructed and returned
> to the caller before initialization is complete, but the code makes no
> distinction between the two cases, as will be shown below.
> processUpdate() called regardless of whether MetastorePlugin a) has been
> initialized (sync mode, success), or b) still being initialized (async mode),
> or c) failed to initialized (either mode). It calls (unconditionally)
> applyLocal() and notifySentry(). aplyLocal() calls authzPaths.updatePartial()
> which immediately fails with NullPointerException if authzPaths == null which
> happens if initThread within the constructor fails.
> SENTRY-1270 avoids NullPointerException by execuitng this before
> dereferencing authzPaths inside applyLocal():
> if(authzPaths == null) {
> LOGGER.error(initializationFailureMsg);
> return;
> }
> However, this leads to hard-to-diagnose behavior, because a) local updates
> fail forever, and b) housekeeping thread running SyncTask to synchronize with
> Sentry-side state fails as well, printing endlessly misleading "
> Metastore Plugin cache has not finished initialization." message suggesting
> its temporary condition as opposed to a permanent failure. Failure to sync up
> with Sentry state can lead to very subtle, often intermittent, problems with
> acls. MetastorePlugin and its caller never get fully aware of the permanent
> nature of init failure.
> Suggestion:
> a) define 3 states in MetastorePlugin: initializing, initialized, init_failed.
> 2) communicate those states to the caller clearly, by _immediately_ throwing
> some kind of exception from public callback APIs in the cases preventing
> those callback from completing successfully:
> a) for state == initializing, throw some kind of
> InitializationInProgressException. This only applies for asynchronous
> initialization configuration. It is unknown whether it is actually deployed
> anywhere, but the code exists and there is no reason why not to support it.
> Async init makes sense as an optimization, as long as
> initialization-in-progress condition is clearly communicated.
>b) for state == init_failed throw InitializationFailedException of some
> sort. For usability, it should contain the cause - the original exception
> originating either from the constructor or from the initThread.run() started
> from the constructor (in sync or async modes alike).
> Although in the sync mode initialization failure could be detected easily by
> constructor failure, we may want to preserve the current design, in which
> constructor always succeeds, so the client code does not need to be aware of
> whether init mode is configured as sync vs async.
> Additional cleanup:
> 1. getClient() is always dereferenced, while it can actually return null
> (exception is swallowed and error is logged). Suggested fix: getClient() may
> still print error message, but it should also re-throw an original exception
> - it is much easier to debug than dealing with inevitable and uninformative
> NullPointerException up the caller stack. General rule - error messages are
> to help with diagnostic, not to replace throwing exception, which is the
[jira] [Commented] (SENTRY-1422) JDO deadlocks while processing grant while a background thread processes Notificationlogs
[
https://issues.apache.org/jira/browse/SENTRY-1422?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15601609#comment-15601609
]
Hadoop QA commented on SENTRY-1422:
---
Here are the results of testing the latest attachment
https://issues.apache.org/jira/secure/attachment/12834896/SENTRY-1422.017-sentry-ha-redesign.patch
against sentry-ha-redesign.
{color:red}Overall:{color} -1 due to 3 errors
{color:red}ERROR:{color} mvn test exited 1
{color:red}ERROR:{color} Failed:
org.apache.sentry.tests.e2e.dbprovider.TestDatabaseProvider
{color:red}ERROR:{color} Failed:
org.apache.sentry.tests.e2e.dbprovider.TestDbPrivilegesAtFunctionScope
Console output:
https://builds.apache.org/job/PreCommit-SENTRY-Build/2083/console
This message is automatically generated.
> JDO deadlocks while processing grant while a background thread processes
> Notificationlogs
> -
>
> Key: SENTRY-1422
> URL: https://issues.apache.org/jira/browse/SENTRY-1422
> Project: Sentry
> Issue Type: Sub-task
> Components: Hdfs Plugin
>Reporter: Sravya Tirukkovalur
>Assignee: Colin Ma
> Fix For: sentry-ha-redesign
>
> Attachments: SENTRY-1422.008-sentry-ha-redesign.patch,
> SENTRY-1422.009-sentry-ha-redesign.patch,
> SENTRY-1422.010-sentry-ha-redesign.patch,
> SENTRY-1422.011-sentry-ha-redesign.patch,
> SENTRY-1422.012-sentry-ha-redesign.patch,
> SENTRY-1422.013-sentry-ha-redesign.patch,
> SENTRY-1422.014-sentry-ha-redesign.patch,
> SENTRY-1422.015-sentry-ha-redesign.patch,
> SENTRY-1422.016-sentry-ha-redesign.patch,
> SENTRY-1422.017-sentry-ha-redesign.patch, sentry_deadlock.png,
> testDeadLock.patch
>
>
> As I was working on Sentry-1321. I see that test case
> TestDbPrivilegeCleanupOnDrop#testDropObjects fails while "Granting select on
> table" with following deadlock.
> {code}
> java.sql.SQLException: Error while processing statement: FAILED: Execution
> Error, return code 1 from
> org.apache.hadoop.hive.ql.exec.SentryGrantRevokeTask. Unknown error for
> request: TAlterSentryRoleGrantPrivilegeRequest(protocol_version:2,
> requestorUserName:admin1, roleName:select_tbl1,
> privileges:[TSentryPrivilege(privilegeScope:TABLE, serverName:server1,
> dbName:db_2, tableName:tb_1, URI:, action:select, createTime:1469562185573,
> grantOption:FALSE, columnName:)]), message: Iteration request failed : SELECT
> 'org.apache.sentry.provider.db.service.model.MSentryPrivilege' AS
> NUCLEUS_TYPE,A1.URI,A1."ACTION",A1."COLUMN_NAME",A1.CREATE_TIME,A1.DB_NAME,A1.WITH_GRANT_OPTION,A1.PRIVILEGE_SCOPE,A1."SERVER_NAME",A1."TABLE_NAME",A1.DB_PRIVILEGE_ID
> FROM SENTRY_ROLE_DB_PRIVILEGE_MAP A0 INNER JOIN SENTRY_DB_PRIVILEGE A1 ON
> A0.DB_PRIVILEGE_ID = A1.DB_PRIVILEGE_ID WHERE A0.ROLE_ID = ?. Server
> Stacktrace: javax.jdo.JDODataStoreException: Iteration request failed :
> SELECT 'org.apache.sentry.provider.db.service.model.MSentryPrivilege' AS
> NUCLEUS_TYPE,A1.URI,A1."ACTION",A1."COLUMN_NAME",A1.CREATE_TIME,A1.DB_NAME,A1.WITH_GRANT_OPTION,A1.PRIVILEGE_SCOPE,A1."SERVER_NAME",A1."TABLE_NAME",A1.DB_PRIVILEGE_ID
> FROM SENTRY_ROLE_DB_PRIVILEGE_MAP A0 INNER JOIN SENTRY_DB_PRIVILEGE A1 ON
> A0.DB_PRIVILEGE_ID = A1.DB_PRIVILEGE_ID WHERE A0.ROLE_ID = ?
> at
> org.datanucleus.api.jdo.NucleusJDOHelper.getJDOExceptionForNucleusException(NucleusJDOHelper.java:451)
> at
> org.datanucleus.api.jdo.JDOTransaction.commit(JDOTransaction.java:165)
> at
> org.apache.sentry.provider.db.service.persistent.SentryStore.commitTransaction(SentryStore.java:279)
> at
> org.apache.sentry.provider.db.service.persistent.SentryStore.commitUpdateTransaction(SentryStore.java:261)
> at
> org.apache.sentry.provider.db.service.persistent.SentryStore.alterSentryRoleGrantPrivileges(SentryStore.java:458)
> at
> org.apache.sentry.provider.db.service.thrift.SentryPolicyStoreProcessor.alter_sentry_role_grant_privilege(SentryPolicyStoreProcessor.java:261)
> at
> org.apache.sentry.provider.db.service.thrift.SentryPolicyService$Processor$alter_sentry_role_grant_privilege.getResult(SentryPolicyService.java:1237)
> at
> org.apache.sentry.provider.db.service.thrift.SentryPolicyService$Processor$alter_sentry_role_grant_privilege.getResult(SentryPolicyService.java:1222)
> at org.apache.thrift.ProcessFunction.process(ProcessFunction.java:39)
> at org.apache.thrift.TBaseProcessor.process(TBaseProcessor.java:39)
> at
> org.apache.sentry.provider.db.service.thrift.SentryProcessorWrapper.process(SentryProcessorWrapper.java:35)
> at
> org.apache.thrift.TMultiplexedProcessor.process(TMultiplexedProcessor.java:123)
> at
> org.apache.thrift.server.TThreadPoolServer$WorkerProcess.run(TThreadPoolServer.java:286)
> at
> java.util.concurr
[jira] [Commented] (SENTRY-1422) JDO deadlocks while processing grant while a background thread processes Notificationlogs
[
https://issues.apache.org/jira/browse/SENTRY-1422?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15601320#comment-15601320
]
Colin Ma commented on SENTRY-1422:
--
[~akolb], thanks for your review, the patch is updated according to your
comments.
> JDO deadlocks while processing grant while a background thread processes
> Notificationlogs
> -
>
> Key: SENTRY-1422
> URL: https://issues.apache.org/jira/browse/SENTRY-1422
> Project: Sentry
> Issue Type: Sub-task
> Components: Hdfs Plugin
>Reporter: Sravya Tirukkovalur
>Assignee: Colin Ma
> Fix For: sentry-ha-redesign
>
> Attachments: SENTRY-1422.008-sentry-ha-redesign.patch,
> SENTRY-1422.009-sentry-ha-redesign.patch,
> SENTRY-1422.010-sentry-ha-redesign.patch,
> SENTRY-1422.011-sentry-ha-redesign.patch,
> SENTRY-1422.012-sentry-ha-redesign.patch,
> SENTRY-1422.013-sentry-ha-redesign.patch,
> SENTRY-1422.014-sentry-ha-redesign.patch,
> SENTRY-1422.015-sentry-ha-redesign.patch,
> SENTRY-1422.016-sentry-ha-redesign.patch,
> SENTRY-1422.017-sentry-ha-redesign.patch, sentry_deadlock.png,
> testDeadLock.patch
>
>
> As I was working on Sentry-1321. I see that test case
> TestDbPrivilegeCleanupOnDrop#testDropObjects fails while "Granting select on
> table" with following deadlock.
> {code}
> java.sql.SQLException: Error while processing statement: FAILED: Execution
> Error, return code 1 from
> org.apache.hadoop.hive.ql.exec.SentryGrantRevokeTask. Unknown error for
> request: TAlterSentryRoleGrantPrivilegeRequest(protocol_version:2,
> requestorUserName:admin1, roleName:select_tbl1,
> privileges:[TSentryPrivilege(privilegeScope:TABLE, serverName:server1,
> dbName:db_2, tableName:tb_1, URI:, action:select, createTime:1469562185573,
> grantOption:FALSE, columnName:)]), message: Iteration request failed : SELECT
> 'org.apache.sentry.provider.db.service.model.MSentryPrivilege' AS
> NUCLEUS_TYPE,A1.URI,A1."ACTION",A1."COLUMN_NAME",A1.CREATE_TIME,A1.DB_NAME,A1.WITH_GRANT_OPTION,A1.PRIVILEGE_SCOPE,A1."SERVER_NAME",A1."TABLE_NAME",A1.DB_PRIVILEGE_ID
> FROM SENTRY_ROLE_DB_PRIVILEGE_MAP A0 INNER JOIN SENTRY_DB_PRIVILEGE A1 ON
> A0.DB_PRIVILEGE_ID = A1.DB_PRIVILEGE_ID WHERE A0.ROLE_ID = ?. Server
> Stacktrace: javax.jdo.JDODataStoreException: Iteration request failed :
> SELECT 'org.apache.sentry.provider.db.service.model.MSentryPrivilege' AS
> NUCLEUS_TYPE,A1.URI,A1."ACTION",A1."COLUMN_NAME",A1.CREATE_TIME,A1.DB_NAME,A1.WITH_GRANT_OPTION,A1.PRIVILEGE_SCOPE,A1."SERVER_NAME",A1."TABLE_NAME",A1.DB_PRIVILEGE_ID
> FROM SENTRY_ROLE_DB_PRIVILEGE_MAP A0 INNER JOIN SENTRY_DB_PRIVILEGE A1 ON
> A0.DB_PRIVILEGE_ID = A1.DB_PRIVILEGE_ID WHERE A0.ROLE_ID = ?
> at
> org.datanucleus.api.jdo.NucleusJDOHelper.getJDOExceptionForNucleusException(NucleusJDOHelper.java:451)
> at
> org.datanucleus.api.jdo.JDOTransaction.commit(JDOTransaction.java:165)
> at
> org.apache.sentry.provider.db.service.persistent.SentryStore.commitTransaction(SentryStore.java:279)
> at
> org.apache.sentry.provider.db.service.persistent.SentryStore.commitUpdateTransaction(SentryStore.java:261)
> at
> org.apache.sentry.provider.db.service.persistent.SentryStore.alterSentryRoleGrantPrivileges(SentryStore.java:458)
> at
> org.apache.sentry.provider.db.service.thrift.SentryPolicyStoreProcessor.alter_sentry_role_grant_privilege(SentryPolicyStoreProcessor.java:261)
> at
> org.apache.sentry.provider.db.service.thrift.SentryPolicyService$Processor$alter_sentry_role_grant_privilege.getResult(SentryPolicyService.java:1237)
> at
> org.apache.sentry.provider.db.service.thrift.SentryPolicyService$Processor$alter_sentry_role_grant_privilege.getResult(SentryPolicyService.java:1222)
> at org.apache.thrift.ProcessFunction.process(ProcessFunction.java:39)
> at org.apache.thrift.TBaseProcessor.process(TBaseProcessor.java:39)
> at
> org.apache.sentry.provider.db.service.thrift.SentryProcessorWrapper.process(SentryProcessorWrapper.java:35)
> at
> org.apache.thrift.TMultiplexedProcessor.process(TMultiplexedProcessor.java:123)
> at
> org.apache.thrift.server.TThreadPoolServer$WorkerProcess.run(TThreadPoolServer.java:286)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
> at java.lang.Thread.run(Thread.java:745)
> NestedThrowablesStackTrace:
> java.sql.SQLTransactionRollbackException: A lock could not be obtained due to
> a deadlock, cycle of locks and waiters is:
> Lock : ROW, SENTRY_ROLE_DB_PRIVILEGE_MAP, (1,10)
> Waiting XID : {712, S} , SENTRY, SELECT
> 'org.apache.sentry.provider.db.service.model.MSentryPriv
[jira] [Updated] (SENTRY-1422) JDO deadlocks while processing grant while a background thread processes Notificationlogs
[
https://issues.apache.org/jira/browse/SENTRY-1422?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Colin Ma updated SENTRY-1422:
-
Attachment: SENTRY-1422.017-sentry-ha-redesign.patch
> JDO deadlocks while processing grant while a background thread processes
> Notificationlogs
> -
>
> Key: SENTRY-1422
> URL: https://issues.apache.org/jira/browse/SENTRY-1422
> Project: Sentry
> Issue Type: Sub-task
> Components: Hdfs Plugin
>Reporter: Sravya Tirukkovalur
>Assignee: Colin Ma
> Fix For: sentry-ha-redesign
>
> Attachments: SENTRY-1422.008-sentry-ha-redesign.patch,
> SENTRY-1422.009-sentry-ha-redesign.patch,
> SENTRY-1422.010-sentry-ha-redesign.patch,
> SENTRY-1422.011-sentry-ha-redesign.patch,
> SENTRY-1422.012-sentry-ha-redesign.patch,
> SENTRY-1422.013-sentry-ha-redesign.patch,
> SENTRY-1422.014-sentry-ha-redesign.patch,
> SENTRY-1422.015-sentry-ha-redesign.patch,
> SENTRY-1422.016-sentry-ha-redesign.patch,
> SENTRY-1422.017-sentry-ha-redesign.patch, sentry_deadlock.png,
> testDeadLock.patch
>
>
> As I was working on Sentry-1321. I see that test case
> TestDbPrivilegeCleanupOnDrop#testDropObjects fails while "Granting select on
> table" with following deadlock.
> {code}
> java.sql.SQLException: Error while processing statement: FAILED: Execution
> Error, return code 1 from
> org.apache.hadoop.hive.ql.exec.SentryGrantRevokeTask. Unknown error for
> request: TAlterSentryRoleGrantPrivilegeRequest(protocol_version:2,
> requestorUserName:admin1, roleName:select_tbl1,
> privileges:[TSentryPrivilege(privilegeScope:TABLE, serverName:server1,
> dbName:db_2, tableName:tb_1, URI:, action:select, createTime:1469562185573,
> grantOption:FALSE, columnName:)]), message: Iteration request failed : SELECT
> 'org.apache.sentry.provider.db.service.model.MSentryPrivilege' AS
> NUCLEUS_TYPE,A1.URI,A1."ACTION",A1."COLUMN_NAME",A1.CREATE_TIME,A1.DB_NAME,A1.WITH_GRANT_OPTION,A1.PRIVILEGE_SCOPE,A1."SERVER_NAME",A1."TABLE_NAME",A1.DB_PRIVILEGE_ID
> FROM SENTRY_ROLE_DB_PRIVILEGE_MAP A0 INNER JOIN SENTRY_DB_PRIVILEGE A1 ON
> A0.DB_PRIVILEGE_ID = A1.DB_PRIVILEGE_ID WHERE A0.ROLE_ID = ?. Server
> Stacktrace: javax.jdo.JDODataStoreException: Iteration request failed :
> SELECT 'org.apache.sentry.provider.db.service.model.MSentryPrivilege' AS
> NUCLEUS_TYPE,A1.URI,A1."ACTION",A1."COLUMN_NAME",A1.CREATE_TIME,A1.DB_NAME,A1.WITH_GRANT_OPTION,A1.PRIVILEGE_SCOPE,A1."SERVER_NAME",A1."TABLE_NAME",A1.DB_PRIVILEGE_ID
> FROM SENTRY_ROLE_DB_PRIVILEGE_MAP A0 INNER JOIN SENTRY_DB_PRIVILEGE A1 ON
> A0.DB_PRIVILEGE_ID = A1.DB_PRIVILEGE_ID WHERE A0.ROLE_ID = ?
> at
> org.datanucleus.api.jdo.NucleusJDOHelper.getJDOExceptionForNucleusException(NucleusJDOHelper.java:451)
> at
> org.datanucleus.api.jdo.JDOTransaction.commit(JDOTransaction.java:165)
> at
> org.apache.sentry.provider.db.service.persistent.SentryStore.commitTransaction(SentryStore.java:279)
> at
> org.apache.sentry.provider.db.service.persistent.SentryStore.commitUpdateTransaction(SentryStore.java:261)
> at
> org.apache.sentry.provider.db.service.persistent.SentryStore.alterSentryRoleGrantPrivileges(SentryStore.java:458)
> at
> org.apache.sentry.provider.db.service.thrift.SentryPolicyStoreProcessor.alter_sentry_role_grant_privilege(SentryPolicyStoreProcessor.java:261)
> at
> org.apache.sentry.provider.db.service.thrift.SentryPolicyService$Processor$alter_sentry_role_grant_privilege.getResult(SentryPolicyService.java:1237)
> at
> org.apache.sentry.provider.db.service.thrift.SentryPolicyService$Processor$alter_sentry_role_grant_privilege.getResult(SentryPolicyService.java:1222)
> at org.apache.thrift.ProcessFunction.process(ProcessFunction.java:39)
> at org.apache.thrift.TBaseProcessor.process(TBaseProcessor.java:39)
> at
> org.apache.sentry.provider.db.service.thrift.SentryProcessorWrapper.process(SentryProcessorWrapper.java:35)
> at
> org.apache.thrift.TMultiplexedProcessor.process(TMultiplexedProcessor.java:123)
> at
> org.apache.thrift.server.TThreadPoolServer$WorkerProcess.run(TThreadPoolServer.java:286)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
> at java.lang.Thread.run(Thread.java:745)
> NestedThrowablesStackTrace:
> java.sql.SQLTransactionRollbackException: A lock could not be obtained due to
> a deadlock, cycle of locks and waiters is:
> Lock : ROW, SENTRY_ROLE_DB_PRIVILEGE_MAP, (1,10)
> Waiting XID : {712, S} , SENTRY, SELECT
> 'org.apache.sentry.provider.db.service.model.MSentryPrivilege' AS
> NUCLEUS_TYPE,A1.URI,A1."ACTION",A1."COLUMN_NAME",A1.CREATE_TIME,A1.DB_NA
[jira] [Commented] (SENTRY-1422) JDO deadlocks while processing grant while a background thread processes Notificationlogs
[
https://issues.apache.org/jira/browse/SENTRY-1422?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15601221#comment-15601221
]
Hadoop QA commented on SENTRY-1422:
---
Here are the results of testing the latest attachment
https://issues.apache.org/jira/secure/attachment/12834883/SENTRY-1422.016-sentry-ha-redesign.patch
against sentry-ha-redesign.
{color:red}Overall:{color} -1 due to 6 errors
{color:red}ERROR:{color} mvn test exited 1
{color:red}ERROR:{color} Failed:
org.apache.sentry.provider.db.generic.service.thrift.TestSentryGenericServiceIntegration
{color:red}ERROR:{color} Failed:
org.apache.sentry.provider.db.generic.service.persistent.TestPrivilegeOperatePersistence
{color:red}ERROR:{color} Failed:
org.apache.sentry.provider.db.generic.service.persistent.TestPrivilegeOperatePersistence
{color:red}ERROR:{color} Failed:
org.apache.sentry.provider.db.generic.service.persistent.TestPrivilegeOperatePersistence
{color:red}ERROR:{color} Failed:
org.apache.sentry.tests.e2e.dbprovider.TestDbPrivilegesAtFunctionScope
Console output:
https://builds.apache.org/job/PreCommit-SENTRY-Build/2082/console
This message is automatically generated.
> JDO deadlocks while processing grant while a background thread processes
> Notificationlogs
> -
>
> Key: SENTRY-1422
> URL: https://issues.apache.org/jira/browse/SENTRY-1422
> Project: Sentry
> Issue Type: Sub-task
> Components: Hdfs Plugin
>Reporter: Sravya Tirukkovalur
>Assignee: Colin Ma
> Fix For: sentry-ha-redesign
>
> Attachments: SENTRY-1422.008-sentry-ha-redesign.patch,
> SENTRY-1422.009-sentry-ha-redesign.patch,
> SENTRY-1422.010-sentry-ha-redesign.patch,
> SENTRY-1422.011-sentry-ha-redesign.patch,
> SENTRY-1422.012-sentry-ha-redesign.patch,
> SENTRY-1422.013-sentry-ha-redesign.patch,
> SENTRY-1422.014-sentry-ha-redesign.patch,
> SENTRY-1422.015-sentry-ha-redesign.patch,
> SENTRY-1422.016-sentry-ha-redesign.patch, sentry_deadlock.png,
> testDeadLock.patch
>
>
> As I was working on Sentry-1321. I see that test case
> TestDbPrivilegeCleanupOnDrop#testDropObjects fails while "Granting select on
> table" with following deadlock.
> {code}
> java.sql.SQLException: Error while processing statement: FAILED: Execution
> Error, return code 1 from
> org.apache.hadoop.hive.ql.exec.SentryGrantRevokeTask. Unknown error for
> request: TAlterSentryRoleGrantPrivilegeRequest(protocol_version:2,
> requestorUserName:admin1, roleName:select_tbl1,
> privileges:[TSentryPrivilege(privilegeScope:TABLE, serverName:server1,
> dbName:db_2, tableName:tb_1, URI:, action:select, createTime:1469562185573,
> grantOption:FALSE, columnName:)]), message: Iteration request failed : SELECT
> 'org.apache.sentry.provider.db.service.model.MSentryPrivilege' AS
> NUCLEUS_TYPE,A1.URI,A1."ACTION",A1."COLUMN_NAME",A1.CREATE_TIME,A1.DB_NAME,A1.WITH_GRANT_OPTION,A1.PRIVILEGE_SCOPE,A1."SERVER_NAME",A1."TABLE_NAME",A1.DB_PRIVILEGE_ID
> FROM SENTRY_ROLE_DB_PRIVILEGE_MAP A0 INNER JOIN SENTRY_DB_PRIVILEGE A1 ON
> A0.DB_PRIVILEGE_ID = A1.DB_PRIVILEGE_ID WHERE A0.ROLE_ID = ?. Server
> Stacktrace: javax.jdo.JDODataStoreException: Iteration request failed :
> SELECT 'org.apache.sentry.provider.db.service.model.MSentryPrivilege' AS
> NUCLEUS_TYPE,A1.URI,A1."ACTION",A1."COLUMN_NAME",A1.CREATE_TIME,A1.DB_NAME,A1.WITH_GRANT_OPTION,A1.PRIVILEGE_SCOPE,A1."SERVER_NAME",A1."TABLE_NAME",A1.DB_PRIVILEGE_ID
> FROM SENTRY_ROLE_DB_PRIVILEGE_MAP A0 INNER JOIN SENTRY_DB_PRIVILEGE A1 ON
> A0.DB_PRIVILEGE_ID = A1.DB_PRIVILEGE_ID WHERE A0.ROLE_ID = ?
> at
> org.datanucleus.api.jdo.NucleusJDOHelper.getJDOExceptionForNucleusException(NucleusJDOHelper.java:451)
> at
> org.datanucleus.api.jdo.JDOTransaction.commit(JDOTransaction.java:165)
> at
> org.apache.sentry.provider.db.service.persistent.SentryStore.commitTransaction(SentryStore.java:279)
> at
> org.apache.sentry.provider.db.service.persistent.SentryStore.commitUpdateTransaction(SentryStore.java:261)
> at
> org.apache.sentry.provider.db.service.persistent.SentryStore.alterSentryRoleGrantPrivileges(SentryStore.java:458)
> at
> org.apache.sentry.provider.db.service.thrift.SentryPolicyStoreProcessor.alter_sentry_role_grant_privilege(SentryPolicyStoreProcessor.java:261)
> at
> org.apache.sentry.provider.db.service.thrift.SentryPolicyService$Processor$alter_sentry_role_grant_privilege.getResult(SentryPolicyService.java:1237)
> at
> org.apache.sentry.provider.db.service.thrift.SentryPolicyService$Processor$alter_sentry_role_grant_privilege.getResult(SentryPolicyService.java:1222)
> at org.apache.thrift.ProcessFunction.process(ProcessFunction.java:39)
> at org.apache.thrift.TBaseProcessor.process(TBaseProcessor.java:3
