[jira] [Issue Comment Deleted] (SENTRY-1422) JDO deadlocks while processing grant while a background thread processes Notificationlogs
[
https://issues.apache.org/jira/browse/SENTRY-1422?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Colin Ma updated SENTRY-1422:
-
Comment: was deleted
(was: Here are the results of testing the latest attachment
https://issues.apache.org/jira/secure/attachment/12833264/SENTRY-1422.011-sentry-ha-redesign.patch
against sentry-ha-redesign.
{color:red}Overall:{color} -1 due to 18 errors
{color:red}ERROR:{color} mvn test exited 1
{color:red}ERROR:{color} Failed:
org.apache.sentry.provider.db.service.thrift.TestSentryServiceWithKerberos
{color:red}ERROR:{color} Failed:
org.apache.sentry.provider.db.service.thrift.TestActivator
{color:red}ERROR:{color} Failed:
org.apache.sentry.provider.db.generic.service.thrift.TestAuditLogForSentryGenericService
{color:red}ERROR:{color} Failed:
org.apache.sentry.provider.db.generic.tools.TestSentryShellSolr
{color:red}ERROR:{color} Failed:
org.apache.sentry.provider.db.service.thrift.TestSentryServiceFailureCase
{color:red}ERROR:{color} Failed:
org.apache.sentry.provider.db.service.thrift.TestSentryServiceMetrics
{color:red}ERROR:{color} Failed:
org.apache.sentry.provider.db.service.thrift.TestSentryServiceImportExport
{color:red}ERROR:{color} Failed:
org.apache.sentry.service.thrift.TestHMSFollower
{color:red}ERROR:{color} Failed:
org.apache.sentry.service.thrift.TestHMSFollower
{color:red}ERROR:{color} Failed:
org.apache.sentry.provider.db.service.thrift.TestSentryWebServerWithKerberos
{color:red}ERROR:{color} Failed:
org.apache.sentry.provider.db.tools.TestSentryShellHive
{color:red}ERROR:{color} Failed:
org.apache.sentry.provider.db.service.thrift.TestSentryServiceWithInvalidMsgSize
{color:red}ERROR:{color} Failed:
org.apache.sentry.provider.db.service.thrift.TestSentryServiceForPoolWithKerberos
{color:red}ERROR:{color} Failed:
org.apache.sentry.provider.db.service.thrift.TestSentryServiceIntegration
{color:red}ERROR:{color} Failed:
org.apache.sentry.provider.db.generic.tools.TestSentryConfigToolSolr
{color:red}ERROR:{color} Failed:
org.apache.sentry.provider.db.service.thrift.TestSentryWebServerWithoutSecurity
{color:red}ERROR:{color} Failed:
org.apache.sentry.tests.e2e.dbprovider.TestDbPrivilegesAtFunctionScope
Console output:
https://builds.apache.org/job/PreCommit-SENTRY-Build/2066/console
This message is automatically generated.)
> JDO deadlocks while processing grant while a background thread processes
> Notificationlogs
> -
>
> Key: SENTRY-1422
> URL: https://issues.apache.org/jira/browse/SENTRY-1422
> Project: Sentry
> Issue Type: Sub-task
> Components: Hdfs Plugin
>Reporter: Sravya Tirukkovalur
>Assignee: Colin Ma
> Fix For: sentry-ha-redesign
>
> Attachments: SENTRY-1422.008-sentry-ha-redesign.patch,
> SENTRY-1422.009-sentry-ha-redesign.patch,
> SENTRY-1422.010-sentry-ha-redesign.patch,
> SENTRY-1422.011-sentry-ha-redesign.patch,
> SENTRY-1422.012-sentry-ha-redesign.patch, sentry_deadlock.png,
> testDeadLock.patch
>
>
> As I was working on Sentry-1321. I see that test case
> TestDbPrivilegeCleanupOnDrop#testDropObjects fails while "Granting select on
> table" with following deadlock.
> {code}
> java.sql.SQLException: Error while processing statement: FAILED: Execution
> Error, return code 1 from
> org.apache.hadoop.hive.ql.exec.SentryGrantRevokeTask. Unknown error for
> request: TAlterSentryRoleGrantPrivilegeRequest(protocol_version:2,
> requestorUserName:admin1, roleName:select_tbl1,
> privileges:[TSentryPrivilege(privilegeScope:TABLE, serverName:server1,
> dbName:db_2, tableName:tb_1, URI:, action:select, createTime:1469562185573,
> grantOption:FALSE, columnName:)]), message: Iteration request failed : SELECT
> 'org.apache.sentry.provider.db.service.model.MSentryPrivilege' AS
> NUCLEUS_TYPE,A1.URI,A1."ACTION",A1."COLUMN_NAME",A1.CREATE_TIME,A1.DB_NAME,A1.WITH_GRANT_OPTION,A1.PRIVILEGE_SCOPE,A1."SERVER_NAME",A1."TABLE_NAME",A1.DB_PRIVILEGE_ID
> FROM SENTRY_ROLE_DB_PRIVILEGE_MAP A0 INNER JOIN SENTRY_DB_PRIVILEGE A1 ON
> A0.DB_PRIVILEGE_ID = A1.DB_PRIVILEGE_ID WHERE A0.ROLE_ID = ?. Server
> Stacktrace: javax.jdo.JDODataStoreException: Iteration request failed :
> SELECT 'org.apache.sentry.provider.db.service.model.MSentryPrivilege' AS
> NUCLEUS_TYPE,A1.URI,A1."ACTION",A1."COLUMN_NAME",A1.CREATE_TIME,A1.DB_NAME,A1.WITH_GRANT_OPTION,A1.PRIVILEGE_SCOPE,A1."SERVER_NAME",A1."TABLE_NAME",A1.DB_PRIVILEGE_ID
> FROM SENTRY_ROLE_DB_PRIVILEGE_MAP A0 INNER JOIN SENTRY_DB_PRIVILEGE A1 ON
> A0.DB_PRIVILEGE_ID = A1.DB_PRIVILEGE_ID WHERE A0.ROLE_ID = ?
> at
> org.datanucleus.api.jdo.NucleusJDOHelper.getJDOExceptionForNucleusException(NucleusJDOHelper.java:451)
> at
> org.datanucleus.api.jdo.JDOTransaction.commit(JDOTransaction.java:165)
> at
>
[jira] [Issue Comment Deleted] (SENTRY-1422) JDO deadlocks while processing grant while a background thread processes Notificationlogs
[
https://issues.apache.org/jira/browse/SENTRY-1422?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Colin Ma updated SENTRY-1422:
-
Comment: was deleted
(was: Here are the results of testing the latest attachment
https://issues.apache.org/jira/secure/attachment/12825190/SENTRY-1422.006-sentry-ha-redesign.patch
against sentry-ha-redesign.
{color:red}Overall:{color} -1 due to 4 errors
{color:red}ERROR:{color} mvn test exited 1
{color:red}ERROR:{color} Failed:
org.apache.sentry.provider.db.generic.tools.TestSentryShellSolr
{color:red}ERROR:{color} Failed:
org.apache.sentry.provider.db.service.thrift.TestSentryServiceIntegration
{color:red}ERROR:{color} Failed:
org.apache.sentry.tests.e2e.dbprovider.TestDbPrivilegesAtFunctionScope
Console output:
https://builds.apache.org/job/PreCommit-SENTRY-Build/1981/console
This message is automatically generated.)
> JDO deadlocks while processing grant while a background thread processes
> Notificationlogs
> -
>
> Key: SENTRY-1422
> URL: https://issues.apache.org/jira/browse/SENTRY-1422
> Project: Sentry
> Issue Type: Sub-task
> Components: Hdfs Plugin
>Reporter: Sravya Tirukkovalur
>Assignee: Colin Ma
> Fix For: sentry-ha-redesign
>
> Attachments: SENTRY-1422.007-sentry-ha-redesign.patch,
> SENTRY-1422.008-sentry-ha-redesign.patch,
> SENTRY-1422.009-sentry-ha-redesign.patch
>
>
> As I was working on Sentry-1321. I see that test case
> TestDbPrivilegeCleanupOnDrop#testDropObjects fails while "Granting select on
> table" with following deadlock.
> {code}
> java.sql.SQLException: Error while processing statement: FAILED: Execution
> Error, return code 1 from
> org.apache.hadoop.hive.ql.exec.SentryGrantRevokeTask. Unknown error for
> request: TAlterSentryRoleGrantPrivilegeRequest(protocol_version:2,
> requestorUserName:admin1, roleName:select_tbl1,
> privileges:[TSentryPrivilege(privilegeScope:TABLE, serverName:server1,
> dbName:db_2, tableName:tb_1, URI:, action:select, createTime:1469562185573,
> grantOption:FALSE, columnName:)]), message: Iteration request failed : SELECT
> 'org.apache.sentry.provider.db.service.model.MSentryPrivilege' AS
> NUCLEUS_TYPE,A1.URI,A1."ACTION",A1."COLUMN_NAME",A1.CREATE_TIME,A1.DB_NAME,A1.WITH_GRANT_OPTION,A1.PRIVILEGE_SCOPE,A1."SERVER_NAME",A1."TABLE_NAME",A1.DB_PRIVILEGE_ID
> FROM SENTRY_ROLE_DB_PRIVILEGE_MAP A0 INNER JOIN SENTRY_DB_PRIVILEGE A1 ON
> A0.DB_PRIVILEGE_ID = A1.DB_PRIVILEGE_ID WHERE A0.ROLE_ID = ?. Server
> Stacktrace: javax.jdo.JDODataStoreException: Iteration request failed :
> SELECT 'org.apache.sentry.provider.db.service.model.MSentryPrivilege' AS
> NUCLEUS_TYPE,A1.URI,A1."ACTION",A1."COLUMN_NAME",A1.CREATE_TIME,A1.DB_NAME,A1.WITH_GRANT_OPTION,A1.PRIVILEGE_SCOPE,A1."SERVER_NAME",A1."TABLE_NAME",A1.DB_PRIVILEGE_ID
> FROM SENTRY_ROLE_DB_PRIVILEGE_MAP A0 INNER JOIN SENTRY_DB_PRIVILEGE A1 ON
> A0.DB_PRIVILEGE_ID = A1.DB_PRIVILEGE_ID WHERE A0.ROLE_ID = ?
> at
> org.datanucleus.api.jdo.NucleusJDOHelper.getJDOExceptionForNucleusException(NucleusJDOHelper.java:451)
> at
> org.datanucleus.api.jdo.JDOTransaction.commit(JDOTransaction.java:165)
> at
> org.apache.sentry.provider.db.service.persistent.SentryStore.commitTransaction(SentryStore.java:279)
> at
> org.apache.sentry.provider.db.service.persistent.SentryStore.commitUpdateTransaction(SentryStore.java:261)
> at
> org.apache.sentry.provider.db.service.persistent.SentryStore.alterSentryRoleGrantPrivileges(SentryStore.java:458)
> at
> org.apache.sentry.provider.db.service.thrift.SentryPolicyStoreProcessor.alter_sentry_role_grant_privilege(SentryPolicyStoreProcessor.java:261)
> at
> org.apache.sentry.provider.db.service.thrift.SentryPolicyService$Processor$alter_sentry_role_grant_privilege.getResult(SentryPolicyService.java:1237)
> at
> org.apache.sentry.provider.db.service.thrift.SentryPolicyService$Processor$alter_sentry_role_grant_privilege.getResult(SentryPolicyService.java:1222)
> at org.apache.thrift.ProcessFunction.process(ProcessFunction.java:39)
> at org.apache.thrift.TBaseProcessor.process(TBaseProcessor.java:39)
> at
> org.apache.sentry.provider.db.service.thrift.SentryProcessorWrapper.process(SentryProcessorWrapper.java:35)
> at
> org.apache.thrift.TMultiplexedProcessor.process(TMultiplexedProcessor.java:123)
> at
> org.apache.thrift.server.TThreadPoolServer$WorkerProcess.run(TThreadPoolServer.java:286)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
> at java.lang.Thread.run(Thread.java:745)
> NestedThrowablesStackTrace:
>
[jira] [Issue Comment Deleted] (SENTRY-1422) JDO deadlocks while processing grant while a background thread processes Notificationlogs
[
https://issues.apache.org/jira/browse/SENTRY-1422?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Colin Ma updated SENTRY-1422:
-
Comment: was deleted
(was: Here are the results of testing the latest attachment
https://issues.apache.org/jira/secure/attachment/12825205/SENTRY-1422.007-sentry-ha-redesign.patch
against sentry-ha-redesign.
{color:red}Overall:{color} -1 due to 2 errors
{color:red}ERROR:{color} mvn test exited 1
{color:red}ERROR:{color} Failed:
org.apache.sentry.tests.e2e.dbprovider.TestDbRuntimeMetadataRetrieval
Console output:
https://builds.apache.org/job/PreCommit-SENTRY-Build/1984/console
This message is automatically generated.)
> JDO deadlocks while processing grant while a background thread processes
> Notificationlogs
> -
>
> Key: SENTRY-1422
> URL: https://issues.apache.org/jira/browse/SENTRY-1422
> Project: Sentry
> Issue Type: Sub-task
> Components: Hdfs Plugin
>Reporter: Sravya Tirukkovalur
>Assignee: Colin Ma
> Fix For: sentry-ha-redesign
>
> Attachments: SENTRY-1422.007-sentry-ha-redesign.patch,
> SENTRY-1422.008-sentry-ha-redesign.patch,
> SENTRY-1422.009-sentry-ha-redesign.patch
>
>
> As I was working on Sentry-1321. I see that test case
> TestDbPrivilegeCleanupOnDrop#testDropObjects fails while "Granting select on
> table" with following deadlock.
> {code}
> java.sql.SQLException: Error while processing statement: FAILED: Execution
> Error, return code 1 from
> org.apache.hadoop.hive.ql.exec.SentryGrantRevokeTask. Unknown error for
> request: TAlterSentryRoleGrantPrivilegeRequest(protocol_version:2,
> requestorUserName:admin1, roleName:select_tbl1,
> privileges:[TSentryPrivilege(privilegeScope:TABLE, serverName:server1,
> dbName:db_2, tableName:tb_1, URI:, action:select, createTime:1469562185573,
> grantOption:FALSE, columnName:)]), message: Iteration request failed : SELECT
> 'org.apache.sentry.provider.db.service.model.MSentryPrivilege' AS
> NUCLEUS_TYPE,A1.URI,A1."ACTION",A1."COLUMN_NAME",A1.CREATE_TIME,A1.DB_NAME,A1.WITH_GRANT_OPTION,A1.PRIVILEGE_SCOPE,A1."SERVER_NAME",A1."TABLE_NAME",A1.DB_PRIVILEGE_ID
> FROM SENTRY_ROLE_DB_PRIVILEGE_MAP A0 INNER JOIN SENTRY_DB_PRIVILEGE A1 ON
> A0.DB_PRIVILEGE_ID = A1.DB_PRIVILEGE_ID WHERE A0.ROLE_ID = ?. Server
> Stacktrace: javax.jdo.JDODataStoreException: Iteration request failed :
> SELECT 'org.apache.sentry.provider.db.service.model.MSentryPrivilege' AS
> NUCLEUS_TYPE,A1.URI,A1."ACTION",A1."COLUMN_NAME",A1.CREATE_TIME,A1.DB_NAME,A1.WITH_GRANT_OPTION,A1.PRIVILEGE_SCOPE,A1."SERVER_NAME",A1."TABLE_NAME",A1.DB_PRIVILEGE_ID
> FROM SENTRY_ROLE_DB_PRIVILEGE_MAP A0 INNER JOIN SENTRY_DB_PRIVILEGE A1 ON
> A0.DB_PRIVILEGE_ID = A1.DB_PRIVILEGE_ID WHERE A0.ROLE_ID = ?
> at
> org.datanucleus.api.jdo.NucleusJDOHelper.getJDOExceptionForNucleusException(NucleusJDOHelper.java:451)
> at
> org.datanucleus.api.jdo.JDOTransaction.commit(JDOTransaction.java:165)
> at
> org.apache.sentry.provider.db.service.persistent.SentryStore.commitTransaction(SentryStore.java:279)
> at
> org.apache.sentry.provider.db.service.persistent.SentryStore.commitUpdateTransaction(SentryStore.java:261)
> at
> org.apache.sentry.provider.db.service.persistent.SentryStore.alterSentryRoleGrantPrivileges(SentryStore.java:458)
> at
> org.apache.sentry.provider.db.service.thrift.SentryPolicyStoreProcessor.alter_sentry_role_grant_privilege(SentryPolicyStoreProcessor.java:261)
> at
> org.apache.sentry.provider.db.service.thrift.SentryPolicyService$Processor$alter_sentry_role_grant_privilege.getResult(SentryPolicyService.java:1237)
> at
> org.apache.sentry.provider.db.service.thrift.SentryPolicyService$Processor$alter_sentry_role_grant_privilege.getResult(SentryPolicyService.java:1222)
> at org.apache.thrift.ProcessFunction.process(ProcessFunction.java:39)
> at org.apache.thrift.TBaseProcessor.process(TBaseProcessor.java:39)
> at
> org.apache.sentry.provider.db.service.thrift.SentryProcessorWrapper.process(SentryProcessorWrapper.java:35)
> at
> org.apache.thrift.TMultiplexedProcessor.process(TMultiplexedProcessor.java:123)
> at
> org.apache.thrift.server.TThreadPoolServer$WorkerProcess.run(TThreadPoolServer.java:286)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
> at java.lang.Thread.run(Thread.java:745)
> NestedThrowablesStackTrace:
> java.sql.SQLTransactionRollbackException: A lock could not be obtained due to
> a deadlock, cycle of locks and waiters is:
> Lock : ROW, SENTRY_ROLE_DB_PRIVILEGE_MAP, (1,10)
> Waiting XID : {712, S} , SENTRY, SELECT
>
[jira] [Issue Comment Deleted] (SENTRY-1422) JDO deadlocks while processing grant while a background thread processes Notificationlogs
[
https://issues.apache.org/jira/browse/SENTRY-1422?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Colin Ma updated SENTRY-1422:
-
Comment: was deleted
(was: Here are the results of testing the latest attachment
https://issues.apache.org/jira/secure/attachment/12825177/SENTRY-1422.005-sentry-ha-redesign.patch
against sentry-ha-redesign.
{color:red}Overall:{color} -1 due to 5 errors
{color:red}ERROR:{color} mvn test exited 1
{color:red}ERROR:{color} Failed:
org.apache.sentry.tests.e2e.dbprovider.TestDbSandboxOps
{color:red}ERROR:{color} Failed:
org.apache.sentry.tests.e2e.dbprovider.TestDbCrossDbOps
{color:red}ERROR:{color} Failed:
org.apache.sentry.tests.e2e.dbprovider.TestDbOperationsPart1
{color:red}ERROR:{color} Failed:
org.apache.sentry.tests.e2e.dbprovider.TestDbPrivilegesAtTableScopePart1
Console output:
https://builds.apache.org/job/PreCommit-SENTRY-Build/1978/console
This message is automatically generated.)
> JDO deadlocks while processing grant while a background thread processes
> Notificationlogs
> -
>
> Key: SENTRY-1422
> URL: https://issues.apache.org/jira/browse/SENTRY-1422
> Project: Sentry
> Issue Type: Sub-task
> Components: Hdfs Plugin
>Reporter: Sravya Tirukkovalur
>Assignee: Colin Ma
> Fix For: sentry-ha-redesign
>
> Attachments: SENTRY-1422.007-sentry-ha-redesign.patch,
> SENTRY-1422.008-sentry-ha-redesign.patch,
> SENTRY-1422.009-sentry-ha-redesign.patch
>
>
> As I was working on Sentry-1321. I see that test case
> TestDbPrivilegeCleanupOnDrop#testDropObjects fails while "Granting select on
> table" with following deadlock.
> {code}
> java.sql.SQLException: Error while processing statement: FAILED: Execution
> Error, return code 1 from
> org.apache.hadoop.hive.ql.exec.SentryGrantRevokeTask. Unknown error for
> request: TAlterSentryRoleGrantPrivilegeRequest(protocol_version:2,
> requestorUserName:admin1, roleName:select_tbl1,
> privileges:[TSentryPrivilege(privilegeScope:TABLE, serverName:server1,
> dbName:db_2, tableName:tb_1, URI:, action:select, createTime:1469562185573,
> grantOption:FALSE, columnName:)]), message: Iteration request failed : SELECT
> 'org.apache.sentry.provider.db.service.model.MSentryPrivilege' AS
> NUCLEUS_TYPE,A1.URI,A1."ACTION",A1."COLUMN_NAME",A1.CREATE_TIME,A1.DB_NAME,A1.WITH_GRANT_OPTION,A1.PRIVILEGE_SCOPE,A1."SERVER_NAME",A1."TABLE_NAME",A1.DB_PRIVILEGE_ID
> FROM SENTRY_ROLE_DB_PRIVILEGE_MAP A0 INNER JOIN SENTRY_DB_PRIVILEGE A1 ON
> A0.DB_PRIVILEGE_ID = A1.DB_PRIVILEGE_ID WHERE A0.ROLE_ID = ?. Server
> Stacktrace: javax.jdo.JDODataStoreException: Iteration request failed :
> SELECT 'org.apache.sentry.provider.db.service.model.MSentryPrivilege' AS
> NUCLEUS_TYPE,A1.URI,A1."ACTION",A1."COLUMN_NAME",A1.CREATE_TIME,A1.DB_NAME,A1.WITH_GRANT_OPTION,A1.PRIVILEGE_SCOPE,A1."SERVER_NAME",A1."TABLE_NAME",A1.DB_PRIVILEGE_ID
> FROM SENTRY_ROLE_DB_PRIVILEGE_MAP A0 INNER JOIN SENTRY_DB_PRIVILEGE A1 ON
> A0.DB_PRIVILEGE_ID = A1.DB_PRIVILEGE_ID WHERE A0.ROLE_ID = ?
> at
> org.datanucleus.api.jdo.NucleusJDOHelper.getJDOExceptionForNucleusException(NucleusJDOHelper.java:451)
> at
> org.datanucleus.api.jdo.JDOTransaction.commit(JDOTransaction.java:165)
> at
> org.apache.sentry.provider.db.service.persistent.SentryStore.commitTransaction(SentryStore.java:279)
> at
> org.apache.sentry.provider.db.service.persistent.SentryStore.commitUpdateTransaction(SentryStore.java:261)
> at
> org.apache.sentry.provider.db.service.persistent.SentryStore.alterSentryRoleGrantPrivileges(SentryStore.java:458)
> at
> org.apache.sentry.provider.db.service.thrift.SentryPolicyStoreProcessor.alter_sentry_role_grant_privilege(SentryPolicyStoreProcessor.java:261)
> at
> org.apache.sentry.provider.db.service.thrift.SentryPolicyService$Processor$alter_sentry_role_grant_privilege.getResult(SentryPolicyService.java:1237)
> at
> org.apache.sentry.provider.db.service.thrift.SentryPolicyService$Processor$alter_sentry_role_grant_privilege.getResult(SentryPolicyService.java:1222)
> at org.apache.thrift.ProcessFunction.process(ProcessFunction.java:39)
> at org.apache.thrift.TBaseProcessor.process(TBaseProcessor.java:39)
> at
> org.apache.sentry.provider.db.service.thrift.SentryProcessorWrapper.process(SentryProcessorWrapper.java:35)
> at
> org.apache.thrift.TMultiplexedProcessor.process(TMultiplexedProcessor.java:123)
> at
> org.apache.thrift.server.TThreadPoolServer$WorkerProcess.run(TThreadPoolServer.java:286)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
> at
[jira] [Issue Comment Deleted] (SENTRY-1422) JDO deadlocks while processing grant while a background thread processes Notificationlogs
[
https://issues.apache.org/jira/browse/SENTRY-1422?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Colin Ma updated SENTRY-1422:
-
Comment: was deleted
(was: Here are the results of testing the latest attachment
https://issues.apache.org/jira/secure/attachment/12825205/SENTRY-1422.007-sentry-ha-redesign.patch
against sentry-ha-redesign.
{color:red}Overall:{color} -1 due to 6 errors
{color:red}ERROR:{color} mvn test exited 1
{color:red}ERROR:{color} Failed:
org.apache.sentry.tests.e2e.dbprovider.TestDbCrossDbOps
{color:red}ERROR:{color} Failed:
org.apache.sentry.tests.e2e.dbprovider.TestDbOperationsPart1
{color:red}ERROR:{color} Failed:
org.apache.sentry.tests.e2e.dbprovider.TestDbPrivilegesAtFunctionScope
{color:red}ERROR:{color} Failed:
org.apache.sentry.tests.e2e.dbprovider.TestDbOperationsPart2
{color:red}ERROR:{color} Failed:
org.apache.sentry.tests.e2e.dbprovider.TestDbSandboxOps
Console output:
https://builds.apache.org/job/PreCommit-SENTRY-Build/1983/console
This message is automatically generated.)
> JDO deadlocks while processing grant while a background thread processes
> Notificationlogs
> -
>
> Key: SENTRY-1422
> URL: https://issues.apache.org/jira/browse/SENTRY-1422
> Project: Sentry
> Issue Type: Sub-task
> Components: Hdfs Plugin
>Reporter: Sravya Tirukkovalur
>Assignee: Colin Ma
> Fix For: sentry-ha-redesign
>
> Attachments: SENTRY-1422.007-sentry-ha-redesign.patch,
> SENTRY-1422.008-sentry-ha-redesign.patch,
> SENTRY-1422.009-sentry-ha-redesign.patch
>
>
> As I was working on Sentry-1321. I see that test case
> TestDbPrivilegeCleanupOnDrop#testDropObjects fails while "Granting select on
> table" with following deadlock.
> {code}
> java.sql.SQLException: Error while processing statement: FAILED: Execution
> Error, return code 1 from
> org.apache.hadoop.hive.ql.exec.SentryGrantRevokeTask. Unknown error for
> request: TAlterSentryRoleGrantPrivilegeRequest(protocol_version:2,
> requestorUserName:admin1, roleName:select_tbl1,
> privileges:[TSentryPrivilege(privilegeScope:TABLE, serverName:server1,
> dbName:db_2, tableName:tb_1, URI:, action:select, createTime:1469562185573,
> grantOption:FALSE, columnName:)]), message: Iteration request failed : SELECT
> 'org.apache.sentry.provider.db.service.model.MSentryPrivilege' AS
> NUCLEUS_TYPE,A1.URI,A1."ACTION",A1."COLUMN_NAME",A1.CREATE_TIME,A1.DB_NAME,A1.WITH_GRANT_OPTION,A1.PRIVILEGE_SCOPE,A1."SERVER_NAME",A1."TABLE_NAME",A1.DB_PRIVILEGE_ID
> FROM SENTRY_ROLE_DB_PRIVILEGE_MAP A0 INNER JOIN SENTRY_DB_PRIVILEGE A1 ON
> A0.DB_PRIVILEGE_ID = A1.DB_PRIVILEGE_ID WHERE A0.ROLE_ID = ?. Server
> Stacktrace: javax.jdo.JDODataStoreException: Iteration request failed :
> SELECT 'org.apache.sentry.provider.db.service.model.MSentryPrivilege' AS
> NUCLEUS_TYPE,A1.URI,A1."ACTION",A1."COLUMN_NAME",A1.CREATE_TIME,A1.DB_NAME,A1.WITH_GRANT_OPTION,A1.PRIVILEGE_SCOPE,A1."SERVER_NAME",A1."TABLE_NAME",A1.DB_PRIVILEGE_ID
> FROM SENTRY_ROLE_DB_PRIVILEGE_MAP A0 INNER JOIN SENTRY_DB_PRIVILEGE A1 ON
> A0.DB_PRIVILEGE_ID = A1.DB_PRIVILEGE_ID WHERE A0.ROLE_ID = ?
> at
> org.datanucleus.api.jdo.NucleusJDOHelper.getJDOExceptionForNucleusException(NucleusJDOHelper.java:451)
> at
> org.datanucleus.api.jdo.JDOTransaction.commit(JDOTransaction.java:165)
> at
> org.apache.sentry.provider.db.service.persistent.SentryStore.commitTransaction(SentryStore.java:279)
> at
> org.apache.sentry.provider.db.service.persistent.SentryStore.commitUpdateTransaction(SentryStore.java:261)
> at
> org.apache.sentry.provider.db.service.persistent.SentryStore.alterSentryRoleGrantPrivileges(SentryStore.java:458)
> at
> org.apache.sentry.provider.db.service.thrift.SentryPolicyStoreProcessor.alter_sentry_role_grant_privilege(SentryPolicyStoreProcessor.java:261)
> at
> org.apache.sentry.provider.db.service.thrift.SentryPolicyService$Processor$alter_sentry_role_grant_privilege.getResult(SentryPolicyService.java:1237)
> at
> org.apache.sentry.provider.db.service.thrift.SentryPolicyService$Processor$alter_sentry_role_grant_privilege.getResult(SentryPolicyService.java:1222)
> at org.apache.thrift.ProcessFunction.process(ProcessFunction.java:39)
> at org.apache.thrift.TBaseProcessor.process(TBaseProcessor.java:39)
> at
> org.apache.sentry.provider.db.service.thrift.SentryProcessorWrapper.process(SentryProcessorWrapper.java:35)
> at
> org.apache.thrift.TMultiplexedProcessor.process(TMultiplexedProcessor.java:123)
> at
> org.apache.thrift.server.TThreadPoolServer$WorkerProcess.run(TThreadPoolServer.java:286)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
> at
>
[jira] [Issue Comment Deleted] (SENTRY-1422) JDO deadlocks while processing grant while a background thread processes Notificationlogs
[
https://issues.apache.org/jira/browse/SENTRY-1422?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Colin Ma updated SENTRY-1422:
-
Comment: was deleted
(was: Here are the results of testing the latest attachment
https://issues.apache.org/jira/secure/attachment/12825003/SENTRY-1422.004-sentry-ha-redesign.patch
against sentry-ha-redesign.
{color:red}Overall:{color} -1 due to 2 errors
{color:red}ERROR:{color} mvn test exited 1
{color:red}ERROR:{color} Failed:
org.apache.sentry.provider.db.generic.service.persistent.TestPrivilegeOperatePersistence
Console output:
https://builds.apache.org/job/PreCommit-SENTRY-Build/1970/console
This message is automatically generated.)
> JDO deadlocks while processing grant while a background thread processes
> Notificationlogs
> -
>
> Key: SENTRY-1422
> URL: https://issues.apache.org/jira/browse/SENTRY-1422
> Project: Sentry
> Issue Type: Sub-task
> Components: Hdfs Plugin
>Reporter: Sravya Tirukkovalur
>Assignee: Colin Ma
> Fix For: sentry-ha-redesign
>
> Attachments: SENTRY-1422.007-sentry-ha-redesign.patch,
> SENTRY-1422.008-sentry-ha-redesign.patch
>
>
> As I was working on Sentry-1321. I see that test case
> TestDbPrivilegeCleanupOnDrop#testDropObjects fails while "Granting select on
> table" with following deadlock.
> {code}
> java.sql.SQLException: Error while processing statement: FAILED: Execution
> Error, return code 1 from
> org.apache.hadoop.hive.ql.exec.SentryGrantRevokeTask. Unknown error for
> request: TAlterSentryRoleGrantPrivilegeRequest(protocol_version:2,
> requestorUserName:admin1, roleName:select_tbl1,
> privileges:[TSentryPrivilege(privilegeScope:TABLE, serverName:server1,
> dbName:db_2, tableName:tb_1, URI:, action:select, createTime:1469562185573,
> grantOption:FALSE, columnName:)]), message: Iteration request failed : SELECT
> 'org.apache.sentry.provider.db.service.model.MSentryPrivilege' AS
> NUCLEUS_TYPE,A1.URI,A1."ACTION",A1."COLUMN_NAME",A1.CREATE_TIME,A1.DB_NAME,A1.WITH_GRANT_OPTION,A1.PRIVILEGE_SCOPE,A1."SERVER_NAME",A1."TABLE_NAME",A1.DB_PRIVILEGE_ID
> FROM SENTRY_ROLE_DB_PRIVILEGE_MAP A0 INNER JOIN SENTRY_DB_PRIVILEGE A1 ON
> A0.DB_PRIVILEGE_ID = A1.DB_PRIVILEGE_ID WHERE A0.ROLE_ID = ?. Server
> Stacktrace: javax.jdo.JDODataStoreException: Iteration request failed :
> SELECT 'org.apache.sentry.provider.db.service.model.MSentryPrivilege' AS
> NUCLEUS_TYPE,A1.URI,A1."ACTION",A1."COLUMN_NAME",A1.CREATE_TIME,A1.DB_NAME,A1.WITH_GRANT_OPTION,A1.PRIVILEGE_SCOPE,A1."SERVER_NAME",A1."TABLE_NAME",A1.DB_PRIVILEGE_ID
> FROM SENTRY_ROLE_DB_PRIVILEGE_MAP A0 INNER JOIN SENTRY_DB_PRIVILEGE A1 ON
> A0.DB_PRIVILEGE_ID = A1.DB_PRIVILEGE_ID WHERE A0.ROLE_ID = ?
> at
> org.datanucleus.api.jdo.NucleusJDOHelper.getJDOExceptionForNucleusException(NucleusJDOHelper.java:451)
> at
> org.datanucleus.api.jdo.JDOTransaction.commit(JDOTransaction.java:165)
> at
> org.apache.sentry.provider.db.service.persistent.SentryStore.commitTransaction(SentryStore.java:279)
> at
> org.apache.sentry.provider.db.service.persistent.SentryStore.commitUpdateTransaction(SentryStore.java:261)
> at
> org.apache.sentry.provider.db.service.persistent.SentryStore.alterSentryRoleGrantPrivileges(SentryStore.java:458)
> at
> org.apache.sentry.provider.db.service.thrift.SentryPolicyStoreProcessor.alter_sentry_role_grant_privilege(SentryPolicyStoreProcessor.java:261)
> at
> org.apache.sentry.provider.db.service.thrift.SentryPolicyService$Processor$alter_sentry_role_grant_privilege.getResult(SentryPolicyService.java:1237)
> at
> org.apache.sentry.provider.db.service.thrift.SentryPolicyService$Processor$alter_sentry_role_grant_privilege.getResult(SentryPolicyService.java:1222)
> at org.apache.thrift.ProcessFunction.process(ProcessFunction.java:39)
> at org.apache.thrift.TBaseProcessor.process(TBaseProcessor.java:39)
> at
> org.apache.sentry.provider.db.service.thrift.SentryProcessorWrapper.process(SentryProcessorWrapper.java:35)
> at
> org.apache.thrift.TMultiplexedProcessor.process(TMultiplexedProcessor.java:123)
> at
> org.apache.thrift.server.TThreadPoolServer$WorkerProcess.run(TThreadPoolServer.java:286)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
> at java.lang.Thread.run(Thread.java:745)
> NestedThrowablesStackTrace:
> java.sql.SQLTransactionRollbackException: A lock could not be obtained due to
> a deadlock, cycle of locks and waiters is:
> Lock : ROW, SENTRY_ROLE_DB_PRIVILEGE_MAP, (1,10)
> Waiting XID : {712, S} , SENTRY, SELECT
>
[jira] [Issue Comment Deleted] (SENTRY-1422) JDO deadlocks while processing grant while a background thread processes Notificationlogs
[
https://issues.apache.org/jira/browse/SENTRY-1422?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Colin Ma updated SENTRY-1422:
-
Comment: was deleted
(was: Here are the results of testing the latest attachment
https://issues.apache.org/jira/secure/attachment/12824968/SENTRY-1422.003-sentry-ha-redesign.patch
against sentry-ha-redesign.
{color:red}Overall:{color} -1 due to 2 errors
{color:red}ERROR:{color} mvn test exited 1
{color:red}ERROR:{color} Failed: org.apache.sentry.tests.e2e.kafka.TestAuthorize
Console output:
https://builds.apache.org/job/PreCommit-SENTRY-Build/1967/console
This message is automatically generated.)
> JDO deadlocks while processing grant while a background thread processes
> Notificationlogs
> -
>
> Key: SENTRY-1422
> URL: https://issues.apache.org/jira/browse/SENTRY-1422
> Project: Sentry
> Issue Type: Sub-task
> Components: Hdfs Plugin
>Reporter: Sravya Tirukkovalur
>Assignee: Colin Ma
> Fix For: sentry-ha-redesign
>
> Attachments: SENTRY-1422.007-sentry-ha-redesign.patch,
> SENTRY-1422.008-sentry-ha-redesign.patch
>
>
> As I was working on Sentry-1321. I see that test case
> TestDbPrivilegeCleanupOnDrop#testDropObjects fails while "Granting select on
> table" with following deadlock.
> {code}
> java.sql.SQLException: Error while processing statement: FAILED: Execution
> Error, return code 1 from
> org.apache.hadoop.hive.ql.exec.SentryGrantRevokeTask. Unknown error for
> request: TAlterSentryRoleGrantPrivilegeRequest(protocol_version:2,
> requestorUserName:admin1, roleName:select_tbl1,
> privileges:[TSentryPrivilege(privilegeScope:TABLE, serverName:server1,
> dbName:db_2, tableName:tb_1, URI:, action:select, createTime:1469562185573,
> grantOption:FALSE, columnName:)]), message: Iteration request failed : SELECT
> 'org.apache.sentry.provider.db.service.model.MSentryPrivilege' AS
> NUCLEUS_TYPE,A1.URI,A1."ACTION",A1."COLUMN_NAME",A1.CREATE_TIME,A1.DB_NAME,A1.WITH_GRANT_OPTION,A1.PRIVILEGE_SCOPE,A1."SERVER_NAME",A1."TABLE_NAME",A1.DB_PRIVILEGE_ID
> FROM SENTRY_ROLE_DB_PRIVILEGE_MAP A0 INNER JOIN SENTRY_DB_PRIVILEGE A1 ON
> A0.DB_PRIVILEGE_ID = A1.DB_PRIVILEGE_ID WHERE A0.ROLE_ID = ?. Server
> Stacktrace: javax.jdo.JDODataStoreException: Iteration request failed :
> SELECT 'org.apache.sentry.provider.db.service.model.MSentryPrivilege' AS
> NUCLEUS_TYPE,A1.URI,A1."ACTION",A1."COLUMN_NAME",A1.CREATE_TIME,A1.DB_NAME,A1.WITH_GRANT_OPTION,A1.PRIVILEGE_SCOPE,A1."SERVER_NAME",A1."TABLE_NAME",A1.DB_PRIVILEGE_ID
> FROM SENTRY_ROLE_DB_PRIVILEGE_MAP A0 INNER JOIN SENTRY_DB_PRIVILEGE A1 ON
> A0.DB_PRIVILEGE_ID = A1.DB_PRIVILEGE_ID WHERE A0.ROLE_ID = ?
> at
> org.datanucleus.api.jdo.NucleusJDOHelper.getJDOExceptionForNucleusException(NucleusJDOHelper.java:451)
> at
> org.datanucleus.api.jdo.JDOTransaction.commit(JDOTransaction.java:165)
> at
> org.apache.sentry.provider.db.service.persistent.SentryStore.commitTransaction(SentryStore.java:279)
> at
> org.apache.sentry.provider.db.service.persistent.SentryStore.commitUpdateTransaction(SentryStore.java:261)
> at
> org.apache.sentry.provider.db.service.persistent.SentryStore.alterSentryRoleGrantPrivileges(SentryStore.java:458)
> at
> org.apache.sentry.provider.db.service.thrift.SentryPolicyStoreProcessor.alter_sentry_role_grant_privilege(SentryPolicyStoreProcessor.java:261)
> at
> org.apache.sentry.provider.db.service.thrift.SentryPolicyService$Processor$alter_sentry_role_grant_privilege.getResult(SentryPolicyService.java:1237)
> at
> org.apache.sentry.provider.db.service.thrift.SentryPolicyService$Processor$alter_sentry_role_grant_privilege.getResult(SentryPolicyService.java:1222)
> at org.apache.thrift.ProcessFunction.process(ProcessFunction.java:39)
> at org.apache.thrift.TBaseProcessor.process(TBaseProcessor.java:39)
> at
> org.apache.sentry.provider.db.service.thrift.SentryProcessorWrapper.process(SentryProcessorWrapper.java:35)
> at
> org.apache.thrift.TMultiplexedProcessor.process(TMultiplexedProcessor.java:123)
> at
> org.apache.thrift.server.TThreadPoolServer$WorkerProcess.run(TThreadPoolServer.java:286)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
> at java.lang.Thread.run(Thread.java:745)
> NestedThrowablesStackTrace:
> java.sql.SQLTransactionRollbackException: A lock could not be obtained due to
> a deadlock, cycle of locks and waiters is:
> Lock : ROW, SENTRY_ROLE_DB_PRIVILEGE_MAP, (1,10)
> Waiting XID : {712, S} , SENTRY, SELECT
> 'org.apache.sentry.provider.db.service.model.MSentryPrivilege' AS
>
[jira] [Issue Comment Deleted] (SENTRY-1422) JDO deadlocks while processing grant while a background thread processes Notificationlogs
[
https://issues.apache.org/jira/browse/SENTRY-1422?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Colin Ma updated SENTRY-1422:
-
Comment: was deleted
(was: Here are the results of testing the latest attachment
https://issues.apache.org/jira/secure/attachment/12824968/SENTRY-1422.003-sentry-ha-redesign.patch
against sentry-ha-redesign.
{color:red}Overall:{color} -1 due to 2 errors
{color:red}ERROR:{color} mvn test exited 1
{color:red}ERROR:{color} Failed:
org.apache.sentry.tests.e2e.solr.db.integration.TestSolrAdminOperations
Console output:
https://builds.apache.org/job/PreCommit-SENTRY-Build/1965/console
This message is automatically generated.)
> JDO deadlocks while processing grant while a background thread processes
> Notificationlogs
> -
>
> Key: SENTRY-1422
> URL: https://issues.apache.org/jira/browse/SENTRY-1422
> Project: Sentry
> Issue Type: Sub-task
> Components: Hdfs Plugin
>Reporter: Sravya Tirukkovalur
>Assignee: Colin Ma
> Fix For: sentry-ha-redesign
>
> Attachments: SENTRY-1422.001-sentry-ha-redesign.patch,
> SENTRY-1422.002-sentry-ha-redesign.patch,
> SENTRY-1422.003-sentry-ha-redesign.patch
>
>
> As I was working on Sentry-1321. I see that test case
> TestDbPrivilegeCleanupOnDrop#testDropObjects fails while "Granting select on
> table" with following deadlock.
> {code}
> java.sql.SQLException: Error while processing statement: FAILED: Execution
> Error, return code 1 from
> org.apache.hadoop.hive.ql.exec.SentryGrantRevokeTask. Unknown error for
> request: TAlterSentryRoleGrantPrivilegeRequest(protocol_version:2,
> requestorUserName:admin1, roleName:select_tbl1,
> privileges:[TSentryPrivilege(privilegeScope:TABLE, serverName:server1,
> dbName:db_2, tableName:tb_1, URI:, action:select, createTime:1469562185573,
> grantOption:FALSE, columnName:)]), message: Iteration request failed : SELECT
> 'org.apache.sentry.provider.db.service.model.MSentryPrivilege' AS
> NUCLEUS_TYPE,A1.URI,A1."ACTION",A1."COLUMN_NAME",A1.CREATE_TIME,A1.DB_NAME,A1.WITH_GRANT_OPTION,A1.PRIVILEGE_SCOPE,A1."SERVER_NAME",A1."TABLE_NAME",A1.DB_PRIVILEGE_ID
> FROM SENTRY_ROLE_DB_PRIVILEGE_MAP A0 INNER JOIN SENTRY_DB_PRIVILEGE A1 ON
> A0.DB_PRIVILEGE_ID = A1.DB_PRIVILEGE_ID WHERE A0.ROLE_ID = ?. Server
> Stacktrace: javax.jdo.JDODataStoreException: Iteration request failed :
> SELECT 'org.apache.sentry.provider.db.service.model.MSentryPrivilege' AS
> NUCLEUS_TYPE,A1.URI,A1."ACTION",A1."COLUMN_NAME",A1.CREATE_TIME,A1.DB_NAME,A1.WITH_GRANT_OPTION,A1.PRIVILEGE_SCOPE,A1."SERVER_NAME",A1."TABLE_NAME",A1.DB_PRIVILEGE_ID
> FROM SENTRY_ROLE_DB_PRIVILEGE_MAP A0 INNER JOIN SENTRY_DB_PRIVILEGE A1 ON
> A0.DB_PRIVILEGE_ID = A1.DB_PRIVILEGE_ID WHERE A0.ROLE_ID = ?
> at
> org.datanucleus.api.jdo.NucleusJDOHelper.getJDOExceptionForNucleusException(NucleusJDOHelper.java:451)
> at
> org.datanucleus.api.jdo.JDOTransaction.commit(JDOTransaction.java:165)
> at
> org.apache.sentry.provider.db.service.persistent.SentryStore.commitTransaction(SentryStore.java:279)
> at
> org.apache.sentry.provider.db.service.persistent.SentryStore.commitUpdateTransaction(SentryStore.java:261)
> at
> org.apache.sentry.provider.db.service.persistent.SentryStore.alterSentryRoleGrantPrivileges(SentryStore.java:458)
> at
> org.apache.sentry.provider.db.service.thrift.SentryPolicyStoreProcessor.alter_sentry_role_grant_privilege(SentryPolicyStoreProcessor.java:261)
> at
> org.apache.sentry.provider.db.service.thrift.SentryPolicyService$Processor$alter_sentry_role_grant_privilege.getResult(SentryPolicyService.java:1237)
> at
> org.apache.sentry.provider.db.service.thrift.SentryPolicyService$Processor$alter_sentry_role_grant_privilege.getResult(SentryPolicyService.java:1222)
> at org.apache.thrift.ProcessFunction.process(ProcessFunction.java:39)
> at org.apache.thrift.TBaseProcessor.process(TBaseProcessor.java:39)
> at
> org.apache.sentry.provider.db.service.thrift.SentryProcessorWrapper.process(SentryProcessorWrapper.java:35)
> at
> org.apache.thrift.TMultiplexedProcessor.process(TMultiplexedProcessor.java:123)
> at
> org.apache.thrift.server.TThreadPoolServer$WorkerProcess.run(TThreadPoolServer.java:286)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
> at java.lang.Thread.run(Thread.java:745)
> NestedThrowablesStackTrace:
> java.sql.SQLTransactionRollbackException: A lock could not be obtained due to
> a deadlock, cycle of locks and waiters is:
> Lock : ROW, SENTRY_ROLE_DB_PRIVILEGE_MAP, (1,10)
> Waiting XID : {712, S} , SENTRY, SELECT
>
[jira] [Issue Comment Deleted] (SENTRY-1422) JDO deadlocks while processing grant while a background thread processes Notificationlogs
[
https://issues.apache.org/jira/browse/SENTRY-1422?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Colin Ma updated SENTRY-1422:
-
Comment: was deleted
(was: Here are the results of testing the latest attachment
https://issues.apache.org/jira/secure/attachment/12824759/SENTRY-1422.001-sentry-ha-redesign.patch
against sentry-ha-redesign.
{color:red}Overall:{color} -1 due to 24 errors
{color:red}ERROR:{color} mvn test exited 1
{color:red}ERROR:{color} Failed:
org.apache.sentry.provider.db.service.thrift.TestSentryServiceWithInvalidMsgSize
{color:red}ERROR:{color} Failed:
org.apache.sentry.provider.db.service.thrift.TestSentryServiceClientPool
{color:red}ERROR:{color} Failed:
org.apache.sentry.provider.db.generic.service.persistent.TestPrivilegeOperatePersistence
{color:red}ERROR:{color} Failed:
org.apache.sentry.provider.db.generic.service.persistent.TestPrivilegeOperatePersistence
{color:red}ERROR:{color} Failed:
org.apache.sentry.provider.db.generic.service.persistent.TestPrivilegeOperatePersistence
{color:red}ERROR:{color} Failed:
org.apache.sentry.provider.db.generic.service.persistent.TestPrivilegeOperatePersistence
{color:red}ERROR:{color} Failed:
org.apache.sentry.provider.db.generic.service.persistent.TestPrivilegeOperatePersistence
{color:red}ERROR:{color} Failed:
org.apache.sentry.provider.db.generic.service.persistent.TestPrivilegeOperatePersistence
{color:red}ERROR:{color} Failed:
org.apache.sentry.provider.db.generic.service.persistent.TestPrivilegeOperatePersistence
{color:red}ERROR:{color} Failed:
org.apache.sentry.provider.db.generic.service.persistent.TestPrivilegeOperatePersistence
{color:red}ERROR:{color} Failed:
org.apache.sentry.provider.db.generic.service.persistent.TestPrivilegeOperatePersistence
{color:red}ERROR:{color} Failed:
org.apache.sentry.provider.db.generic.service.persistent.TestPrivilegeOperatePersistence
{color:red}ERROR:{color} Failed:
org.apache.sentry.provider.db.generic.service.persistent.TestPrivilegeOperatePersistence
{color:red}ERROR:{color} Failed:
org.apache.sentry.provider.db.generic.service.persistent.TestPrivilegeOperatePersistence
{color:red}ERROR:{color} Failed:
org.apache.sentry.provider.db.generic.service.persistent.TestPrivilegeOperatePersistence
{color:red}ERROR:{color} Failed:
org.apache.sentry.provider.db.generic.service.persistent.TestPrivilegeOperatePersistence
{color:red}ERROR:{color} Failed:
org.apache.sentry.provider.db.generic.service.persistent.TestPrivilegeOperatePersistence
{color:red}ERROR:{color} Failed:
org.apache.sentry.provider.db.generic.service.persistent.TestPrivilegeOperatePersistence
{color:red}ERROR:{color} Failed:
org.apache.sentry.provider.db.generic.service.persistent.TestPrivilegeOperatePersistence
{color:red}ERROR:{color} Failed:
org.apache.sentry.provider.db.generic.service.persistent.TestPrivilegeOperatePersistence
{color:red}ERROR:{color} Failed:
org.apache.sentry.provider.db.generic.service.persistent.TestPrivilegeOperatePersistence
{color:red}ERROR:{color} Failed:
org.apache.sentry.provider.db.generic.service.persistent.TestPrivilegeOperatePersistence
{color:red}ERROR:{color} Failed:
org.apache.sentry.provider.db.generic.service.persistent.TestPrivilegeOperatePersistence
Console output:
https://builds.apache.org/job/PreCommit-SENTRY-Build/1962/console
This message is automatically generated.)
> JDO deadlocks while processing grant while a background thread processes
> Notificationlogs
> -
>
> Key: SENTRY-1422
> URL: https://issues.apache.org/jira/browse/SENTRY-1422
> Project: Sentry
> Issue Type: Sub-task
> Components: Hdfs Plugin
>Reporter: Sravya Tirukkovalur
>Assignee: Colin Ma
> Fix For: sentry-ha-redesign
>
> Attachments: SENTRY-1422.001-sentry-ha-redesign.patch,
> SENTRY-1422.002-sentry-ha-redesign.patch,
> SENTRY-1422.003-sentry-ha-redesign.patch
>
>
> As I was working on Sentry-1321. I see that test case
> TestDbPrivilegeCleanupOnDrop#testDropObjects fails while "Granting select on
> table" with following deadlock.
> {code}
> java.sql.SQLException: Error while processing statement: FAILED: Execution
> Error, return code 1 from
> org.apache.hadoop.hive.ql.exec.SentryGrantRevokeTask. Unknown error for
> request: TAlterSentryRoleGrantPrivilegeRequest(protocol_version:2,
> requestorUserName:admin1, roleName:select_tbl1,
> privileges:[TSentryPrivilege(privilegeScope:TABLE, serverName:server1,
> dbName:db_2, tableName:tb_1, URI:, action:select, createTime:1469562185573,
> grantOption:FALSE, columnName:)]), message: Iteration request failed : SELECT
> 'org.apache.sentry.provider.db.service.model.MSentryPrivilege' AS
>
[jira] [Issue Comment Deleted] (SENTRY-1422) JDO deadlocks while processing grant while a background thread processes Notificationlogs
[
https://issues.apache.org/jira/browse/SENTRY-1422?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Colin Ma updated SENTRY-1422:
-
Comment: was deleted
(was: Here are the results of testing the latest attachment
https://issues.apache.org/jira/secure/attachment/12824790/SENTRY-1422.002-sentry-ha-redesign.patch
against sentry-ha-redesign.
{color:red}Overall:{color} -1 due to 6 errors
{color:red}ERROR:{color} mvn test exited 1
{color:red}ERROR:{color} Failed:
org.apache.sentry.provider.db.service.thrift.TestSentryServiceWithInvalidMsgSize
{color:red}ERROR:{color} Failed:
org.apache.sentry.service.thrift.TestHMSFollower
{color:red}ERROR:{color} Failed:
org.apache.sentry.service.thrift.TestHMSFollower
{color:red}ERROR:{color} Failed:
org.apache.sentry.provider.db.service.thrift.TestSentryServiceClientPool
{color:red}ERROR:{color} Failed:
org.apache.sentry.provider.db.service.thrift.TestSentryServiceClientPool
Console output:
https://builds.apache.org/job/PreCommit-SENTRY-Build/1963/console
This message is automatically generated.)
> JDO deadlocks while processing grant while a background thread processes
> Notificationlogs
> -
>
> Key: SENTRY-1422
> URL: https://issues.apache.org/jira/browse/SENTRY-1422
> Project: Sentry
> Issue Type: Sub-task
> Components: Hdfs Plugin
>Reporter: Sravya Tirukkovalur
>Assignee: Colin Ma
> Fix For: sentry-ha-redesign
>
> Attachments: SENTRY-1422.001-sentry-ha-redesign.patch,
> SENTRY-1422.002-sentry-ha-redesign.patch,
> SENTRY-1422.003-sentry-ha-redesign.patch
>
>
> As I was working on Sentry-1321. I see that test case
> TestDbPrivilegeCleanupOnDrop#testDropObjects fails while "Granting select on
> table" with following deadlock.
> {code}
> java.sql.SQLException: Error while processing statement: FAILED: Execution
> Error, return code 1 from
> org.apache.hadoop.hive.ql.exec.SentryGrantRevokeTask. Unknown error for
> request: TAlterSentryRoleGrantPrivilegeRequest(protocol_version:2,
> requestorUserName:admin1, roleName:select_tbl1,
> privileges:[TSentryPrivilege(privilegeScope:TABLE, serverName:server1,
> dbName:db_2, tableName:tb_1, URI:, action:select, createTime:1469562185573,
> grantOption:FALSE, columnName:)]), message: Iteration request failed : SELECT
> 'org.apache.sentry.provider.db.service.model.MSentryPrivilege' AS
> NUCLEUS_TYPE,A1.URI,A1."ACTION",A1."COLUMN_NAME",A1.CREATE_TIME,A1.DB_NAME,A1.WITH_GRANT_OPTION,A1.PRIVILEGE_SCOPE,A1."SERVER_NAME",A1."TABLE_NAME",A1.DB_PRIVILEGE_ID
> FROM SENTRY_ROLE_DB_PRIVILEGE_MAP A0 INNER JOIN SENTRY_DB_PRIVILEGE A1 ON
> A0.DB_PRIVILEGE_ID = A1.DB_PRIVILEGE_ID WHERE A0.ROLE_ID = ?. Server
> Stacktrace: javax.jdo.JDODataStoreException: Iteration request failed :
> SELECT 'org.apache.sentry.provider.db.service.model.MSentryPrivilege' AS
> NUCLEUS_TYPE,A1.URI,A1."ACTION",A1."COLUMN_NAME",A1.CREATE_TIME,A1.DB_NAME,A1.WITH_GRANT_OPTION,A1.PRIVILEGE_SCOPE,A1."SERVER_NAME",A1."TABLE_NAME",A1.DB_PRIVILEGE_ID
> FROM SENTRY_ROLE_DB_PRIVILEGE_MAP A0 INNER JOIN SENTRY_DB_PRIVILEGE A1 ON
> A0.DB_PRIVILEGE_ID = A1.DB_PRIVILEGE_ID WHERE A0.ROLE_ID = ?
> at
> org.datanucleus.api.jdo.NucleusJDOHelper.getJDOExceptionForNucleusException(NucleusJDOHelper.java:451)
> at
> org.datanucleus.api.jdo.JDOTransaction.commit(JDOTransaction.java:165)
> at
> org.apache.sentry.provider.db.service.persistent.SentryStore.commitTransaction(SentryStore.java:279)
> at
> org.apache.sentry.provider.db.service.persistent.SentryStore.commitUpdateTransaction(SentryStore.java:261)
> at
> org.apache.sentry.provider.db.service.persistent.SentryStore.alterSentryRoleGrantPrivileges(SentryStore.java:458)
> at
> org.apache.sentry.provider.db.service.thrift.SentryPolicyStoreProcessor.alter_sentry_role_grant_privilege(SentryPolicyStoreProcessor.java:261)
> at
> org.apache.sentry.provider.db.service.thrift.SentryPolicyService$Processor$alter_sentry_role_grant_privilege.getResult(SentryPolicyService.java:1237)
> at
> org.apache.sentry.provider.db.service.thrift.SentryPolicyService$Processor$alter_sentry_role_grant_privilege.getResult(SentryPolicyService.java:1222)
> at org.apache.thrift.ProcessFunction.process(ProcessFunction.java:39)
> at org.apache.thrift.TBaseProcessor.process(TBaseProcessor.java:39)
> at
> org.apache.sentry.provider.db.service.thrift.SentryProcessorWrapper.process(SentryProcessorWrapper.java:35)
> at
> org.apache.thrift.TMultiplexedProcessor.process(TMultiplexedProcessor.java:123)
> at
> org.apache.thrift.server.TThreadPoolServer$WorkerProcess.run(TThreadPoolServer.java:286)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
> at
>
