[jira] [Assigned] (SENTRY-2179) Sentry should provide its own HMS listener
[ https://issues.apache.org/jira/browse/SENTRY-2179?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alexander Kolbasov reassigned SENTRY-2179: -- Assignee: (was: Alexander Kolbasov) > Sentry should provide its own HMS listener > -- > > Key: SENTRY-2179 > URL: https://issues.apache.org/jira/browse/SENTRY-2179 > Project: Sentry > Issue Type: Bug > Components: Sentry >Affects Versions: 2.1.0 >Reporter: Alexander Kolbasov >Priority: Major > > Right now Sentry is using DbNotificationListener from Hive which listens to > lots of things that are not relevant for Sentry. Instead Sentry should > provide its own listener. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Assigned] (SENTRY-2179) Sentry should provide its own HMS listener
[ https://issues.apache.org/jira/browse/SENTRY-2179?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alexander Kolbasov reassigned SENTRY-2179: -- Assignee: Alexander Kolbasov > Sentry should provide its own HMS listener > -- > > Key: SENTRY-2179 > URL: https://issues.apache.org/jira/browse/SENTRY-2179 > Project: Sentry > Issue Type: Bug > Components: Sentry >Affects Versions: 2.1.0 >Reporter: Alexander Kolbasov >Assignee: Alexander Kolbasov >Priority: Major > > Right now Sentry is using DbNotificationListener from Hive which listens to > lots of things that are not relevant for Sentry. Instead Sentry should > provide its own listener. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (SENTRY-2178) Sentry permissions for Solr are deleted as part of migration process
[
https://issues.apache.org/jira/browse/SENTRY-2178?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16394061#comment-16394061
]
Hrishikesh Gadre commented on SENTRY-2178:
--
[~kkalyan] could you please review this change?
[https://reviews.apache.org/r/66019/]
> Sentry permissions for Solr are deleted as part of migration process
>
>
> Key: SENTRY-2178
> URL: https://issues.apache.org/jira/browse/SENTRY-2178
> Project: Sentry
> Issue Type: Bug
> Components: Solr Plugin
>Affects Versions: 2.0.0
>Reporter: Hrishikesh Gadre
>Assignee: Hrishikesh Gadre
>Priority: Critical
> Attachments: sentry2178.patch
>
>
> SENTRY-1480 implemented a command-line tool to migrate Sentry permissions
> from 1.x to 2.x. During upgrade testing I found a bug in the migration
> process where the pre-upgrade permissions are deleted. Specifically following
> permission was configured on Sentry v1
> {noformat}
> collection=*->action=*
> {noformat}
> After the migration, following permissions were available on Sentry service
> {noformat}
> admin=collections->action=*
> admin=cores->action=*
> {noformat}
> Note that the original permission is missing. From the following log snippet
> of Sentry service, it looks like the original permission was incorrectly
> revoked.
> {noformat}
> 2018-03-08 21:40:20,856 INFO DataNucleus.Datastore: Collection field
> "org.apache.sentry.provider.db.service.model.MSentryRole.gmPrivileges" of
> "org.apache.sentry.provider.db.service.model.MSentryRole@4dc76fa5" was asked
> to add element "MSentryGMPrivilege [serverName=service1, componentName=solr,
> authorizables=Admin=collections, scope=Admin, action=*, roles=[...],
> createTime=1520574020823, grantOption=false]" to the M-N bidirectional
> relation but the element already has this field in its collection (maybe
> added from the other side)
> 2018-03-08 21:40:20,997 INFO sentry.generic.authorization.ddl.logger:
> \{"serviceName":"Sentry-Service","userName":"solr","impersonator":"s...@gce.cloudera.com","ipAddress":"/172.31.117.188","operation":"GRANT_PRIVILEGE","eventTime":"1520574020991","operationText":"GRANT
> ALL ON Admin collections TO ROLE solr-admin ON COMPONENT
> SOLR","allowed":"true","databaseName":null,"tableName":null,"column":null,"resourcePath":null,"objectType":"PRINCIPAL"}
> 2018-03-08 21:40:21,015 INFO DataNucleus.Datastore: Collection field
> "org.apache.sentry.provider.db.service.model.MSentryRole.gmPrivileges" of
> "org.apache.sentry.provider.db.service.model.MSentryRole@46f3fe41" was asked
> to add element "MSentryGMPrivilege [serverName=service1, componentName=solr,
> authorizables=Admin=cores, scope=Admin, action=*, roles=[...],
> createTime=1520574021011, grantOption=false]" to the M-N bidirectional
> relation but the element already has this field in its collection (maybe
> added from the other side)
> 2018-03-08 21:40:21,022 INFO sentry.generic.authorization.ddl.logger:
> \{"serviceName":"Sentry-Service","userName":"solr","impersonator":"s...@gce.cloudera.com","ipAddress":"/172.31.117.188","operation":"GRANT_PRIVILEGE","eventTime":"1520574021022","operationText":"GRANT
> ALL ON Admin cores TO ROLE solr-admin ON COMPONENT
> SOLR","allowed":"true","databaseName":null,"tableName":null,"column":null,"resourcePath":null,"objectType":"PRINCIPAL"}
> 2018-03-08 21:40:21,035 INFO sentry.generic.authorization.ddl.logger:
> \{"serviceName":"Sentry-Service","userName":"solr","impersonator":"s...@gce.cloudera.com","ipAddress":"/172.31.117.188","operation":"GRANT_PRIVILEGE","eventTime":"1520574021035","operationText":"GRANT
> ALL ON Collection * TO ROLE solr-admin ON COMPONENT
> SOLR","allowed":"true","databaseName":null,"tableName":null,"column":null,"resourcePath":null,"objectType":"PRINCIPAL"}
> 2018-03-08 21:40:21,080 INFO sentry.generic.authorization.ddl.logger:
> \{"serviceName":"Sentry-Service","userName":"solr","impersonator":"s...@gce.cloudera.com","ipAddress":"/172.31.117.188","operation":"REVOKE_PRIVILEGE","eventTime":"1520574021080","operationText":"REVOKE
> ALL ON Collection * FROM ROLE solr-admin ON COMPONENT
> SOLR","allowed":"true","databaseName":null,"tableName":null,"column":null,"resourcePath":null,"objectType":"PRINCIPAL"}
> {noformat}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (SENTRY-2178) Sentry permissions for Solr are deleted as part of migration process
[
https://issues.apache.org/jira/browse/SENTRY-2178?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Hrishikesh Gadre updated SENTRY-2178:
-
Status: Patch Available (was: Open)
> Sentry permissions for Solr are deleted as part of migration process
>
>
> Key: SENTRY-2178
> URL: https://issues.apache.org/jira/browse/SENTRY-2178
> Project: Sentry
> Issue Type: Bug
> Components: Solr Plugin
>Affects Versions: 2.0.0
>Reporter: Hrishikesh Gadre
>Assignee: Hrishikesh Gadre
>Priority: Critical
> Attachments: sentry2178.patch
>
>
> SENTRY-1480 implemented a command-line tool to migrate Sentry permissions
> from 1.x to 2.x. During upgrade testing I found a bug in the migration
> process where the pre-upgrade permissions are deleted. Specifically following
> permission was configured on Sentry v1
> {noformat}
> collection=*->action=*
> {noformat}
> After the migration, following permissions were available on Sentry service
> {noformat}
> admin=collections->action=*
> admin=cores->action=*
> {noformat}
> Note that the original permission is missing. From the following log snippet
> of Sentry service, it looks like the original permission was incorrectly
> revoked.
> {noformat}
> 2018-03-08 21:40:20,856 INFO DataNucleus.Datastore: Collection field
> "org.apache.sentry.provider.db.service.model.MSentryRole.gmPrivileges" of
> "org.apache.sentry.provider.db.service.model.MSentryRole@4dc76fa5" was asked
> to add element "MSentryGMPrivilege [serverName=service1, componentName=solr,
> authorizables=Admin=collections, scope=Admin, action=*, roles=[...],
> createTime=1520574020823, grantOption=false]" to the M-N bidirectional
> relation but the element already has this field in its collection (maybe
> added from the other side)
> 2018-03-08 21:40:20,997 INFO sentry.generic.authorization.ddl.logger:
> \{"serviceName":"Sentry-Service","userName":"solr","impersonator":"s...@gce.cloudera.com","ipAddress":"/172.31.117.188","operation":"GRANT_PRIVILEGE","eventTime":"1520574020991","operationText":"GRANT
> ALL ON Admin collections TO ROLE solr-admin ON COMPONENT
> SOLR","allowed":"true","databaseName":null,"tableName":null,"column":null,"resourcePath":null,"objectType":"PRINCIPAL"}
> 2018-03-08 21:40:21,015 INFO DataNucleus.Datastore: Collection field
> "org.apache.sentry.provider.db.service.model.MSentryRole.gmPrivileges" of
> "org.apache.sentry.provider.db.service.model.MSentryRole@46f3fe41" was asked
> to add element "MSentryGMPrivilege [serverName=service1, componentName=solr,
> authorizables=Admin=cores, scope=Admin, action=*, roles=[...],
> createTime=1520574021011, grantOption=false]" to the M-N bidirectional
> relation but the element already has this field in its collection (maybe
> added from the other side)
> 2018-03-08 21:40:21,022 INFO sentry.generic.authorization.ddl.logger:
> \{"serviceName":"Sentry-Service","userName":"solr","impersonator":"s...@gce.cloudera.com","ipAddress":"/172.31.117.188","operation":"GRANT_PRIVILEGE","eventTime":"1520574021022","operationText":"GRANT
> ALL ON Admin cores TO ROLE solr-admin ON COMPONENT
> SOLR","allowed":"true","databaseName":null,"tableName":null,"column":null,"resourcePath":null,"objectType":"PRINCIPAL"}
> 2018-03-08 21:40:21,035 INFO sentry.generic.authorization.ddl.logger:
> \{"serviceName":"Sentry-Service","userName":"solr","impersonator":"s...@gce.cloudera.com","ipAddress":"/172.31.117.188","operation":"GRANT_PRIVILEGE","eventTime":"1520574021035","operationText":"GRANT
> ALL ON Collection * TO ROLE solr-admin ON COMPONENT
> SOLR","allowed":"true","databaseName":null,"tableName":null,"column":null,"resourcePath":null,"objectType":"PRINCIPAL"}
> 2018-03-08 21:40:21,080 INFO sentry.generic.authorization.ddl.logger:
> \{"serviceName":"Sentry-Service","userName":"solr","impersonator":"s...@gce.cloudera.com","ipAddress":"/172.31.117.188","operation":"REVOKE_PRIVILEGE","eventTime":"1520574021080","operationText":"REVOKE
> ALL ON Collection * FROM ROLE solr-admin ON COMPONENT
> SOLR","allowed":"true","databaseName":null,"tableName":null,"column":null,"resourcePath":null,"objectType":"PRINCIPAL"}
> {noformat}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (SENTRY-2178) Sentry permissions for Solr are deleted as part of migration process
[
https://issues.apache.org/jira/browse/SENTRY-2178?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Hrishikesh Gadre updated SENTRY-2178:
-
Attachment: sentry2178.patch
> Sentry permissions for Solr are deleted as part of migration process
>
>
> Key: SENTRY-2178
> URL: https://issues.apache.org/jira/browse/SENTRY-2178
> Project: Sentry
> Issue Type: Bug
> Components: Solr Plugin
>Affects Versions: 2.0.0
>Reporter: Hrishikesh Gadre
>Assignee: Hrishikesh Gadre
>Priority: Critical
> Attachments: sentry2178.patch
>
>
> SENTRY-1480 implemented a command-line tool to migrate Sentry permissions
> from 1.x to 2.x. During upgrade testing I found a bug in the migration
> process where the pre-upgrade permissions are deleted. Specifically following
> permission was configured on Sentry v1
> {noformat}
> collection=*->action=*
> {noformat}
> After the migration, following permissions were available on Sentry service
> {noformat}
> admin=collections->action=*
> admin=cores->action=*
> {noformat}
> Note that the original permission is missing. From the following log snippet
> of Sentry service, it looks like the original permission was incorrectly
> revoked.
> {noformat}
> 2018-03-08 21:40:20,856 INFO DataNucleus.Datastore: Collection field
> "org.apache.sentry.provider.db.service.model.MSentryRole.gmPrivileges" of
> "org.apache.sentry.provider.db.service.model.MSentryRole@4dc76fa5" was asked
> to add element "MSentryGMPrivilege [serverName=service1, componentName=solr,
> authorizables=Admin=collections, scope=Admin, action=*, roles=[...],
> createTime=1520574020823, grantOption=false]" to the M-N bidirectional
> relation but the element already has this field in its collection (maybe
> added from the other side)
> 2018-03-08 21:40:20,997 INFO sentry.generic.authorization.ddl.logger:
> \{"serviceName":"Sentry-Service","userName":"solr","impersonator":"s...@gce.cloudera.com","ipAddress":"/172.31.117.188","operation":"GRANT_PRIVILEGE","eventTime":"1520574020991","operationText":"GRANT
> ALL ON Admin collections TO ROLE solr-admin ON COMPONENT
> SOLR","allowed":"true","databaseName":null,"tableName":null,"column":null,"resourcePath":null,"objectType":"PRINCIPAL"}
> 2018-03-08 21:40:21,015 INFO DataNucleus.Datastore: Collection field
> "org.apache.sentry.provider.db.service.model.MSentryRole.gmPrivileges" of
> "org.apache.sentry.provider.db.service.model.MSentryRole@46f3fe41" was asked
> to add element "MSentryGMPrivilege [serverName=service1, componentName=solr,
> authorizables=Admin=cores, scope=Admin, action=*, roles=[...],
> createTime=1520574021011, grantOption=false]" to the M-N bidirectional
> relation but the element already has this field in its collection (maybe
> added from the other side)
> 2018-03-08 21:40:21,022 INFO sentry.generic.authorization.ddl.logger:
> \{"serviceName":"Sentry-Service","userName":"solr","impersonator":"s...@gce.cloudera.com","ipAddress":"/172.31.117.188","operation":"GRANT_PRIVILEGE","eventTime":"1520574021022","operationText":"GRANT
> ALL ON Admin cores TO ROLE solr-admin ON COMPONENT
> SOLR","allowed":"true","databaseName":null,"tableName":null,"column":null,"resourcePath":null,"objectType":"PRINCIPAL"}
> 2018-03-08 21:40:21,035 INFO sentry.generic.authorization.ddl.logger:
> \{"serviceName":"Sentry-Service","userName":"solr","impersonator":"s...@gce.cloudera.com","ipAddress":"/172.31.117.188","operation":"GRANT_PRIVILEGE","eventTime":"1520574021035","operationText":"GRANT
> ALL ON Collection * TO ROLE solr-admin ON COMPONENT
> SOLR","allowed":"true","databaseName":null,"tableName":null,"column":null,"resourcePath":null,"objectType":"PRINCIPAL"}
> 2018-03-08 21:40:21,080 INFO sentry.generic.authorization.ddl.logger:
> \{"serviceName":"Sentry-Service","userName":"solr","impersonator":"s...@gce.cloudera.com","ipAddress":"/172.31.117.188","operation":"REVOKE_PRIVILEGE","eventTime":"1520574021080","operationText":"REVOKE
> ALL ON Collection * FROM ROLE solr-admin ON COMPONENT
> SOLR","allowed":"true","databaseName":null,"tableName":null,"column":null,"resourcePath":null,"objectType":"PRINCIPAL"}
> {noformat}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Created] (SENTRY-2179) Sentry should provide its own HMS listener
Alexander Kolbasov created SENTRY-2179: -- Summary: Sentry should provide its own HMS listener Key: SENTRY-2179 URL: https://issues.apache.org/jira/browse/SENTRY-2179 Project: Sentry Issue Type: Bug Components: Sentry Affects Versions: 2.1.0 Reporter: Alexander Kolbasov Right now Sentry is using DbNotificationListener from Hive which listens to lots of things that are not relevant for Sentry. Instead Sentry should provide its own listener. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (SENTRY-2178) Sentry permissions for Solr are deleted as part of migration process
[
https://issues.apache.org/jira/browse/SENTRY-2178?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16393944#comment-16393944
]
Hrishikesh Gadre commented on SENTRY-2178:
--
It looks like the following defensive check in the code failed to handle this
case,
[https://github.com/apache/sentry/blob/f557a0380cf0ddc250010d4eb1b49e127f86f271/sentry-provider/sentry-provider-db/src/main/java/org/apache/sentry/provider/db/generic/tools/PermissionsMigrationToolCommon.java#L269-L272]
The root cause is that the equals method in TSentryPrivilege object performs
case sensitive comparison of component string. When we convert TSentryPrivilege
object from string representation, the component name is set as SOLR. On the
other hand TSentryPrivilege object prepared from database state has the
component name as solr. Due to this the original permission can not be found in
the converted permissions resulting in incorrect revocation.
The fix would be to ensure that we use case insensitive comparison of
permission strings (instead of TSentryPrivilege objects).
> Sentry permissions for Solr are deleted as part of migration process
>
>
> Key: SENTRY-2178
> URL: https://issues.apache.org/jira/browse/SENTRY-2178
> Project: Sentry
> Issue Type: Bug
> Components: Solr Plugin
>Affects Versions: 2.0.0
>Reporter: Hrishikesh Gadre
>Assignee: Hrishikesh Gadre
>Priority: Critical
>
> SENTRY-1480 implemented a command-line tool to migrate Sentry permissions
> from 1.x to 2.x. During upgrade testing I found a bug in the migration
> process where the pre-upgrade permissions are deleted. Specifically following
> permission was configured on Sentry v1
> {noformat}
> collection=*->action=*
> {noformat}
> After the migration, following permissions were available on Sentry service
> {noformat}
> admin=collections->action=*
> admin=cores->action=*
> {noformat}
> Note that the original permission is missing. From the following log snippet
> of Sentry service, it looks like the original permission was incorrectly
> revoked.
> {noformat}
> 2018-03-08 21:40:20,856 INFO DataNucleus.Datastore: Collection field
> "org.apache.sentry.provider.db.service.model.MSentryRole.gmPrivileges" of
> "org.apache.sentry.provider.db.service.model.MSentryRole@4dc76fa5" was asked
> to add element "MSentryGMPrivilege [serverName=service1, componentName=solr,
> authorizables=Admin=collections, scope=Admin, action=*, roles=[...],
> createTime=1520574020823, grantOption=false]" to the M-N bidirectional
> relation but the element already has this field in its collection (maybe
> added from the other side)
> 2018-03-08 21:40:20,997 INFO sentry.generic.authorization.ddl.logger:
> \{"serviceName":"Sentry-Service","userName":"solr","impersonator":"s...@gce.cloudera.com","ipAddress":"/172.31.117.188","operation":"GRANT_PRIVILEGE","eventTime":"1520574020991","operationText":"GRANT
> ALL ON Admin collections TO ROLE solr-admin ON COMPONENT
> SOLR","allowed":"true","databaseName":null,"tableName":null,"column":null,"resourcePath":null,"objectType":"PRINCIPAL"}
> 2018-03-08 21:40:21,015 INFO DataNucleus.Datastore: Collection field
> "org.apache.sentry.provider.db.service.model.MSentryRole.gmPrivileges" of
> "org.apache.sentry.provider.db.service.model.MSentryRole@46f3fe41" was asked
> to add element "MSentryGMPrivilege [serverName=service1, componentName=solr,
> authorizables=Admin=cores, scope=Admin, action=*, roles=[...],
> createTime=1520574021011, grantOption=false]" to the M-N bidirectional
> relation but the element already has this field in its collection (maybe
> added from the other side)
> 2018-03-08 21:40:21,022 INFO sentry.generic.authorization.ddl.logger:
> \{"serviceName":"Sentry-Service","userName":"solr","impersonator":"s...@gce.cloudera.com","ipAddress":"/172.31.117.188","operation":"GRANT_PRIVILEGE","eventTime":"1520574021022","operationText":"GRANT
> ALL ON Admin cores TO ROLE solr-admin ON COMPONENT
> SOLR","allowed":"true","databaseName":null,"tableName":null,"column":null,"resourcePath":null,"objectType":"PRINCIPAL"}
> 2018-03-08 21:40:21,035 INFO sentry.generic.authorization.ddl.logger:
> \{"serviceName":"Sentry-Service","userName":"solr","impersonator":"s...@gce.cloudera.com","ipAddress":"/172.31.117.188","operation":"GRANT_PRIVILEGE","eventTime":"1520574021035","operationText":"GRANT
> ALL ON Collection * TO ROLE solr-admin ON COMPONENT
> SOLR","allowed":"true","databaseName":null,"tableName":null,"column":null,"resourcePath":null,"objectType":"PRINCIPAL"}
> 2018-03-08 21:40:21,080 INFO sentry.generic.authorization.ddl.logger:
>
[jira] [Updated] (SENTRY-2178) Sentry permissions for Solr are deleted as part of migration process
[
https://issues.apache.org/jira/browse/SENTRY-2178?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Hrishikesh Gadre updated SENTRY-2178:
-
Description:
SENTRY-1480 implemented a command-line tool to migrate Sentry permissions from
1.x to 2.x. During upgrade testing I found a bug in the migration process where
the pre-upgrade permissions are deleted. Specifically following permission was
configured on Sentry v1
{noformat}
collection=*->action=*
{noformat}
After the migration, following permissions were available on Sentry service
{noformat}
admin=collections->action=*
admin=cores->action=*
{noformat}
Note that the original permission is missing. From the following log snippet of
Sentry service, it looks like the original permission was incorrectly revoked.
{noformat}
2018-03-08 21:40:20,856 INFO DataNucleus.Datastore: Collection field
"org.apache.sentry.provider.db.service.model.MSentryRole.gmPrivileges" of
"org.apache.sentry.provider.db.service.model.MSentryRole@4dc76fa5" was asked to
add element "MSentryGMPrivilege [serverName=service1, componentName=solr,
authorizables=Admin=collections, scope=Admin, action=*, roles=[...],
createTime=1520574020823, grantOption=false]" to the M-N bidirectional relation
but the element already has this field in its collection (maybe added from the
other side)
2018-03-08 21:40:20,997 INFO sentry.generic.authorization.ddl.logger:
\{"serviceName":"Sentry-Service","userName":"solr","impersonator":"s...@gce.cloudera.com","ipAddress":"/172.31.117.188","operation":"GRANT_PRIVILEGE","eventTime":"1520574020991","operationText":"GRANT
ALL ON Admin collections TO ROLE solr-admin ON COMPONENT
SOLR","allowed":"true","databaseName":null,"tableName":null,"column":null,"resourcePath":null,"objectType":"PRINCIPAL"}
2018-03-08 21:40:21,015 INFO DataNucleus.Datastore: Collection field
"org.apache.sentry.provider.db.service.model.MSentryRole.gmPrivileges" of
"org.apache.sentry.provider.db.service.model.MSentryRole@46f3fe41" was asked to
add element "MSentryGMPrivilege [serverName=service1, componentName=solr,
authorizables=Admin=cores, scope=Admin, action=*, roles=[...],
createTime=1520574021011, grantOption=false]" to the M-N bidirectional relation
but the element already has this field in its collection (maybe added from the
other side)
2018-03-08 21:40:21,022 INFO sentry.generic.authorization.ddl.logger:
\{"serviceName":"Sentry-Service","userName":"solr","impersonator":"s...@gce.cloudera.com","ipAddress":"/172.31.117.188","operation":"GRANT_PRIVILEGE","eventTime":"1520574021022","operationText":"GRANT
ALL ON Admin cores TO ROLE solr-admin ON COMPONENT
SOLR","allowed":"true","databaseName":null,"tableName":null,"column":null,"resourcePath":null,"objectType":"PRINCIPAL"}
2018-03-08 21:40:21,035 INFO sentry.generic.authorization.ddl.logger:
\{"serviceName":"Sentry-Service","userName":"solr","impersonator":"s...@gce.cloudera.com","ipAddress":"/172.31.117.188","operation":"GRANT_PRIVILEGE","eventTime":"1520574021035","operationText":"GRANT
ALL ON Collection * TO ROLE solr-admin ON COMPONENT
SOLR","allowed":"true","databaseName":null,"tableName":null,"column":null,"resourcePath":null,"objectType":"PRINCIPAL"}
2018-03-08 21:40:21,080 INFO sentry.generic.authorization.ddl.logger:
\{"serviceName":"Sentry-Service","userName":"solr","impersonator":"s...@gce.cloudera.com","ipAddress":"/172.31.117.188","operation":"REVOKE_PRIVILEGE","eventTime":"1520574021080","operationText":"REVOKE
ALL ON Collection * FROM ROLE solr-admin ON COMPONENT
SOLR","allowed":"true","databaseName":null,"tableName":null,"column":null,"resourcePath":null,"objectType":"PRINCIPAL"}
{noformat}
was:
SENTRY-1480 implemented a command-line tool to migrate Sentry permissions from
1.x to 2.x. During upgrade testing I found a bug in the migration process where
the pre-upgrade permissions are deleted. Specifically following permission was
configured on Sentry 2
{noformat}
collection=*->action=*
{noformat}
After the migration, following permissions were available on Sentry service
{noformat}
admin=collections->action=*
admin=cores->action=*
{noformat}
Note that the original permission is missing. From the following log snippet of
Sentry service, it looks like the original permission was incorrectly revoked.
{noformat}
2018-03-08 21:40:20,856 INFO DataNucleus.Datastore: Collection field
"org.apache.sentry.provider.db.service.model.MSentryRole.gmPrivileges" of
"org.apache.sentry.provider.db.service.model.MSentryRole@4dc76fa5" was asked to
add element "MSentryGMPrivilege [serverName=service1, componentName=solr,
authorizables=Admin=collections, scope=Admin, action=*, roles=[...],
createTime=1520574020823, grantOption=false]" to the M-N bidirectional relation
but the element already has this field in its collection (maybe added from the
other side)
2018-03-08 21:40:20,997 INFO sentry.generic.authorization.ddl.logger:
[jira] [Created] (SENTRY-2178) Sentry permissions for Solr are deleted as part of migration process
Hrishikesh Gadre created SENTRY-2178:
Summary: Sentry permissions for Solr are deleted as part of
migration process
Key: SENTRY-2178
URL: https://issues.apache.org/jira/browse/SENTRY-2178
Project: Sentry
Issue Type: Bug
Components: Solr Plugin
Affects Versions: 2.0.0
Reporter: Hrishikesh Gadre
Assignee: Hrishikesh Gadre
SENTRY-1480 implemented a command-line tool to migrate Sentry permissions from
1.x (i.e. CDH5.x) to 2.x (i.e. C6). During upgrade testing I found a bug in the
migration process where the pre-upgrade permissions are deleted. Specifically
following permission was configured on CDH5 cluster
{noformat}
collection=*->action=*
{noformat}
After the migration, following permissions were available on Sentry service
{noformat}
admin=collections->action=*
admin=cores->action=*
{noformat}
Note that the original permission is missing. From the following log snippet of
Sentry service, it looks like the original permission was incorrectly revoked.
{noformat}
2018-03-08 21:40:20,856 INFO DataNucleus.Datastore: Collection field
"org.apache.sentry.provider.db.service.model.MSentryRole.gmPrivileges" of
"org.apache.sentry.provider.db.service.model.MSentryRole@4dc76fa5" was asked to
add element "MSentryGMPrivilege [serverName=service1, componentName=solr,
authorizables=Admin=collections, scope=Admin, action=*, roles=[...],
createTime=1520574020823, grantOption=false]" to the M-N bidirectional relation
but the element already has this field in its collection (maybe added from the
other side)
2018-03-08 21:40:20,997 INFO sentry.generic.authorization.ddl.logger:
\{"serviceName":"Sentry-Service","userName":"solr","impersonator":"s...@gce.cloudera.com","ipAddress":"/172.31.117.188","operation":"GRANT_PRIVILEGE","eventTime":"1520574020991","operationText":"GRANT
ALL ON Admin collections TO ROLE solr-admin ON COMPONENT
SOLR","allowed":"true","databaseName":null,"tableName":null,"column":null,"resourcePath":null,"objectType":"PRINCIPAL"}
2018-03-08 21:40:21,015 INFO DataNucleus.Datastore: Collection field
"org.apache.sentry.provider.db.service.model.MSentryRole.gmPrivileges" of
"org.apache.sentry.provider.db.service.model.MSentryRole@46f3fe41" was asked to
add element "MSentryGMPrivilege [serverName=service1, componentName=solr,
authorizables=Admin=cores, scope=Admin, action=*, roles=[...],
createTime=1520574021011, grantOption=false]" to the M-N bidirectional relation
but the element already has this field in its collection (maybe added from the
other side)
2018-03-08 21:40:21,022 INFO sentry.generic.authorization.ddl.logger:
\{"serviceName":"Sentry-Service","userName":"solr","impersonator":"s...@gce.cloudera.com","ipAddress":"/172.31.117.188","operation":"GRANT_PRIVILEGE","eventTime":"1520574021022","operationText":"GRANT
ALL ON Admin cores TO ROLE solr-admin ON COMPONENT
SOLR","allowed":"true","databaseName":null,"tableName":null,"column":null,"resourcePath":null,"objectType":"PRINCIPAL"}
2018-03-08 21:40:21,035 INFO sentry.generic.authorization.ddl.logger:
\{"serviceName":"Sentry-Service","userName":"solr","impersonator":"s...@gce.cloudera.com","ipAddress":"/172.31.117.188","operation":"GRANT_PRIVILEGE","eventTime":"1520574021035","operationText":"GRANT
ALL ON Collection * TO ROLE solr-admin ON COMPONENT
SOLR","allowed":"true","databaseName":null,"tableName":null,"column":null,"resourcePath":null,"objectType":"PRINCIPAL"}
2018-03-08 21:40:21,080 INFO sentry.generic.authorization.ddl.logger:
\{"serviceName":"Sentry-Service","userName":"solr","impersonator":"s...@gce.cloudera.com","ipAddress":"/172.31.117.188","operation":"REVOKE_PRIVILEGE","eventTime":"1520574021080","operationText":"REVOKE
ALL ON Collection * FROM ROLE solr-admin ON COMPONENT
SOLR","allowed":"true","databaseName":null,"tableName":null,"column":null,"resourcePath":null,"objectType":"PRINCIPAL"}
{noformat}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (SENTRY-2178) Sentry permissions for Solr are deleted as part of migration process
[
https://issues.apache.org/jira/browse/SENTRY-2178?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Hrishikesh Gadre updated SENTRY-2178:
-
Description:
SENTRY-1480 implemented a command-line tool to migrate Sentry permissions from
1.x to 2.x. During upgrade testing I found a bug in the migration process where
the pre-upgrade permissions are deleted. Specifically following permission was
configured on Sentry 2
{noformat}
collection=*->action=*
{noformat}
After the migration, following permissions were available on Sentry service
{noformat}
admin=collections->action=*
admin=cores->action=*
{noformat}
Note that the original permission is missing. From the following log snippet of
Sentry service, it looks like the original permission was incorrectly revoked.
{noformat}
2018-03-08 21:40:20,856 INFO DataNucleus.Datastore: Collection field
"org.apache.sentry.provider.db.service.model.MSentryRole.gmPrivileges" of
"org.apache.sentry.provider.db.service.model.MSentryRole@4dc76fa5" was asked to
add element "MSentryGMPrivilege [serverName=service1, componentName=solr,
authorizables=Admin=collections, scope=Admin, action=*, roles=[...],
createTime=1520574020823, grantOption=false]" to the M-N bidirectional relation
but the element already has this field in its collection (maybe added from the
other side)
2018-03-08 21:40:20,997 INFO sentry.generic.authorization.ddl.logger:
\{"serviceName":"Sentry-Service","userName":"solr","impersonator":"s...@gce.cloudera.com","ipAddress":"/172.31.117.188","operation":"GRANT_PRIVILEGE","eventTime":"1520574020991","operationText":"GRANT
ALL ON Admin collections TO ROLE solr-admin ON COMPONENT
SOLR","allowed":"true","databaseName":null,"tableName":null,"column":null,"resourcePath":null,"objectType":"PRINCIPAL"}
2018-03-08 21:40:21,015 INFO DataNucleus.Datastore: Collection field
"org.apache.sentry.provider.db.service.model.MSentryRole.gmPrivileges" of
"org.apache.sentry.provider.db.service.model.MSentryRole@46f3fe41" was asked to
add element "MSentryGMPrivilege [serverName=service1, componentName=solr,
authorizables=Admin=cores, scope=Admin, action=*, roles=[...],
createTime=1520574021011, grantOption=false]" to the M-N bidirectional relation
but the element already has this field in its collection (maybe added from the
other side)
2018-03-08 21:40:21,022 INFO sentry.generic.authorization.ddl.logger:
\{"serviceName":"Sentry-Service","userName":"solr","impersonator":"s...@gce.cloudera.com","ipAddress":"/172.31.117.188","operation":"GRANT_PRIVILEGE","eventTime":"1520574021022","operationText":"GRANT
ALL ON Admin cores TO ROLE solr-admin ON COMPONENT
SOLR","allowed":"true","databaseName":null,"tableName":null,"column":null,"resourcePath":null,"objectType":"PRINCIPAL"}
2018-03-08 21:40:21,035 INFO sentry.generic.authorization.ddl.logger:
\{"serviceName":"Sentry-Service","userName":"solr","impersonator":"s...@gce.cloudera.com","ipAddress":"/172.31.117.188","operation":"GRANT_PRIVILEGE","eventTime":"1520574021035","operationText":"GRANT
ALL ON Collection * TO ROLE solr-admin ON COMPONENT
SOLR","allowed":"true","databaseName":null,"tableName":null,"column":null,"resourcePath":null,"objectType":"PRINCIPAL"}
2018-03-08 21:40:21,080 INFO sentry.generic.authorization.ddl.logger:
\{"serviceName":"Sentry-Service","userName":"solr","impersonator":"s...@gce.cloudera.com","ipAddress":"/172.31.117.188","operation":"REVOKE_PRIVILEGE","eventTime":"1520574021080","operationText":"REVOKE
ALL ON Collection * FROM ROLE solr-admin ON COMPONENT
SOLR","allowed":"true","databaseName":null,"tableName":null,"column":null,"resourcePath":null,"objectType":"PRINCIPAL"}
{noformat}
was:
SENTRY-1480 implemented a command-line tool to migrate Sentry permissions from
1.x to 2.x. During upgrade testing I found a bug in the migration process where
the pre-upgrade permissions are deleted. Specifically following permission was
configured on CDH5 cluster
{noformat}
collection=*->action=*
{noformat}
After the migration, following permissions were available on Sentry service
{noformat}
admin=collections->action=*
admin=cores->action=*
{noformat}
Note that the original permission is missing. From the following log snippet of
Sentry service, it looks like the original permission was incorrectly revoked.
{noformat}
2018-03-08 21:40:20,856 INFO DataNucleus.Datastore: Collection field
"org.apache.sentry.provider.db.service.model.MSentryRole.gmPrivileges" of
"org.apache.sentry.provider.db.service.model.MSentryRole@4dc76fa5" was asked to
add element "MSentryGMPrivilege [serverName=service1, componentName=solr,
authorizables=Admin=collections, scope=Admin, action=*, roles=[...],
createTime=1520574020823, grantOption=false]" to the M-N bidirectional relation
but the element already has this field in its collection (maybe added from the
other side)
2018-03-08 21:40:20,997 INFO sentry.generic.authorization.ddl.logger:
[jira] [Updated] (SENTRY-2178) Sentry permissions for Solr are deleted as part of migration process
[
https://issues.apache.org/jira/browse/SENTRY-2178?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Hrishikesh Gadre updated SENTRY-2178:
-
Description:
SENTRY-1480 implemented a command-line tool to migrate Sentry permissions from
1.x to 2.x. During upgrade testing I found a bug in the migration process where
the pre-upgrade permissions are deleted. Specifically following permission was
configured on CDH5 cluster
{noformat}
collection=*->action=*
{noformat}
After the migration, following permissions were available on Sentry service
{noformat}
admin=collections->action=*
admin=cores->action=*
{noformat}
Note that the original permission is missing. From the following log snippet of
Sentry service, it looks like the original permission was incorrectly revoked.
{noformat}
2018-03-08 21:40:20,856 INFO DataNucleus.Datastore: Collection field
"org.apache.sentry.provider.db.service.model.MSentryRole.gmPrivileges" of
"org.apache.sentry.provider.db.service.model.MSentryRole@4dc76fa5" was asked to
add element "MSentryGMPrivilege [serverName=service1, componentName=solr,
authorizables=Admin=collections, scope=Admin, action=*, roles=[...],
createTime=1520574020823, grantOption=false]" to the M-N bidirectional relation
but the element already has this field in its collection (maybe added from the
other side)
2018-03-08 21:40:20,997 INFO sentry.generic.authorization.ddl.logger:
\{"serviceName":"Sentry-Service","userName":"solr","impersonator":"s...@gce.cloudera.com","ipAddress":"/172.31.117.188","operation":"GRANT_PRIVILEGE","eventTime":"1520574020991","operationText":"GRANT
ALL ON Admin collections TO ROLE solr-admin ON COMPONENT
SOLR","allowed":"true","databaseName":null,"tableName":null,"column":null,"resourcePath":null,"objectType":"PRINCIPAL"}
2018-03-08 21:40:21,015 INFO DataNucleus.Datastore: Collection field
"org.apache.sentry.provider.db.service.model.MSentryRole.gmPrivileges" of
"org.apache.sentry.provider.db.service.model.MSentryRole@46f3fe41" was asked to
add element "MSentryGMPrivilege [serverName=service1, componentName=solr,
authorizables=Admin=cores, scope=Admin, action=*, roles=[...],
createTime=1520574021011, grantOption=false]" to the M-N bidirectional relation
but the element already has this field in its collection (maybe added from the
other side)
2018-03-08 21:40:21,022 INFO sentry.generic.authorization.ddl.logger:
\{"serviceName":"Sentry-Service","userName":"solr","impersonator":"s...@gce.cloudera.com","ipAddress":"/172.31.117.188","operation":"GRANT_PRIVILEGE","eventTime":"1520574021022","operationText":"GRANT
ALL ON Admin cores TO ROLE solr-admin ON COMPONENT
SOLR","allowed":"true","databaseName":null,"tableName":null,"column":null,"resourcePath":null,"objectType":"PRINCIPAL"}
2018-03-08 21:40:21,035 INFO sentry.generic.authorization.ddl.logger:
\{"serviceName":"Sentry-Service","userName":"solr","impersonator":"s...@gce.cloudera.com","ipAddress":"/172.31.117.188","operation":"GRANT_PRIVILEGE","eventTime":"1520574021035","operationText":"GRANT
ALL ON Collection * TO ROLE solr-admin ON COMPONENT
SOLR","allowed":"true","databaseName":null,"tableName":null,"column":null,"resourcePath":null,"objectType":"PRINCIPAL"}
2018-03-08 21:40:21,080 INFO sentry.generic.authorization.ddl.logger:
\{"serviceName":"Sentry-Service","userName":"solr","impersonator":"s...@gce.cloudera.com","ipAddress":"/172.31.117.188","operation":"REVOKE_PRIVILEGE","eventTime":"1520574021080","operationText":"REVOKE
ALL ON Collection * FROM ROLE solr-admin ON COMPONENT
SOLR","allowed":"true","databaseName":null,"tableName":null,"column":null,"resourcePath":null,"objectType":"PRINCIPAL"}
{noformat}
was:
SENTRY-1480 implemented a command-line tool to migrate Sentry permissions from
1.x (i.e. CDH5.x) to 2.x (i.e. C6). During upgrade testing I found a bug in the
migration process where the pre-upgrade permissions are deleted. Specifically
following permission was configured on CDH5 cluster
{noformat}
collection=*->action=*
{noformat}
After the migration, following permissions were available on Sentry service
{noformat}
admin=collections->action=*
admin=cores->action=*
{noformat}
Note that the original permission is missing. From the following log snippet of
Sentry service, it looks like the original permission was incorrectly revoked.
{noformat}
2018-03-08 21:40:20,856 INFO DataNucleus.Datastore: Collection field
"org.apache.sentry.provider.db.service.model.MSentryRole.gmPrivileges" of
"org.apache.sentry.provider.db.service.model.MSentryRole@4dc76fa5" was asked to
add element "MSentryGMPrivilege [serverName=service1, componentName=solr,
authorizables=Admin=collections, scope=Admin, action=*, roles=[...],
createTime=1520574020823, grantOption=false]" to the M-N bidirectional relation
but the element already has this field in its collection (maybe added from the
other side)
2018-03-08 21:40:20,997 INFO
[jira] [Commented] (SENTRY-2154) Update schema to grant privileges to user
[
https://issues.apache.org/jira/browse/SENTRY-2154?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16393617#comment-16393617
]
Sergio Peña commented on SENTRY-2154:
-
I see the same circular dependency with roles and dependencies
privileges->roles->privileges
{noformat}
MSentryPrivilege {
Set roles;
}
MSentryRole {
Set privileges;
Set users;
}{noformat}
How does DN work differently between these two JDO objects?
privileges->roles->users->privileges?
{noformat}
MSentryPrivilege {
Set roles;
}
MSentryRole {
Set privileges;
Set users;
}
MSentryUser {
Set privileges;
Set roles;
}{noformat}
> Update schema to grant privileges to user
> -
>
> Key: SENTRY-2154
> URL: https://issues.apache.org/jira/browse/SENTRY-2154
> Project: Sentry
> Issue Type: Sub-task
> Components: Sentry
>Affects Versions: 2.1.0
>Reporter: Na Li
>Priority: Major
> Fix For: 2.1.0
>
>
> Need to add new DB table to support grant user to privileges
> Also, a flag should be added in privilege table to indicate the privilege is
> created by user, or created by sentry implicitly. User can view the implicit
> privileges, but cannot change it directly
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (SENTRY-1242) Enable getting all privileges on a hive object
[
https://issues.apache.org/jira/browse/SENTRY-1242?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16393615#comment-16393615
]
Hadoop QA commented on SENTRY-1242:
---
Here are the results of testing the latest attachment
https://issues.apache.org/jira/secure/attachment/12913807/SENTRY-1242-003.patch
against master.
{color:green}Overall:{color} +1 all checks pass
{color:green}SUCCESS:{color} all tests passed
Console output:
https://builds.apache.org/job/PreCommit-SENTRY-Build/3693/console
This message is automatically generated.
> Enable getting all privileges on a hive object
> --
>
> Key: SENTRY-1242
> URL: https://issues.apache.org/jira/browse/SENTRY-1242
> Project: Sentry
> Issue Type: New Feature
>Affects Versions: 2.0.0
>Reporter: Sravya Tirukkovalur
>Assignee: Steve Moist
>Priority: Major
> Fix For: 2.1.0
>
> Attachments: SENTRY-1242-001.patch, SENTRY-1242-002.patch,
> SENTRY-1242-003.patch
>
>
> Enable show grant on table/db . This syntax is already supported by
> hive.
> This would be really useful for the admin to find out all policies on a hive
> object.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (SENTRY-1242) Enable getting all privileges on a hive object
[ https://issues.apache.org/jira/browse/SENTRY-1242?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Moist updated SENTRY-1242: Attachment: SENTRY-1242-003.patch > Enable getting all privileges on a hive object > -- > > Key: SENTRY-1242 > URL: https://issues.apache.org/jira/browse/SENTRY-1242 > Project: Sentry > Issue Type: New Feature >Affects Versions: 2.0.0 >Reporter: Sravya Tirukkovalur >Assignee: Steve Moist >Priority: Major > Fix For: 2.1.0 > > Attachments: SENTRY-1242-001.patch, SENTRY-1242-002.patch, > SENTRY-1242-003.patch > > > Enable show grant on table/db . This syntax is already supported by > hive. > This would be really useful for the admin to find out all policies on a hive > object. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (SENTRY-2140) Attribute based access control
[ https://issues.apache.org/jira/browse/SENTRY-2140?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16393263#comment-16393263 ] Steve Moist commented on SENTRY-2140: - >1) More formal definition of tags and their interaction with Hive privilege >model Attributes are sent in as a JSON object, the implementation will parse it and use a hierarchical Java object representation. The current examples show a String attribute, and this is what we plan to implement, but we leave it for other types to be added later for a fuller ABAC solution. >2) Some discussion of how it all applies (or doesn't apply) to generic >privilege model Currently there are no plans for it. >3) Proposed changes to Sentry thrift API (after all, CLI examples that you >mention just speak Sentrish). Main changes would be add_attribute_to_role, remove_attribute_from_role, list_attributes_for_role type commands. >4) Proposed changes to the Hive privilege model Currently there is none, it’s going to use the existing roles table, but add a new table linking attributes to roles and an additional table linking columns to attributes. >1) Can you add more specific details on how ABAC work with Role Based Access >Control? In my opinion, it happens at "Enforcement point for attribute >privileges in Sentry bindings for Hive and Impala" ABAC will be enforced after RBAC is applied. It mainly will happen in the bindings when a column is being accessed. >2) "Means for user to specify attribute privileges for roles (and users?)" It >seems you only use attribute on table column, Can we use attribute on user and >session? For example, can we grant access on accessing table column with PII >only for user with clearance > 4, during working hour and user country matches >the value of the "Country" column? Bringing a full ABAC implementation like you have described is a much larger effort. It would require much retooling of Sentry to properly process all of the rules on order of precedence that then get applied. We have someone on the team that is drafting a proposal on how to improve the "rule" flow and where things are enforced. >3) How is the info from "Attribute Ingestion" used in "Enforcement point for >attribute privileges"? An example that shows the whole work flow would be very >helpful. It’s pulled from a configured endpoint, processed and persisted in the database. Once Hive makes a query, the binding calls into the DefaultSentryController at the HiveAuthorizer.applyRowFilterAndColumnMasking to query if the columns have ABAC enforcement in them. There they are applied. > Attribute based access control > -- > > Key: SENTRY-2140 > URL: https://issues.apache.org/jira/browse/SENTRY-2140 > Project: Sentry > Issue Type: New Feature > Components: Core >Reporter: Steve Moist >Priority: Major > Attachments: Sentry ABAC Proposal.pdf > > > As a user, I want to have finer grain control over which users/roles can view > data in Hive. Some information such as Social Security Number is considered > very confidential information. I want to be able to tag columns in Hive with > "attributes" that prevent users/roles from not accessing or seeing the data. > For users/roles that have that attribute, they should be able to see that > information. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Resolved] (SENTRY-2176) IllegalFormatConversionException while trying to convert AtomicLong to int
[ https://issues.apache.org/jira/browse/SENTRY-2176?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Arjun Mishra resolved SENTRY-2176. -- Resolution: Fixed Has been resolved with SENTRY-2124 > IllegalFormatConversionException while trying to convert AtomicLong to int > -- > > Key: SENTRY-2176 > URL: https://issues.apache.org/jira/browse/SENTRY-2176 > Project: Sentry > Issue Type: Improvement > Components: Sentry >Affects Versions: 2.1.0 >Reporter: Arjun Mishra >Assignee: Arjun Mishra >Priority: Major > > Bug introduced by SENTRY-2078 in the toString method while trying to convert > AtomicLong "leaderCount" to int > SLF4J: Failed toString() invocation on an object of type > [org.apache.sentry.service.thrift.LeaderStatusMonitor] > java.util.IllegalFormatConversionException: d != > java.util.concurrent.atomic.AtomicLong > at > java.util.Formatter$FormatSpecifier.failConversion(Formatter.java:4302) > at java.util.Formatter$FormatSpecifier.printInteger(Formatter.java:2793) > at java.util.Formatter$FormatSpecifier.print(Formatter.java:2747) > at java.util.Formatter.format(Formatter.java:2520) > at java.util.Formatter.format(Formatter.java:2455) > at java.lang.String.format(String.java:2940) > at > org.apache.sentry.service.thrift.LeaderStatusMonitor.toString(LeaderStatusMonitor.java:284) > at > org.slf4j.helpers.MessageFormatter.safeObjectAppend(MessageFormatter.java:304) > at > org.slf4j.helpers.MessageFormatter.deeplyAppendParameter(MessageFormatter.java:276) > at > org.slf4j.helpers.MessageFormatter.arrayFormat(MessageFormatter.java:230) > at org.slf4j.helpers.MessageFormatter.format(MessageFormatter.java:124) > at org.slf4j.impl.Log4jLoggerAdapter.info(Log4jLoggerAdapter.java:322) > at > org.apache.sentry.service.thrift.LeaderStatusMonitor.takeLeadership(LeaderStatusMonitor.java:254) > at > sentry.org.apache.curator.framework.recipes.leader.LeaderSelector$WrappedListener.takeLeadership(LeaderSelector.java:537) > at > sentry.org.apache.curator.framework.recipes.leader.LeaderSelector.doWork(LeaderSelector.java:399) > at > sentry.org.apache.curator.framework.recipes.leader.LeaderSelector.doWorkLoop(LeaderSelector.java:444) > at > sentry.org.apache.curator.framework.recipes.leader.LeaderSelector.access$100(LeaderSelector.java:64) > at > sentry.org.apache.curator.framework.recipes.leader.LeaderSelector$2.call(LeaderSelector.java:245) > at > sentry.org.apache.curator.framework.recipes.leader.LeaderSelector$2.call(LeaderSelector.java:239) > at java.util.concurrent.FutureTask.run(FutureTask.java:266) > at > java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) > at java.util.concurrent.FutureTask.run(FutureTask.java:266) > at > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) > at > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) > at java.lang.Thread.run(Thread.java:745) -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (SENTRY-2109) Fix the logic of identifying HMS out of Sync and handle gaps and out-of-sequence notifications.
[ https://issues.apache.org/jira/browse/SENTRY-2109?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16392803#comment-16392803 ] kalyan kumar kalvagadda commented on SENTRY-2109: - [~LinaAtAustin] Scenario you mentioned in your comment is already handled by the patch. > Fix the logic of identifying HMS out of Sync and handle gaps and > out-of-sequence notifications. > --- > > Key: SENTRY-2109 > URL: https://issues.apache.org/jira/browse/SENTRY-2109 > Project: Sentry > Issue Type: Bug > Components: Sentry >Affects Versions: 2.1.0 >Reporter: kalyan kumar kalvagadda >Assignee: kalyan kumar kalvagadda >Priority: Major > Fix For: 2.1.0 > > Attachments: SENTRY-2109-rebase-SENTRY-2106.014.patch, > SENTRY-2109.001.patch, SENTRY-2109.002.patch, SENTRY-2109.003.patch, > SENTRY-2109.004.patch, SENTRY-2109.005.patch, SENTRY-2109.006.patch, > SENTRY-2109.007.patch, SENTRY-2109.008.patch, SENTRY-2109.009.patch, > SENTRY-2109.010.patch, SENTRY-2109.010.patch, SENTRY-2109.011.patch, > SENTRY-2109.012.patch, SENTRY-2109.012.patch, SENTRY-2109.012.patch, > SENTRY-2109.013.patch, Screenshot_HMS_NOTIFICATION_LOG.png > > > Currently HMSFollower proactively checks if sentry is out of sync with HMS > and initiates full snapshot, if needed. > There will be false positives with the current logic if there are gaps in the > event-id in the notification log sequence. > This jira is aimed at making that logic robust. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (SENTRY-2109) Fix the logic of identifying HMS out of Sync and handle gaps and out-of-sequence notifications.
[
https://issues.apache.org/jira/browse/SENTRY-2109?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16392789#comment-16392789
]
Hadoop QA commented on SENTRY-2109:
---
Here are the results of testing the latest attachment
https://issues.apache.org/jira/secure/attachment/12913741/SENTRY-2109-rebase-SENTRY-2106.014.patch
against master.
{color:green}Overall:{color} +1 all checks pass
{color:green}SUCCESS:{color} all tests passed
Console output:
https://builds.apache.org/job/PreCommit-SENTRY-Build/3692/console
This message is automatically generated.
> Fix the logic of identifying HMS out of Sync and handle gaps and
> out-of-sequence notifications.
> ---
>
> Key: SENTRY-2109
> URL: https://issues.apache.org/jira/browse/SENTRY-2109
> Project: Sentry
> Issue Type: Bug
> Components: Sentry
>Affects Versions: 2.1.0
>Reporter: kalyan kumar kalvagadda
>Assignee: kalyan kumar kalvagadda
>Priority: Major
> Fix For: 2.1.0
>
> Attachments: SENTRY-2109-rebase-SENTRY-2106.014.patch,
> SENTRY-2109.001.patch, SENTRY-2109.002.patch, SENTRY-2109.003.patch,
> SENTRY-2109.004.patch, SENTRY-2109.005.patch, SENTRY-2109.006.patch,
> SENTRY-2109.007.patch, SENTRY-2109.008.patch, SENTRY-2109.009.patch,
> SENTRY-2109.010.patch, SENTRY-2109.010.patch, SENTRY-2109.011.patch,
> SENTRY-2109.012.patch, SENTRY-2109.012.patch, SENTRY-2109.012.patch,
> SENTRY-2109.013.patch, Screenshot_HMS_NOTIFICATION_LOG.png
>
>
> Currently HMSFollower proactively checks if sentry is out of sync with HMS
> and initiates full snapshot, if needed.
> There will be false positives with the current logic if there are gaps in the
> event-id in the notification log sequence.
> This jira is aimed at making that logic robust.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Created] (SENTRY-2177) Allow suffix wildcards for Kafka topic management
Jim Halfpenny created SENTRY-2177: - Summary: Allow suffix wildcards for Kafka topic management Key: SENTRY-2177 URL: https://issues.apache.org/jira/browse/SENTRY-2177 Project: Sentry Issue Type: Improvement Components: kafka-integration, Sentry Reporter: Jim Halfpenny -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (SENTRY-2176) IllegalFormatConversionException while trying to convert AtomicLong to int
[ https://issues.apache.org/jira/browse/SENTRY-2176?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Arjun Mishra updated SENTRY-2176: - Description: Bug introduced by SENTRY-2078 in the toString method while trying to convert AtomicLong "leaderCount" to int SLF4J: Failed toString() invocation on an object of type [org.apache.sentry.service.thrift.LeaderStatusMonitor] java.util.IllegalFormatConversionException: d != java.util.concurrent.atomic.AtomicLong at java.util.Formatter$FormatSpecifier.failConversion(Formatter.java:4302) at java.util.Formatter$FormatSpecifier.printInteger(Formatter.java:2793) at java.util.Formatter$FormatSpecifier.print(Formatter.java:2747) at java.util.Formatter.format(Formatter.java:2520) at java.util.Formatter.format(Formatter.java:2455) at java.lang.String.format(String.java:2940) at org.apache.sentry.service.thrift.LeaderStatusMonitor.toString(LeaderStatusMonitor.java:284) at org.slf4j.helpers.MessageFormatter.safeObjectAppend(MessageFormatter.java:304) at org.slf4j.helpers.MessageFormatter.deeplyAppendParameter(MessageFormatter.java:276) at org.slf4j.helpers.MessageFormatter.arrayFormat(MessageFormatter.java:230) at org.slf4j.helpers.MessageFormatter.format(MessageFormatter.java:124) at org.slf4j.impl.Log4jLoggerAdapter.info(Log4jLoggerAdapter.java:322) at org.apache.sentry.service.thrift.LeaderStatusMonitor.takeLeadership(LeaderStatusMonitor.java:254) at sentry.org.apache.curator.framework.recipes.leader.LeaderSelector$WrappedListener.takeLeadership(LeaderSelector.java:537) at sentry.org.apache.curator.framework.recipes.leader.LeaderSelector.doWork(LeaderSelector.java:399) at sentry.org.apache.curator.framework.recipes.leader.LeaderSelector.doWorkLoop(LeaderSelector.java:444) at sentry.org.apache.curator.framework.recipes.leader.LeaderSelector.access$100(LeaderSelector.java:64) at sentry.org.apache.curator.framework.recipes.leader.LeaderSelector$2.call(LeaderSelector.java:245) at sentry.org.apache.curator.framework.recipes.leader.LeaderSelector$2.call(LeaderSelector.java:239) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) at java.lang.Thread.run(Thread.java:745) was: Bug introduced by SENTRY-2078 in the toString method while trying to convert AtomicLong to int SLF4J: Failed toString() invocation on an object of type [org.apache.sentry.service.thrift.LeaderStatusMonitor] java.util.IllegalFormatConversionException: d != java.util.concurrent.atomic.AtomicLong at java.util.Formatter$FormatSpecifier.failConversion(Formatter.java:4302) at java.util.Formatter$FormatSpecifier.printInteger(Formatter.java:2793) at java.util.Formatter$FormatSpecifier.print(Formatter.java:2747) at java.util.Formatter.format(Formatter.java:2520) at java.util.Formatter.format(Formatter.java:2455) at java.lang.String.format(String.java:2940) at org.apache.sentry.service.thrift.LeaderStatusMonitor.toString(LeaderStatusMonitor.java:284) at org.slf4j.helpers.MessageFormatter.safeObjectAppend(MessageFormatter.java:304) at org.slf4j.helpers.MessageFormatter.deeplyAppendParameter(MessageFormatter.java:276) at org.slf4j.helpers.MessageFormatter.arrayFormat(MessageFormatter.java:230) at org.slf4j.helpers.MessageFormatter.format(MessageFormatter.java:124) at org.slf4j.impl.Log4jLoggerAdapter.info(Log4jLoggerAdapter.java:322) at org.apache.sentry.service.thrift.LeaderStatusMonitor.takeLeadership(LeaderStatusMonitor.java:254) at sentry.org.apache.curator.framework.recipes.leader.LeaderSelector$WrappedListener.takeLeadership(LeaderSelector.java:537) at sentry.org.apache.curator.framework.recipes.leader.LeaderSelector.doWork(LeaderSelector.java:399) at sentry.org.apache.curator.framework.recipes.leader.LeaderSelector.doWorkLoop(LeaderSelector.java:444) at sentry.org.apache.curator.framework.recipes.leader.LeaderSelector.access$100(LeaderSelector.java:64) at sentry.org.apache.curator.framework.recipes.leader.LeaderSelector$2.call(LeaderSelector.java:245) at sentry.org.apache.curator.framework.recipes.leader.LeaderSelector$2.call(LeaderSelector.java:239) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) at
[jira] [Updated] (SENTRY-2176) IllegalFormatConversionException while trying to convert AtomicLong to int
[ https://issues.apache.org/jira/browse/SENTRY-2176?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Arjun Mishra updated SENTRY-2176: - Description: Bug introduced by SENTRY-2078 in the toString method while trying to convert AtomicLong to int SLF4J: Failed toString() invocation on an object of type [org.apache.sentry.service.thrift.LeaderStatusMonitor] java.util.IllegalFormatConversionException: d != java.util.concurrent.atomic.AtomicLong at java.util.Formatter$FormatSpecifier.failConversion(Formatter.java:4302) at java.util.Formatter$FormatSpecifier.printInteger(Formatter.java:2793) at java.util.Formatter$FormatSpecifier.print(Formatter.java:2747) at java.util.Formatter.format(Formatter.java:2520) at java.util.Formatter.format(Formatter.java:2455) at java.lang.String.format(String.java:2940) at org.apache.sentry.service.thrift.LeaderStatusMonitor.toString(LeaderStatusMonitor.java:284) at org.slf4j.helpers.MessageFormatter.safeObjectAppend(MessageFormatter.java:304) at org.slf4j.helpers.MessageFormatter.deeplyAppendParameter(MessageFormatter.java:276) at org.slf4j.helpers.MessageFormatter.arrayFormat(MessageFormatter.java:230) at org.slf4j.helpers.MessageFormatter.format(MessageFormatter.java:124) at org.slf4j.impl.Log4jLoggerAdapter.info(Log4jLoggerAdapter.java:322) at org.apache.sentry.service.thrift.LeaderStatusMonitor.takeLeadership(LeaderStatusMonitor.java:254) at sentry.org.apache.curator.framework.recipes.leader.LeaderSelector$WrappedListener.takeLeadership(LeaderSelector.java:537) at sentry.org.apache.curator.framework.recipes.leader.LeaderSelector.doWork(LeaderSelector.java:399) at sentry.org.apache.curator.framework.recipes.leader.LeaderSelector.doWorkLoop(LeaderSelector.java:444) at sentry.org.apache.curator.framework.recipes.leader.LeaderSelector.access$100(LeaderSelector.java:64) at sentry.org.apache.curator.framework.recipes.leader.LeaderSelector$2.call(LeaderSelector.java:245) at sentry.org.apache.curator.framework.recipes.leader.LeaderSelector$2.call(LeaderSelector.java:239) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) at java.lang.Thread.run(Thread.java:745) was:Bug introduced by SENTRY-2078 in the toString method while trying to convert AtomicLong to int > IllegalFormatConversionException while trying to convert AtomicLong to int > -- > > Key: SENTRY-2176 > URL: https://issues.apache.org/jira/browse/SENTRY-2176 > Project: Sentry > Issue Type: Improvement > Components: Sentry >Affects Versions: 2.1.0 >Reporter: Arjun Mishra >Assignee: Arjun Mishra >Priority: Major > > Bug introduced by SENTRY-2078 in the toString method while trying to convert > AtomicLong to int > SLF4J: Failed toString() invocation on an object of type > [org.apache.sentry.service.thrift.LeaderStatusMonitor] > java.util.IllegalFormatConversionException: d != > java.util.concurrent.atomic.AtomicLong > at > java.util.Formatter$FormatSpecifier.failConversion(Formatter.java:4302) > at java.util.Formatter$FormatSpecifier.printInteger(Formatter.java:2793) > at java.util.Formatter$FormatSpecifier.print(Formatter.java:2747) > at java.util.Formatter.format(Formatter.java:2520) > at java.util.Formatter.format(Formatter.java:2455) > at java.lang.String.format(String.java:2940) > at > org.apache.sentry.service.thrift.LeaderStatusMonitor.toString(LeaderStatusMonitor.java:284) > at > org.slf4j.helpers.MessageFormatter.safeObjectAppend(MessageFormatter.java:304) > at > org.slf4j.helpers.MessageFormatter.deeplyAppendParameter(MessageFormatter.java:276) > at > org.slf4j.helpers.MessageFormatter.arrayFormat(MessageFormatter.java:230) > at org.slf4j.helpers.MessageFormatter.format(MessageFormatter.java:124) > at org.slf4j.impl.Log4jLoggerAdapter.info(Log4jLoggerAdapter.java:322) > at > org.apache.sentry.service.thrift.LeaderStatusMonitor.takeLeadership(LeaderStatusMonitor.java:254) > at > sentry.org.apache.curator.framework.recipes.leader.LeaderSelector$WrappedListener.takeLeadership(LeaderSelector.java:537) > at > sentry.org.apache.curator.framework.recipes.leader.LeaderSelector.doWork(LeaderSelector.java:399) > at >
[jira] [Created] (SENTRY-2176) IllegalFormatConversionException while trying to convert AtomicLong to int
Arjun Mishra created SENTRY-2176: Summary: IllegalFormatConversionException while trying to convert AtomicLong to int Key: SENTRY-2176 URL: https://issues.apache.org/jira/browse/SENTRY-2176 Project: Sentry Issue Type: Improvement Components: Sentry Affects Versions: 2.1.0 Reporter: Arjun Mishra Assignee: Arjun Mishra Bug introduced by SENTRY-2078 in the toString method while trying to convert AtomicLong to int -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (SENTRY-2106) If Sentry is ahead do not trigger a full snapshot
[ https://issues.apache.org/jira/browse/SENTRY-2106?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Arjun Mishra updated SENTRY-2106: - Resolution: Information Provided Status: Resolved (was: Patch Available) > If Sentry is ahead do not trigger a full snapshot > - > > Key: SENTRY-2106 > URL: https://issues.apache.org/jira/browse/SENTRY-2106 > Project: Sentry > Issue Type: New Feature > Components: Sentry >Affects Versions: 2.1.0 >Reporter: Arjun Mishra >Assignee: Arjun Mishra >Priority: Major > Labels: features > Fix For: 2.1.0 > > Attachments: SENTRY-2106.01.patch, SENTRY-2106.02.patch, > SENTRY-2106.03.patch, SENTRY-2106.04.patch, SENTRY-2106.05.patch, > SENTRY-2106.06.patch > > > After steady state, a full snapshot is triggered by mainly 2 cases > # Sentry is "ahead" > # Sentry is behind > Case 1 has a dependency on NOTIFICATION_SEQUENCE table. This is not reliable > as it was observed that sometimes NOTIFICATION_SEQUENCE and NOTIFICATION_LOG > are not in sync. As a result of this unnecessary full snapshots can be > triggered. > The fix is to remove this dependency > > PLEASE NOTE THAT THIS SHOULD BE COMMITTED WITH SENTRY-2109 -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (SENTRY-2106) If Sentry is ahead do not trigger a full snapshot
[ https://issues.apache.org/jira/browse/SENTRY-2106?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16392643#comment-16392643 ] Arjun Mishra commented on SENTRY-2106: -- Rebased with SENTRY-2109 > If Sentry is ahead do not trigger a full snapshot > - > > Key: SENTRY-2106 > URL: https://issues.apache.org/jira/browse/SENTRY-2106 > Project: Sentry > Issue Type: New Feature > Components: Sentry >Affects Versions: 2.1.0 >Reporter: Arjun Mishra >Assignee: Arjun Mishra >Priority: Major > Labels: features > Fix For: 2.1.0 > > Attachments: SENTRY-2106.01.patch, SENTRY-2106.02.patch, > SENTRY-2106.03.patch, SENTRY-2106.04.patch, SENTRY-2106.05.patch, > SENTRY-2106.06.patch > > > After steady state, a full snapshot is triggered by mainly 2 cases > # Sentry is "ahead" > # Sentry is behind > Case 1 has a dependency on NOTIFICATION_SEQUENCE table. This is not reliable > as it was observed that sometimes NOTIFICATION_SEQUENCE and NOTIFICATION_LOG > are not in sync. As a result of this unnecessary full snapshots can be > triggered. > The fix is to remove this dependency > > PLEASE NOTE THAT THIS SHOULD BE COMMITTED WITH SENTRY-2109 -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (SENTRY-2109) Fix the logic of identifying HMS out of Sync and handle gaps and out-of-sequence notifications.
[ https://issues.apache.org/jira/browse/SENTRY-2109?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Arjun Mishra updated SENTRY-2109: - Attachment: SENTRY-2109-rebase-SENTRY-2106.014.patch > Fix the logic of identifying HMS out of Sync and handle gaps and > out-of-sequence notifications. > --- > > Key: SENTRY-2109 > URL: https://issues.apache.org/jira/browse/SENTRY-2109 > Project: Sentry > Issue Type: Bug > Components: Sentry >Affects Versions: 2.1.0 >Reporter: kalyan kumar kalvagadda >Assignee: kalyan kumar kalvagadda >Priority: Major > Fix For: 2.1.0 > > Attachments: SENTRY-2109-rebase-SENTRY-2106.014.patch, > SENTRY-2109.001.patch, SENTRY-2109.002.patch, SENTRY-2109.003.patch, > SENTRY-2109.004.patch, SENTRY-2109.005.patch, SENTRY-2109.006.patch, > SENTRY-2109.007.patch, SENTRY-2109.008.patch, SENTRY-2109.009.patch, > SENTRY-2109.010.patch, SENTRY-2109.010.patch, SENTRY-2109.011.patch, > SENTRY-2109.012.patch, SENTRY-2109.012.patch, SENTRY-2109.012.patch, > SENTRY-2109.013.patch, Screenshot_HMS_NOTIFICATION_LOG.png > > > Currently HMSFollower proactively checks if sentry is out of sync with HMS > and initiates full snapshot, if needed. > There will be false positives with the current logic if there are gaps in the > event-id in the notification log sequence. > This jira is aimed at making that logic robust. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (SENTRY-2106) If Sentry is ahead do not trigger a full snapshot
[ https://issues.apache.org/jira/browse/SENTRY-2106?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Arjun Mishra updated SENTRY-2106: - Summary: If Sentry is ahead do not trigger a full snapshot (was: Remove sentry dependency on HMS table NOTIFICATION_SEQUENCE when checking if Sentry is out-of-sync with HMS) > If Sentry is ahead do not trigger a full snapshot > - > > Key: SENTRY-2106 > URL: https://issues.apache.org/jira/browse/SENTRY-2106 > Project: Sentry > Issue Type: New Feature > Components: Sentry >Affects Versions: 2.1.0 >Reporter: Arjun Mishra >Assignee: Arjun Mishra >Priority: Major > Labels: features > Fix For: 2.1.0 > > Attachments: SENTRY-2106.01.patch, SENTRY-2106.02.patch, > SENTRY-2106.03.patch, SENTRY-2106.04.patch, SENTRY-2106.05.patch, > SENTRY-2106.06.patch > > > After steady state, a full snapshot is triggered by mainly 2 cases > # Sentry is "ahead" > # Sentry is behind > Case 1 has a dependency on NOTIFICATION_SEQUENCE table. This is not reliable > as it was observed that sometimes NOTIFICATION_SEQUENCE and NOTIFICATION_LOG > are not in sync. As a result of this unnecessary full snapshots can be > triggered. > The fix is to remove this dependency > > PLEASE NOTE THAT THIS SHOULD BE COMMITTED WITH SENTRY-2109 -- This message was sent by Atlassian JIRA (v7.6.3#76005)
