[jira] [Updated] (SENTRY-1649) Initialize HMSFollower when sentry server actually starts
[ https://issues.apache.org/jira/browse/SENTRY-1649?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Na Li updated SENTRY-1649: -- Attachment: SENTRY-1649.007-sentry-ha-redesign.patch update for suggestions in code review by Sasha. > Initialize HMSFollower when sentry server actually starts > - > > Key: SENTRY-1649 > URL: https://issues.apache.org/jira/browse/SENTRY-1649 > Project: Sentry > Issue Type: Sub-task > Components: Hdfs Plugin >Affects Versions: sentry-ha-redesign >Reporter: Hao Hao >Assignee: Na Li >Priority: Critical > Fix For: sentry-ha-redesign > > Attachments: SENTRY-1649.001-sentry-ha-redesign.patch, > SENTRY-1649.002-sentry-ha-redesign.patch, > SENTRY-1649.003-sentry-ha-redesign.patch, > SENTRY-1649.004-sentry-ha-redesign.patch, > SENTRY-1649.005-sentry-ha-redesign.patch, > SENTRY-1649.006-sentry-ha-redesign.patch, > SENTRY-1649.007-sentry-ha-redesign.patch > > > Now HMSFollower has been initialized at the constructor of SentryService. It > would be better to initialize it when the service starts, e.g runServer(). -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (SENTRY-1649) Initialize HMSFollower when sentry server actually starts
[ https://issues.apache.org/jira/browse/SENTRY-1649?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15965384#comment-15965384 ] Alexander Kolbasov commented on SENTRY-1649: Looking more at this it looks like it is possible to cancel execution using future's cancel method, but all this seems to be rather complicated - in this case shutting down the executor seems simpler. And you do need to shut it down on exit anyway. > Initialize HMSFollower when sentry server actually starts > - > > Key: SENTRY-1649 > URL: https://issues.apache.org/jira/browse/SENTRY-1649 > Project: Sentry > Issue Type: Sub-task > Components: Hdfs Plugin >Affects Versions: sentry-ha-redesign >Reporter: Hao Hao >Assignee: Na Li >Priority: Critical > Fix For: sentry-ha-redesign > > Attachments: SENTRY-1649.001-sentry-ha-redesign.patch, > SENTRY-1649.002-sentry-ha-redesign.patch, > SENTRY-1649.003-sentry-ha-redesign.patch, > SENTRY-1649.004-sentry-ha-redesign.patch, > SENTRY-1649.005-sentry-ha-redesign.patch, > SENTRY-1649.006-sentry-ha-redesign.patch > > > Now HMSFollower has been initialized at the constructor of SentryService. It > would be better to initialize it when the service starts, e.g runServer(). -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (SENTRY-1649) Initialize HMSFollower when sentry server actually starts
[ https://issues.apache.org/jira/browse/SENTRY-1649?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15965376#comment-15965376 ] Alexander Kolbasov commented on SENTRY-1649: Why do you want to use future for de-scheduling? Are you sure that it does the right thing? What's wrong with the simple shutdown()? > Initialize HMSFollower when sentry server actually starts > - > > Key: SENTRY-1649 > URL: https://issues.apache.org/jira/browse/SENTRY-1649 > Project: Sentry > Issue Type: Sub-task > Components: Hdfs Plugin >Affects Versions: sentry-ha-redesign >Reporter: Hao Hao >Assignee: Na Li >Priority: Critical > Fix For: sentry-ha-redesign > > Attachments: SENTRY-1649.001-sentry-ha-redesign.patch, > SENTRY-1649.002-sentry-ha-redesign.patch, > SENTRY-1649.003-sentry-ha-redesign.patch, > SENTRY-1649.004-sentry-ha-redesign.patch, > SENTRY-1649.005-sentry-ha-redesign.patch, > SENTRY-1649.006-sentry-ha-redesign.patch > > > Now HMSFollower has been initialized at the constructor of SentryService. It > would be better to initialize it when the service starts, e.g runServer(). -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Updated] (SENTRY-1649) Initialize HMSFollower when sentry server actually starts
[ https://issues.apache.org/jira/browse/SENTRY-1649?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Na Li updated SENTRY-1649: -- Attachment: SENTRY-1649.006-sentry-ha-redesign.patch version 006 is patch after rebase that includes all latest commites from remote sentry-ha-redesign > Initialize HMSFollower when sentry server actually starts > - > > Key: SENTRY-1649 > URL: https://issues.apache.org/jira/browse/SENTRY-1649 > Project: Sentry > Issue Type: Sub-task > Components: Hdfs Plugin >Affects Versions: sentry-ha-redesign >Reporter: Hao Hao >Assignee: Na Li >Priority: Critical > Fix For: sentry-ha-redesign > > Attachments: SENTRY-1649.001-sentry-ha-redesign.patch, > SENTRY-1649.002-sentry-ha-redesign.patch, > SENTRY-1649.003-sentry-ha-redesign.patch, > SENTRY-1649.004-sentry-ha-redesign.patch, > SENTRY-1649.005-sentry-ha-redesign.patch, > SENTRY-1649.006-sentry-ha-redesign.patch > > > Now HMSFollower has been initialized at the constructor of SentryService. It > would be better to initialize it when the service starts, e.g runServer(). -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (SENTRY-1649) Initialize HMSFollower when sentry server actually starts
[
https://issues.apache.org/jira/browse/SENTRY-1649?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15965337#comment-15965337
]
Hadoop QA commented on SENTRY-1649:
---
Here are the results of testing the latest attachment
https://issues.apache.org/jira/secure/attachment/12862969/SENTRY-1649.005-sentry-ha-redesign.patch
against sentry-ha-redesign.
{color:red}Overall:{color} -1 due to 3 errors
{color:red}ERROR:{color} mvn test exited 1
{color:red}ERROR:{color} Failed:
org.apache.sentry.provider.db.service.persistent.TestSentryStore
{color:red}ERROR:{color} Failed:
org.apache.sentry.provider.db.service.persistent.TestSentryStore
Console output:
https://builds.apache.org/job/PreCommit-SENTRY-Build/2499/console
This message is automatically generated.
> Initialize HMSFollower when sentry server actually starts
> -
>
> Key: SENTRY-1649
> URL: https://issues.apache.org/jira/browse/SENTRY-1649
> Project: Sentry
> Issue Type: Sub-task
> Components: Hdfs Plugin
>Affects Versions: sentry-ha-redesign
>Reporter: Hao Hao
>Assignee: Na Li
>Priority: Critical
> Fix For: sentry-ha-redesign
>
> Attachments: SENTRY-1649.001-sentry-ha-redesign.patch,
> SENTRY-1649.002-sentry-ha-redesign.patch,
> SENTRY-1649.003-sentry-ha-redesign.patch,
> SENTRY-1649.004-sentry-ha-redesign.patch,
> SENTRY-1649.005-sentry-ha-redesign.patch
>
>
> Now HMSFollower has been initialized at the constructor of SentryService. It
> would be better to initialize it when the service starts, e.g runServer().
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Updated] (SENTRY-1649) Initialize HMSFollower when sentry server actually starts
[ https://issues.apache.org/jira/browse/SENTRY-1649?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Na Li updated SENTRY-1649: -- Attachment: SENTRY-1649.005-sentry-ha-redesign.patch 1. Implement HMSFollower.close(). Remove its start() and stop(). 2. Add SentryService.close(), executor service shutdown are moved there. in stop(), cancel the future of the tasks instead of shutting down executor services. 3. In SentryService$CommandImpl.run(), call SentryService.stop() and close() instead of just call serviceExecutor.shutdown(). > Initialize HMSFollower when sentry server actually starts > - > > Key: SENTRY-1649 > URL: https://issues.apache.org/jira/browse/SENTRY-1649 > Project: Sentry > Issue Type: Sub-task > Components: Hdfs Plugin >Affects Versions: sentry-ha-redesign >Reporter: Hao Hao >Assignee: Na Li >Priority: Critical > Fix For: sentry-ha-redesign > > Attachments: SENTRY-1649.001-sentry-ha-redesign.patch, > SENTRY-1649.002-sentry-ha-redesign.patch, > SENTRY-1649.003-sentry-ha-redesign.patch, > SENTRY-1649.004-sentry-ha-redesign.patch, > SENTRY-1649.005-sentry-ha-redesign.patch > > > Now HMSFollower has been initialized at the constructor of SentryService. It > would be better to initialize it when the service starts, e.g runServer(). -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Updated] (SENTRY-1703) Solr-Sentry in kerberos mode makes too many KDC requests and returns unauthorized on KDC timeout
[
https://issues.apache.org/jira/browse/SENTRY-1703?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Tushar I updated SENTRY-1703:
-
Attachment: stacktrace3.log.txt
stacktrace2.log.txt
stacktrace1.log.txt
kdc.log.txt
> Solr-Sentry in kerberos mode makes too many KDC requests and returns
> unauthorized on KDC timeout
>
>
> Key: SENTRY-1703
> URL: https://issues.apache.org/jira/browse/SENTRY-1703
> Project: Sentry
> Issue Type: Bug
> Components: Solr Plugin
>Affects Versions: 1.5.1
>Reporter: Tushar I
> Attachments: kdc.log.txt, stacktrace1.log.txt, stacktrace2.log.txt,
> stacktrace3.log.txt
>
>
> Sentry Version: 1.5.1-cdh5.8.0
> We are seeing intermittent authorization failures with Sentry Solr plugin in
> a Kerberos environment.
> 1. We are writing to Solr using the SolrJ client from within Spark jobs in a
> multi-node Spark/Hadoop cluster and frequently get authorization errors from
> Solr in individual spark tasks saying "User XX does not have privileges for
> YYcollection" which are generated by the Solr-Sentry plugin. (The user very
> well has access to the collection and it works fine rest of the times).
> 2. The root cause seems to be that on every Solr call from the client, Sentry
> reaches out to KDC on behalf of solr/hostname to check if user XX has
> permission on the YYcollection, thereby drowning the KDC in tons of requests
> per second, and at some point fails on a KDC timeout, throwing the exception:
> {{org.apache.sentry.binding.solr.authz.SentrySolrAuthorizationException: User
> XX does not have privileges for YYcollection}} to the calling client.
> I didn't get enough time to investigate why Sentry is making so many KDC
> calls, maybe it's doing it for each document in a batched Solr operation, or
> it logs in using keytab each time and doesn't cache the ticket, etc.
> Caching the result of {{authProvider.hasAccess()}} in SolrAuthzBinding.java
> for a reasonably short time might not be a bad idea.
> My question in the meantime is: Are there any tuning knobs to somehow reduce
> the load on KDC, or increase the KDC request timeout value, or anything along
> these lines?
> Relevant stacktraces captured from Solr Admin are attached:
> 1. stacktrace1.log : The timeout from KDC for sentry call
> 2. stacktrace2.log: When Sentry cannot authenticate with KDC due to # 1 above
> 3. stacktrace3.log: SolrException when {{authProvider.hasAccess()}} returns
> false due to # 2 above.
> Also attached is a _snippet_ from the KDC log - the full log bloats to 17 MB
> within a minute, full of messages like:
> {code}
> Apr 10 17:06:37 a0 krb5kdc[20427](info): TGS_REQ (1 etypes {23}) 10.0.0.1:
> ISSUE: authtime 1491818430, etypes {rep=23 tkt=23 ses=23},
> solr/a...@realm.com for sentry/a...@realm.com
> {code}
> This is reproducible in two separate clusters with different environments:
> CDH 5.10.1 and
> CDH 5.8.0
> Please let me know if I've left out any key information.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Updated] (SENTRY-1643) AutoIncrement ChangeID of MSentryPermChange/MSentryPathChange may be error-prone
[
https://issues.apache.org/jira/browse/SENTRY-1643?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Alexander Kolbasov updated SENTRY-1643:
---
Resolution: Fixed
Status: Resolved (was: Patch Available)
[~eddyxu] Thank you for your contribution!
> AutoIncrement ChangeID of MSentryPermChange/MSentryPathChange may be
> error-prone
>
>
> Key: SENTRY-1643
> URL: https://issues.apache.org/jira/browse/SENTRY-1643
> Project: Sentry
> Issue Type: Sub-task
> Components: Hdfs Plugin
>Affects Versions: sentry-ha-redesign
>Reporter: Hao Hao
>Assignee: Lei (Eddy) Xu
>Priority: Critical
> Fix For: sentry-ha-redesign
>
> Attachments: SENTRY-1643.00-sentry-ha-redesign.patch,
> SENTRY-1643.01-sentry-ha-redesign.patch,
> SENTRY-1643.02-sentry-ha-redesign.patch
>
>
> In MSentryPermChange/MSentryPathChange table, the changeID field is
> auto-increment.
> {noformat}
>
>
> {noformat}
> However, found through the integration unit test TestHDFSIntegration, the
> value does not seem to be correctly auto increased. e.g Instead of increasing
> by one for each new entry, it increased by some unexpected number.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Updated] (SENTRY-1629) Current MAuthzPathsMapping table definition may cause error 'Duplicate entry XX for key PRIMARY'
[
https://issues.apache.org/jira/browse/SENTRY-1629?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
kalyan kumar kalvagadda updated SENTRY-1629:
Attachment: SENTRY-1629.002-sentry-ha-redesign.patch
> Current MAuthzPathsMapping table definition may cause error 'Duplicate entry
> XX for key PRIMARY'
>
>
> Key: SENTRY-1629
> URL: https://issues.apache.org/jira/browse/SENTRY-1629
> Project: Sentry
> Issue Type: Sub-task
> Components: Hdfs Plugin
>Affects Versions: sentry-ha-redesign
>Reporter: Hao Hao
>Assignee: kalyan kumar kalvagadda
>Priority: Critical
> Fix For: sentry-ha-redesign
>
> Attachments: SENTRY-1629.001-sentry-ha-redesign.patch,
> SENTRY-1629.001-sentry-ha-redesign.patch,
> SENTRY-1629.002-sentry-ha-redesign.patch
>
>
> In MAuthzPathsMapping table defines 1-N mapping of authzObjName to HMSPaths.
> However, current JDO metadata/SQL definition may cause duplicate entry error.
> {noformat} 2017-02-13 07:03:06,909 ERROR
> org.apache.sentry.provider.db.service.persistent.TransactionManager: The
> transaction has reached max retry numbe, rAdd request failed : INSERT INTO
> `MAUTHZPATHSMAPPING_PATHS` (`AUTHZ_OBJ_ID_OID`,`PATHS`) VALUES (?,?)
> javax.jdo.JDODataStoreException: Add request failed : INSERT INTO
> `MAUTHZPATHSMAPPING_PATHS` (`AUTHZ_OBJ_ID_OID`,`PATHS`) VALUES (?,?)
> FailedObject:MSentryPathsUpdate [authzObj=default.web_logs],
> paths=[[user/admin/2015_11_21, user/admin/2015_11_20,
> user/hive/warehouse/web_logs, user/admin/2015_11_18, user/admin/2015_11_19]],
> createTimeMs=[1486969386906]
> at
> org.datanucleus.api.jdo.NucleusJDOHelper.getJDOExceptionForNucleusException(NucleusJDOHelper.java:449)
> at
> org.datanucleus.api.jdo.JDOPersistenceManager.jdoMakePersistent(JDOPersistenceManager.java:732)
> at
> org.datanucleus.api.jdo.JDOPersistenceManager.makePersistent(JDOPersistenceManager.java:752)
> at
> org.apache.sentry.provider.db.service.persistent.SentryStore.createAuthzPathsMappingCore(SentryStore.java:2200)
> at
> org.apache.sentry.provider.db.service.persistent.SentryStore.access$2500(SentryStore.java:88)
> at
> org.apache.sentry.provider.db.service.persistent.SentryStore$31.execute(SentryStore.java:2174)
> at
> org.apache.sentry.provider.db.service.persistent.TransactionManager.executeTransaction(TransactionManager.java:114)
> at
> org.apache.sentry.provider.db.service.persistent.TransactionManager.executeTransactionWithRetry(TransactionManager.java:183)
> at
> org.apache.sentry.provider.db.service.persistent.SentryStore.persistFullPathsImage(SentryStore.java:2170)
> at
> org.apache.sentry.service.thrift.HMSFollower.run(HMSFollower.java:263)
> at
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
> at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:304)
> at
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:178)
> at
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
> at java.lang.Thread.run(Thread.java:745)
> NestedThrowablesStackTrace:
> org.datanucleus.store.rdbms.exceptions.MappedDatastoreException: INSERT INTO
> `MAUTHZPATHSMAPPING_PATHS` (`AUTHZ_OBJ_ID_OID`,`PATHS`) VALUES (?,?)
> at
> org.datanucleus.store.rdbms.scostore.JoinSetStore.doInternalAdd(JoinSetStore.java:754)
> at
> org.datanucleus.store.rdbms.scostore.JoinSetStore.internalAdd(JoinSetStore.java:502)
> at
> org.datanucleus.store.rdbms.scostore.JoinSetStore.addAll(JoinSetStore.java:433)
> at
> org.datanucleus.store.rdbms.mapping.java.CollectionMapping.postInsert(CollectionMapping.java:136)
> at
> org.datanucleus.store.rdbms.request.InsertRequest.execute(InsertRequest.java:519)
> at
> org.datanucleus.store.rdbms.RDBMSPersistenceHandler.insertTable(RDBMSPersistenceHandler.java:167)
> at
> org.datanucleus.store.rdbms.RDBMSPersistenceHandler.insertObject(RDBMSPersistenceHandler.java:143)
> at
> org.datanucleus.state.JDOStateManager.internalMakePersistent(JDOStateManager.java:3784)
> at
> org.datanucleus.state.JDOStateManager.makePersistent(JDOStateManager.java:3760)
> at
> org.datanucleus.ExecutionContextImpl.persistObjectInternal(ExecutionContextImpl.java:2219)
> at
> org.datanucleus.ExecutionContextImpl.persistObjectWork(ExecutionContextImpl.java:2065)
> at
>
[jira] [Commented] (SENTRY-1640) Implement HMS Notification barrier on the HMS plugin side
[ https://issues.apache.org/jira/browse/SENTRY-1640?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15964831#comment-15964831 ] Sergio Peña commented on SENTRY-1640: - This issue requires a newer version of Hive that has support for the new ListenerEvent API that gets the event id. Hive 2.3 is the version that this API is included, but it is not released yet. See HIVE-16164 I will attach a patch in the meantime for code review, and when we bump our Hive dependency, then we can commit it. > Implement HMS Notification barrier on the HMS plugin side > - > > Key: SENTRY-1640 > URL: https://issues.apache.org/jira/browse/SENTRY-1640 > Project: Sentry > Issue Type: Sub-task > Components: Sentry >Affects Versions: sentry-ha-redesign >Reporter: Alexander Kolbasov >Assignee: Sergio Peña >Priority: Critical > Fix For: sentry-ha-redesign > > > This covers HMS plugin work for the notification barrier code from > SENTRY-1600 and SENTRY-1601 -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (SENTRY-1643) AutoIncrement ChangeID of MSentryPermChange/MSentryPathChange may be error-prone
[
https://issues.apache.org/jira/browse/SENTRY-1643?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15964823#comment-15964823
]
Hadoop QA commented on SENTRY-1643:
---
Here are the results of testing the latest attachment
https://issues.apache.org/jira/secure/attachment/12862753/SENTRY-1643.02-sentry-ha-redesign.patch
against sentry-ha-redesign.
{color:green}Overall:{color} +1 all checks pass
{color:green}SUCCESS:{color} all tests passed
Console output:
https://builds.apache.org/job/PreCommit-SENTRY-Build/2497/console
This message is automatically generated.
> AutoIncrement ChangeID of MSentryPermChange/MSentryPathChange may be
> error-prone
>
>
> Key: SENTRY-1643
> URL: https://issues.apache.org/jira/browse/SENTRY-1643
> Project: Sentry
> Issue Type: Sub-task
> Components: Hdfs Plugin
>Affects Versions: sentry-ha-redesign
>Reporter: Hao Hao
>Assignee: Lei (Eddy) Xu
>Priority: Critical
> Fix For: sentry-ha-redesign
>
> Attachments: SENTRY-1643.00-sentry-ha-redesign.patch,
> SENTRY-1643.01-sentry-ha-redesign.patch,
> SENTRY-1643.02-sentry-ha-redesign.patch
>
>
> In MSentryPermChange/MSentryPathChange table, the changeID field is
> auto-increment.
> {noformat}
>
>
> {noformat}
> However, found through the integration unit test TestHDFSIntegration, the
> value does not seem to be correctly auto increased. e.g Instead of increasing
> by one for each new entry, it increased by some unexpected number.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Created] (SENTRY-1702) Revoke on Server Causes Broken URI Privilege
Johndee Burks created SENTRY-1702:
-
Summary: Revoke on Server Causes Broken URI Privilege
Key: SENTRY-1702
URL: https://issues.apache.org/jira/browse/SENTRY-1702
Project: Sentry
Issue Type: Bug
Components: Sentry
Environment: CDH5.9
Reporter: Johndee Burks
== Issue ==
SENTRY-281 can create a situation in which a URI privilege is not removable
using revoke.
== Reproduction Steps ==
If you do the following you end up with a privilege that cannot be revoked on a
URI.
1. Create Role and Grant all on server:
{code}
0: jdbc:hive2://jreposec-1.gce.cloudera.com:1> create role turi;
0: jdbc:hive2://jreposec-1.gce.cloudera.com:1> grant all on server server1 to
role turi;
0: jdbc:hive2://jreposec-1.gce.cloudera.com:1> show grant role turi;
+---+++-+-+-++---+---+--+--+
| database | table | partition | column | principal_name | principal_type
| privilege | grant_option |grant_time | grantor |
+---+++-+-+-++---+---+--+--+
| * ||| | turi| ROLE
| * | false | 1486508699269000 | -- |
+---+++-+-+-++---+---+--+--+
{code}
2. Grant all on URI:
{code}
0: jdbc:hive2://jreposec-1.gce.cloudera.com:1> grant all on uri
"hdfs://jreposec-1.gce.cloudera.com:8020/tmp" to role turi;
0: jdbc:hive2://jreposec-1.gce.cloudera.com:1> show grant role turi;
+--+++-+-+-++---+---+--+--+
| database | table | partition | column
| principal_name | principal_type | privilege | grant_option |
grant_time | grantor |
+--+++-+-+-++---+---+--+--+
| *|||
| turi| ROLE| * | false |
1486508699269000 | -- |
| hdfs://jreposec-1.gce.cloudera.com:8020/tmp |||
| turi| ROLE| * | false |
1491867083637000 | -- |
+--+++-+-+-++---+---+--+--+
{code}
3. Now revoke insert from that role on server
{code}
0: jdbc:hive2://jreposec-1.gce.cloudera.com:1> revoke insert on server server1
from role turi;
0: jdbc:hive2://jreposec-1.gce.cloudera.com:1> show grant role turi;
+--+++-+-+-++---+---+--+--+
| database | table | partition | column
| principal_name | principal_type | privilege | grant_option |
grant_time | grantor |
+--+++-+-+-++---+---+--+--+
| *|||
| turi| ROLE| select | false |
1491867142657000 | -- |
| hdfs://jreposec-1.gce.cloudera.com:8020/tmp |||
| turi| ROLE| select | false |
1491867142646000 | -- |
+--+++-+-+-++---+---+--+--+
{code}
4. Attempt to revoke the URI.
{code}
0: jdbc:hive2://jreposec-1.gce.cloudera.com:1> revoke all on uri
"hdfs://jreposec-1.gce.cloudera.com:8020/tmp" from role turi;
0: jdbc:hive2://jreposec-1.gce.cloudera.com:1> revoke select on uri
"hdfs://jreposec-1.gce.cloudera.com:8020/tmp" from role turi;
0: jdbc:hive2://jreposec-1.gce.cloudera.com:1> revoke insert on uri
"hdfs://jreposec-1.gce.cloudera.com:8020/tmp" from role turi;
0: jdbc:hive2://jreposec-1.gce.cloudera.com:1> show grant role turi;
+--+++-+-+-++---+---+--+--+
| database | table | partition | column
|
[jira] [Commented] (SENTRY-1643) AutoIncrement ChangeID of MSentryPermChange/MSentryPathChange may be error-prone
[
https://issues.apache.org/jira/browse/SENTRY-1643?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15964745#comment-15964745
]
Hao Hao commented on SENTRY-1643:
-
[~kkalyan] unfortunately I did not have a good idea how to solve it in the way
you suggest, otherwise would comment it in the first place. Please let me know
if you have something in mind after you get more familiar with namenode
plugin-in part. Thanks!
> AutoIncrement ChangeID of MSentryPermChange/MSentryPathChange may be
> error-prone
>
>
> Key: SENTRY-1643
> URL: https://issues.apache.org/jira/browse/SENTRY-1643
> Project: Sentry
> Issue Type: Sub-task
> Components: Hdfs Plugin
>Affects Versions: sentry-ha-redesign
>Reporter: Hao Hao
>Assignee: Lei (Eddy) Xu
>Priority: Critical
> Fix For: sentry-ha-redesign
>
> Attachments: SENTRY-1643.00-sentry-ha-redesign.patch,
> SENTRY-1643.01-sentry-ha-redesign.patch,
> SENTRY-1643.02-sentry-ha-redesign.patch
>
>
> In MSentryPermChange/MSentryPathChange table, the changeID field is
> auto-increment.
> {noformat}
>
>
> {noformat}
> However, found through the integration unit test TestHDFSIntegration, the
> value does not seem to be correctly auto increased. e.g Instead of increasing
> by one for each new entry, it increased by some unexpected number.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Commented] (SENTRY-1643) AutoIncrement ChangeID of MSentryPermChange/MSentryPathChange may be error-prone
[
https://issues.apache.org/jira/browse/SENTRY-1643?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15964734#comment-15964734
]
kalyan kumar kalvagadda commented on SENTRY-1643:
-
[~hahao] I don't have a solution in mind. I never looked deeper into the
namenode plug-in code so could not comment if we could make namenode plug-in
work with monotonically increasing change ID's. Wanted to hear from you.
> AutoIncrement ChangeID of MSentryPermChange/MSentryPathChange may be
> error-prone
>
>
> Key: SENTRY-1643
> URL: https://issues.apache.org/jira/browse/SENTRY-1643
> Project: Sentry
> Issue Type: Sub-task
> Components: Hdfs Plugin
>Affects Versions: sentry-ha-redesign
>Reporter: Hao Hao
>Assignee: Lei (Eddy) Xu
>Priority: Critical
> Fix For: sentry-ha-redesign
>
> Attachments: SENTRY-1643.00-sentry-ha-redesign.patch,
> SENTRY-1643.01-sentry-ha-redesign.patch,
> SENTRY-1643.02-sentry-ha-redesign.patch
>
>
> In MSentryPermChange/MSentryPathChange table, the changeID field is
> auto-increment.
> {noformat}
>
>
> {noformat}
> However, found through the integration unit test TestHDFSIntegration, the
> value does not seem to be correctly auto increased. e.g Instead of increasing
> by one for each new entry, it increased by some unexpected number.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Commented] (SENTRY-1643) AutoIncrement ChangeID of MSentryPermChange/MSentryPathChange may be error-prone
[
https://issues.apache.org/jira/browse/SENTRY-1643?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15964728#comment-15964728
]
Hao Hao commented on SENTRY-1643:
-
[~kkalyan] Yeah, this is only one way to solve it. We can benchmark the
performance if we continue on this solution. And see if the performance is
really bad. Do you have another concrete solution in mind?
> AutoIncrement ChangeID of MSentryPermChange/MSentryPathChange may be
> error-prone
>
>
> Key: SENTRY-1643
> URL: https://issues.apache.org/jira/browse/SENTRY-1643
> Project: Sentry
> Issue Type: Sub-task
> Components: Hdfs Plugin
>Affects Versions: sentry-ha-redesign
>Reporter: Hao Hao
>Assignee: Lei (Eddy) Xu
>Priority: Critical
> Fix For: sentry-ha-redesign
>
> Attachments: SENTRY-1643.00-sentry-ha-redesign.patch,
> SENTRY-1643.01-sentry-ha-redesign.patch,
> SENTRY-1643.02-sentry-ha-redesign.patch
>
>
> In MSentryPermChange/MSentryPathChange table, the changeID field is
> auto-increment.
> {noformat}
>
>
> {noformat}
> However, found through the integration unit test TestHDFSIntegration, the
> value does not seem to be correctly auto increased. e.g Instead of increasing
> by one for each new entry, it increased by some unexpected number.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Updated] (SENTRY-1643) AutoIncrement ChangeID of MSentryPermChange/MSentryPathChange may be error-prone
[
https://issues.apache.org/jira/browse/SENTRY-1643?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Hao Hao updated SENTRY-1643:
Status: Patch Available (was: Open)
> AutoIncrement ChangeID of MSentryPermChange/MSentryPathChange may be
> error-prone
>
>
> Key: SENTRY-1643
> URL: https://issues.apache.org/jira/browse/SENTRY-1643
> Project: Sentry
> Issue Type: Sub-task
> Components: Hdfs Plugin
>Affects Versions: sentry-ha-redesign
>Reporter: Hao Hao
>Assignee: Lei (Eddy) Xu
>Priority: Critical
> Fix For: sentry-ha-redesign
>
> Attachments: SENTRY-1643.00-sentry-ha-redesign.patch,
> SENTRY-1643.01-sentry-ha-redesign.patch,
> SENTRY-1643.02-sentry-ha-redesign.patch
>
>
> In MSentryPermChange/MSentryPathChange table, the changeID field is
> auto-increment.
> {noformat}
>
>
> {noformat}
> However, found through the integration unit test TestHDFSIntegration, the
> value does not seem to be correctly auto increased. e.g Instead of increasing
> by one for each new entry, it increased by some unexpected number.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Updated] (SENTRY-1323) Bump the hive version to 1.2.0
[ https://issues.apache.org/jira/browse/SENTRY-1323?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mathew Crocker updated SENTRY-1323: --- Fix Version/s: (was: sentry-ha-redesign) > Bump the hive version to 1.2.0 > -- > > Key: SENTRY-1323 > URL: https://issues.apache.org/jira/browse/SENTRY-1323 > Project: Sentry > Issue Type: Sub-task > Components: Hive V2 >Reporter: Sravya Tirukkovalur >Assignee: Sergio Peña > Labels: masking > Attachments: SENTRY-1323.0.patch > > > Deserializer as part of Hive-7973 work has some bugs in 1.1.0. For example: > deserializer.getAlterTableMessage fails with JsonMappingException > After some debugging, I confirmed that it is due to the fact that > JSONAlterTableMessage does not have a default constructor. > Seems like this is fixed as part of HIVE-10227 (1.2.0). So would be best to > move to hive 1.2.0. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Comment Edited] (SENTRY-1323) Bump the hive version to 1.2.0
[ https://issues.apache.org/jira/browse/SENTRY-1323?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15964528#comment-15964528 ] Mathew Crocker edited comment on SENTRY-1323 at 4/11/17 3:37 PM: - talked about this at length with [~spena] and [~kkalyan] , this is not tied to the sentry ha efforts. This task enables column masking, and row filtering. We should strongly consider this as apart of the HIVE V2 Migration , as the work to use these additional fields is integrated only in HIVE 2.x was (Author: mat.crocker): talked about this at length with [~spena] and [~kkalyan] , this is not tied to the sentry ha efforts. This task enables column masking, and row filtering. > Bump the hive version to 1.2.0 > -- > > Key: SENTRY-1323 > URL: https://issues.apache.org/jira/browse/SENTRY-1323 > Project: Sentry > Issue Type: Sub-task > Components: Hive V2 >Reporter: Sravya Tirukkovalur >Assignee: Sergio Peña > Labels: masking > Attachments: SENTRY-1323.0.patch > > > Deserializer as part of Hive-7973 work has some bugs in 1.1.0. For example: > deserializer.getAlterTableMessage fails with JsonMappingException > After some debugging, I confirmed that it is due to the fact that > JSONAlterTableMessage does not have a default constructor. > Seems like this is fixed as part of HIVE-10227 (1.2.0). So would be best to > move to hive 1.2.0. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Updated] (SENTRY-1323) Bump the hive version to 1.2.0
[ https://issues.apache.org/jira/browse/SENTRY-1323?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mathew Crocker updated SENTRY-1323: --- Component/s: Hive V2 > Bump the hive version to 1.2.0 > -- > > Key: SENTRY-1323 > URL: https://issues.apache.org/jira/browse/SENTRY-1323 > Project: Sentry > Issue Type: Sub-task > Components: Hive V2 >Reporter: Sravya Tirukkovalur >Assignee: Sergio Peña > Labels: masking > Attachments: SENTRY-1323.0.patch > > > Deserializer as part of Hive-7973 work has some bugs in 1.1.0. For example: > deserializer.getAlterTableMessage fails with JsonMappingException > After some debugging, I confirmed that it is due to the fact that > JSONAlterTableMessage does not have a default constructor. > Seems like this is fixed as part of HIVE-10227 (1.2.0). So would be best to > move to hive 1.2.0. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (SENTRY-1323) Bump the hive version to 1.2.0
[ https://issues.apache.org/jira/browse/SENTRY-1323?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15964528#comment-15964528 ] Mathew Crocker commented on SENTRY-1323: talked about this at length with [~spena] and [~kkalyan] , this is not tied to the sentry ha efforts. This task enables column masking, and row filtering. > Bump the hive version to 1.2.0 > -- > > Key: SENTRY-1323 > URL: https://issues.apache.org/jira/browse/SENTRY-1323 > Project: Sentry > Issue Type: Sub-task >Reporter: Sravya Tirukkovalur >Assignee: Sergio Peña > Fix For: sentry-ha-redesign > > Attachments: SENTRY-1323.0.patch > > > Deserializer as part of Hive-7973 work has some bugs in 1.1.0. For example: > deserializer.getAlterTableMessage fails with JsonMappingException > After some debugging, I confirmed that it is due to the fact that > JSONAlterTableMessage does not have a default constructor. > Seems like this is fixed as part of HIVE-10227 (1.2.0). So would be best to > move to hive 1.2.0. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Updated] (SENTRY-1649) Initialize HMSFollower when sentry server actually starts
[ https://issues.apache.org/jira/browse/SENTRY-1649?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Na Li updated SENTRY-1649: -- Attachment: SENTRY-1649.004-sentry-ha-redesign.patch fix the build error by including all changes in a single patch file. > Initialize HMSFollower when sentry server actually starts > - > > Key: SENTRY-1649 > URL: https://issues.apache.org/jira/browse/SENTRY-1649 > Project: Sentry > Issue Type: Sub-task > Components: Hdfs Plugin >Affects Versions: sentry-ha-redesign >Reporter: Hao Hao >Assignee: Na Li >Priority: Critical > Fix For: sentry-ha-redesign > > Attachments: SENTRY-1649.001-sentry-ha-redesign.patch, > SENTRY-1649.002-sentry-ha-redesign.patch, > SENTRY-1649.003-sentry-ha-redesign.patch, > SENTRY-1649.004-sentry-ha-redesign.patch > > > Now HMSFollower has been initialized at the constructor of SentryService. It > would be better to initialize it when the service starts, e.g runServer(). -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (SENTRY-1649) Initialize HMSFollower when sentry server actually starts
[ https://issues.apache.org/jira/browse/SENTRY-1649?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15964409#comment-15964409 ] Na Li commented on SENTRY-1649: --- My latest update does not destroy one instance. You can take a look at "https://reviews.apache.org/r/58221/diff/3#1; sentry/service/thrift/SentryService.java. It is created in SentryService constructor. > Initialize HMSFollower when sentry server actually starts > - > > Key: SENTRY-1649 > URL: https://issues.apache.org/jira/browse/SENTRY-1649 > Project: Sentry > Issue Type: Sub-task > Components: Hdfs Plugin >Affects Versions: sentry-ha-redesign >Reporter: Hao Hao >Assignee: Na Li >Priority: Critical > Fix For: sentry-ha-redesign > > Attachments: SENTRY-1649.001-sentry-ha-redesign.patch, > SENTRY-1649.002-sentry-ha-redesign.patch, > SENTRY-1649.003-sentry-ha-redesign.patch > > > Now HMSFollower has been initialized at the constructor of SentryService. It > would be better to initialize it when the service starts, e.g runServer(). -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (SENTRY-1697) Deprecate feature flag for enabling notification log
[
https://issues.apache.org/jira/browse/SENTRY-1697?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15963946#comment-15963946
]
Hadoop QA commented on SENTRY-1697:
---
Here are the results of testing the latest attachment
https://issues.apache.org/jira/secure/attachment/12862784/SENTRY-1697.001-sentry-ha-redesign.patch
against sentry-ha-redesign.
{color:red}Overall:{color} -1 due to 5 errors
{color:red}ERROR:{color} mvn test exited 1
{color:red}ERROR:{color} Failed:
org.apache.sentry.tests.e2e.dbprovider.TestDbPrivilegeCleanupOnDrop
{color:red}ERROR:{color} Failed:
org.apache.sentry.tests.e2e.dbprovider.TestDbPrivilegeCleanupOnDrop
{color:red}ERROR:{color} Failed:
org.apache.sentry.tests.e2e.dbprovider.TestDbPrivilegeCleanupOnDrop
{color:red}ERROR:{color} Failed:
org.apache.sentry.tests.e2e.dbprovider.TestDbPrivilegeCleanupOnDrop
Console output:
https://builds.apache.org/job/PreCommit-SENTRY-Build/2495/console
This message is automatically generated.
> Deprecate feature flag for enabling notification log
>
>
> Key: SENTRY-1697
> URL: https://issues.apache.org/jira/browse/SENTRY-1697
> Project: Sentry
> Issue Type: Sub-task
> Components: Sentry
>Affects Versions: sentry-ha-redesign
>Reporter: Alexander Kolbasov
>Assignee: Alexander Kolbasov
> Attachments: SENTRY-1697.001-sentry-ha-redesign.patch
>
>
> We tried to use the feature flag to support old way of talking between HMS
> and Sentry but it isn't quite supported and it isn't trivial to make it work,
> so we should deprecate the feature flag.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
