Sorry for my late answer, I had to work on a different problem.
In the meantime, I realized that I am using anger 0.4, and not Ranger 0.5.
So this problem may have been solved in Ranger 0.5.
Here is all the the logs I get when my user toto tries to access chaneldb
on which he should have permission to read because he belongs to the group
sysadmin which has all the rights (including admin) on the database :
2015-07-30 11:50:49,891 INFO [HiveServer2-Handler-Pool: Thread-48]:
parse.ParseDriver (ParseDriver.java:parse(185)) - Parsing command: use
chaneldb
2015-07-30 11:50:50,295 INFO [HiveServer2-Handler-Pool: Thread-48]:
parse.ParseDriver (ParseDriver.java:parse(206)) - Parse Completed
2015-07-30 11:50:50,297 INFO [HiveServer2-Handler-Pool: Thread-48]:
log.PerfLogger (PerfLogger.java:PerfLogEnd(135)) - </PERFLOG method=parse
start=1438249849885 end=1438249850297 duration=412
from=org.apache.hadoop.hive.ql.Driver>
2015-07-30 11:50:50,302 INFO [HiveServer2-Handler-Pool: Thread-48]:
log.PerfLogger (PerfLogger.java:PerfLogBegin(108)) - <PERFLOG
method=semanticAnalyze from=org.apache.hadoop.hive.ql.Driver>
2015-07-30 11:50:50,347 INFO [HiveServer2-Handler-Pool: Thread-48]:
metastore.HiveMetaStore (HiveMetaStore.java:logInfo(714)) - 2:
get_database: chaneldb
2015-07-30 11:50:50,347 INFO [HiveServer2-Handler-Pool: Thread-48]:
HiveMetaStore.audit (HiveMetaStore.java:logAuditEvent(340)) - ugi=toto
ip=unknown-ip-addr cmd=get_database: chaneldb
2015-07-30 11:50:50,348 INFO [HiveServer2-Handler-Pool: Thread-48]:
metastore.HiveMetaStore (HiveMetaStore.java:newRawStore(557)) - 2: Opening
raw store with implemenation
class:org.apache.hadoop.hive.metastore.ObjectStore
2015-07-30 11:50:50,350 INFO [HiveServer2-Handler-Pool: Thread-48]:
metastore.ObjectStore (ObjectStore.java:initialize(262)) - ObjectStore,
initialize called
2015-07-30 11:50:50,371 INFO [HiveServer2-Handler-Pool: Thread-48]:
metastore.MetaStoreDirectSql (MetaStoreDirectSql.java:<init>(131)) - Using
direct SQL, underlying DB is MYSQL
2015-07-30 11:50:50,371 INFO [HiveServer2-Handler-Pool: Thread-48]:
metastore.ObjectStore (ObjectStore.java:setConf(245)) - Initialized
ObjectStore
2015-07-30 11:50:50,391 INFO [HiveServer2-Handler-Pool: Thread-48]:
metadata.HiveUtils
(HiveUtils.java:getMetaStoreAuthorizeProviderManagers(353)) - Adding
metastore authorization provider:
org.apache.hadoop.hive.ql.security.authorization.StorageBasedAuthorizationProvider
2015-07-30 11:50:50,395 INFO [HiveServer2-Handler-Pool: Thread-48]:
metadata.HiveUtils
(HiveUtils.java:getMetaStoreAuthorizeProviderManagers(353)) - Adding
metastore authorization provider:
org.apache.hadoop.hive.ql.security.authorization.MetaStoreAuthzAPIAuthorizerEmbedOnly
2015-07-30 11:50:50,427 INFO [HiveServer2-Handler-Pool: Thread-48]:
ql.Driver (Driver.java:compile(429)) - Semantic Analysis Completed
2015-07-30 11:50:50,427 INFO [HiveServer2-Handler-Pool: Thread-48]:
log.PerfLogger (PerfLogger.java:PerfLogEnd(135)) - </PERFLOG
method=semanticAnalyze start=1438249850302 end=1438249850427 duration=125
from=org.apache.hadoop.hive.ql.Driver>
2015-07-30 11:50:50,440 INFO [HiveServer2-Handler-Pool: Thread-48]:
ql.Driver (Driver.java:getSchema(237)) - Returning Hive schema:
Schema(fieldSchemas:null, properties:null)
2015-07-30 11:50:50,440 INFO [HiveServer2-Handler-Pool: Thread-48]:
log.PerfLogger (PerfLogger.java:PerfLogBegin(108)) - <PERFLOG
method=doAuthorization from=org.apache.hadoop.hive.ql.Driver>
2015-07-30 11:50:50,486 INFO [HiveServer2-Handler-Pool: Thread-48]:
log.PerfLogger (PerfLogger.java:PerfLogEnd(135)) - </PERFLOG
method=doAuthorization start=1438249850440 end=1438249850486 duration=46
from=org.apache.hadoop.hive.ql.Driver>
==> /var/log/hive/hive-server2.log <==
FAILED: HiveAccessControlException Permission denied: user [toto] does not
have [USE] privilege on [chaneldb]
==> /var/log/hive/hiveserver2.log <==
2015-07-30 11:50:50,487 ERROR [HiveServer2-Handler-Pool: Thread-48]:
ql.Driver (SessionState.java:printError(833)) - FAILED:
HiveAccessControlException Permission denied: user [toto] does not have
[USE] privilege on [chaneldb]
org.apache.hadoop.hive.ql.security.authorization.plugin.HiveAccessControlException:
Permission denied: user [toto] does not have [USE] privilege on [chaneldb]
at
com.xasecure.authorization.hive.authorizer.XaSecureHiveAuthorizer.checkPrivileges(XaSecureHiveAuthorizer.java:254)
at
org.apache.hadoop.hive.ql.Driver.doAuthorizationV2(Driver.java:727)
at org.apache.hadoop.hive.ql.Driver.doAuthorization(Driver.java:520)
at org.apache.hadoop.hive.ql.Driver.compile(Driver.java:457)
at org.apache.hadoop.hive.ql.Driver.compile(Driver.java:305)
at
org.apache.hadoop.hive.ql.Driver.compileInternal(Driver.java:1069)
at
org.apache.hadoop.hive.ql.Driver.compileAndRespond(Driver.java:1063)
at
org.apache.hive.service.cli.operation.SQLOperation.prepare(SQLOperation.java:109)
at
org.apache.hive.service.cli.operation.SQLOperation.runInternal(SQLOperation.java:180)
at
org.apache.hive.service.cli.operation.Operation.run(Operation.java:256)
at
org.apache.hive.service.cli.session.HiveSessionImpl.executeStatementInternal(HiveSessionImpl.java:376)
at
org.apache.hive.service.cli.session.HiveSessionImpl.executeStatementAsync(HiveSessionImpl.java:363)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source)
at java.lang.reflect.Method.invoke(Unknown Source)
at
org.apache.hive.service.cli.session.HiveSessionProxy.invoke(HiveSessionProxy.java:79)
at
org.apache.hive.service.cli.session.HiveSessionProxy.access$000(HiveSessionProxy.java:37)
at
org.apache.hive.service.cli.session.HiveSessionProxy$1.run(HiveSessionProxy.java:64)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Unknown Source)
at
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1628)
at
org.apache.hadoop.hive.shims.HadoopShimsSecure.doAs(HadoopShimsSecure.java:536)
at
org.apache.hive.service.cli.session.HiveSessionProxy.invoke(HiveSessionProxy.java:60)
at com.sun.proxy.$Proxy28.executeStatementAsync(Unknown Source)
at
org.apache.hive.service.cli.CLIService.executeStatementAsync(CLIService.java:270)
at
org.apache.hive.service.cli.thrift.ThriftCLIService.ExecuteStatement(ThriftCLIService.java:401)
at
org.apache.hive.service.cli.thrift.TCLIService$Processor$ExecuteStatement.getResult(TCLIService.java:1313)
at
org.apache.hive.service.cli.thrift.TCLIService$Processor$ExecuteStatement.getResult(TCLIService.java:1298)
at
org.apache.thrift.ProcessFunction.process(ProcessFunction.java:39)
at org.apache.thrift.TBaseProcessor.process(TBaseProcessor.java:39)
at
org.apache.hive.service.auth.TSetIpAddressProcessor.process(TSetIpAddressProcessor.java:56)
at
org.apache.thrift.server.TThreadPoolServer$WorkerProcess.run(TThreadPoolServer.java:206)
at java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown
Source)
at java.lang.Thread.run(Unknown Source)
2015-07-30 11:50:50,488 INFO [HiveServer2-Handler-Pool: Thread-48]:
log.PerfLogger (PerfLogger.java:PerfLogEnd(135)) - </PERFLOG method=compile
start=1438249849844 end=1438249850488 duration=644
from=org.apache.hadoop.hive.ql.Driver>
2015-07-30 11:50:50,488 INFO [HiveServer2-Handler-Pool: Thread-48]:
log.PerfLogger (PerfLogger.java:PerfLogBegin(108)) - <PERFLOG
method=releaseLocks from=org.apache.hadoop.hive.ql.Driver>
2015-07-30 11:50:50,488 INFO [HiveServer2-Handler-Pool: Thread-48]:
log.PerfLogger (PerfLogger.java:PerfLogEnd(135)) - </PERFLOG
method=releaseLocks start=1438249850488 end=1438249850488 duration=0
from=org.apache.hadoop.hive.ql.Driver>
2015-07-30 11:50:50,490 WARN [HiveServer2-Handler-Pool: Thread-48]:
thrift.ThriftCLIService (ThriftCLIService.java:ExecuteStatement(407)) -
Error executing statement:
org.apache.hive.service.cli.HiveSQLException: Error while compiling
statement: FAILED: HiveAccessControlException Permission denied: user
[toto] does not have [USE] privilege on [chaneldb]
at
org.apache.hive.service.cli.operation.Operation.toSQLException(Operation.java:314)
at
org.apache.hive.service.cli.operation.SQLOperation.prepare(SQLOperation.java:111)
at
org.apache.hive.service.cli.operation.SQLOperation.runInternal(SQLOperation.java:180)
at
org.apache.hive.service.cli.operation.Operation.run(Operation.java:256)
at
org.apache.hive.service.cli.session.HiveSessionImpl.executeStatementInternal(HiveSessionImpl.java:376)
at
org.apache.hive.service.cli.session.HiveSessionImpl.executeStatementAsync(HiveSessionImpl.java:363)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source)
at java.lang.reflect.Method.invoke(Unknown Source)
at
org.apache.hive.service.cli.session.HiveSessionProxy.invoke(HiveSessionProxy.java:79)
at
org.apache.hive.service.cli.session.HiveSessionProxy.access$000(HiveSessionProxy.java:37)
at
org.apache.hive.service.cli.session.HiveSessionProxy$1.run(HiveSessionProxy.java:64)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Unknown Source)
at
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1628)
at
org.apache.hadoop.hive.shims.HadoopShimsSecure.doAs(HadoopShimsSecure.java:536)
at
org.apache.hive.service.cli.session.HiveSessionProxy.invoke(HiveSessionProxy.java:60)
at com.sun.proxy.$Proxy28.executeStatementAsync(Unknown Source)
at
org.apache.hive.service.cli.CLIService.executeStatementAsync(CLIService.java:270)
at
org.apache.hive.service.cli.thrift.ThriftCLIService.ExecuteStatement(ThriftCLIService.java:401)
at
org.apache.hive.service.cli.thrift.TCLIService$Processor$ExecuteStatement.getResult(TCLIService.java:1313)
at
org.apache.hive.service.cli.thrift.TCLIService$Processor$ExecuteStatement.getResult(TCLIService.java:1298)
at
org.apache.thrift.ProcessFunction.process(ProcessFunction.java:39)
at org.apache.thrift.TBaseProcessor.process(TBaseProcessor.java:39)
at
org.apache.hive.service.auth.TSetIpAddressProcessor.process(TSetIpAddressProcessor.java:56)
at
org.apache.thrift.server.TThreadPoolServer$WorkerProcess.run(TThreadPoolServer.java:206)
at java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown
Source)
at java.lang.Thread.run(Unknown Source)
Caused by:
org.apache.hadoop.hive.ql.security.authorization.plugin.HiveAccessControlException:
Permission denied: user [toto] does not have [USE] privilege on [chaneldb]
at
com.xasecure.authorization.hive.authorizer.XaSecureHiveAuthorizer.checkPrivileges(XaSecureHiveAuthorizer.java:254)
at
org.apache.hadoop.hive.ql.Driver.doAuthorizationV2(Driver.java:727)
at org.apache.hadoop.hive.ql.Driver.doAuthorization(Driver.java:520)
at org.apache.hadoop.hive.ql.Driver.compile(Driver.java:457)
at org.apache.hadoop.hive.ql.Driver.compile(Driver.java:305)
at
org.apache.hadoop.hive.ql.Driver.compileInternal(Driver.java:1069)
at
org.apache.hadoop.hive.ql.Driver.compileAndRespond(Driver.java:1063)
at
org.apache.hive.service.cli.operation.SQLOperation.prepare(SQLOperation.java:109)
... 28 more
And when I'm trying the command "groups" or even "hdfs groups" on the host
running HiveServer I get "toto : nobody UsrSysAdmin SysAdmin ..."
Do you or anyone else see where the problem might come from ?
Thanks in advance,
Loïc
Loïc CHANEL
Engineering student at TELECOM Nancy
Trainee at Worldline - Villeurbanne
2015-07-24 17:30 GMT+02:00 Alok Lal <[email protected]>:
> Perhaps. It is hard to say definitively without taking a look at the logs.
>
> From: Loïc Chanel <[email protected]>
> Reply-To: "[email protected]" <
> [email protected]>
> Date: Friday, July 24, 2015 at 8:10 AM
> To: "[email protected]" <[email protected]>
>
> Subject: Re: Hive server identity assertion
>
> Exactly !
>
> And I've checked the logs once again, but I can't see any groups
> mentioned. Does this reveal a special issue ?
>
> Thanks,
>
>
> Loïc
>
> Loïc CHANEL
> Engineering student at TELECOM Nancy
> Trainee at Worldline - Villeurbanne
>
> 2015-07-24 16:23 GMT+02:00 Alok Lal <[email protected]>:
>
>> If the user groups couldn't be asserted, would I see a log indicating
>> that the user cannot be impersonated (like Knox prompts) ?
>>
>> Yes log should show the user and group info being sent to policy engine.
>> For authorizing. I presume you are using ranger 0.5 to connect via beeline
>> to a hiveserver2 instance. Right? (Not that these matter, just to set
>> context.)
>>
>> Thanks
>>
>> From: Loïc Chanel <[email protected]>
>> Reply-To: "[email protected]" <
>> [email protected]>
>> Date: Friday, July 24, 2015 at 12:53 AM
>> To: "[email protected]" <[email protected]>
>> Subject: Re: Hive server identity assertion
>>
>> Well, that's what I thought, but the command hdfs groups returns me a
>> group that I use for a policy giving access to a database, and as I get the
>> message "HiveAccessControlException Permission denied" when accessing this
>> database, I think Hive cannot assert the groups the user belongs to.
>>
>> I'm using Hive 0.14.0.2.2.
>> As the problem might come from this, I think it's important to mention
>> that the users are synchronized from a LDAP via SSSD.
>>
>> If the user groups couldn't be asserted, would I see a log indicating
>> that the user cannot be impersonated (like Knox prompts) ?
>>
>> Thanks,
>>
>>
>> Loïc
>>
>> Loïc CHANEL
>> Engineering student at TELECOM Nancy
>> Trainee at Worldline - Villeurbanne
>>
>> 2015-07-23 20:09 GMT+02:00 Don Bosco Durai <[email protected]>:
>>
>>> Hive uses the same core-site.xml settings as HDFS. So if the group
>>> mapping work in HDFS, then it should work in Hive also.
>>>
>>> And if the user and groups are in linux/unix, then it should have been
>>> support out of the box.
>>>
>>> What version of Hive are you using? (It shouldn’t matter)
>>>
>>> Thanks
>>>
>>> Bosco
>>>
>>>
>>> From: Loïc Chanel <[email protected]>
>>> Reply-To: "[email protected]" <
>>> [email protected]>
>>> Date: Thursday, July 23, 2015 at 3:10 AM
>>> To: "[email protected]" <[email protected]
>>> >
>>> Subject: Hive server identity assertion
>>>
>>> Hi all,
>>>
>>> As I am now exploring how Ranger works with Hive, I made some policies,
>>> but it seems that group policies are not enforced.
>>> Therefore, I was wondering how the Ranger plugin running on Hive was
>>> asserting the user's identity.
>>>
>>> I am even more surprised by the fact that I do not have any problem with
>>> Ranger plugin working on HDFS, which is running on the exact same node.
>>>
>>> In parallel, I know that Know plugin, for example, runs in a totally
>>> different way, but as it seems that, as does HBase, Hive does not provide
>>> with any user mapping function, I thought the identity would be asserted on
>>> the node Hive Server is running on, as if the user was a Unix one.
>>>
>>> Do someone as an idea about how the user groups can be founded by Hive
>>> Ranger plugin ?
>>> Thanks in advance,
>>>
>>>
>>> Loïc
>>>
>>> Loïc CHANEL
>>> Engineering student at TELECOM Nancy
>>> Trainee at Worldline - Villeurbanne
>>>
>>>
>>
>
