At this point, I am starting to think Sentry does not support LDAP
Integration without Kerberos for HiveServer2 as I am not able to get it up
and keep on getting errors like below:
=========
2014-06-10 19:50:11,621 DEBUG transport.TSaslServerTransport
(TSaslServerTransport.java:getTransport(218)) - failed to open server
transport
org.apache.thrift.transport.TTransportException: Error validating the login
at
org.apache.thrift.transport.TSaslTransport.sendAndThrowMessage(TSaslTransport.java:221)
at
org.apache.thrift.transport.TSaslTransport.open(TSaslTransport.java:297)
at
org.apache.thrift.transport.TSaslServerTransport.open(TSaslServerTransport.java:1)
at
org.apache.thrift.transport.TSaslServerTransport$Factory.getTransport(TSaslServerTransport.java:216)
at
org.apache.thrift.server.TThreadPoolServer$WorkerProcess.run(TThreadPoolServer.java:227)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:722)
2014-06-10 19:50:11,621 DEBUG log.LogManager
(Log4jLoggerAdapter.java:debug(286)) - Operation log assocaited with
thread: pool-1-thread-2 couldn't be found.
2014-06-10 19:50:11,621 DEBUG log.LogDivertAppender
(Log4jLoggerAdapter.java:debug(286)) - ---+++=== Dropped log event from
thread pool-1-thread-2
2014-06-10 19:50:11,622 ERROR server.TThreadPoolServer
(TThreadPoolServer.java:run(253)) - Error occurred during processing of
message.
java.lang.RuntimeException:
org.apache.thrift.transport.TTransportException: Error validating the login
at
org.apache.thrift.transport.TSaslServerTransport$Factory.getTransport(TSaslServerTransport.java:219)
at
org.apache.thrift.server.TThreadPoolServer$WorkerProcess.run(TThreadPoolServer.java:227)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:722)
Caused by: org.apache.thrift.transport.TTransportException: Error
validating the login
at
org.apache.thrift.transport.TSaslTransport.sendAndThrowMessage(TSaslTransport.java:221)
at
org.apache.thrift.transport.TSaslTransport.open(TSaslTransport.java:297)
at
org.apache.thrift.transport.TSaslServerTransport.open(TSaslServerTransport.java:1)
at
org.apache.thrift.transport.TSaslServerTransport$Factory.getTransport(TSaslServerTransport.java:216)
... 4 more
================
It would be nice if Sentry team can confirm this and save me the agony of
keep on trying to configure something that is not even there :)
Thanks,
Anurag Tangri
On Tue, Jun 10, 2014 at 12:16 AM, Anurag Tangri <[email protected]> wrote:
> Hi Prasad,
> I tried it and the setting hive.server2.authentication.ldap.Domain does
> not change anything.
>
> Thanks,
> Anurag Tangri
>
>
> On Thu, Jun 5, 2014 at 10:45 AM, Anurag Tangri <[email protected]>
> wrote:
>
>> That's correct Prasad.
>>
>> Let me try this and see if it works.
>>
>>
>> Thanks,
>> Anurag Tangri
>>
>>
>> On Thu, Jun 5, 2014 at 10:35 AM, Prasad Mujumdar <[email protected]>
>> wrote:
>>
>>> Hi Anurag,
>>>
>>> If this is active directory, then try setting a config property
>>> hive.server2.authentication.ldap.Domain to your LDAP domain (a.b). Then you
>>> can login with the short userid atangri.
>>> Are you using LDAP group mapping in Hadoop as well ?
>>>
>>> thanks
>>> Prasad
>>>
>>>
>>>
>>> On Thu, Jun 5, 2014 at 7:43 AM, Anurag <[email protected]> wrote:
>>>
>>>>
>>>> Any response anyone?
>>>>
>>>> Sent from my iPhone
>>>> > On Jun 4, 2014, at 3:20 PM, Anurag Tangri <[email protected]>
>>>> wrote:
>>>> >
>>>> > + sentry mailing list
>>>> >
>>>> >
>>>> >> On Wed, Jun 4, 2014 at 12:27 AM, Anurag Tangri <[email protected]>
>>>> wrote:
>>>> >> So, I am working on setting this up.
>>>> >>
>>>> >> I have HDFS integrated with LDAP and working fine.
>>>> >>
>>>> >> I also added ldap params in hive-site along with sentry conf :
>>>> >>
>>>> >> ====
>>>> >> <property>
>>>> >> <name>hive.server2.authentication</name>
>>>> >> <value>LDAP</value>
>>>> >> </property>
>>>> >> <property>
>>>> >> <name>hive.server2.authentication.ldap.url</name>
>>>> >> <value>ldap://<ip></value>
>>>> >> </property>
>>>> >>
>>>> >> <property>
>>>> >> <name>hive.server2.authentication.ldap.baseDN</name>
>>>> >> <value>dc=a,dc=b</value>
>>>> >> </property>
>>>> >>
>>>> >> ======
>>>> >>
>>>> >> Now when I go to beeline, I type:
>>>> >>
>>>> >> /usr/local/lib/hive/bin/beeline -u jdbc:hive2://<host>:10000
>>>> >> Beeline version 0.10.0-cdh4.6.0 by Apache Hive
>>>> >> 0: jdbc:hive2://<host>:10000> !connect jdbc:hive2://<host>:10000
>>>> >>
>>>> >>
>>>> >> It asks for username and password. I give LDAP credentials as:
>>>> >>
>>>> >> Connecting to jdbc:hive2://<host>:10000
>>>> >> Enter username for jdbc:hive2://<host>:10000: [email protected]
>>>> >> Enter password for jdbc:hive2://<host>:10000: **********
>>>> >> Error: Invalid URL: jdbc:hive2://<host>:10000 (state=08S01,code=0)
>>>> >> 1: jdbc:hive2://<host>:10000>
>>>> >>
>>>> >>
>>>> >> Now if I type, hive queries, i see error in hive log:
>>>> >>
>>>> >> 2014-06-04 07:15:13,211 WARN file.HadoopGroupMappingService
>>>> (HadoopGroupMappingService.java:getGroups(42)) - Unable to obtain groups
>>>> for [email protected]
>>>> >> java.io.IOException: No groups found for user [email protected]
>>>> >> at org.apache.hadoop.security.Groups.getGroups(Groups.java:105)
>>>> >> at
>>>> org.apache.sentry.provider.file.HadoopGroupMappingService.getGroups(HadoopGroupMappingService.java:40)
>>>> >> at
>>>> org.apache.sentry.provider.file.ResourceAuthorizationProvider.doHasAccess(ResourceAuthorizationProvider.java:98)
>>>> >> at
>>>> org.apache.sentry.provider.file.ResourceAuthorizationProvider.hasAccess(ResourceAuthorizationProvider.java:93)
>>>> >> at
>>>> org.apache.sentry.binding.hive.authz.HiveAuthzBinding.authorize(HiveAuthzBinding.java:179)
>>>> >> at
>>>> org.apache.sentry.binding.hive.HiveAuthzBindingHook.filterShowTables(HiveAuthzBindingHook.java:561)
>>>> >> at
>>>> org.apache.sentry.binding.hive.HiveAuthzBindingHook.postDriverFetch(HiveAuthzBindingHook.java:634)
>>>> >> at
>>>> org.apache.hadoop.hive.ql.Driver.fireFilterHooks(Driver.java:1455)
>>>> >> at org.apache.hadoop.hive.ql.Driver.getResults(Driver.java:1475)
>>>> >> at
>>>> org.apache.hive.service.cli.operation.SQLOperation.getNextRowSet(SQLOperation.java:200)
>>>> >> at
>>>> org.apache.hive.service.cli.operation.OperationManager.getOperationNextRowSet(OperationManager.java:179)
>>>> >> at
>>>> org.apache.hive.service.cli.session.HiveSessionImpl.fetchResults(HiveSessionImpl.java:468)
>>>> >> at
>>>> org.apache.hive.service.cli.CLIService.fetchResults(CLIService.java:318)
>>>> >> at
>>>> org.apache.hive.service.cli.thrift.ThriftCLIService.FetchResults(ThriftCLIService.java:398)
>>>> >> at
>>>> org.apache.hive.service.cli.thrift.TCLIService$Processor$FetchResults.getResult(TCLIService.java:1613)
>>>> >> at
>>>> org.apache.hive.service.cli.thrift.TCLIService$Processor$FetchResults.getResult(TCLIService.java:1598)
>>>> >> at
>>>> org.apache.thrift.ProcessFunction.process(ProcessFunction.java:39)
>>>> >> at
>>>> org.apache.thrift.TBaseProcessor.process(TBaseProcessor.java:39)
>>>> >> at
>>>> org.apache.hive.service.cli.thrift.TSetIpAddressProcessor.process(TSetIpAddressProcessor.java:38)
>>>> >> at
>>>> org.apache.thrift.server.TThreadPoolServer$WorkerProcess.run(TThreadPoolServer.java:244)
>>>> >> at
>>>> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
>>>> >> at
>>>> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
>>>> >> at java.lang.Thread.run(Thread.java:722)
>>>> >> =======
>>>> >>
>>>> >>
>>>> >> I think the issue is my account on LDAP is atangri, not [email protected].
>>>> >>
>>>> >> Any thoughts on how to fix this ?
>>>> >>
>>>> >>
>>>> >> Thanks,
>>>> >> Anurag Tangri
>>>> >>
>>>> >>
>>>> >>
>>>> >>
>>>> >>
>>>> >>
>>>> >> =====
>>>> >> 2014-06-04 07:15:13,211 WARN file.HadoopGroupMappingService
>>>> (HadoopGroupMappingService.java:getGroups(42)) - Unable to obtain groups
>>>> for [email protected]
>>>> >> java.io.IOException: No groups found for user [email protected]
>>>> >>
>>>> >>
>>>> >>> On Tue, Jun 3, 2014 at 12:49 PM, Anurag <[email protected]>
>>>> wrote:
>>>> >>> Thanks Philippe!
>>>> >>>
>>>> >>> This answers almost all of my questions.
>>>> >>>
>>>> >>> Thanks,
>>>> >>> Anurag Tangri
>>>> >>>
>>>> >>>> On Jun 3, 2014, at 12:08 PM, Philippe Marseille <
>>>> [email protected]> wrote:
>>>> >>>>
>>>> >>>> I think the Hive CLI (Command line interface tool) is now
>>>> deprecated. see
>>>> http://blog.cloudera.com/blog/2014/02/migrating-from-hive-cli-to-beeline-a-primer/
>>>> >>>>
>>>> >>>>
>>>> >>>> The metastore has nothing to do with it. The Metastore Daemon is
>>>> used by all clients.
>>>> >>>>
>>>> >>>> The new flow is :
>>>> >>>>
>>>> >>>> Beeline -> HiveServer2 -> Metastore Daemon -> Metastore Database
>>>> (MySql for example).
>>>> >>>>
>>>> >>>> The previous flow was :
>>>> >>>>
>>>> >>>> Hive CLI (shell ) -> Metastore Daemon -> Metastore Database
>>>> >>>>
>>>> >>>> And before the Metastore Daemon ever existed, it was :
>>>> >>>>
>>>> >>>> Hive CLI -> Metastore Database
>>>> >>>>
>>>> >>>>> On Tuesday, 3 June 2014 13:06:22 UTC-4, Anurag Tangri wrote:
>>>> >>>>> Thanks Casey.
>>>> >>>>>
>>>> >>>>> This is going to be big shift to ask all our users to go to
>>>> beeline from hive.
>>>> >>>>>
>>>> >>>>> Is there no way to have consistency between what roles and
>>>> policies are seen in HUE via hiveserver2 and hive metastore on hive shell ?
>>>> >>>>> Also, is there a reason why hive metastore was excluded from
>>>> Sentry design ?
>>>> >>>>>
>>>> >>>>>
>>>> >>>>>
>>>> >>>>> Thanks,
>>>> >>>>> Anurag Tangri
>>>> >>>>>
>>>> >>>>>
>>>> >>>>>
>>>> >>>>>
>>>> >>>>>> On Tue, Jun 3, 2014 at 9:54 AM, Casey Brotherton <
>>>> [email protected]> wrote:
>>>> >>>>>> Hello Anurag,
>>>> >>>>>>
>>>> >>>>>> To use Sentry, as Johndee mentioned, you must use Hiveserver2.
>>>> That means you will need to transition users from the hive command line to
>>>> the beeline command line.
>>>> >>>>>>
>>>> >>>>>> This link has more information for CDH4.5
>>>> >>>>>>
>>>> http://www.cloudera.com/content/cloudera-content/cloudera-docs/CDH4/4.5.0/CDH4-Security-Guide/cdh4sg_Sentry.html
>>>> >>>>>>
>>>> >>>>>> There is a link to "Securing the Hive Metastore" which suggests
>>>> to limit connections using iptables, and granting of permissions.
>>>> >>>>>>
>>>> >>>>>> Hope this helps,
>>>> >>>>>> Casey
>>>> >>>>>>
>>>> >>>>>>
>>>> >>>>>>
>>>> >>>>>>> On Tue, Jun 3, 2014 at 11:45 AM, Anurag Tangri <
>>>> [email protected]> wrote:
>>>> >>>>>>> Hey Philippe,
>>>> >>>>>>> No, HUE should be fine as per Johndee's explanation as it can
>>>> be pointed to hiveserver2.
>>>> >>>>>>>
>>>> >>>>>>> By metastore, I mean mysql database that hosts hive metadata.
>>>> >>>>>>>
>>>> >>>>>>> This is the database that people go against when they login to
>>>> hive shell for a hadoop client box.
>>>> >>>>>>>
>>>> >>>>>>>
>>>> >>>>>>> Thanks,
>>>> >>>>>>> Anurag Tangri
>>>> >>>>>>>
>>>> >>>>>>>
>>>> >>>>>>>
>>>> >>>>>>>> On Tue, Jun 3, 2014 at 8:11 AM, Philippe Marseille <
>>>> [email protected]> wrote:
>>>> >>>>>>>> By "Hive Metastore", are you referring to the app in Hue?
>>>> >>>>>>>>
>>>> >>>>>>>>
>>>> >>>>>>>>> On Tuesday, 3 June 2014 00:32:53 UTC-4, Anurag Tangri wrote:
>>>> >>>>>>>>> Thanks Johndee.
>>>> >>>>>>>>>
>>>> >>>>>>>>> Can you point to some documentation on hive group and
>>>> rejecting requests from metastore ?
>>>> >>>>>>>>>
>>>> >>>>>>>>> Hiveserver2 is mainly for jdbc interface, hue etc.
>>>> >>>>>>>>>
>>>> >>>>>>>>> For people going via hive metastore, we will need to enforce
>>>> same set of roles and policies as via hiveserver2.
>>>> >>>>>>>>>
>>>> >>>>>>>>> Thanks,
>>>> >>>>>>>>> Anurag Tangri
>>>> >>>>>>>>>
>>>> >>>>>>>>>> On Jun 2, 2014, at 7:57 PM, Johndee Cloudera <
>>>> [email protected]> wrote:
>>>> >>>>>>>>>>
>>>> >>>>>>>>>> When it comes to hive and sentry you need at least Hive
>>>> Server 2. Hive Server 2 is what actually runs the Sentry daemon. The Hive
>>>> Metastore Server handles metadata requests, and in the setup guide is
>>>> configured to reject all connections from users not in the hive group
>>>> typically.
>>>> >>>>>>>>>>
>>>> >>>>>>>>>>
>>>> >>>>>>>>>>> On Mon, Jun 2, 2014 at 10:15 PM, Anurag Tangri <
>>>> [email protected]> wrote:
>>>> >>>>>>>>>>> Hi all,
>>>> >>>>>>>>>>> From what I read everywhere, sentry is to be used in
>>>> conjunction with hiveserver2.
>>>> >>>>>>>>>>>
>>>> >>>>>>>>>>> Is it true that it can be used with hive metastore ? If
>>>> yes, Any links explaining how to do so would be great.
>>>> >>>>>>>>>>>
>>>> >>>>>>>>>>> Thanks,
>>>> >>>>>>>>>>> Anurag Tangri
>>>> >>>>>>>>>>>
>>>> >>>>>>>>>>> --
>>>> >>>>>>>>>>>
>>>> >>>>>>>>>>> ---
>>>> >>>>>>>>>>> You received this message because you are subscribed to the
>>>> Google Groups "CDH Users" group.
>>>> >>>>>>>>>>> To unsubscribe from this group and stop receiving emails
>>>> from it, send an email to [email protected].
>>>> >>>>>>>>>>>
>>>> >>>>>>>>>>> For more options, visit
>>>> https://groups.google.com/a/cloudera.org/d/optout.
>>>> >>>>>>>>>>
>>>> >>>>>>>>>>
>>>> >>>>>>>>>>
>>>> >>>>>>>>>> --
>>>> >>>>>>>>>> - JRB
>>>> >>>>>>>>>> --
>>>> >>>>>>>>>>
>>>> >>>>>>>>>> ---
>>>> >>>>>>>>>> You received this message because you are subscribed to the
>>>> Google Groups "CDH Users" group.
>>>> >>>>>>>>>> To unsubscribe from this group and stop receiving emails
>>>> from it, send an email to [email protected].
>>>> >>>>>>>>>>
>>>> >>>>>>>>>> For more options, visit
>>>> https://groups.google.com/a/cloudera.org/d/optout.
>>>> >>>>>>>>
>>>> >>>>>>>> --
>>>> >>>>>>>>
>>>> >>>>>>>> ---
>>>> >>>>>>>> You received this message because you are subscribed to the
>>>> Google Groups "CDH Users" group.
>>>> >>>>>>>> To unsubscribe from this group and stop receiving emails from
>>>> it, send an email to [email protected].
>>>> >>>>>>>> For more options, visit
>>>> https://groups.google.com/a/cloudera.org/d/optout.
>>>> >>>>>>>
>>>> >>>>>>> --
>>>> >>>>>>>
>>>> >>>>>>> ---
>>>> >>>>>>> You received this message because you are subscribed to the
>>>> Google Groups "CDH Users" group.
>>>> >>>>>>> To unsubscribe from this group and stop receiving emails from
>>>> it, send an email to [email protected].
>>>> >>>>>>> For more options, visit
>>>> https://groups.google.com/a/cloudera.org/d/optout.
>>>> >>>>>>
>>>> >>>>>>
>>>> >>>>>>
>>>> >>>>>> --
>>>> >>>>>> Casey J. Brotherton
>>>> >>>>>> Customer Operations Engineer
>>>> >>>>>>
>>>> >>>>>> --
>>>> >>>>>>
>>>> >>>>>> ---
>>>> >>>>>> You received this message because you are subscribed to the
>>>> Google Groups "CDH Users" group.
>>>> >>>>>> To unsubscribe from this group and stop receiving emails from
>>>> it, send an email to [email protected].
>>>> >>>>>> For more options, visit
>>>> https://groups.google.com/a/cloudera.org/d/optout.
>>>> >>>>
>>>> >>>> --
>>>> >>>>
>>>> >>>> ---
>>>> >>>> You received this message because you are subscribed to the Google
>>>> Groups "CDH Users" group.
>>>> >>>> To unsubscribe from this group and stop receiving emails from it,
>>>> send an email to [email protected].
>>>> >>>> For more options, visit
>>>> https://groups.google.com/a/cloudera.org/d/optout.
>>>> >
>>>> > --
>>>> >
>>>> > ---
>>>> > You received this message because you are subscribed to the Google
>>>> Groups "CDH Users" group.
>>>> > To unsubscribe from this group and stop receiving emails from it,
>>>> send an email to [email protected].
>>>> > For more options, visit
>>>> https://groups.google.com/a/cloudera.org/d/optout.
>>>>
>>>
>>> --
>>>
>>> ---
>>> You received this message because you are subscribed to the Google
>>> Groups "CDH Users" group.
>>> To unsubscribe from this group and stop receiving emails from it, send
>>> an email to [email protected].
>>> For more options, visit
>>> https://groups.google.com/a/cloudera.org/d/optout.
>>>
>>
>>
>
