Re: Drills' Hbase storage plugin
Dear Sorabh, About putting a hbase-site.xml in classpath, can you please explain a bit more about this point? Regards On Mon, Aug 21, 2017 at 3:38 PM, Ascot Moss <ascot.m...@gmail.com> wrote: > Dear Sorabh, > > Your reply helps a lot!! > > I have copied hbase-site.xml to $DRILL_HOME/conf, > restarted drillbit, ran below: > > kinit > sqlline -n "jdbc:drill:drillbit=n1.testserver:31010;auth=kerberos" -n " > u1@MY_DOMAIN.COM" > > use hbase; > +--+---+ > | ok| summary| > +--+---+ > | true | Default schema chnaged to [hbase] | > +--+---+ > > > show tables; > (no result, took long time) <=== any idea how to solve it? > > > Regards > > > On Tue, Aug 15, 2017 at 10:14 AM, Sorabh Hamirwasia <shamirwa...@mapr.com> > wrote: > >> Hi Ascot, >> >> I am not sure if Kerberos between Hbase storage plugin and Hbase server >> has been tried before or not. Link in [1] talks about Kerberos setup >> between Drill Client and Drillbit. However looking into the code for Hbase >> and link at [2], it looks like if you can put a hbase-site.xml in classpath >> with corresponding setting, then it should work, since internally Drill's >> hbase storage plugin creates a HBaseConfiguration instance which loads this >> file in class-path and set corresponding settings for authentication. You >> need to create TGT for Drill process user (by running kinit, also >> information provided in link [2]) which will be running as HBase client to >> the server. >> >> >> HBaseConnectionKey::getHBaseConf() { >> >> return storeConfig.getHBaseConf(); >> } >> >> >> Under HBaseStoragePluginConfig you can find this is how configuration is >> created: >> >> >> public Configuration getHBaseConf() { >> if (hbaseConf == null) { >> hbaseConf = HBaseConfiguration.create(); >> ... >> >> ... >> >> } >> >> ... >> >> ... >> >> } >> >> >> >> [1]: https://drill.apache.org/docs/configuring-kerberos-authentication/< >> https://drill.apache.org/docs/configuring-kerberos-authentication/>< >> https://drill.apache.org/docs/configuring-kerberos-authentication/> >> [2]: http://hbase.apache.org/0.94/book/security.html >> >> >> Thanks, >> Sorabh >> >
Re: Drills' Hbase storage plugin
Dear Sorabh, Your reply helps a lot!! I have copied hbase-site.xml to $DRILL_HOME/conf, restarted drillbit, ran below: kinit sqlline -n "jdbc:drill:drillbit=n1.testserver:31010;auth=kerberos" -n " u1@MY_DOMAIN.COM" use hbase; +--+---+ | ok| summary| +--+---+ | true | Default schema chnaged to [hbase] | +--+---+ show tables; (no result, took long time) <=== any idea how to solve it? Regards On Tue, Aug 15, 2017 at 10:14 AM, Sorabh Hamirwasiawrote: > Hi Ascot, > > I am not sure if Kerberos between Hbase storage plugin and Hbase server > has been tried before or not. Link in [1] talks about Kerberos setup > between Drill Client and Drillbit. However looking into the code for Hbase > and link at [2], it looks like if you can put a hbase-site.xml in classpath > with corresponding setting, then it should work, since internally Drill's > hbase storage plugin creates a HBaseConfiguration instance which loads this > file in class-path and set corresponding settings for authentication. You > need to create TGT for Drill process user (by running kinit, also > information provided in link [2]) which will be running as HBase client to > the server. > > > HBaseConnectionKey::getHBaseConf() { > > return storeConfig.getHBaseConf(); > } > > > Under HBaseStoragePluginConfig you can find this is how configuration is > created: > > > public Configuration getHBaseConf() { > if (hbaseConf == null) { > hbaseConf = HBaseConfiguration.create(); > ... > > ... > > } > > ... > > ... > > } > > > > [1]: https://drill.apache.org/docs/configuring-kerberos-authentication/< > https://drill.apache.org/docs/configuring-kerberos-authentication/> s://drill.apache.org/docs/configuring-kerberos-authentication/> > [2]: http://hbase.apache.org/0.94/book/security.html > > > Thanks, > Sorabh >
Re: Drill on secure HDFS
Hi Robert, "This will work for JDBC clients." Great! any sample command for reference? e.g. any beeline !connect command sample about the parameters required for jdbc connection on Hadoop-kerberos? Regards
Error: SYSTEM ERROR: IOException: Can't get Master Kerberos principal for use as renewer
Hi, I use Drill 1.11 on Hive 2.1.1 on Hadoop 2.7.2 with Kerberos. sqlline -u "jdbc:drill:drillbit=n1.server:31010;auth=kerberos;principal=drill/server" -n "drill/n1.server" use hive; +---+---+ | ok | summary | +---+---+ | true | Default schema changed to [hive] | +---+---+ show tables; +---+--+ | TABLE_SCHEMA |TABLE_NAME| +---+--+ | hive.default | test01| +---+--+ select * from `test01`; Error: SYSTEM ERROR: IOException: Can't get Master Kerberos principal for use as renewer How can I fix this? Regards
Error: SYSTEM ERROR: IOException: Can't get Master Kerberos principal for use as renewer
Hi, I use Drill 1.11 on Hive 2.1.1 on Hadoop 2.7.2 with Kerberos. sqlline -u "jdbc:drill:drillbit=n1.server:31010;auth=kerberos;principal=drill/server" -n "drill/n1.server " use hive; +---+---+ | ok | summary | +---+---+ | true | Default schema changed to [hive] | +---+---+ show tables; +---+--+ | TABLE_SCHEMA |TABLE_NAME| +---+--+ | hive.default | test01| +---+--+ select * from `test01`; Error: SYSTEM ERROR: IOException: Can't get Master Kerberos principal for use as renewer How can I fix this? Regards
Re: Drill on secure HDFS
Does Drill's File Storage Plugin (DFS Plugin) support Hadoop-Kerberos? I cannot find any link about this. https://drill.apache.org/docs/file-system-storage-plugin/ This document does not provide to much detail about whether Drill supports Hadoop-Kerberos or not. On Sat, Aug 12, 2017 at 4:09 PM, Ascot Moss <ascot.m...@gmail.com> wrote: > Hi, > > I have HDFS with Kerberos enabled, how to configure drill's dfs plugin in > order to enable Kerberos authentication? > > regards >
Re: Drills' Hbase storage plugin
Does Drill's Hbase Storage support Hbase-Kerberos? I cannot find any link about this. https://drill.apache.org/docs/hbase-storage-plugin/ This link does not provide too much detail about Drills' Hbase Storage on HBase-Kerberos. On Sat, Aug 12, 2017 at 4:11 PM, Ascot Moss <ascot.m...@gmail.com> wrote: > Hi, > > I have Hbase with Kerberos enabled, how to configure drill's hbase storage > plugin in order to access kerberos-Hbase? > > Regards >
Re: Drill UI (8047) Default User and Password
Thanks! By the way, how to configure hbase plugin to query kerberos-hbase? On Fri, Aug 11, 2017 at 1:38 PM, Sorabh Hamirwasia <shamirwa...@mapr.com> wrote: > To add on Arina's reply: > > > Authentication on WebServer side happens using PLAIN mechanism with PAM > authenticator. So when authentication is enabled [1], you can login to > WebUI using credentials of any user present on the node to which you are > connecting to. > > For example: If you are connecting to WebServer running on > http://10.10.10.10:8047, then you can login using user bob credentials, > provided bob is a valid user on node 10.10.10.10 > > > [1] https://drill.apache.org/docs/configuring-plain-security/ > > > Thanks, > Sorabh > > > From: Arina Yelchiyeva <arina.yelchiy...@gmail.com> > Sent: Thursday, August 10, 2017 5:03:49 AM > To: user > Subject: Re: Drill UI (8047) Default User and Password > > Currently if you did not configure authorization, Drill does not require > entering login and password. All queries will be run under the user who > started the drillbit. On Web UI, user who ran the queries will be shown as > anonymous. > Though if you did than it would depend on those users whom you have allowed > access [1]. > > [1] https://drill.apache.org/docs/securing-drill-introduction/ > > Kind regards > Arina > > On Thu, Aug 10, 2017 at 2:50 PM, Ascot Moss <ascot.m...@gmail.com> wrote: > > > Hi, > > > > What is the default login name and password for drill 8047 port UI? > > > > (Drill version: 1.11.0) > > > > regards > > >
Drills' Hbase storage plugin
Hi, I have Hbase with Kerberos enabled, how to configure drill's hbase storage plugin in order to access kerberos-Hbase? Regards
Drill on secure HDFS
Hi, I have HDFS with Kerberos enabled, how to configure drill's dfs plugin in order to enable Kerberos authentication? regards
Drill UI (8047) Default User and Password
Hi, What is the default login name and password for drill 8047 port UI? (Drill version: 1.11.0) regards
Hive Metastore
Hi Drill team, On the Drill documentation page, https://drill.apache.org/docs/file-system-storage-plugin/, it states that the IP address is required if HDFS is used. • HDFS connection: hdfs://IP Address:Port/ e.g. connection: hdfs://10.10.30.156:8020/ My question is that: Does the current version of Apache Drill support HDFS High Availability Using the Quorum ? i.e. Does current Drill version support dfs.nameservices from the Hadoop High Availability Cluster? Regards