Re: Drills' Hbase storage plugin
Hi Ascot, When Drill's HBase storage plugin creates a HBase connection it uses the storage plugin configuration to get all the required configs. The way storage plugin config is formed is based on code I shared with you below or you can look in [1]. If you look into HBaseConfiguration.create() then it first loads all the HBase related configurations provided as (hbase-site.xml / hbase-default.xml) and then override those with Drill's Storage Plugin configs. So the reason of suggesting hbase-site.xml with all proper configuration in classpath was that the security related configurations will be picked up properly from that file and other configuration will be picked from Storage Plugin configuration and hence it should work. Hope it helps. [1]: https://github.com/apache/drill/blob/master/contrib/storage-hbase/src/main/java/org/apache/drill/exec/store/hbase/HBaseStoragePluginConfig.java#L91 Thanks, Sorabh From: Ascot Moss <ascot.m...@gmail.com> Sent: Monday, August 21, 2017 12:50:59 AM To: user@drill.apache.org Subject: Re: Drills' Hbase storage plugin Dear Sorabh, About putting a hbase-site.xml in classpath, can you please explain a bit more about this point? Regards On Mon, Aug 21, 2017 at 3:38 PM, Ascot Moss <ascot.m...@gmail.com> wrote: > Dear Sorabh, > > Your reply helps a lot!! > > I have copied hbase-site.xml to $DRILL_HOME/conf, > restarted drillbit, ran below: > > kinit > sqlline -n "jdbc:drill:drillbit=n1.testserver:31010;auth=kerberos" -n " > u1@MY_DOMAIN.COM" > > use hbase; > +--+---+ > | ok| summary| > +--+---+ > | true | Default schema chnaged to [hbase] | > +--+---+ > > > show tables; > (no result, took long time) <=== any idea how to solve it? > > > Regards > > > On Tue, Aug 15, 2017 at 10:14 AM, Sorabh Hamirwasia <shamirwa...@mapr.com> > wrote: > >> Hi Ascot, >> >> I am not sure if Kerberos between Hbase storage plugin and Hbase server >> has been tried before or not. Link in [1] talks about Kerberos setup >> between Drill Client and Drillbit. However looking into the code for Hbase >> and link at [2], it looks like if you can put a hbase-site.xml in classpath >> with corresponding setting, then it should work, since internally Drill's >> hbase storage plugin creates a HBaseConfiguration instance which loads this >> file in class-path and set corresponding settings for authentication. You >> need to create TGT for Drill process user (by running kinit, also >> information provided in link [2]) which will be running as HBase client to >> the server. >> >> >> HBaseConnectionKey::getHBaseConf() { >> >> return storeConfig.getHBaseConf(); >> } >> >> >> Under HBaseStoragePluginConfig you can find this is how configuration is >> created: >> >> >> public Configuration getHBaseConf() { >> if (hbaseConf == null) { >> hbaseConf = HBaseConfiguration.create(); >> ... >> >> ... >> >> } >> >> ... >> >> ... >> >> } >> >> >> >> [1]: https://drill.apache.org/docs/configuring-kerberos-authentication/< >> https://drill.apache.org/docs/configuring-kerberos-authentication/>< >> https://drill.apache.org/docs/configuring-kerberos-authentication/> >> [2]: http://hbase.apache.org/0.94/book/security.html >> >> >> Thanks, >> Sorabh >> >
Re: Drills' Hbase storage plugin
Dear Sorabh, About putting a hbase-site.xml in classpath, can you please explain a bit more about this point? Regards On Mon, Aug 21, 2017 at 3:38 PM, Ascot Mosswrote: > Dear Sorabh, > > Your reply helps a lot!! > > I have copied hbase-site.xml to $DRILL_HOME/conf, > restarted drillbit, ran below: > > kinit > sqlline -n "jdbc:drill:drillbit=n1.testserver:31010;auth=kerberos" -n " > u1@MY_DOMAIN.COM" > > use hbase; > +--+---+ > | ok| summary| > +--+---+ > | true | Default schema chnaged to [hbase] | > +--+---+ > > > show tables; > (no result, took long time) <=== any idea how to solve it? > > > Regards > > > On Tue, Aug 15, 2017 at 10:14 AM, Sorabh Hamirwasia > wrote: > >> Hi Ascot, >> >> I am not sure if Kerberos between Hbase storage plugin and Hbase server >> has been tried before or not. Link in [1] talks about Kerberos setup >> between Drill Client and Drillbit. However looking into the code for Hbase >> and link at [2], it looks like if you can put a hbase-site.xml in classpath >> with corresponding setting, then it should work, since internally Drill's >> hbase storage plugin creates a HBaseConfiguration instance which loads this >> file in class-path and set corresponding settings for authentication. You >> need to create TGT for Drill process user (by running kinit, also >> information provided in link [2]) which will be running as HBase client to >> the server. >> >> >> HBaseConnectionKey::getHBaseConf() { >> >> return storeConfig.getHBaseConf(); >> } >> >> >> Under HBaseStoragePluginConfig you can find this is how configuration is >> created: >> >> >> public Configuration getHBaseConf() { >> if (hbaseConf == null) { >> hbaseConf = HBaseConfiguration.create(); >> ... >> >> ... >> >> } >> >> ... >> >> ... >> >> } >> >> >> >> [1]: https://drill.apache.org/docs/configuring-kerberos-authentication/< >> https://drill.apache.org/docs/configuring-kerberos-authentication/>< >> https://drill.apache.org/docs/configuring-kerberos-authentication/> >> [2]: http://hbase.apache.org/0.94/book/security.html >> >> >> Thanks, >> Sorabh >> >
Re: Drills' Hbase storage plugin
Dear Sorabh, Your reply helps a lot!! I have copied hbase-site.xml to $DRILL_HOME/conf, restarted drillbit, ran below: kinit sqlline -n "jdbc:drill:drillbit=n1.testserver:31010;auth=kerberos" -n " u1@MY_DOMAIN.COM" use hbase; +--+---+ | ok| summary| +--+---+ | true | Default schema chnaged to [hbase] | +--+---+ show tables; (no result, took long time) <=== any idea how to solve it? Regards On Tue, Aug 15, 2017 at 10:14 AM, Sorabh Hamirwasiawrote: > Hi Ascot, > > I am not sure if Kerberos between Hbase storage plugin and Hbase server > has been tried before or not. Link in [1] talks about Kerberos setup > between Drill Client and Drillbit. However looking into the code for Hbase > and link at [2], it looks like if you can put a hbase-site.xml in classpath > with corresponding setting, then it should work, since internally Drill's > hbase storage plugin creates a HBaseConfiguration instance which loads this > file in class-path and set corresponding settings for authentication. You > need to create TGT for Drill process user (by running kinit, also > information provided in link [2]) which will be running as HBase client to > the server. > > > HBaseConnectionKey::getHBaseConf() { > > return storeConfig.getHBaseConf(); > } > > > Under HBaseStoragePluginConfig you can find this is how configuration is > created: > > > public Configuration getHBaseConf() { > if (hbaseConf == null) { > hbaseConf = HBaseConfiguration.create(); > ... > > ... > > } > > ... > > ... > > } > > > > [1]: https://drill.apache.org/docs/configuring-kerberos-authentication/< > https://drill.apache.org/docs/configuring-kerberos-authentication/> s://drill.apache.org/docs/configuring-kerberos-authentication/> > [2]: http://hbase.apache.org/0.94/book/security.html > > > Thanks, > Sorabh >
Re: Drills' Hbase storage plugin
Hi Ascot, I am not sure if Kerberos between Hbase storage plugin and Hbase server has been tried before or not. Link in [1] talks about Kerberos setup between Drill Client and Drillbit. However looking into the code for Hbase and link at [2], it looks like if you can put a hbase-site.xml in classpath with corresponding setting, then it should work, since internally Drill's hbase storage plugin creates a HBaseConfiguration instance which loads this file in class-path and set corresponding settings for authentication. You need to create TGT for Drill process user (by running kinit, also information provided in link [2]) which will be running as HBase client to the server. HBaseConnectionKey::getHBaseConf() { return storeConfig.getHBaseConf(); } Under HBaseStoragePluginConfig you can find this is how configuration is created: public Configuration getHBaseConf() { if (hbaseConf == null) { hbaseConf = HBaseConfiguration.create(); ... ... } ... ... } [1]: https://drill.apache.org/docs/configuring-kerberos-authentication/<https://drill.apache.org/docs/configuring-kerberos-authentication/><https://drill.apache.org/docs/configuring-kerberos-authentication/> [2]: http://hbase.apache.org/0.94/book/security.html Thanks, Sorabh From: Ascot Moss <ascot.m...@gmail.com> Sent: Saturday, August 12, 2017 5:01 AM To: user@drill.apache.org Subject: Re: Drills' Hbase storage plugin Does Drill's Hbase Storage support Hbase-Kerberos? I cannot find any link about this. https://drill.apache.org/docs/hbase-storage-plugin/ This link does not provide too much detail about Drills' Hbase Storage on HBase-Kerberos. On Sat, Aug 12, 2017 at 4:11 PM, Ascot Moss <ascot.m...@gmail.com> wrote: > Hi, > > I have Hbase with Kerberos enabled, how to configure drill's hbase storage > plugin in order to access kerberos-Hbase? > > Regards >
Re: Drills' Hbase storage plugin
Does Drill's Hbase Storage support Hbase-Kerberos? I cannot find any link about this. https://drill.apache.org/docs/hbase-storage-plugin/ This link does not provide too much detail about Drills' Hbase Storage on HBase-Kerberos. On Sat, Aug 12, 2017 at 4:11 PM, Ascot Mosswrote: > Hi, > > I have Hbase with Kerberos enabled, how to configure drill's hbase storage > plugin in order to access kerberos-Hbase? > > Regards >