Re: Validating Hive statements
Hi, I can confirm as Elliot was mentioning, the CLI takes care of comments (justed tested that) and splitting the statements, but ADDs and SETs are still kept, and the parser breaks when trying to parse them. Is there any other middle step I should be aware of? Thanks. On Wed, 19 Feb 2020 at 14:51, Odon Copon wrote: > Hi Elliot, > Thanks for your quick response. > Are you saying that things like SETs and other stuff is handled by the CLI > and doesn't reach the parser? Is there any example or testing I could check > to see how does this work? > > Thanks. > > On Wed, 19 Feb 2020 at 14:21, Elliot West wrote: > >> Hi, >> >> If I recall correctly, not all script input is handled by the parser, and >> the CLI takes care of some statements prior to parsing of SQL - comments >> are an example. Also, there is some divergence between Beeline and hive >> CLI. In HiveRunner we handled this by providing different CLI emulations: >> >> >> https://github.com/klarna/HiveRunner/tree/master/src/main/java/com/klarna/hiverunner/sql/cli >> >> Elliot. >> >> On Wed, 19 Feb 2020 at 13:55, Odon Copon wrote: >> >>> Hi, >>> I was wondering what would be the easiest way to validate a Hive script >>> with multiple query statements offline. I thought it was possible to do >>> that will the following Java code but it doesn't look like is possible for >>> all of them: >>> >>> --- >>> import org.apache.hadoop.hive.ql.parse.ParseDriver; >>> >>> ParseDriver pd = new ParseDriver(); >>> pd.parse(query); >>> --- >>> >>> With that tiny snippet, I'm able to validate some queries but fails with >>> other statements like: >>> - ADD JAR ; >>> >>> or: >>> - SET =true; >>> >>> I would like not to omit those and be able to parse a script with >>> multiple query statements. >>> Are there any tips you could give me to help me with this? Currently, >>> I'm splitting by semicolon and discarding lines with ADD and SET >>> statements, but there must something there I'm missing. >>> >>> Thanks. >>> >>
Re: Validating Hive statements
Hi Elliot, Thanks for your quick response. Are you saying that things like SETs and other stuff is handled by the CLI and doesn't reach the parser? Is there any example or testing I could check to see how does this work? Thanks. On Wed, 19 Feb 2020 at 14:21, Elliot West wrote: > Hi, > > If I recall correctly, not all script input is handled by the parser, and > the CLI takes care of some statements prior to parsing of SQL - comments > are an example. Also, there is some divergence between Beeline and hive > CLI. In HiveRunner we handled this by providing different CLI emulations: > > > https://github.com/klarna/HiveRunner/tree/master/src/main/java/com/klarna/hiverunner/sql/cli > > Elliot. > > On Wed, 19 Feb 2020 at 13:55, Odon Copon wrote: > >> Hi, >> I was wondering what would be the easiest way to validate a Hive script >> with multiple query statements offline. I thought it was possible to do >> that will the following Java code but it doesn't look like is possible for >> all of them: >> >> --- >> import org.apache.hadoop.hive.ql.parse.ParseDriver; >> >> ParseDriver pd = new ParseDriver(); >> pd.parse(query); >> --- >> >> With that tiny snippet, I'm able to validate some queries but fails with >> other statements like: >> - ADD JAR ; >> >> or: >> - SET =true; >> >> I would like not to omit those and be able to parse a script with >> multiple query statements. >> Are there any tips you could give me to help me with this? Currently, I'm >> splitting by semicolon and discarding lines with ADD and SET statements, >> but there must something there I'm missing. >> >> Thanks. >> >
Validating Hive statements
Hi, I was wondering what would be the easiest way to validate a Hive script with multiple query statements offline. I thought it was possible to do that will the following Java code but it doesn't look like is possible for all of them: --- import org.apache.hadoop.hive.ql.parse.ParseDriver; ParseDriver pd = new ParseDriver(); pd.parse(query); --- With that tiny snippet, I'm able to validate some queries but fails with other statements like: - ADD JAR ; or: - SET =true; I would like not to omit those and be able to parse a script with multiple query statements. Are there any tips you could give me to help me with this? Currently, I'm splitting by semicolon and discarding lines with ADD and SET statements, but there must something there I'm missing. Thanks.
Re: Property url when using beeline
Would it work on a jdbc connection different than Hive? As you can see, I'm trying to connect to Presto over jdbc. Thanks On Fri, 26 Apr 2019, 18:36 Odon Copon, wrote: > Sure, the command is the following: > > beeline> !addlocaldriverjar /Users/odon/jars/presto-jdbc-0.212.jar > beeline> !addlocaldrivername com.facebook.presto.jdbc.PrestoDriver > beeline> !connect jdbc:presto://presto-host:8080/catalog/schema > > On Fri, 26 Apr 2019, 17:59 Suresh Kumar Sethuramaswamy, > wrote: > >> Could you please provide your beeline command >> >> Regards >> Suresh >> >> On Fri, Apr 26, 2019, 12:30 PM Odon Copon > >>> Hi, >>> I'm trying to understand the following error the beeline is giving me: >>> >>> Fail to connect with a local driver due to the >>> exception:java.sql.SQLException: Unrecognized connection property 'url'. >>> >>> How does the beeline generate this property? >>> Thanks. >>> >>
Re: Property url when using beeline
Sure, the command is the following: beeline> !addlocaldriverjar /Users/odon/jars/presto-jdbc-0.212.jar beeline> !addlocaldrivername com.facebook.presto.jdbc.PrestoDriver beeline> !connect jdbc:presto://presto-host:8080/catalog/schema On Fri, 26 Apr 2019, 17:59 Suresh Kumar Sethuramaswamy, wrote: > Could you please provide your beeline command > > Regards > Suresh > > On Fri, Apr 26, 2019, 12:30 PM Odon Copon >> Hi, >> I'm trying to understand the following error the beeline is giving me: >> >> Fail to connect with a local driver due to the >> exception:java.sql.SQLException: Unrecognized connection property 'url'. >> >> How does the beeline generate this property? >> Thanks. >> >
Property url when using beeline
Hi, I'm trying to understand the following error the beeline is giving me: Fail to connect with a local driver due to the exception:java.sql.SQLException: Unrecognized connection property 'url'. How does the beeline generate this property? Thanks.
Re: Hive metastore service
Thanks Shawn, Yeah, I was talking about that. In that documentation didn't see any authentication so was wondering if there were options to protect it. But doesn't seem as you said. Thanks On Wed, 17 Apr 2019, 13:21 Shawn Weeks, wrote: > I think they misunderstood, your talking about this > https://cwiki.apache.org/confluence/display/Hive/AdminManual+Metastore+Administration. > As far as I know the only authentication it supports is Kerberos. > > > > Thanks > > Shawn > > > > *From:* Odon Copon > *Sent:* Wednesday, April 17, 2019 3:16 AM > *To:* user@hive.apache.org > *Subject:* Re: Hive metastore service > > > > Hi, > > I don't know if I am not understanding properly the design, but, isn't > there supposed to be a Hive metastore service that Hive server uses to > access the metastore db? > > > > On Tue, 16 Apr 2019, 22:26 Mich Talebzadeh, > wrote: > > Try this > > > > Assuming that you are talking about Hive Thrift server > > > > beeline -u jdbc:hive2://rhes75:10099/default > org.apache.hive.jdbc.HiveDriver *-n USERNAME -p PASSWORD* -i > /home/hduser/dba/bin/add_jars.hql' > > > > HTH > > > > Dr Mich Talebzadeh > > > > LinkedIn > *https://www.linkedin.com/profile/view?id=AAEWh2gBxianrbJd6zP6AcPCCdOABUrV8Pw > <https://www.linkedin.com/profile/view?id=AAEWh2gBxianrbJd6zP6AcPCCdOABUrV8Pw>* > > > > http://talebzadehmich.wordpress.com > > > > *Disclaimer:* Use it at your own risk. Any and all responsibility for any > loss, damage or destruction of data or any other property which may arise > from relying on this email's technical content is explicitly disclaimed. > The author will in no case be liable for any monetary damages arising from > such loss, damage or destruction. > > > > > > > > On Tue, 16 Apr 2019 at 16:11, Odon Copon wrote: > > Hi, > > would be possible to add authentication to the Thrift Hive metastore > service? like user and password? > > I cannot find any documentation on how to protect this endpoint. > > Thanks. > >
Re: Hive metastore service
Hi, I don't know if I am not understanding properly the design, but, isn't there supposed to be a Hive metastore service that Hive server uses to access the metastore db? On Tue, 16 Apr 2019, 22:26 Mich Talebzadeh, wrote: > Try this > > Assuming that you are talking about Hive Thrift server > > beeline -u jdbc:hive2://rhes75:10099/default > org.apache.hive.jdbc.HiveDriver *-n USERNAME -p PASSWORD* -i > /home/hduser/dba/bin/add_jars.hql' > > HTH > > Dr Mich Talebzadeh > > > > LinkedIn * > https://www.linkedin.com/profile/view?id=AAEWh2gBxianrbJd6zP6AcPCCdOABUrV8Pw > <https://www.linkedin.com/profile/view?id=AAEWh2gBxianrbJd6zP6AcPCCdOABUrV8Pw>* > > > > http://talebzadehmich.wordpress.com > > > *Disclaimer:* Use it at your own risk. Any and all responsibility for any > loss, damage or destruction of data or any other property which may arise > from relying on this email's technical content is explicitly disclaimed. > The author will in no case be liable for any monetary damages arising from > such loss, damage or destruction. > > > > > On Tue, 16 Apr 2019 at 16:11, Odon Copon wrote: > >> Hi, >> would be possible to add authentication to the Thrift Hive metastore >> service? like user and password? >> I cannot find any documentation on how to protect this endpoint. >> Thanks. >> >
Hive metastore service
Hi, would be possible to add authentication to the Thrift Hive metastore service? like user and password? I cannot find any documentation on how to protect this endpoint. Thanks.
Re: Beeline and password
Ok, I managed to get it working by not specifying driver and user while calling !connect. If I specify the driver or user I don't get the prompt to input the credentials. Thanks. On Tue, 9 Apr 2019, 09:41 Odon Copon, wrote: > What's the difference between using "beeline -u" and beeline and then > connect? It doesn't seem to behave equally. > > On Mon, 8 Apr 2019, 22:49 Odon Copon, wrote: > >> Using "beeline -u -p" I can make it to prompt and be asked for the >> password, but it does not connect, seems the authentication it does is not >> working for me. On the error I see the jdbc string gets appended the >> "user=. >> When running it with !connect it works, but I don't get the same prompt >> as before to input the password. >> >> Thanks. >> >> On Mon, 8 Apr 2019, 22:08 Odon Copon, wrote: >> >>> Hi Raj, >>> That doesn't work for me. It doesn't ask for the password, it fails >>> directly without prompting. >>> >>> Thanks >>> >>> On Mon, 8 Apr 2019, 21:05 Raj K Singh, wrote: >>> >>>> if you try beeline -u URL_TO_CONNECT -n hive -p then it should ask you >>>> for password, if you are using hive-3.1 then possibly you need fix for >>>> https://issues.apache.org/jira/browse/HIVE-21538 >>>> >>>> Raj K Singh >>>> >>>> >>>> On Mon, Apr 8, 2019 at 12:22 PM Odon Copon wrote: >>>> >>>>> Hi, >>>>> I'm using LDAP at the moment. But, from your answer, seems that using >>>>> user and password there's no way around to try to hide the password, >>>>> right? >>>>> >>>>> Thanks >>>>> >>>>> On Mon, 8 Apr 2019, 19:29 Will Du, wrote: >>>>> >>>>>> You can consider use other authentication methods rather than >>>>>> username/password. >>>>>> >>>>>> Sent from my iPhone >>>>>> >>>>>> > On Apr 8, 2019, at 14:04, Odon Copon wrote: >>>>>> > >>>>>> > Hi, >>>>>> > Is there a way to make Beeline to prompt for the password? >>>>>> > I see that all the options require the password to be visibly >>>>>> inserted in plain text. The only option seems to store it in a file and >>>>>> the >>>>>> file path instead, but that would also require to store the password in >>>>>> plain text format. >>>>>> > Any suggestions? >>>>>> > >>>>>> > Thanks >>>>>> >>>>>
Re: Beeline and password
What's the difference between using "beeline -u" and beeline and then connect? It doesn't seem to behave equally. On Mon, 8 Apr 2019, 22:49 Odon Copon, wrote: > Using "beeline -u -p" I can make it to prompt and be asked for the > password, but it does not connect, seems the authentication it does is not > working for me. On the error I see the jdbc string gets appended the > "user=. > When running it with !connect it works, but I don't get the same prompt as > before to input the password. > > Thanks. > > On Mon, 8 Apr 2019, 22:08 Odon Copon, wrote: > >> Hi Raj, >> That doesn't work for me. It doesn't ask for the password, it fails >> directly without prompting. >> >> Thanks >> >> On Mon, 8 Apr 2019, 21:05 Raj K Singh, wrote: >> >>> if you try beeline -u URL_TO_CONNECT -n hive -p then it should ask you >>> for password, if you are using hive-3.1 then possibly you need fix for >>> https://issues.apache.org/jira/browse/HIVE-21538 >>> >>> Raj K Singh >>> >>> >>> On Mon, Apr 8, 2019 at 12:22 PM Odon Copon wrote: >>> >>>> Hi, >>>> I'm using LDAP at the moment. But, from your answer, seems that using >>>> user and password there's no way around to try to hide the password, right? >>>> >>>> Thanks >>>> >>>> On Mon, 8 Apr 2019, 19:29 Will Du, wrote: >>>> >>>>> You can consider use other authentication methods rather than >>>>> username/password. >>>>> >>>>> Sent from my iPhone >>>>> >>>>> > On Apr 8, 2019, at 14:04, Odon Copon wrote: >>>>> > >>>>> > Hi, >>>>> > Is there a way to make Beeline to prompt for the password? >>>>> > I see that all the options require the password to be visibly >>>>> inserted in plain text. The only option seems to store it in a file and >>>>> the >>>>> file path instead, but that would also require to store the password in >>>>> plain text format. >>>>> > Any suggestions? >>>>> > >>>>> > Thanks >>>>> >>>>
Re: Beeline and password
Using "beeline -u -p" I can make it to prompt and be asked for the password, but it does not connect, seems the authentication it does is not working for me. On the error I see the jdbc string gets appended the "user=. When running it with !connect it works, but I don't get the same prompt as before to input the password. Thanks. On Mon, 8 Apr 2019, 22:08 Odon Copon, wrote: > Hi Raj, > That doesn't work for me. It doesn't ask for the password, it fails > directly without prompting. > > Thanks > > On Mon, 8 Apr 2019, 21:05 Raj K Singh, wrote: > >> if you try beeline -u URL_TO_CONNECT -n hive -p then it should ask you >> for password, if you are using hive-3.1 then possibly you need fix for >> https://issues.apache.org/jira/browse/HIVE-21538 >> :::: >> Raj K Singh >> >> >> On Mon, Apr 8, 2019 at 12:22 PM Odon Copon wrote: >> >>> Hi, >>> I'm using LDAP at the moment. But, from your answer, seems that using >>> user and password there's no way around to try to hide the password, right? >>> >>> Thanks >>> >>> On Mon, 8 Apr 2019, 19:29 Will Du, wrote: >>> >>>> You can consider use other authentication methods rather than >>>> username/password. >>>> >>>> Sent from my iPhone >>>> >>>> > On Apr 8, 2019, at 14:04, Odon Copon wrote: >>>> > >>>> > Hi, >>>> > Is there a way to make Beeline to prompt for the password? >>>> > I see that all the options require the password to be visibly >>>> inserted in plain text. The only option seems to store it in a file and the >>>> file path instead, but that would also require to store the password in >>>> plain text format. >>>> > Any suggestions? >>>> > >>>> > Thanks >>>> >>>
Re: Beeline and password
Hi Raj, That doesn't work for me. It doesn't ask for the password, it fails directly without prompting. Thanks On Mon, 8 Apr 2019, 21:05 Raj K Singh, wrote: > if you try beeline -u URL_TO_CONNECT -n hive -p then it should ask you > for password, if you are using hive-3.1 then possibly you need fix for > https://issues.apache.org/jira/browse/HIVE-21538 > > Raj K Singh > > > On Mon, Apr 8, 2019 at 12:22 PM Odon Copon wrote: > >> Hi, >> I'm using LDAP at the moment. But, from your answer, seems that using >> user and password there's no way around to try to hide the password, right? >> >> Thanks >> >> On Mon, 8 Apr 2019, 19:29 Will Du, wrote: >> >>> You can consider use other authentication methods rather than >>> username/password. >>> >>> Sent from my iPhone >>> >>> > On Apr 8, 2019, at 14:04, Odon Copon wrote: >>> > >>> > Hi, >>> > Is there a way to make Beeline to prompt for the password? >>> > I see that all the options require the password to be visibly inserted >>> in plain text. The only option seems to store it in a file and the file >>> path instead, but that would also require to store the password in plain >>> text format. >>> > Any suggestions? >>> > >>> > Thanks >>> >>
Re: Beeline and password
Hi, I'm using LDAP at the moment. But, from your answer, seems that using user and password there's no way around to try to hide the password, right? Thanks On Mon, 8 Apr 2019, 19:29 Will Du, wrote: > You can consider use other authentication methods rather than > username/password. > > Sent from my iPhone > > > On Apr 8, 2019, at 14:04, Odon Copon wrote: > > > > Hi, > > Is there a way to make Beeline to prompt for the password? > > I see that all the options require the password to be visibly inserted > in plain text. The only option seems to store it in a file and the file > path instead, but that would also require to store the password in plain > text format. > > Any suggestions? > > > > Thanks >
Beeline and password
Hi, Is there a way to make Beeline to prompt for the password? I see that all the options require the password to be visibly inserted in plain text. The only option seems to store it in a file and the file path instead, but that would also require to store the password in plain text format. Any suggestions? Thanks
