Re: Using comments in query for detailed monitoring

[Greg Rahn]

The driver with this fix is now available.
http://community.cloudera.com/t5/Community-News-Release/ANNOUNCE-New-Impala-JDBC-Driver-Released/m-p/65325#M220


On Fri, Feb 16, 2018 at 5:59 PM, Greg Rahn  wrote:

> Unfortunately there is not a JIRA to track because the drivers are
> maintained by a 3rd party and are not part of the Apache Impala project.
> FWIW the version I tested was 2.5.43.1063
>
> On Fri, Feb 16, 2018 at 4:52 PM, Sunil Parmar 
> wrote:
>
>> Thanks is there a JIRA we can track ?
>>
>> Sunil Parmar
>>
>> On Tue, Feb 13, 2018 at 3:18 PM, Greg Rahn  wrote:
>>
>>> An upcoming release of the Impala driver will include a fix for not
>>> removing comments when UseNativeQuery=1 is used.
>>>
>>> On Fri, Jan 26, 2018 at 10:29 AM, Greg Rahn  wrote:
>>>
 I was able to repro this as well and checking in with Simba who authors
 the driver.


 On Wed, Jan 17, 2018 at 9:03 PM, Sunil Parmar 
 wrote:

> Tried with case change but no luck.  From the connection log.
>
> Jan 17 20:33:38.163 TRACE 1 
> com.cloudera.jdbc.common.SConnection.nativeSQL("select
> sum(1) from dwh.event_detail where org_group_id > 10; /*abcdefg*/"): +
> enter +
> Jan 17 20:33:38.163 TRACE 1 com.cloudera.hivecommon.core.H
> iveJDBCCommonConnection.toNativeSQL("select sum(1) from
> dwh.event_detail where org_group_id > 10; /*abcdefg*/"): + enter +
> Jan 17 20:33:38.164 TRACE 1 com.cloudera.hivecommon.dataen
> gine.HiveJDBCDataEngine.prepare("select sum(1) from dwh.event_detail
> where org_group_id > 10; "): + enter +
> Jan 17 20:33:38.166 TRACE 1 com.cloudera.hivecommon.dataen
> gine.HiveJDBCNativeQueryExecutor.HiveJDBCNativeQueryExecutor
> (com.cloudera.dsi.core.impl.DSILogger@1d057a39,
> com.cloudera.hivecommon.api.HS2Client@26be92ad, "select sum(1) from
> dwh.event_detail where org_group_id > 10;"): + enter +
> ...
>
> Sunil Parmar
>
> On Wed, Jan 17, 2018 at 5:52 PM, Vincent Tran 
> wrote:
>
>> The properties are case sensitive so I would try UseNativeQuery just
>> to dot the i's and cross the t's.
>>
>> Ensure that the property is recognized and applied by the driver by
>> checking the JDBC log. (LogLevel=6;LogPath=/path/to/foo;)
>>
>> On Jan 17, 2018 6:45 PM, "Sunil Parmar" 
>> wrote:
>>
>>> useNativeQuery didn't help.
>>>
>>> This is my code.
>>>
>>> public static void main(String[] args) throws IOException,
>>> SQLException {
>>> connectionUrl =
>>>  "jdbc:impala://hadoop102.qa1.sac.int.company.com:21050/dwh;
>>> AuthMech=3;UID=ldaptest;PWD=test;useNativeQuery=1";
>>> jdbcDriverName = "com.cloudera.impala.jdbc41.Driver";
>>> System.out.println("\n==
>>> ===");
>>> System.out.println("Cloudera Impala JDBC Example");
>>> System.out.println("Using Connection URL: " + connectionUrl);
>>> // System.out.println("Running Query: " + sqlStatement);
>>> Connection con = null;
>>> try {
>>> Class.forName(jdbcDriverName);
>>> con = DriverManager.getConnection(connectionUrl);
>>> String query="select count( distinct mmdd) from
>>> dwh.event_detail limit 1/*abcdefg1*/";
>>> System.out.println("\n== Begin Query Results
>>> ==");
>>> System.out.println("\n== Running query=== " + query);
>>> Statement st = con.createStatement();
>>> ResultSet rs = st.executeQuery(query);
>>> QueryTask.printResultSet(rs);
>>>
>>> } catch (SQLException e) {
>>> e.printStackTrace();
>>> } catch (Exception e) {
>>> e.printStackTrace();
>>> } finally {
>>> con.close();
>>>
>>> }
>>> }
>>>
>>> Same query when executed from HUE shows comments. Look at the
>>> screenshot from CM.
>>>
>>>
>>> ​
>>>
>>> Sunil Parmar
>>>
>>> On Wed, Dec 13, 2017 at 11:30 PM, Mostafa Mokhtar <
>>> mmokh...@cloudera.com> wrote:
>>>
 +1 on what Alex recommends.
 Highly recommend using useNativeQuery=1 all the time.

 Thanks
 Mostafa

 On Dec 13, 2017, at 9:42 PM, Alexander Behm 
 wrote:

 Not sure if this will work, but you could try enabling
 useNativeQuery in the driver. Theoretically that should send the query
 string including comments verbatim to Impala.

 On Wed, Dec 13, 2017 at 7:57 PM, Philip Zeyliger <
 

Re: Using comments in query for detailed monitoring

[Greg Rahn]

Unfortunately there is not a JIRA to track because the drivers are
maintained by a 3rd party and are not part of the Apache Impala project.
FWIW the version I tested was 2.5.43.1063

On Fri, Feb 16, 2018 at 4:52 PM, Sunil Parmar  wrote:

> Thanks is there a JIRA we can track ?
>
> Sunil Parmar
>
> On Tue, Feb 13, 2018 at 3:18 PM, Greg Rahn  wrote:
>
>> An upcoming release of the Impala driver will include a fix for not
>> removing comments when UseNativeQuery=1 is used.
>>
>> On Fri, Jan 26, 2018 at 10:29 AM, Greg Rahn  wrote:
>>
>>> I was able to repro this as well and checking in with Simba who authors
>>> the driver.
>>>
>>>
>>> On Wed, Jan 17, 2018 at 9:03 PM, Sunil Parmar 
>>> wrote:
>>>
 Tried with case change but no luck.  From the connection log.

 Jan 17 20:33:38.163 TRACE 1 
 com.cloudera.jdbc.common.SConnection.nativeSQL("select
 sum(1) from dwh.event_detail where org_group_id > 10; /*abcdefg*/"): +
 enter +
 Jan 17 20:33:38.163 TRACE 1 com.cloudera.hivecommon.core.H
 iveJDBCCommonConnection.toNativeSQL("select sum(1) from
 dwh.event_detail where org_group_id > 10; /*abcdefg*/"): + enter +
 Jan 17 20:33:38.164 TRACE 1 com.cloudera.hivecommon.dataen
 gine.HiveJDBCDataEngine.prepare("select sum(1) from dwh.event_detail
 where org_group_id > 10; "): + enter +
 Jan 17 20:33:38.166 TRACE 1 com.cloudera.hivecommon.dataen
 gine.HiveJDBCNativeQueryExecutor.HiveJDBCNativeQueryExecutor
 (com.cloudera.dsi.core.impl.DSILogger@1d057a39,
 com.cloudera.hivecommon.api.HS2Client@26be92ad, "select sum(1) from
 dwh.event_detail where org_group_id > 10;"): + enter +
 ...

 Sunil Parmar

 On Wed, Jan 17, 2018 at 5:52 PM, Vincent Tran 
 wrote:

> The properties are case sensitive so I would try UseNativeQuery just
> to dot the i's and cross the t's.
>
> Ensure that the property is recognized and applied by the driver by
> checking the JDBC log. (LogLevel=6;LogPath=/path/to/foo;)
>
> On Jan 17, 2018 6:45 PM, "Sunil Parmar"  wrote:
>
>> useNativeQuery didn't help.
>>
>> This is my code.
>>
>> public static void main(String[] args) throws IOException,
>> SQLException {
>> connectionUrl =
>>  "jdbc:impala://hadoop102.qa1.sac.int.company.com:21050/dwh;
>> AuthMech=3;UID=ldaptest;PWD=test;useNativeQuery=1";
>> jdbcDriverName = "com.cloudera.impala.jdbc41.Driver";
>> System.out.println("\n==
>> ===");
>> System.out.println("Cloudera Impala JDBC Example");
>> System.out.println("Using Connection URL: " + connectionUrl);
>> // System.out.println("Running Query: " + sqlStatement);
>> Connection con = null;
>> try {
>> Class.forName(jdbcDriverName);
>> con = DriverManager.getConnection(connectionUrl);
>> String query="select count( distinct mmdd) from
>> dwh.event_detail limit 1/*abcdefg1*/";
>> System.out.println("\n== Begin Query Results
>> ==");
>> System.out.println("\n== Running query=== " + query);
>> Statement st = con.createStatement();
>> ResultSet rs = st.executeQuery(query);
>> QueryTask.printResultSet(rs);
>>
>> } catch (SQLException e) {
>> e.printStackTrace();
>> } catch (Exception e) {
>> e.printStackTrace();
>> } finally {
>> con.close();
>>
>> }
>> }
>>
>> Same query when executed from HUE shows comments. Look at the
>> screenshot from CM.
>>
>>
>> ​
>>
>> Sunil Parmar
>>
>> On Wed, Dec 13, 2017 at 11:30 PM, Mostafa Mokhtar <
>> mmokh...@cloudera.com> wrote:
>>
>>> +1 on what Alex recommends.
>>> Highly recommend using useNativeQuery=1 all the time.
>>>
>>> Thanks
>>> Mostafa
>>>
>>> On Dec 13, 2017, at 9:42 PM, Alexander Behm 
>>> wrote:
>>>
>>> Not sure if this will work, but you could try enabling
>>> useNativeQuery in the driver. Theoretically that should send the query
>>> string including comments verbatim to Impala.
>>>
>>> On Wed, Dec 13, 2017 at 7:57 PM, Philip Zeyliger <
>>> phi...@cloudera.com> wrote:
>>>
 I’m a little surprised comments are being stripped, but adding (1=1
 or “”=“your comment”) to a where clause would work similarly. Impala 
 will
 simplify that away during query compilation.

 P

 On Wed, Dec 13, 2017 at 7:22 PM Sunil Parmar 
 

Re: Using comments in query for detailed monitoring

[Sunil Parmar]

Thanks is there a JIRA we can track ?

Sunil Parmar

On Tue, Feb 13, 2018 at 3:18 PM, Greg Rahn  wrote:

> An upcoming release of the Impala driver will include a fix for not
> removing comments when UseNativeQuery=1 is used.
>
> On Fri, Jan 26, 2018 at 10:29 AM, Greg Rahn  wrote:
>
>> I was able to repro this as well and checking in with Simba who authors
>> the driver.
>>
>>
>> On Wed, Jan 17, 2018 at 9:03 PM, Sunil Parmar 
>> wrote:
>>
>>> Tried with case change but no luck.  From the connection log.
>>>
>>> Jan 17 20:33:38.163 TRACE 1 
>>> com.cloudera.jdbc.common.SConnection.nativeSQL("select
>>> sum(1) from dwh.event_detail where org_group_id > 10; /*abcdefg*/"): +
>>> enter +
>>> Jan 17 20:33:38.163 TRACE 1 com.cloudera.hivecommon.core.H
>>> iveJDBCCommonConnection.toNativeSQL("select sum(1) from
>>> dwh.event_detail where org_group_id > 10; /*abcdefg*/"): + enter +
>>> Jan 17 20:33:38.164 TRACE 1 com.cloudera.hivecommon.dataen
>>> gine.HiveJDBCDataEngine.prepare("select sum(1) from dwh.event_detail
>>> where org_group_id > 10; "): + enter +
>>> Jan 17 20:33:38.166 TRACE 1 com.cloudera.hivecommon.dataen
>>> gine.HiveJDBCNativeQueryExecutor.HiveJDBCNativeQueryExecutor
>>> (com.cloudera.dsi.core.impl.DSILogger@1d057a39,
>>> com.cloudera.hivecommon.api.HS2Client@26be92ad, "select sum(1) from
>>> dwh.event_detail where org_group_id > 10;"): + enter +
>>> ...
>>>
>>> Sunil Parmar
>>>
>>> On Wed, Jan 17, 2018 at 5:52 PM, Vincent Tran 
>>> wrote:
>>>
 The properties are case sensitive so I would try UseNativeQuery just to
 dot the i's and cross the t's.

 Ensure that the property is recognized and applied by the driver by
 checking the JDBC log. (LogLevel=6;LogPath=/path/to/foo;)

 On Jan 17, 2018 6:45 PM, "Sunil Parmar"  wrote:

> useNativeQuery didn't help.
>
> This is my code.
>
> public static void main(String[] args) throws IOException,
> SQLException {
> connectionUrl =
>  "jdbc:impala://hadoop102.qa1.sac.int.company.com:21050/dwh;
> AuthMech=3;UID=ldaptest;PWD=test;useNativeQuery=1";
> jdbcDriverName = "com.cloudera.impala.jdbc41.Driver";
> System.out.println("\n==
> ===");
> System.out.println("Cloudera Impala JDBC Example");
> System.out.println("Using Connection URL: " + connectionUrl);
> // System.out.println("Running Query: " + sqlStatement);
> Connection con = null;
> try {
> Class.forName(jdbcDriverName);
> con = DriverManager.getConnection(connectionUrl);
> String query="select count( distinct mmdd) from
> dwh.event_detail limit 1/*abcdefg1*/";
> System.out.println("\n== Begin Query Results
> ==");
> System.out.println("\n== Running query=== " + query);
> Statement st = con.createStatement();
> ResultSet rs = st.executeQuery(query);
> QueryTask.printResultSet(rs);
>
> } catch (SQLException e) {
> e.printStackTrace();
> } catch (Exception e) {
> e.printStackTrace();
> } finally {
> con.close();
>
> }
> }
>
> Same query when executed from HUE shows comments. Look at the
> screenshot from CM.
>
>
> ​
>
> Sunil Parmar
>
> On Wed, Dec 13, 2017 at 11:30 PM, Mostafa Mokhtar <
> mmokh...@cloudera.com> wrote:
>
>> +1 on what Alex recommends.
>> Highly recommend using useNativeQuery=1 all the time.
>>
>> Thanks
>> Mostafa
>>
>> On Dec 13, 2017, at 9:42 PM, Alexander Behm 
>> wrote:
>>
>> Not sure if this will work, but you could try enabling useNativeQuery
>> in the driver. Theoretically that should send the query string including
>> comments verbatim to Impala.
>>
>> On Wed, Dec 13, 2017 at 7:57 PM, Philip Zeyliger > > wrote:
>>
>>> I’m a little surprised comments are being stripped, but adding (1=1
>>> or “”=“your comment”) to a where clause would work similarly. Impala 
>>> will
>>> simplify that away during query compilation.
>>>
>>> P
>>>
>>> On Wed, Dec 13, 2017 at 7:22 PM Sunil Parmar 
>>> wrote:
>>>
 Server version: impalad version 2.9.0-cdh5.12.1 RELEASE (build
 5131a031f4aa38c1e50c430373c55ca53e0517b9)

 One of the primary use of of Impala is it acts as a backend to a
 web application exposed to lot of users. We are trying to build 
 monitoring
 metrics per web page, logged in user and other things that are not

Re: Using comments in query for detailed monitoring

[Greg Rahn]

An upcoming release of the Impala driver will include a fix for not
removing comments when UseNativeQuery=1 is used.

On Fri, Jan 26, 2018 at 10:29 AM, Greg Rahn  wrote:

> I was able to repro this as well and checking in with Simba who authors
> the driver.
>
>
> On Wed, Jan 17, 2018 at 9:03 PM, Sunil Parmar 
> wrote:
>
>> Tried with case change but no luck.  From the connection log.
>>
>> Jan 17 20:33:38.163 TRACE 1 
>> com.cloudera.jdbc.common.SConnection.nativeSQL("select
>> sum(1) from dwh.event_detail where org_group_id > 10; /*abcdefg*/"): +
>> enter +
>> Jan 17 20:33:38.163 TRACE 1 com.cloudera.hivecommon.core.H
>> iveJDBCCommonConnection.toNativeSQL("select sum(1) from dwh.event_detail
>> where org_group_id > 10; /*abcdefg*/"): + enter +
>> Jan 17 20:33:38.164 TRACE 1 com.cloudera.hivecommon.dataen
>> gine.HiveJDBCDataEngine.prepare("select sum(1) from dwh.event_detail
>> where org_group_id > 10; "): + enter +
>> Jan 17 20:33:38.166 TRACE 1 com.cloudera.hivecommon.dataen
>> gine.HiveJDBCNativeQueryExecutor.HiveJDBCNativeQueryExecutor
>> (com.cloudera.dsi.core.impl.DSILogger@1d057a39,
>> com.cloudera.hivecommon.api.HS2Client@26be92ad, "select sum(1) from
>> dwh.event_detail where org_group_id > 10;"): + enter +
>> ...
>>
>> Sunil Parmar
>>
>> On Wed, Jan 17, 2018 at 5:52 PM, Vincent Tran 
>> wrote:
>>
>>> The properties are case sensitive so I would try UseNativeQuery just to
>>> dot the i's and cross the t's.
>>>
>>> Ensure that the property is recognized and applied by the driver by
>>> checking the JDBC log. (LogLevel=6;LogPath=/path/to/foo;)
>>>
>>> On Jan 17, 2018 6:45 PM, "Sunil Parmar"  wrote:
>>>
 useNativeQuery didn't help.

 This is my code.

 public static void main(String[] args) throws IOException, SQLException
 {
 connectionUrl =
  "jdbc:impala://hadoop102.qa1.sac.int.company.com:21050/dwh;
 AuthMech=3;UID=ldaptest;PWD=test;useNativeQuery=1";
 jdbcDriverName = "com.cloudera.impala.jdbc41.Driver";
 System.out.println("\n==
 ===");
 System.out.println("Cloudera Impala JDBC Example");
 System.out.println("Using Connection URL: " + connectionUrl);
 // System.out.println("Running Query: " + sqlStatement);
 Connection con = null;
 try {
 Class.forName(jdbcDriverName);
 con = DriverManager.getConnection(connectionUrl);
 String query="select count( distinct mmdd) from
 dwh.event_detail limit 1/*abcdefg1*/";
 System.out.println("\n== Begin Query Results
 ==");
 System.out.println("\n== Running query=== " + query);
 Statement st = con.createStatement();
 ResultSet rs = st.executeQuery(query);
 QueryTask.printResultSet(rs);

 } catch (SQLException e) {
 e.printStackTrace();
 } catch (Exception e) {
 e.printStackTrace();
 } finally {
 con.close();

 }
 }

 Same query when executed from HUE shows comments. Look at the
 screenshot from CM.


 ​

 Sunil Parmar

 On Wed, Dec 13, 2017 at 11:30 PM, Mostafa Mokhtar <
 mmokh...@cloudera.com> wrote:

> +1 on what Alex recommends.
> Highly recommend using useNativeQuery=1 all the time.
>
> Thanks
> Mostafa
>
> On Dec 13, 2017, at 9:42 PM, Alexander Behm 
> wrote:
>
> Not sure if this will work, but you could try enabling useNativeQuery
> in the driver. Theoretically that should send the query string including
> comments verbatim to Impala.
>
> On Wed, Dec 13, 2017 at 7:57 PM, Philip Zeyliger 
> wrote:
>
>> I’m a little surprised comments are being stripped, but adding (1=1
>> or “”=“your comment”) to a where clause would work similarly. Impala will
>> simplify that away during query compilation.
>>
>> P
>>
>> On Wed, Dec 13, 2017 at 7:22 PM Sunil Parmar 
>> wrote:
>>
>>> Server version: impalad version 2.9.0-cdh5.12.1 RELEASE (build
>>> 5131a031f4aa38c1e50c430373c55ca53e0517b9)
>>>
>>> One of the primary use of of Impala is it acts as a backend to a web
>>> application exposed to lot of users. We are trying to build monitoring
>>> metrics per web page, logged in user and other things that are not
>>> typically supplied in the query string. We were thinking to use query
>>> comments to pass such information along with query. It worked fine when
>>> query is executed using impala-shell; I can see the comments in the 
>>> impala
>>> monitoring page 

Re: Using comments in query for detailed monitoring

[Greg Rahn]

I was able to repro this as well and checking in with Simba who authors the
driver.


On Wed, Jan 17, 2018 at 9:03 PM, Sunil Parmar  wrote:

> Tried with case change but no luck.  From the connection log.
>
> Jan 17 20:33:38.163 TRACE 1 
> com.cloudera.jdbc.common.SConnection.nativeSQL("select
> sum(1) from dwh.event_detail where org_group_id > 10; /*abcdefg*/"): +
> enter +
> Jan 17 20:33:38.163 TRACE 1 com.cloudera.hivecommon.core.H
> iveJDBCCommonConnection.toNativeSQL("select sum(1) from dwh.event_detail
> where org_group_id > 10; /*abcdefg*/"): + enter +
> Jan 17 20:33:38.164 TRACE 1 com.cloudera.hivecommon.dataen
> gine.HiveJDBCDataEngine.prepare("select sum(1) from dwh.event_detail
> where org_group_id > 10; "): + enter +
> Jan 17 20:33:38.166 TRACE 1 com.cloudera.hivecommon.dataen
> gine.HiveJDBCNativeQueryExecutor.HiveJDBCNativeQueryExecutor
> (com.cloudera.dsi.core.impl.DSILogger@1d057a39,
> com.cloudera.hivecommon.api.HS2Client@26be92ad, "select sum(1) from
> dwh.event_detail where org_group_id > 10;"): + enter +
> ...
>
> Sunil Parmar
>
> On Wed, Jan 17, 2018 at 5:52 PM, Vincent Tran  wrote:
>
>> The properties are case sensitive so I would try UseNativeQuery just to
>> dot the i's and cross the t's.
>>
>> Ensure that the property is recognized and applied by the driver by
>> checking the JDBC log. (LogLevel=6;LogPath=/path/to/foo;)
>>
>> On Jan 17, 2018 6:45 PM, "Sunil Parmar"  wrote:
>>
>>> useNativeQuery didn't help.
>>>
>>> This is my code.
>>>
>>> public static void main(String[] args) throws IOException, SQLException {
>>> connectionUrl =
>>>  "jdbc:impala://hadoop102.qa1.sac.int.company.com:21050/dwh;
>>> AuthMech=3;UID=ldaptest;PWD=test;useNativeQuery=1";
>>> jdbcDriverName = "com.cloudera.impala.jdbc41.Driver";
>>> System.out.println("\n==
>>> ===");
>>> System.out.println("Cloudera Impala JDBC Example");
>>> System.out.println("Using Connection URL: " + connectionUrl);
>>> // System.out.println("Running Query: " + sqlStatement);
>>> Connection con = null;
>>> try {
>>> Class.forName(jdbcDriverName);
>>> con = DriverManager.getConnection(connectionUrl);
>>> String query="select count( distinct mmdd) from
>>> dwh.event_detail limit 1/*abcdefg1*/";
>>> System.out.println("\n== Begin Query Results
>>> ==");
>>> System.out.println("\n== Running query=== " + query);
>>> Statement st = con.createStatement();
>>> ResultSet rs = st.executeQuery(query);
>>> QueryTask.printResultSet(rs);
>>>
>>> } catch (SQLException e) {
>>> e.printStackTrace();
>>> } catch (Exception e) {
>>> e.printStackTrace();
>>> } finally {
>>> con.close();
>>>
>>> }
>>> }
>>>
>>> Same query when executed from HUE shows comments. Look at the screenshot
>>> from CM.
>>>
>>>
>>> ​
>>>
>>> Sunil Parmar
>>>
>>> On Wed, Dec 13, 2017 at 11:30 PM, Mostafa Mokhtar >> > wrote:
>>>
 +1 on what Alex recommends.
 Highly recommend using useNativeQuery=1 all the time.

 Thanks
 Mostafa

 On Dec 13, 2017, at 9:42 PM, Alexander Behm 
 wrote:

 Not sure if this will work, but you could try enabling useNativeQuery
 in the driver. Theoretically that should send the query string including
 comments verbatim to Impala.

 On Wed, Dec 13, 2017 at 7:57 PM, Philip Zeyliger 
 wrote:

> I’m a little surprised comments are being stripped, but adding (1=1 or
> “”=“your comment”) to a where clause would work similarly. Impala will
> simplify that away during query compilation.
>
> P
>
> On Wed, Dec 13, 2017 at 7:22 PM Sunil Parmar 
> wrote:
>
>> Server version: impalad version 2.9.0-cdh5.12.1 RELEASE (build
>> 5131a031f4aa38c1e50c430373c55ca53e0517b9)
>>
>> One of the primary use of of Impala is it acts as a backend to a web
>> application exposed to lot of users. We are trying to build monitoring
>> metrics per web page, logged in user and other things that are not
>> typically supplied in the query string. We were thinking to use query
>> comments to pass such information along with query. It worked fine when
>> query is executed using impala-shell; I can see the comments in the 
>> impala
>> monitoring page i.e. on port #25000 but when comments are sent over JDBC 
>> ;
>> they're not visible to Impalad.
>>
>> 1. Is there an configuration in Impala driver ? We're using
>>  
>>   com.cloudera.impala
>>   impala-jdbc
>>   2.5.30
>>   compile
>> 
>>
>> 2. Is 

Re: Using comments in query for detailed monitoring

[Sunil Parmar]

Tried with case change but no luck.  From the connection log.

Jan 17 20:33:38.163 TRACE 1
com.cloudera.jdbc.common.SConnection.nativeSQL("select
sum(1) from dwh.event_detail where org_group_id > 10; /*abcdefg*/"): +
enter +
Jan 17 20:33:38.163 TRACE 1 com.cloudera.hivecommon.core.
HiveJDBCCommonConnection.toNativeSQL("select sum(1) from dwh.event_detail
where org_group_id > 10; /*abcdefg*/"): + enter +
Jan 17 20:33:38.164 TRACE 1 com.cloudera.hivecommon.
dataengine.HiveJDBCDataEngine.prepare("select sum(1) from dwh.event_detail
where org_group_id > 10; "): + enter +
Jan 17 20:33:38.166 TRACE 1 com.cloudera.hivecommon.dataengine.
HiveJDBCNativeQueryExecutor.HiveJDBCNativeQueryExecutor(
com.cloudera.dsi.core.impl.DSILogger@1d057a39, com.cloudera.hivecommon.api.
HS2Client@26be92ad, "select sum(1) from dwh.event_detail where org_group_id
> 10;"): + enter +
...

Sunil Parmar

On Wed, Jan 17, 2018 at 5:52 PM, Vincent Tran  wrote:

> The properties are case sensitive so I would try UseNativeQuery just to
> dot the i's and cross the t's.
>
> Ensure that the property is recognized and applied by the driver by
> checking the JDBC log. (LogLevel=6;LogPath=/path/to/foo;)
>
> On Jan 17, 2018 6:45 PM, "Sunil Parmar"  wrote:
>
>> useNativeQuery didn't help.
>>
>> This is my code.
>>
>> public static void main(String[] args) throws IOException, SQLException {
>> connectionUrl =
>>  "jdbc:impala://hadoop102.qa1.sac.int.company.com:21050/dwh;
>> AuthMech=3;UID=ldaptest;PWD=test;useNativeQuery=1";
>> jdbcDriverName = "com.cloudera.impala.jdbc41.Driver";
>> System.out.println("\n==
>> ===");
>> System.out.println("Cloudera Impala JDBC Example");
>> System.out.println("Using Connection URL: " + connectionUrl);
>> // System.out.println("Running Query: " + sqlStatement);
>> Connection con = null;
>> try {
>> Class.forName(jdbcDriverName);
>> con = DriverManager.getConnection(connectionUrl);
>> String query="select count( distinct mmdd) from
>> dwh.event_detail limit 1/*abcdefg1*/";
>> System.out.println("\n== Begin Query Results
>> ==");
>> System.out.println("\n== Running query=== " + query);
>> Statement st = con.createStatement();
>> ResultSet rs = st.executeQuery(query);
>> QueryTask.printResultSet(rs);
>>
>> } catch (SQLException e) {
>> e.printStackTrace();
>> } catch (Exception e) {
>> e.printStackTrace();
>> } finally {
>> con.close();
>>
>> }
>> }
>>
>> Same query when executed from HUE shows comments. Look at the screenshot
>> from CM.
>>
>>
>> ​
>>
>> Sunil Parmar
>>
>> On Wed, Dec 13, 2017 at 11:30 PM, Mostafa Mokhtar 
>> wrote:
>>
>>> +1 on what Alex recommends.
>>> Highly recommend using useNativeQuery=1 all the time.
>>>
>>> Thanks
>>> Mostafa
>>>
>>> On Dec 13, 2017, at 9:42 PM, Alexander Behm 
>>> wrote:
>>>
>>> Not sure if this will work, but you could try enabling useNativeQuery in
>>> the driver. Theoretically that should send the query string including
>>> comments verbatim to Impala.
>>>
>>> On Wed, Dec 13, 2017 at 7:57 PM, Philip Zeyliger 
>>> wrote:
>>>
 I’m a little surprised comments are being stripped, but adding (1=1 or
 “”=“your comment”) to a where clause would work similarly. Impala will
 simplify that away during query compilation.

 P

 On Wed, Dec 13, 2017 at 7:22 PM Sunil Parmar 
 wrote:

> Server version: impalad version 2.9.0-cdh5.12.1 RELEASE (build
> 5131a031f4aa38c1e50c430373c55ca53e0517b9)
>
> One of the primary use of of Impala is it acts as a backend to a web
> application exposed to lot of users. We are trying to build monitoring
> metrics per web page, logged in user and other things that are not
> typically supplied in the query string. We were thinking to use query
> comments to pass such information along with query. It worked fine when
> query is executed using impala-shell; I can see the comments in the impala
> monitoring page i.e. on port #25000 but when comments are sent over JDBC ;
> they're not visible to Impalad.
>
> 1. Is there an configuration in Impala driver ? We're using
>  
>   com.cloudera.impala
>   impala-jdbc
>   2.5.30
>   compile
> 
>
> 2. Is there any other way to send such hints along with query ?
>
>
> Thanks,
> Sunil
>
>
>>>
>>

Re: Using comments in query for detailed monitoring

[Philip Zeyliger]

I’m a little surprised comments are being stripped, but adding (1=1 or
“”=“your comment”) to a where clause would work similarly. Impala will
simplify that away during query compilation.

P

On Wed, Dec 13, 2017 at 7:22 PM Sunil Parmar  wrote:

> Server version: impalad version 2.9.0-cdh5.12.1 RELEASE (build
> 5131a031f4aa38c1e50c430373c55ca53e0517b9)
>
> One of the primary use of of Impala is it acts as a backend to a web
> application exposed to lot of users. We are trying to build monitoring
> metrics per web page, logged in user and other things that are not
> typically supplied in the query string. We were thinking to use query
> comments to pass such information along with query. It worked fine when
> query is executed using impala-shell; I can see the comments in the impala
> monitoring page i.e. on port #25000 but when comments are sent over JDBC ;
> they're not visible to Impalad.
>
> 1. Is there an configuration in Impala driver ? We're using
>  
>   com.cloudera.impala
>   impala-jdbc
>   2.5.30
>   compile
> 
>
> 2. Is there any other way to send such hints along with query ?
>
>
> Thanks,
> Sunil
>
>