If this is specific to the CLoudera-provided xDBC dirvers, probably the CLoudera forums are the best place to proceed:
community.cloudera.com/t5/Interactive-Short-cycle-SQL/bd-p/Impala On Sat, Jun 16, 2018 at 4:17 AM, Fawze Abujaber <fawz...@gmail.com> wrote: > Hi Mostafa, > > Can these pre statements avoided? > > Any configuration or any set statement that can pypass these statements? > > On Fri, 15 Jun 2018 at 18:48 Mostafa Mokhtar <mmokh...@cloudera.com> wrote: >> >> @Lars Volker >> Many JDBC/ODBC drivers issue show tables & describe statements ahead of >> executing a query by default. >> >> >> >> On Fri, Jun 15, 2018 at 8:45 AM Lars Volker <l...@cloudera.com> wrote: >>> >>> As far as I know the driver should not generate additional statements. >>> Can you share what software you're using to connect to Impala through the >>> driver? I suspect that that software generated these queries, possibly to do >>> some schema discovery. >>> >>> Cheers, Lars >>> >>> On Thu, Jun 14, 2018 at 10:14 PM Jim Apple <jbap...@cloudera.com> wrote: >>>> >>>> I don’t think I understand the statement. Under what conditions are >>>> additional DDL statements generated by the driver? What exact query did you >>>> enter and what was generated instead? >>>> >>>> On Thu, Jun 14, 2018 at 5:44 PM Sunil Parmar <sunilosu...@gmail.com> >>>> wrote: >>>>> >>>>> The Impala JDBC driver generates additional DDL statements. >>>>> >>>>> select column1,column2 from table limit 0 >>>>> or >>>>> show tables >>>>> or >>>>> use dwh; >>>>> or >>>>> describe table >>>>> >>>>> If DDL are expensive; is there a way to avoid this ? >>>>> >>>>> Sunil Parmar >>>>> >>>>> >>>>> On Mon, May 21, 2018 at 5:48 PM Tim Armstrong <tarmstr...@cloudera.com> >>>>> wrote: >>>>>> >>>>>> SET is very cheap because it just changes a value in the user's >>>>>> session. There's no interaction with any other services. >>>>>> >>>>>> DDL operations can be a lot more expensive, although they don't >>>>>> compete with executing queries for resources. For the most part those DDL >>>>>> operations you mentioned consume resources in Java, generate load on >>>>>> metadata services like the HDFS namenode and Hive Metastore, and can >>>>>> block >>>>>> other DDL operations. We don't have great visibility at the moment into >>>>>> those resources consumed by metadata operation. >>>>>> >>>>>> On Mon, May 21, 2018 at 11:21 AM, Fawze Abujaber <fawz...@gmail.com> >>>>>> wrote: >>>>>>> >>>>>>> Thanks Tim, >>>>>>> >>>>>>> If i got your point right, then SET operation is affecting the client >>>>>>> Java memory and not considered as part of the impala daemon memory >>>>>>> limit, >>>>>>> right? >>>>>>> >>>>>>> Is this correct also for invalidate meta data and Refresh or alter >>>>>>> table ... recover partitions? Are all of these client operations? Are >>>>>>> they >>>>>>> use any resources assigned for impala daemon or impala resource pools? >>>>>>> >>>>>>> If they are client operations then I can use the used resources using >>>>>>> the Linux TOP command, if they are taking any resources from impala >>>>>>> daemon >>>>>>> memory limit or resource pool, I will be happy to know where I can >>>>>>> track the >>>>>>> resource usage of these DDL operations. >>>>>>> >>>>>>> On Mon, 21 May 2018 at 20:45 Tim Armstrong <tarmstr...@cloudera.com> >>>>>>> wrote: >>>>>>>> >>>>>>>> "SET" is very cheap - it only affects the client session on the >>>>>>>> Impala server that you're connected to. DDL operations are often more >>>>>>>> expensive because they require updating metadata globally. That can >>>>>>>> sometimes involve a bit of work (e.g. gather metadata about existing >>>>>>>> files >>>>>>>> on HDFS) or can involve the operation getting queued behind other >>>>>>>> metadata >>>>>>>> operations. >>>>>>>> >>>>>>>> On Sun, May 20, 2018 at 4:09 AM, Fawze Abujaber <fawz...@gmail.com> >>>>>>>> wrote: >>>>>>>>> >>>>>>>>> Hi Community, >>>>>>>>> >>>>>>>>> Does the DDL operations like alter, drop and create consume >>>>>>>>> resources? and does the set operations like set resource_pool=xxx also >>>>>>>>> consume resources? >>>>>>>>> >>>>>>>>> Yes, i'm aware these operations are quick but once they are running >>>>>>>>> from interfaces like Hue or MSTR through ODBC it's running till it get >>>>>>>>> timeout .... which may exceed few minutes >>>>>>>>> >>>>>>>>> -- >>>>>>>>> Take Care >>>>>>>>> Fawze Abujaber >>>>>>>> >>>>>>>> >>>>>>> -- >>>>>>> Take Care >>>>>>> Fawze Abujaber >>>>>> >>>>>> > -- > Take Care > Fawze Abujaber
