Hi Mostafa, Can these pre statements avoided?
Any configuration or any set statement that can pypass these statements? On Fri, 15 Jun 2018 at 18:48 Mostafa Mokhtar <mmokh...@cloudera.com> wrote: > @Lars Volker <l...@cloudera.com> > Many JDBC/ODBC drivers issue show tables & describe statements ahead of > executing a query by default. > > > > On Fri, Jun 15, 2018 at 8:45 AM Lars Volker <l...@cloudera.com> wrote: > >> As far as I know the driver should not generate additional statements. >> Can you share what software you're using to connect to Impala through the >> driver? I suspect that that software generated these queries, possibly to >> do some schema discovery. >> >> Cheers, Lars >> >> On Thu, Jun 14, 2018 at 10:14 PM Jim Apple <jbap...@cloudera.com> wrote: >> >>> I don’t think I understand the statement. Under what conditions are >>> additional DDL statements generated by the driver? What exact query did you >>> enter and what was generated instead? >>> >>> On Thu, Jun 14, 2018 at 5:44 PM Sunil Parmar <sunilosu...@gmail.com> >>> wrote: >>> >>>> The Impala JDBC driver generates additional DDL statements. >>>> >>>> select column1,column2 from table limit 0 >>>> or >>>> show tables >>>> or >>>> use dwh; >>>> or >>>> describe table >>>> >>>> If DDL are expensive; is there a way to avoid this ? >>>> >>>> Sunil Parmar >>>> >>>> >>>> On Mon, May 21, 2018 at 5:48 PM Tim Armstrong <tarmstr...@cloudera.com> >>>> wrote: >>>> >>>>> SET is very cheap because it just changes a value in the user's >>>>> session. There's no interaction with any other services. >>>>> >>>>> DDL operations can be a lot more expensive, although they don't >>>>> compete with executing queries for resources. For the most part those DDL >>>>> operations you mentioned consume resources in Java, generate load on >>>>> metadata services like the HDFS namenode and Hive Metastore, and can block >>>>> other DDL operations. We don't have great visibility at the moment into >>>>> those resources consumed by metadata operation. >>>>> >>>>> On Mon, May 21, 2018 at 11:21 AM, Fawze Abujaber <fawz...@gmail.com> >>>>> wrote: >>>>> >>>>>> Thanks Tim, >>>>>> >>>>>> If i got your point right, then SET operation is affecting the client >>>>>> Java memory and not considered as part of the impala daemon memory limit, >>>>>> right? >>>>>> >>>>>> Is this correct also for invalidate meta data and Refresh or alter >>>>>> table ... recover partitions? Are all of these client operations? Are >>>>>> they >>>>>> use any resources assigned for impala daemon or impala resource pools? >>>>>> >>>>>> If they are client operations then I can use the used resources using >>>>>> the Linux TOP command, if they are taking any resources from impala >>>>>> daemon >>>>>> memory limit or resource pool, I will be happy to know where I can track >>>>>> the resource usage of these DDL operations. >>>>>> >>>>>> On Mon, 21 May 2018 at 20:45 Tim Armstrong <tarmstr...@cloudera.com> >>>>>> wrote: >>>>>> >>>>>>> "SET" is very cheap - it only affects the client session on the >>>>>>> Impala server that you're connected to. DDL operations are often more >>>>>>> expensive because they require updating metadata globally. That can >>>>>>> sometimes involve a bit of work (e.g. gather metadata about existing >>>>>>> files >>>>>>> on HDFS) or can involve the operation getting queued behind other >>>>>>> metadata >>>>>>> operations. >>>>>>> >>>>>>> On Sun, May 20, 2018 at 4:09 AM, Fawze Abujaber <fawz...@gmail.com> >>>>>>> wrote: >>>>>>> >>>>>>>> Hi Community, >>>>>>>> >>>>>>>> Does the DDL operations like alter, drop and create consume >>>>>>>> resources? and does the set operations like set resource_pool=xxx also >>>>>>>> consume resources? >>>>>>>> >>>>>>>> Yes, i'm aware these operations are quick but once they are running >>>>>>>> from interfaces like Hue or MSTR through ODBC it's running till it get >>>>>>>> timeout .... which may exceed few minutes >>>>>>>> >>>>>>>> -- >>>>>>>> Take Care >>>>>>>> Fawze Abujaber >>>>>>>> >>>>>>> >>>>>>> -- >>>>>> Take Care >>>>>> Fawze Abujaber >>>>>> >>>>> >>>>> -- Take Care Fawze Abujaber
