SET is very cheap because it just changes a value in the user's session. There's no interaction with any other services.
DDL operations can be a lot more expensive, although they don't compete with executing queries for resources. For the most part those DDL operations you mentioned consume resources in Java, generate load on metadata services like the HDFS namenode and Hive Metastore, and can block other DDL operations. We don't have great visibility at the moment into those resources consumed by metadata operation. On Mon, May 21, 2018 at 11:21 AM, Fawze Abujaber <fawz...@gmail.com> wrote: > Thanks Tim, > > If i got your point right, then SET operation is affecting the client Java > memory and not considered as part of the impala daemon memory limit, right? > > Is this correct also for invalidate meta data and Refresh or alter table > ... recover partitions? Are all of these client operations? Are they use > any resources assigned for impala daemon or impala resource pools? > > If they are client operations then I can use the used resources using the > Linux TOP command, if they are taking any resources from impala daemon > memory limit or resource pool, I will be happy to know where I can track > the resource usage of these DDL operations. > > On Mon, 21 May 2018 at 20:45 Tim Armstrong <tarmstr...@cloudera.com> > wrote: > >> "SET" is very cheap - it only affects the client session on the Impala >> server that you're connected to. DDL operations are often more expensive >> because they require updating metadata globally. That can sometimes involve >> a bit of work (e.g. gather metadata about existing files on HDFS) or can >> involve the operation getting queued behind other metadata operations. >> >> On Sun, May 20, 2018 at 4:09 AM, Fawze Abujaber <fawz...@gmail.com> >> wrote: >> >>> Hi Community, >>> >>> Does the DDL operations like alter, drop and create consume resources? >>> and does the set operations like set resource_pool=xxx also consume >>> resources? >>> >>> Yes, i'm aware these operations are quick but once they are running from >>> interfaces like Hue or MSTR through ODBC it's running till it get timeout >>> .... which may exceed few minutes >>> >>> -- >>> Take Care >>> Fawze Abujaber >>> >> >> -- > Take Care > Fawze Abujaber >
