Sure. Currently we duplicate state in fragment instances of a single query, for example each plan fragment executor creates its own descriptor table, which could actually be shared among all fragment instances. For RPC batching (IMPALA-2550) we will transmit the descriptor table only once, and it seems wasteful to re-build it for every fragment instance. This probably holds true for other information as well, that is actually the same for multiple fragment instances of a query, such as runtime filters.
On Wed, Jul 20, 2016 at 7:15 PM, Henry Robinson <[email protected]> wrote: > Could you say something about the use cases you're trying to address? > That'll make it easier to review the design. > > On 20 July 2016 at 10:08, Lars Volker <[email protected]> wrote: > > > Hi all, > > > > I started drafting a header file for a query-wide QueryExecState class > > here: https://gerrit.cloudera.org/#/c/3686/2 > > > > It's not much and I couldn't figure out how much of the overall changes > > needed for RPC batching should be included in this change. Most notably > we > > might want to replace FragmentExecState in FragmentMgr (now QueryExecMgr) > > with QueryExecState altogether and drive execution from that class. On > the > > other hand keeping the QueryExecState as simple as possible for now would > > make the change less intrusive. > > > > I'd be glad for feedback on this first draft. > > > > Thanks, Lars > > > > > > -- > Henry Robinson > Software Engineer > Cloudera > 415-994-6679 >
