What is the new naming hierarchy? Do we plan to rename all existing relevant classes to better distinguish whether they are talking about e.g. instances vs. fragments? Might be nice to do that first to make the convention clear and make it easier to follow exactly what state lives where and how it's being moved around.
On Wed, Jul 20, 2016 at 10:59 AM, Marcel Kornacker <[email protected]> wrote: > Regarding FragmentExecState: > > We could possibly do away with that and migrate whatever pieces of it > are still needed into RuntimeState (and possibly rename RuntimeState > to make it conform to the naming hierarchy). FragmentExecState is a > bit of a misnomer at the moment anyway, because it's really > FragmentInstanceExecState. If we really need a per-fragment struct, we > can always introduce that. > > However, that's a larger change, so maybe stage that after the > introduction of QueryExecMgr/QueryExecState. > > On Wed, Jul 20, 2016 at 10:24 AM, Lars Volker <[email protected]> wrote: > > Sure. Currently we duplicate state in fragment instances of a single > query, > > for example each plan fragment executor creates its own descriptor table, > > which could actually be shared among all fragment instances. For RPC > > batching (IMPALA-2550) we will transmit the descriptor table only once, > and > > it seems wasteful to re-build it for every fragment instance. This > probably > > holds true for other information as well, that is actually the same for > > multiple fragment instances of a query, such as runtime filters. > > > > On Wed, Jul 20, 2016 at 7:15 PM, Henry Robinson <[email protected]> > wrote: > > > >> Could you say something about the use cases you're trying to address? > >> That'll make it easier to review the design. > >> > >> On 20 July 2016 at 10:08, Lars Volker <[email protected]> wrote: > >> > >> > Hi all, > >> > > >> > I started drafting a header file for a query-wide QueryExecState class > >> > here: https://gerrit.cloudera.org/#/c/3686/2 > >> > > >> > It's not much and I couldn't figure out how much of the overall > changes > >> > needed for RPC batching should be included in this change. Most > notably > >> we > >> > might want to replace FragmentExecState in FragmentMgr (now > QueryExecMgr) > >> > with QueryExecState altogether and drive execution from that class. On > >> the > >> > other hand keeping the QueryExecState as simple as possible for now > would > >> > make the change less intrusive. > >> > > >> > I'd be glad for feedback on this first draft. > >> > > >> > Thanks, Lars > >> > > >> > >> > >> > >> -- > >> Henry Robinson > >> Software Engineer > >> Cloudera > >> 415-994-6679 > >> >
