+cc:[email protected] On Wed, Mar 16, 2016 at 10:38 PM, 林言 <[email protected]> wrote:
> We know that each planfragment has only one destination node in Impala. > Now we want to send the intermidiate results of this fragment to more than > one destination node. But we're only familiar with the data structure and > execution flow in the frontend. So we wonder where we should modify in > the thrift and backend to make it work. > Can you share some design document? So we can know more design details of > Impala. > If you are interested in multi-query adaption in Impala, would you like to > work with us in a new branch of Impala? > > > ------------------------------ > Yan Lin > > > *From:* Jim Apple <[email protected]> > *Date:* 2016-03-17 01:06 > *To:* Impala Dev <[email protected]> > *CC:* bbbbaai <[email protected]> > *Subject:* Re: About Cooperating For A Better Impala > I'm sure everyone will be delighted to have more communication and > cooperation, including reading the papers and the code. Can you share those > today, or is that part of the "puzzle" of "sharing intermediate results"? > Is there anything we can do to help with your puzzlement? > > On Wednesday, March 16, 2016 at 12:11:05 AM UTC-7, 林言 wrote: >> >> Dear Sir/Madam： >> Hello! I am Yan Lin, a master candidate in ZheJiang >> University(CHN) in laboratory "PCL" (http://percom.zju.edu.cn/). Our lab >> has done many works on Impala, as follows: >> 1. We proposed an Impala query optimization method based >> on bushy-tree and an IMPROVED-MCCHYP algorithm [1]. And we implemented >> our method and algorithm in Impala. >> 2. We proposed a replication-selection based scheduling >> algorithm and implemented it in Impala [2]. >> 3. Some of my fellows are now developing a simulator of >> Impala called ImpalaSim and writing the corresponding paper [3]. >> Recently, we put our focus on multi-query optimization which >> sufficiently exploits >> common sub-expressions of batched queries and improves the efficiency. We >> have modified some source code, and the modified Impala can already >> execute multiple queries in the same query context. But we still feel >> puzzled with sharing intermediate results. We hope for more >> communication and cooperation in every aspect. We all want a better >> Impala. >> Thank you for your attention! Hope to hear from you soon! >> >> >> Yours Sincerely, >> >> >> Yan Lin >> >> Reference: >> [1] >> Bushy Tree and Improved-McCHyp Algorithm Based Impala Query Optimization >> [2] >> Replication-Selection based Scheduling for Impala Parallel Query Execution >> [3] ImpalaSim：Discrete Event Simulation Platform for Impala >> System >> ------------------------------ >> Yan Lin >> >
