Hello folks, 

 

Would love to have a discussion of what are the next set of technical things to 
work on. To begin, here is a summary of what has gone into the PRs recently. 

 

#1: A large amount of focus has been on the query execution and optimization 
methods. The key PRs here are PR #113, Issue #69/PR172. These added, as noted 
in the PR discussion, an aggressive semi-join technique. A paper describing 
this work will appear in VLDB 2017 at the end of the summer. The paper is up at 
http://www.vldb.org/pvldb/vol10/p889-zhu.pdf. There are continual changes to 
the optimizer that largely Jianqiao fixes. AFAIK, there are no major features 
that are currently planned on this component. Please chime in if this is not 
true. 

 

#2: There is work on adding support for union/intersection operator lead by our 
newest committer, Tianrun Li. This originated  from him putting GRAIL on top of 
Quickstep. GRAIL is a way to map BSP-style graph analytics on a relational 
core, and was published (as a separate project) two years ago. See 
http://cidrdb.org/cidr2015/Papers/CIDR15_Paper20.pdf 

 

#3: The big active feature right now is on the distributed version, lead by 
Zuyu. (Zuyu please step in to correct the notes below as necessary.) In various 
discussions and PRs the plan he has put forward is to: 
Work on the partitioning in Quickstep as that is critical in the distributed 
version. There was robust discussion on this around Issue 73/PR 217
Moving data around nodes. The idea here is to use the design that we have with 
blocks having unique id, and letting a node pull a block from a remote node 
when it can’t find a block. This would use the GPRC mechanism that we have. I 
think it would be good to have a discussion of this design and get input from 
everyone who can help. This is clearly a big feature. Zuyu: Do you want to 
summarize your design in more detail when you have worked it out?
 

#4: Most of the Wisconsin folks are off for the summer, so I think things are 
going to slow down a bit on the development pace.  

 

As a note, we have been slack in having discussions of design on the dev list. 
I apologize for my part on this front. Having a discussion on the dev list is a 
great idea, and healthy for all in the long-run. So please feel free to chime 
in with your views and feel free to make a design proposal for specific parts 
that you think need work on.  

 

Happy Memorial Day everyone! 

 

Cheers,

Jignesh 

Reply via email to