Seems like folks are in agreement. I will go ahead and create a branch.
On Mon, Jun 23, 2014 at 11:40 AM, John Pullokkaran
jpullokka...@hortonworks.com wrote:
I see that design doc doesn't talk about plug and play aspect of cost
model; and it also doesn't make it clear that cost model
Created a branch at https://svn.apache.org/repos/asf/hive/branches/cbo
Folks interested to contribute feel free to create jiras and provide
patches for branch.
Lets advance Hive's optimizer by teaching it how to cost different plans!
Thanks,
Ashutosh
On Mon, Jun 23, 2014 at 11:13 PM, Ashutosh
Hi Ashutosh,
Thanks for your information. I do have some questions, but my concern is
more on the design doc than branching. Nevertheless, I think it would be
very helpful to clarify in the design before we actually put a lot of
effort.
From the design doc, it seems that the cost estimation is
Following may help in reducing the confusion:
1. In design doc the cost formula is for choosing Join Algorithm. The cost
formula as described in the doc assumes Tez execution.
2. However current work on CBO doesn’t include Join algorithm selection.
Instead it rearranges Join based on Join
Thanks for the clarification. I'm happily on board with this as long as our
approach takes account of the differences between execution engines. While
MR and Tez might be similar, there could be new execution engines in the
future which might not be that similar. Ideally, all execution engines
I see that design doc doesn't talk about plug and play aspect of cost
model; and it also doesn't make it clear that cost model described is for
Join Algorithm selection; also it doesn't have cost model for MR.
I will update the doc appropriately.
Thanks
John
--
CONFIDENTIALITY NOTICE
NOTICE:
Hi all,
Some of you may have noticed that cost based optimizer work is going on at
HIVE-5775 John has put up an initial patch there as well. But there is a
lot more work that needs to be done. Following our tradition of large
feature work in branch, I propose that we create a branch and commit
