[GitHub] spark issue #15363: [SPARK-17791][SQL] Join reordering using star schema det...

2017-03-20 Thread cloud-fan
Github user cloud-fan commented on the issue: https://github.com/apache/spark/pull/15363 thanks, merging to master! The nest step is consolidating this with CBO, looking forward to it :) --- If your project is set up for it, you can reply to this email and have your reply

[GitHub] spark issue #15363: [SPARK-17791][SQL] Join reordering using star schema det...

2017-03-20 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15363 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/74847/ Test PASSed. ---

[GitHub] spark issue #15363: [SPARK-17791][SQL] Join reordering using star schema det...

2017-03-20 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15363 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #15363: [SPARK-17791][SQL] Join reordering using star schema det...

2017-03-20 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15363 **[Test build #74847 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/74847/testReport)** for PR 15363 at commit

[GitHub] spark issue #15363: [SPARK-17791][SQL] Join reordering using star schema det...

2017-03-19 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15363 **[Test build #74847 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/74847/testReport)** for PR 15363 at commit

[GitHub] spark issue #15363: [SPARK-17791][SQL] Join reordering using star schema det...

2017-03-19 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/15363 retest this please --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes

[GitHub] spark issue #15363: [SPARK-17791][SQL] Join reordering using star schema det...

2017-03-19 Thread ioana-delaney
Github user ioana-delaney commented on the issue: https://github.com/apache/spark/pull/15363 retest this please --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and

[GitHub] spark issue #15363: [SPARK-17791][SQL] Join reordering using star schema det...

2017-03-19 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15363 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #15363: [SPARK-17791][SQL] Join reordering using star schema det...

2017-03-19 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15363 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/74842/ Test FAILed. ---

[GitHub] spark issue #15363: [SPARK-17791][SQL] Join reordering using star schema det...

2017-03-19 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15363 **[Test build #74842 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/74842/testReport)** for PR 15363 at commit

[GitHub] spark issue #15363: [SPARK-17791][SQL] Join reordering using star schema det...

2017-03-19 Thread ioana-delaney
Github user ioana-delaney commented on the issue: https://github.com/apache/spark/pull/15363 @gatorsmile @cloud-fan I rewrote the test cases to align to the join reorder suite. Please take a look. Thanks. --- If your project is set up for it, you can reply to this email and have

[GitHub] spark issue #15363: [SPARK-17791][SQL] Join reordering using star schema det...

2017-03-19 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15363 **[Test build #74842 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/74842/testReport)** for PR 15363 at commit

[GitHub] spark issue #15363: [SPARK-17791][SQL] Join reordering using star schema det...

2017-03-19 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15363 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/74804/ Test PASSed. ---

[GitHub] spark issue #15363: [SPARK-17791][SQL] Join reordering using star schema det...

2017-03-19 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15363 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #15363: [SPARK-17791][SQL] Join reordering using star schema det...

2017-03-19 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15363 **[Test build #74804 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/74804/testReport)** for PR 15363 at commit

[GitHub] spark issue #15363: [SPARK-17791][SQL] Join reordering using star schema det...

2017-03-18 Thread ioana-delaney
Github user ioana-delaney commented on the issue: https://github.com/apache/spark/pull/15363 @gatorsmile Thank you. It fails on a clean build as well. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does

[GitHub] spark issue #15363: [SPARK-17791][SQL] Join reordering using star schema det...

2017-03-18 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/15363 Let me try to fix this flaky test. See the failure history of this test case.

[GitHub] spark issue #15363: [SPARK-17791][SQL] Join reordering using star schema det...

2017-03-18 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15363 **[Test build #74804 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/74804/testReport)** for PR 15363 at commit

[GitHub] spark issue #15363: [SPARK-17791][SQL] Join reordering using star schema det...

2017-03-18 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/15363 retest this please --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes

[GitHub] spark issue #15363: [SPARK-17791][SQL] Join reordering using star schema det...

2017-03-18 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15363 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/74803/ Test FAILed. ---

[GitHub] spark issue #15363: [SPARK-17791][SQL] Join reordering using star schema det...

2017-03-18 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15363 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #15363: [SPARK-17791][SQL] Join reordering using star schema det...

2017-03-18 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15363 **[Test build #74803 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/74803/testReport)** for PR 15363 at commit

[GitHub] spark issue #15363: [SPARK-17791][SQL] Join reordering using star schema det...

2017-03-18 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15363 **[Test build #74803 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/74803/testReport)** for PR 15363 at commit

[GitHub] spark issue #15363: [SPARK-17791][SQL] Join reordering using star schema det...

2017-03-18 Thread ioana-delaney
Github user ioana-delaney commented on the issue: https://github.com/apache/spark/pull/15363 @cloud-fan Thank you for the comments. I am looking at them. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project

[GitHub] spark issue #15363: [SPARK-17791][SQL] Join reordering using star schema det...

2017-03-17 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15363 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/74749/ Test PASSed. ---

[GitHub] spark issue #15363: [SPARK-17791][SQL] Join reordering using star schema det...

2017-03-17 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15363 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #15363: [SPARK-17791][SQL] Join reordering using star schema det...

2017-03-17 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15363 **[Test build #74749 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/74749/testReport)** for PR 15363 at commit

[GitHub] spark issue #15363: [SPARK-17791][SQL] Join reordering using star schema det...

2017-03-17 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15363 **[Test build #74749 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/74749/testReport)** for PR 15363 at commit

[GitHub] spark issue #15363: [SPARK-17791][SQL] Join reordering using star schema det...

2017-03-16 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/15363 It sounds like all the comments have been addressed. LGTM except one minor comment. cc @sameeragarwal @cloud-fan @hvanhovell --- If your project is set up for it, you can reply to

[GitHub] spark issue #15363: [SPARK-17791][SQL] Join reordering using star schema det...

2017-03-16 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15363 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/74685/ Test PASSed. ---

[GitHub] spark issue #15363: [SPARK-17791][SQL] Join reordering using star schema det...

2017-03-16 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15363 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #15363: [SPARK-17791][SQL] Join reordering using star schema det...

2017-03-16 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15363 **[Test build #74685 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/74685/testReport)** for PR 15363 at commit

[GitHub] spark issue #15363: [SPARK-17791][SQL] Join reordering using star schema det...

2017-03-16 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15363 **[Test build #74685 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/74685/testReport)** for PR 15363 at commit

[GitHub] spark issue #15363: [SPARK-17791][SQL] Join reordering using star schema det...

2017-03-15 Thread ioana-delaney
Github user ioana-delaney commented on the issue: https://github.com/apache/spark/pull/15363 @gatorsmile @hvanhovell @wzhfy @ron8hu Please let me know if we can move forward with this review. @wzhfy I removed the star-join call from CostBasedJoinReorder until the two are integrated.

[GitHub] spark issue #15363: [SPARK-17791][SQL] Join reordering using star schema det...

2017-03-15 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15363 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/74628/ Test PASSed. ---

[GitHub] spark issue #15363: [SPARK-17791][SQL] Join reordering using star schema det...

2017-03-15 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15363 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #15363: [SPARK-17791][SQL] Join reordering using star schema det...

2017-03-15 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15363 **[Test build #74628 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/74628/testReport)** for PR 15363 at commit

[GitHub] spark issue #15363: [SPARK-17791][SQL] Join reordering using star schema det...

2017-03-15 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15363 **[Test build #74628 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/74628/testReport)** for PR 15363 at commit

[GitHub] spark issue #15363: [SPARK-17791][SQL] Join reordering using star schema det...

2017-03-15 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/15363 See [the list of paper that cited K. Ono](https://scholar.google.com/scholar?start=0=en_sdt=0,5=0,5=5144610119819043766=) of `Measuring the Complexity of Join Enumeration in Query Optimization`.

[GitHub] spark issue #15363: [SPARK-17791][SQL] Join reordering using star schema det...

2017-03-14 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15363 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #15363: [SPARK-17791][SQL] Join reordering using star schema det...

2017-03-14 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15363 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/74551/ Test PASSed. ---

[GitHub] spark issue #15363: [SPARK-17791][SQL] Join reordering using star schema det...

2017-03-14 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15363 **[Test build #74551 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/74551/testReport)** for PR 15363 at commit

[GitHub] spark issue #15363: [SPARK-17791][SQL] Join reordering using star schema det...

2017-03-14 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15363 **[Test build #74551 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/74551/testReport)** for PR 15363 at commit

[GitHub] spark issue #15363: [SPARK-17791][SQL] Join reordering using star schema det...

2017-03-10 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15363 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #15363: [SPARK-17791][SQL] Join reordering using star schema det...

2017-03-10 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15363 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/74364/ Test PASSed. ---

[GitHub] spark issue #15363: [SPARK-17791][SQL] Join reordering using star schema det...

2017-03-10 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15363 **[Test build #74364 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/74364/testReport)** for PR 15363 at commit

[GitHub] spark issue #15363: [SPARK-17791][SQL] Join reordering using star schema det...

2017-03-10 Thread ioana-delaney
Github user ioana-delaney commented on the issue: https://github.com/apache/spark/pull/15363 @wzhfy I made some changes to call star join detection from ```CostBasedJoinReorder```. I didn’t integrate with the DP algorithm though. For now, I only made the star join available to cbo.

[GitHub] spark issue #15363: [SPARK-17791][SQL] Join reordering using star schema det...

2017-03-10 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15363 **[Test build #74364 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/74364/testReport)** for PR 15363 at commit

[GitHub] spark issue #15363: [SPARK-17791][SQL] Join reordering using star schema det...

2017-03-09 Thread ioana-delaney
Github user ioana-delaney commented on the issue: https://github.com/apache/spark/pull/15363 @wzhfy Thank you, Zhenhua. With the cost based optimizer in place, yes, it make sense to only call Star schema detection in the context of the new algorithm. There are two parts to

[GitHub] spark issue #15363: [SPARK-17791][SQL] Join reordering using star schema det...

2017-03-09 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15363 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #15363: [SPARK-17791][SQL] Join reordering using star schema det...

2017-03-09 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15363 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/74273/ Test PASSed. ---

[GitHub] spark issue #15363: [SPARK-17791][SQL] Join reordering using star schema det...

2017-03-09 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15363 **[Test build #74273 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/74273/testReport)** for PR 15363 at commit

[GitHub] spark issue #15363: [SPARK-17791][SQL] Join reordering using star schema det...

2017-03-09 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15363 **[Test build #74273 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/74273/testReport)** for PR 15363 at commit

[GitHub] spark issue #15363: [SPARK-17791][SQL] Join reordering using star schema det...

2017-03-09 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/15363 retest this please --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes

[GitHub] spark issue #15363: [SPARK-17791][SQL] Join reordering using star schema det...

2017-03-09 Thread wzhfy
Github user wzhfy commented on the issue: https://github.com/apache/spark/pull/15363 Can you move the entry point of `DetectStarSchemaJoin` from `ReorderJoin` to `CostBasedJoinReorder`? `ReorderJoin` is in a big batch and it will iterate multiple times. We don't have to optimize join

[GitHub] spark issue #15363: [SPARK-17791][SQL] Join reordering using star schema det...

2017-03-09 Thread wzhfy
Github user wzhfy commented on the issue: https://github.com/apache/spark/pull/15363 @ioana-delaney Yes, this is also what I'm thinking :) We can combine two algorithms in this way. --- If your project is set up for it, you can reply to this email and have your reply appear on

[GitHub] spark issue #15363: [SPARK-17791][SQL] Join reordering using star schema det...

2017-03-09 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15363 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #15363: [SPARK-17791][SQL] Join reordering using star schema det...

2017-03-09 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15363 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/74244/ Test FAILed. ---

[GitHub] spark issue #15363: [SPARK-17791][SQL] Join reordering using star schema det...

2017-03-08 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15363 **[Test build #74244 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/74244/testReport)** for PR 15363 at commit

[GitHub] spark issue #15363: [SPARK-17791][SQL] Join reordering using star schema det...

2017-03-08 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15363 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #15363: [SPARK-17791][SQL] Join reordering using star schema det...

2017-03-08 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15363 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/74221/ Test PASSed. ---

[GitHub] spark issue #15363: [SPARK-17791][SQL] Join reordering using star schema det...

2017-03-08 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15363 **[Test build #74221 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/74221/testReport)** for PR 15363 at commit

[GitHub] spark issue #15363: [SPARK-17791][SQL] Join reordering using star schema det...

2017-03-08 Thread ioana-delaney
Github user ioana-delaney commented on the issue: https://github.com/apache/spark/pull/15363 @wzhfy I've looked at the new CBO join reordering. The star schema detection can be used as follows: Assume a four-way join: A, B, C, D. Star schema join detection is called

[GitHub] spark issue #15363: [SPARK-17791][SQL] Join reordering using star schema det...

2017-03-08 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15363 **[Test build #74221 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/74221/testReport)** for PR 15363 at commit

[GitHub] spark issue #15363: [SPARK-17791][SQL] Join reordering using star schema det...

2017-03-08 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/15363 https://github.com/apache/spark/pull/17138 has been merged. Maybe we can start reviewing this PR @wzhfy @hvanhovell @cloud-fan @sameeragarwal ? --- If your project is set up for it, you can

[GitHub] spark issue #15363: [SPARK-17791][SQL] Join reordering using star schema det...

2017-03-07 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15363 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/74175/ Test PASSed. ---

[GitHub] spark issue #15363: [SPARK-17791][SQL] Join reordering using star schema det...

2017-03-07 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15363 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #15363: [SPARK-17791][SQL] Join reordering using star schema det...

2017-03-07 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15363 **[Test build #74175 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/74175/testReport)** for PR 15363 at commit

[GitHub] spark issue #15363: [SPARK-17791][SQL] Join reordering using star schema det...

2017-03-07 Thread ioana-delaney
Github user ioana-delaney commented on the issue: https://github.com/apache/spark/pull/15363 @wzhfy Star schema detection works with both positional join and cost based optimizer as it finds relationships among the joined tables. So I don't see a major reason for postponing its

[GitHub] spark issue #15363: [SPARK-17791][SQL] Join reordering using star schema det...

2017-03-07 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15363 **[Test build #74175 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/74175/testReport)** for PR 15363 at commit

[GitHub] spark issue #15363: [SPARK-17791][SQL] Join reordering using star schema det...

2017-03-07 Thread wzhfy
Github user wzhfy commented on the issue: https://github.com/apache/spark/pull/15363 Hi @ioana-delaney , I've discussed with databricks about this. Star schema definitely has its value. We will take a detailed review on this after the general join reorder goes in, and see how to

[GitHub] spark issue #15363: [SPARK-17791][SQL] Join reordering using star schema det...

2017-03-07 Thread ioana-delaney
Github user ioana-delaney commented on the issue: https://github.com/apache/spark/pull/15363 @gatorsmile @wzhfy Would you please review this PR. Thank you. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project

[GitHub] spark issue #15363: [SPARK-17791][SQL] Join reordering using star schema det...

2017-03-06 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15363 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/74068/ Test PASSed. ---

[GitHub] spark issue #15363: [SPARK-17791][SQL] Join reordering using star schema det...

2017-03-06 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15363 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #15363: [SPARK-17791][SQL] Join reordering using star schema det...

2017-03-06 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15363 **[Test build #74068 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/74068/testReport)** for PR 15363 at commit

[GitHub] spark issue #15363: [SPARK-17791][SQL] Join reordering using star schema det...

2017-03-06 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15363 **[Test build #74068 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/74068/testReport)** for PR 15363 at commit

[GitHub] spark issue #15363: [SPARK-17791][SQL] Join reordering using star schema det...

2017-01-03 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15363 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/70839/ Test PASSed. ---

[GitHub] spark issue #15363: [SPARK-17791][SQL] Join reordering using star schema det...

2017-01-03 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15363 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #15363: [SPARK-17791][SQL] Join reordering using star schema det...

2017-01-03 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15363 **[Test build #70839 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/70839/testReport)** for PR 15363 at commit

[GitHub] spark issue #15363: [SPARK-17791][SQL] Join reordering using star schema det...

2017-01-03 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15363 **[Test build #70839 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/70839/testReport)** for PR 15363 at commit

[GitHub] spark issue #15363: [SPARK-17791][SQL] Join reordering using star schema det...

2016-12-14 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15363 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/70159/ Test PASSed. ---

[GitHub] spark issue #15363: [SPARK-17791][SQL] Join reordering using star schema det...

2016-12-14 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15363 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #15363: [SPARK-17791][SQL] Join reordering using star schema det...

2016-12-14 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15363 **[Test build #70159 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/70159/testReport)** for PR 15363 at commit

[GitHub] spark issue #15363: [SPARK-17791][SQL] Join reordering using star schema det...

2016-12-14 Thread ioana-delaney
Github user ioana-delaney commented on the issue: https://github.com/apache/spark/pull/15363 The following updates were made: 1. Incorporate table and column statistics into the star join detection algorithm. Fact table is chosen based on table cardinality, and dimensions are

[GitHub] spark issue #15363: [SPARK-17791][SQL] Join reordering using star schema det...

2016-12-14 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15363 **[Test build #70159 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/70159/testReport)** for PR 15363 at commit

[GitHub] spark issue #15363: [SPARK-17791][SQL] Join reordering using star schema det...

2016-11-03 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15363 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #15363: [SPARK-17791][SQL] Join reordering using star schema det...

2016-11-03 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15363 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/68099/ Test PASSed. ---

[GitHub] spark issue #15363: [SPARK-17791][SQL] Join reordering using star schema det...

2016-11-03 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15363 **[Test build #68099 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/68099/consoleFull)** for PR 15363 at commit

[GitHub] spark issue #15363: [SPARK-17791][SQL] Join reordering using star schema det...

2016-11-03 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15363 **[Test build #68099 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/68099/consoleFull)** for PR 15363 at commit

[GitHub] spark issue #15363: [SPARK-17791][SQL] Join reordering using star schema det...

2016-11-03 Thread ioana-delaney
Github user ioana-delaney commented on the issue: https://github.com/apache/spark/pull/15363 retest this please --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and

[GitHub] spark issue #15363: [SPARK-17791][SQL] Join reordering using star schema det...

2016-11-03 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15363 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #15363: [SPARK-17791][SQL] Join reordering using star schema det...

2016-11-03 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15363 **[Test build #68089 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/68089/consoleFull)** for PR 15363 at commit

[GitHub] spark issue #15363: [SPARK-17791][SQL] Join reordering using star schema det...

2016-11-03 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15363 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/68089/ Test FAILed. ---

[GitHub] spark issue #15363: [SPARK-17791][SQL] Join reordering using star schema det...

2016-11-03 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15363 **[Test build #68089 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/68089/consoleFull)** for PR 15363 at commit

[GitHub] spark issue #15363: [SPARK-17791][SQL] Join reordering using star schema det...

2016-10-28 Thread ioana-delaney
Github user ioana-delaney commented on the issue: https://github.com/apache/spark/pull/15363 @davies Thank you for reviewing the code! I see this work as evolving and improving with the support of CBO. Without statistics and features such as cardinality and selectivity, we cannot

[GitHub] spark issue #15363: [SPARK-17791][SQL] Join reordering using star schema det...

2016-10-24 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15363 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/67474/ Test PASSed. ---

[GitHub] spark issue #15363: [SPARK-17791][SQL] Join reordering using star schema det...

2016-10-24 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15363 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #15363: [SPARK-17791][SQL] Join reordering using star schema det...

2016-10-24 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15363 **[Test build #67474 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/67474/consoleFull)** for PR 15363 at commit

[GitHub] spark issue #15363: [SPARK-17791][SQL] Join reordering using star schema det...

2016-10-24 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15363 **[Test build #67474 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/67474/consoleFull)** for PR 15363 at commit

[GitHub] spark issue #15363: [SPARK-17791][SQL] Join reordering using star schema det...

2016-10-05 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15363 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

  1   2   >