Re: query planner design doc?

2015-01-23 Thread Michael Armbrust
No, are you looking for something in particular? On Fri, Jan 23, 2015 at 9:44 AM, Nicholas Murphy wrote: > Okay, thanks. The design document mostly details the infrastructure for > optimization strategies but doesn’t detail the strategies themselves. I > take it the set of strategies are basic

Re: query planner design doc?

2015-01-23 Thread Nicholas Murphy
Okay, thanks. The design document mostly details the infrastructure for optimization strategies but doesn’t detail the strategies themselves. I take it the set of strategies are basically embodied in SparkStrategies.scala...is there a design doc/roadmap/JIRA issue detailing what strategies exi

Re: query planner design doc?

2015-01-22 Thread Michael Armbrust
Here is the initial design document for catalyst : https://docs.google.com/document/d/1Hc_Ehtr0G8SQUg69cmViZsMi55_Kf3tISD9GPGU5M1Y/edit Strategies (many of which are in SparkStragegies.scala) are the part that creates the physical operators from a catalyst logical plan. These operators have execu

query planner design doc?

2015-01-22 Thread Nicholas Murphy
Hi- Quick question: is there a design doc (or something more than “look at the code”) for the query planner for Spark SQL (i.e., the component that takes…Catalyst?…operator trees and translates them into SPARK operations)? Thanks, Nick ---