Okay, thanks. The design document mostly details the infrastructure for optimization strategies but doesn’t detail the strategies themselves. I take it the set of strategies are basically embodied in SparkStrategies.scala...is there a design doc/roadmap/JIRA issue detailing what strategies exist and which are planned?
Thanks, Nick > On Jan 22, 2015, at 7:45 PM, Michael Armbrust <[email protected]> wrote: > > Here is the initial design document for catalyst : > https://docs.google.com/document/d/1Hc_Ehtr0G8SQUg69cmViZsMi55_Kf3tISD9GPGU5M1Y/edit > > <https://docs.google.com/document/d/1Hc_Ehtr0G8SQUg69cmViZsMi55_Kf3tISD9GPGU5M1Y/edit> > > Strategies (many of which are in SparkStragegies.scala) are the part that > creates the physical operators from a catalyst logical plan. These operators > have execute() methods that actually call RDD operations. > > On Thu, Jan 22, 2015 at 3:19 PM, Nicholas Murphy <[email protected] > <mailto:[email protected]>> wrote: > Hi- > > Quick question: is there a design doc (or something more than “look at the > code”) for the query planner for Spark SQL (i.e., the component that > takes…Catalyst?…operator trees and translates them into SPARK operations)? > > Thanks, > Nick > --------------------------------------------------------------------- > To unsubscribe, e-mail: [email protected] > <mailto:[email protected]> > For additional commands, e-mail: [email protected] > <mailto:[email protected]> > >
