Github user marmbrus commented on the pull request:
https://github.com/apache/spark/pull/13147#issuecomment-221968384
Unfortunately, I don't have the code anymore, but I can try to sketch out
what I think the right solution looks like. Basically, the problem is that
`planLater` is a lie. It eagerly plans, taking the first available path.
The reason that a `Strategy` is able to return more than one plan is to
allow us to explore multiple valid options and evaluate their cost relative to
the other options. My proposal would be to change plan later from a function
to a special linking node:
```scala
case class PlanLater(tree: LogicalPlan) extends SparkPlan
```
In order for this to work we will need to make the query planner into an
iterator. When you call `next()` it does a depth first exploration using the
list of strategies until there are no more `PlanLater` nodes left. Along the
way it should remember the branches that it has not taken, so that they can be
returned for subsequent `next()` calls. What do you think?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [email protected] or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
