Github user marmbrus commented on the issue:
https://github.com/apache/spark/pull/17770
Some thoughts:
- We shouldn't have multiple optimizations for avoiding repeated analysis.
So if we decide to go this way then we should get rid of `resolveOperators`.
- I agree with Reynold, that the dummy operator could have confusing
side-effects down the road.
- The proposed generalization of `resolveOperators` to take a generic
stopping condition is reasonable, but I would only do it if we have more than
one use case.
Have we tried just using `resolveOperators` in more places? Does that fix
the performance issue?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [email protected] or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
