Github user gatorsmile commented on the issue:
https://github.com/apache/spark/pull/18687
So far, we do not support dynamic SQL statement, but this is a potential
feature we can explore in the future. A global statement cache and management
can reduce the optimization costs, especially when our CBO optimizer is more
advanced.
At the same time, it also resolves a more general issue. We can invalidate
all the physical plans that are built based on the stale info. Data cache is
just one of the examples. It could also include out-of-dated statistics. The
previous optimized plan might not make sense any more when the underlying
tables inserts/deletes a large amount of data.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [email protected] or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
