Github user wzhfy commented on a diff in the pull request:
https://github.com/apache/spark/pull/16401#discussion_r94714976
--- Diff:
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/plans/logical/LogicalPlan.scala
---
@@ -95,6 +96,29 @@ abstract class LogicalPlan extends
QueryPlan[LogicalPlan] with Logging {
}
/**
+ * Returns the default statistics or statistics estimated by cbo based
on configuration.
+ */
+ final def planStats(conf: CatalystConf): Statistics = {
+ if (conf.cboEnabled) {
+ if (estimatedStats.isEmpty) {
+ estimatedStats = Some(cboStatistics(conf))
+ }
+ estimatedStats.get
+ } else {
+ statistics
+ }
+ }
+
+ /**
+ * Returns statistics estimated by cbo. If the plan doesn't override
this, it returns the
+ * default statistics.
+ */
+ protected def cboStatistics(conf: CatalystConf): Statistics = statistics
+
+ /** A cache for the estimated statistics, such that it will only be
computed once. */
+ private var estimatedStats: Option[Statistics] = None
--- End diff --
`estimatedStats` is a cache for the `def cboStatistics()` where stats is
calculated by using column stats. And because I want to pass conf into
cboStatistics(some parameters may be needed during estimation in the future), I
can't use a lazy val, and I do this cache explicitly with `estimatedStats`.
Yes, the naming causes ambiguity, can you give it a better name?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [email protected] or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
