[ https://issues.apache.org/jira/browse/SPARK-28217?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Apache Spark reassigned SPARK-28217: ------------------------------------ Assignee: (was: Apache Spark) > Allow a custom statistics logical plan visitor to be plugged in. > ---------------------------------------------------------------- > > Key: SPARK-28217 > URL: https://issues.apache.org/jira/browse/SPARK-28217 > Project: Spark > Issue Type: Improvement > Components: SQL > Affects Versions: 3.0.0 > Reporter: Terry Kim > Priority: Major > Original Estimate: 120h > Remaining Estimate: 120h > > Spark currently has two built-in statistics plan visitor: > SizeInBytesOnlyStatsPlanVisitor and BasicStatsPlanVisitor. However, this is a > bit limited since there is no way to plug in a custom plan visitor - from > which a custom query optimizer can benefit from. > We can provide a Spark conf that the user can specify to override the > built-in plan visitor: > {code:scala} > // First create your custom stat plan visitor. > class MyStatsPlanVisitor extends LogicalPlanVisitor[Statistics] { > // Implement LogicalPlanVisitor[Statistics] trait > } > // Set the visitor via Spark conf. > spark.conf.set("spark.sql.catalyst.statsPlanVisitorClass", > "MyStatsPlanVisitor") > // Now, stat() on a LogicalPlan object will use MyStatsPlanVisitor as a stat > plan visitor. > {code} -- This message was sent by Atlassian JIRA (v7.6.3#76005) --------------------------------------------------------------------- To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org