[ https://issues.apache.org/jira/browse/SPARK-27655?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Yuming Wang updated SPARK-27655: -------------------------------- Attachment: disableFallBackToHdfs.png enableFallBackToHdfs.png > Persistent the table statistics to metadata after fall back to hdfs > ------------------------------------------------------------------- > > Key: SPARK-27655 > URL: https://issues.apache.org/jira/browse/SPARK-27655 > Project: Spark > Issue Type: Improvement > Components: SQL > Affects Versions: 3.0.0 > Reporter: Yuming Wang > Priority: Major > Attachments: disableFallBackToHdfs.png, enableFallBackToHdfs.png > > > It's a real case. We need to join many times from different tables, but the > statistics of some tables are incorrect. So this job need 43 min, It only > need 3.7min after set > {{spark.sql.statistics.persistentStatsAfterFallBack=true}}. -- This message was sent by Atlassian JIRA (v7.6.3#76005) --------------------------------------------------------------------- To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org