Yuming Wang created SPARK-27655: ----------------------------------- Summary: Persistent the table statistics to metadata after fall back to hdfs Key: SPARK-27655 URL: https://issues.apache.org/jira/browse/SPARK-27655 Project: Spark Issue Type: Improvement Components: SQL Affects Versions: 3.0.0 Reporter: Yuming Wang
It's a real case. We need to join many times from different tables, but the statistics of some tables are incorrect. So this job need 43 min, It only need 3.7min after set {{spark.sql.statistics.persistentStatsAfterFallBack=true}}. -- This message was sent by Atlassian JIRA (v7.6.3#76005) --------------------------------------------------------------------- To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org