Yuming Wang created SPARK-27655:
-----------------------------------
Summary: Persistent the table statistics to metadata after fall
back to hdfs
Key: SPARK-27655
URL: https://issues.apache.org/jira/browse/SPARK-27655
Project: Spark
Issue Type: Improvement
Components: SQL
Affects Versions: 3.0.0
Reporter: Yuming Wang
It's a real case. We need to join many times from different tables, but the
statistics of some tables are incorrect. So this job need 43 min, It only need
3.7min after set {{spark.sql.statistics.persistentStatsAfterFallBack=true}}.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
