Liang-Chi Hsieh created SPARK-8756:
--------------------------------------
Summary: Keep cached information and avoid re-calculating footers
in ParquetRelation2
Key: SPARK-8756
URL: https://issues.apache.org/jira/browse/SPARK-8756
Project: Spark
Issue Type: Bug
Components: SQL
Reporter: Liang-Chi Hsieh
Currently, in ParquetRelation2, footers are re-read every time refresh() is
called. But we can check if it is possibly changed before we do the reading
because reading all footers will be expensive when there are too many
partitions.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
