yucai created SPARK-25207: ----------------------------- Summary: Case-insensitve field resolution for filter pushdown when reading Parquet Key: SPARK-25207 URL: https://issues.apache.org/jira/browse/SPARK-25207 Project: Spark Issue Type: Bug Components: SQL Affects Versions: 2.4.0 Reporter: yucai
Currently, filter pushdown will not work if Parquet schema and Hive metastore schema are in different letter cases even spark.sql.caseSensitive is false. Like the below case: {code:java} spark.range(10).write.parquet("/tmp/data") sql("DROP TABLE t") sql("CREATE TABLE t (ID LONG) USING parquet LOCATION '/tmp/data'") sql("select * from t where id > 0").show{code} No filter will be pushed down. -- This message was sent by Atlassian JIRA (v7.6.3#76005) --------------------------------------------------------------------- To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org