[jira] [Updated] (SPARK-6123) Parquet reader should use the schema of every file to create converter
[ https://issues.apache.org/jira/browse/SPARK-6123?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen updated SPARK-6123: - Assignee: Cheng Lian Parquet reader should use the schema of every file to create converter -- Key: SPARK-6123 URL: https://issues.apache.org/jira/browse/SPARK-6123 Project: Spark Issue Type: Bug Components: SQL Reporter: Yin Huai Assignee: Cheng Lian Priority: Critical Fix For: 1.5.0 For two parquet files for the same table having an array column, if values of the array in one file was created when containsNull was true and those in another file was created when containsNull was false, the containsNull in the merged schema will be true and we cannot correctly read data from the table created with containsNull=false. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Updated] (SPARK-6123) Parquet reader should use the schema of every file to create converter
[ https://issues.apache.org/jira/browse/SPARK-6123?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yin Huai updated SPARK-6123: Target Version/s: 1.5.0 (was: 1.4.0) Parquet reader should use the schema of every file to create converter -- Key: SPARK-6123 URL: https://issues.apache.org/jira/browse/SPARK-6123 Project: Spark Issue Type: Bug Components: SQL Reporter: Yin Huai Priority: Critical For two parquet files for the same table having an array column, if values of the array in one file was created when containsNull was true and those in another file was created when containsNull was false, the containsNull in the merged schema will be true and we cannot correctly read data from the table created with containsNull=false. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Updated] (SPARK-6123) Parquet reader should use the schema of every file to create converter
[ https://issues.apache.org/jira/browse/SPARK-6123?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Lian updated SPARK-6123: -- Assignee: (was: Cheng Lian) Parquet reader should use the schema of every file to create converter -- Key: SPARK-6123 URL: https://issues.apache.org/jira/browse/SPARK-6123 Project: Spark Issue Type: Bug Components: SQL Reporter: Yin Huai Priority: Critical For two parquet files for the same table having an array column, if values of the array in one file was created when containsNull was true and those in another file was created when containsNull was false, the containsNull in the merged schema will be true and we cannot correctly read data from the table created with containsNull=false. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Updated] (SPARK-6123) Parquet reader should use the schema of every file to create converter
[ https://issues.apache.org/jira/browse/SPARK-6123?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Lian updated SPARK-6123: -- Priority: Critical (was: Major) Parquet reader should use the schema of every file to create converter -- Key: SPARK-6123 URL: https://issues.apache.org/jira/browse/SPARK-6123 Project: Spark Issue Type: Bug Components: SQL Reporter: Yin Huai Assignee: Cheng Lian Priority: Critical For two parquet files for the same table having an array column, if values of the array in one file was created when containsNull was true and those in another file was created when containsNull was false, the containsNull in the merged schema will be true and we cannot correctly read data from the table created with containsNull=false. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
