Yuming Wang created SPARK-34897:
-----------------------------------
Summary: The given data schema has less fields than the actual ORC
physical schema
Key: SPARK-34897
URL: https://issues.apache.org/jira/browse/SPARK-34897
Project: Spark
Issue Type: Bug
Components: SQL
Affects Versions: 3.1.1, 3.0.2, 3.2.0
Reporter: Yuming Wang
How to reproduce this issue:
{code:scala}
spark.sql(
"""
|CREATE TABLE `t1` (
| `_col0` INT,
| `_col1` STRING,
| `_col2` STRUCT<`c1`: STRING, `c2`: STRING, `c3`: STRING, `c4`: BIGINT>,
| `_col3` STRING)
|USING orc
|PARTITIONED BY (_col3)
|""".stripMargin)
spark.sql("INSERT INTO `t1` values(1, '2', null, '2021-02-01')")
spark.sql("SELECT _col2.c1, _col0 FROM `t1` WHERE _col3 = '2021-02-01'").show
{code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
