Thomas Prelle created SPARK-34528: ------------------------------------- Summary: View result are not consistent after a modification inside a struct of the table Key: SPARK-34528 URL: https://issues.apache.org/jira/browse/SPARK-34528 Project: Spark Issue Type: Bug Components: Spark Core Affects Versions: 3.2.0 Reporter: Thomas Prelle
After [https://github.com/apache/spark/pull/31368] work to simplify hive view resolution I found a bug because Hive allow you to change the order inside a struct 1) You create a table in hive with a struct: CREATE table test_struct (id int, sub STRUCT <a :INT, b:STRING>); 2) You insert data into it : INSERT INTO TABLE test_struct select 1, named_struct("a",1,"b","v1"); 3) Create a view on top of it : CREATE view test_view_struct as select id, sub from test_view_struct 4) Change the table struct reodoring the struct ALTER TABLE test_struct CHANGE COLUMN sub sub STRUCT < b:STRING,a :INT>; 5) Spark can not anymore query the view because struct in spark it's based on the position not on the name of the column. If the changement it's castable you can even have a silent failed -- This message was sent by Atlassian Jira (v8.3.4#803005) --------------------------------------------------------------------- To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org