xiarixiaoyao commented on issue #6209:
URL: https://github.com/apache/hudi/issues/6209#issuecomment-1198804788
@yihua @lz1984sh
yes, schema evolution can support that case;
```
spark.sql("set hoodie.schema.on.read.enable=true")
// NOTE: This is required since as this tests use type coercions
which were only permitted in Spark 2.x
// and are disallowed now by default in Spark 3.x
spark.sql("set spark.sql.storeAssignmentPolicy=legacy")
spark.sql(
s"""
|create table $tableName (
| id int, comb int, col0 int, col1 bigint, col2 float, col3
double, col4 decimal(10,4), par date
|) using hudi
| location '$tablePath'
| options (
| type = '$tableType',
| primaryKey = 'id',
| preCombineField = 'comb'
| )
| partitioned by (par)
""".stripMargin)
spark.sql(
s"""
| insert into $tableName values
|
(1,1,11,100001,101.01,1001.0001,100001.0001,TIMESTAMP'2021-12-25'),
|
(2,2,12,100002,102.02,1002.0002,100002.0002,TIMESTAMP'2021-12-25')
|""".stripMargin)
// alter table change
spark.sql(s"alter table $tableName alter column col4 type
decimal(18, 6)")
spark.sql(
s"""
| insert into $tableName values
|
(1,1,11,100001,101.01,1001.0001,200001.000001,TIMESTAMP'2021-12-25'),
|
(2,2,12,100002,102.02,1002.0002,300002.000002,TIMESTAMP'2021-12-25')
|""".stripMargin)
spark.sql(s"select col4 from $tableName").show(false)
spark.sql(s"select * from $tableName").printSchema()
```
+-------------+
|col4 |
+-------------+
|300002.000002|
|200001.000001|
+-------------+
root
|-- _hoodie_commit_time: string (nullable = true)
|-- _hoodie_commit_seqno: string (nullable = true)
|-- _hoodie_record_key: string (nullable = true)
|-- _hoodie_partition_path: string (nullable = true)
|-- _hoodie_file_name: string (nullable = true)
|-- id: integer (nullable = true)
|-- comb: integer (nullable = true)
|-- col0: integer (nullable = true)
|-- col1: long (nullable = true)
|-- col2: float (nullable = true)
|-- col3: double (nullable = true)
|-- col4: decimal(18,6) (nullable = true)
|-- par: date (nullable = true)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
