[
https://issues.apache.org/jira/browse/HUDI-8827?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Davis Zhang updated HUDI-8827:
------------------------------
Description:
[https://github.com/apache/hudi/pull/12577]
test repro that create a table with precombine key column "ts bigint" and
insert 1 record, it always complains for the ts column value it cannot cast
string to bigint, even though the insert explicitly cast the value as bigint.
The issue is because spark enforce the partition column to be the last column.
Need to follow up with doc update.
```
spark.sql(
s"""
|create table $tableName (|
|id bigint,|
|name string,|
|price double,|
|ts bigint,|
|dt string|
|) using hudi|
|tblproperties (|
|type = 'mor',|
|primaryKey = 'id',|
|precombineKey = 'ts'|
|)|
|partitioned by(dt)|
|location '${tmp.getCanonicalPath}'
""".stripMargin)
```
|
was:
[https://github.com/apache/hudi/pull/12577]
test repro that create a table with precombine key column "ts bigint" and
insert 1 record, it always complains for the ts column value it cannot cast
string to bigint, even though the insert explicitly cast the value as bigint.
The issue is because spark enforce the partition column to be the last column.
```
spark.sql(
s"""
| create table $tableName (
| id bigint,
| name string,
| price double,
| ts bigint,
| dt string
| ) using hudi
| tblproperties (
| type = 'mor',
| primaryKey = 'id',
| precombineKey = 'ts'
| )
| partitioned by(dt)
| location '${tmp.getCanonicalPath}'
""".stripMargin)
```
> Insert type wrongly interpreted insert value as string while it is int
> ----------------------------------------------------------------------
>
> Key: HUDI-8827
> URL: https://issues.apache.org/jira/browse/HUDI-8827
> Project: Apache Hudi
> Issue Type: Sub-task
> Reporter: Davis Zhang
> Priority: Major
> Fix For: 1.0.1
>
>
>
> [https://github.com/apache/hudi/pull/12577]
> test repro that create a table with precombine key column "ts bigint" and
> insert 1 record, it always complains for the ts column value it cannot cast
> string to bigint, even though the insert explicitly cast the value as bigint.
>
> The issue is because spark enforce the partition column to be the last
> column. Need to follow up with doc update.
> ```
> spark.sql(
> s"""
> |create table $tableName (|
> |id bigint,|
> |name string,|
> |price double,|
> |ts bigint,|
> |dt string|
> |) using hudi|
> |tblproperties (|
> |type = 'mor',|
> |primaryKey = 'id',|
> |precombineKey = 'ts'|
> |)|
> |partitioned by(dt)|
> |location '${tmp.getCanonicalPath}'
> """.stripMargin)
> ```
> |
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
