fujianhua168 opened a new issue, #6285:
URL: https://github.com/apache/hudi/issues/6285
**Describe the problem you faced**
table description: a cow non-partition table (tblproperties with
primaryKey and preCombineField) .
sql environment: spark sql
describe:
1. if I action the **insert or update** action with same primary key
and preCombineField field value change to smaller, Unexpectedly the action
effect (note: preCombineField field value change smaller should not effect,
but actual this is effect);
2. if I action the **merge into** action with same primary key and
preCombineField field value change to smaller yet, it action not effect
expectedly( note: I think this is normal);
the 1 and 2 abover upsert action seem to conflict，Strangely enough.
**To Reproduce**
Steps to reproduce the behavior with SPARKSQL:
--step 1: create table
drop table hudi_cow_pk_cbfield_tbl;
create table hudi_cow_pk_cbfield_tbl (
id bigint,
name string,
ts bigint
) using hudi
tblproperties (
type = 'cow',
primaryKey = 'id',
preCombineField = 'ts',
hoodie.datasource.write.hive_style_partitioning = false
)
;
--step 2: insert into a recored with primaryKey=1, preCombineField=1000
insert into hudi_cow_pk_cbfield_tbl select 1, 'a0', 1000;
--step 3: 'insert' action with same primaryKey, but change the
preCombineField value to the smaller value 100, Unexpectedly the action effect
(note:is not normal)
insert into hudi_cow_pk_cbfield_tbl select 1, 'a0_new', 100;
select * from hudi_cow_pk_cbfield_tbl;
--step 4: 'update' action with same primaryKey, but change the
preCombineField value to the smaller value 20, Unexpectedly the action effect
yet (note:is not normal)
update hudi_cow_pk_cbfield_tbl set name='a1_new',ts=20 where id= 1;
select * from hudi_cow_pk_cbfield_tbl;
--step 5: 'merge into' action with same primaryKey, but change the
preCombineField value to the smaller value 10, the action effect(note: I think
this is normal)
merge into hudi_cow_pk_cbfield_tbl as target
using (select 1 as id,'a1_merge' as name,10 as ts) as source
on target.id = source.id
when matched then update set *
when not matched then insert *
;
select * from hudi_cow_pk_cbfield_tbl;
**Expected behavior**
if the preCombineField value chagne to smaller, the row data upsert
action should not effect( i experiment in the cow-partition table, mor-table,
it's normal)
**Environment Description**
* Hudi version :0.11.1
* Spark version :3.2.1
* Hive version :3.1.0
* Hadoop version :3.1.1
* Storage (HDFS/S3/GCS..) :HDFS
* Running on Docker? (yes/no) :no
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
