[GitHub] [iceberg] mengshangxun opened a new issue, #6996: use uuid as primary key not in first column,flinksql wirte delete event to iceberg,spark read table not delete

Thu, 02 Mar 2023 19:52:07 -0800 


mengshangxun opened a new issue, #6996:
URL: https://github.com/apache/iceberg/issues/6996

   ### Apache Iceberg version
   
   1.1.0 (latest release)
   
   ### Query engine
   
   Spark
   
   ### Please describe the bug 🐞
   
   env:
   spark3.3,hadoop3.2.2,flink1.16,iceberg1.1.0
   desc:
   when primary key is uuid ,and not in first column,delete event after 
commited ,using spark query data,delete data is still exists.
   when change primary key to first column delete event commit success,spark 
query is normal.
   
   
   mysql source table :
   create table test (name varchar(50),id varchar(50));
   insert into test select '1',uuid();
   
   error flinksql ：
   `create database if not exists tempdb;
   create table ods_test_source (
   name varchar(50),
   id varcha(50),
   primary key (id) not enforced
   ) with (
   'connector' = 'mysql-cdc',
   'hostname'='xxx',
   'port'='3306',
   'username'='xxx',
   'password'='xxx',
   'database-name'='test',
   'table-name'='test'
   );
   create table tempdb.ods_test(
   name string,
   id string,
   primary key (id) not enforced
   ) with (
   'connector' = 'iceberg',
   'catalog-name'='iceberg_catalog',
   'catalog-type'='hadoop',
   'format-version'='2',
   'warehouse'='xxx',
   'write.upsert.enabled'='true'
   );
   set 'execution.checkpointing.interval'='120sec';
   set 'table.exec.sink.not-null-enforcer'='drop';
   insert into tempdb.ods_test select * from ods_test_source;
   `
   
   
![image](https://user-images.githubusercontent.com/7135066/222623055-0b5c32fd-2b91-43bd-bc39-0c0311d12259.png)
   
![image](https://user-images.githubusercontent.com/7135066/222626880-69394c47-2733-4c4a-8512-bf27eb386c35.png)
   
![image](https://user-images.githubusercontent.com/7135066/222626490-f76d8893-6a5f-4183-9e99-673346783029.png)
   
   
   
   
   
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]


---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]

Reply via email to