SamealD opened a new issue, #5661: URL: https://github.com/apache/seatunnel/issues/5661
### Search before asking - [X] I had searched in the [issues](https://github.com/apache/seatunnel/issues?q=is%3Aissue+label%3A%22bug%22) and found no similar issues. ### What happened Iceberg Source use multiple parallelism encountering lost data. when parallelism=1 ,it won't be lost data. But when parallelism=2 or more ,it will lost data. ### SeaTunnel Version SeaTunnel 2.3.3 ### SeaTunnel Config ```conf env { parallelism = 2 job.mode = "BATCH" checkpoint.interval = 50000 } source { Iceberg { catalog_name = "hadoop_prod" catalog_type = "hadoop" warehouse="hdfs://***:8020/warehouse/hive/test-iceberg" namespace = "test01" table = "test_table01" } } sink { Console { } } ``` ### Running Command ```shell bin/seatunnel.sh --config jobconf/iceberg_to_local.conf ``` ### Error Exception ```log no Error Exception ``` ### Zeta or Flink or Spark Version _No response_ ### Java or Scala Version _No response_ ### Screenshots this is my iceberg table data count: ![Uploading image.png…]() if I set parallelism = 1, The information I have obtained is as follows: *********************************************** Job Statistic Information *********************************************** Start Time : 2023-10-19 14:41:10 End Time : 2023-10-19 14:41:16 Total Time(s) : 5 Total Read Count : 2000002 Total Write Count : 2000002 Total Failed Count : 0 *********************************************** if I set parallelism = 2, The information I have obtained is as follows: *********************************************** Job Statistic Information *********************************************** Start Time : 2023-10-19 14:48:58 End Time : 2023-10-19 14:49:01 Total Time(s) : 3 Total Read Count : 1000001 Total Write Count : 1000001 Total Failed Count : 0 *********************************************** ### Are you willing to submit PR? - [ ] Yes I am willing to submit a PR! ### Code of Conduct - [X] I agree to follow this project's [Code of Conduct](https://www.apache.org/foundation/policies/conduct) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
