luoyuxia opened a new issue, #2309: URL: https://github.com/apache/fluss/issues/2309
### Search before asking - [x] I searched in the [issues](https://github.com/apache/fluss/issues) and found nothing similar. ### Fluss version 0.8.0 (latest release) ### Please describe the bug 🐞 Consider the following case: 1: create fluss table `fluss_a` with lake enable, so `paimon_a` is create 2: tiering sevice read `fluss_a` and write data files for `paimon_a` 3: drop`fluss_a`, `paimon_a` 4: create fluss table `fluss_a` with lake enable again , so `paimon_a` is create 5: commit the data files to `paimon_a`, but the `paimon_a` is a new table, so dirty commit happens Although we have check in https://github.com/apache/fluss/blob/f90b33e838eaae39b1945a62acd0c05a78576f07/fluss-flink/fluss-flink-common/src/main/java/org/apache/fluss/flink/tiering/source/TieringSplitReader.java#L234 to ensure `TieringReader` fast fail when found the fluss table id changed for same table name. But the corner case still happen that: 1: it passes the check, and start to read fluss records 2: read all fluss records, write to data files 3: drop the table and create a new table 4: committer operator collect the data files and commit since step 2 may cost some time, it's not a rare case that it happens.. ### Solution I think paimon itself should do the check during commit, but currently it's hard to make paimon do the check now according to paimon's limit. Maybe we do check again in step 4 to make the commit is for the right table id. If we find the table current table id is not equal to the table id in commitable, we should fail this commit directly. Although it may still happen that after check pass, users still drop the table and create a new table, and commit is still a dirty commit. But I believe it should happen real rare. ### Are you willing to submit a PR? - [ ] I'm willing to submit a PR! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
