Stephen-Robin edited a comment on issue #2195: URL: https://github.com/apache/iceberg/issues/2195#issuecomment-771781055
> That does seem pretty dangerous although I haven't seen this happen before. Do you have the snapshot summary for the compaction operation? It should hopefully also provide some hints as to what is going on It would also be great if you had a programmatic reproduction of the issue. @RussellSpitzer Thanks for your reply I just ran my previous experiment again and will introduce my reproduction process in detail below. Creating tables and inserting data have been mentioned above, Then   data files as follows  Then execute compaction:  During the execution, I debugged.  The variable `combinedScanTasks` includes the divided small file part B (1407488 bytes), but there is no part A (10M) file.  The Variable `currentDataFiles` includes an initial file of 11MB in size. The file of 11MB will be deleted during commit, but the 10M file B has not been rewritten. Then snapshot file concerned as follows:    show the rows of table   We can find that the 11MB data file has been lost data files as follows:  The red box is the new rewritten data I think the problem described in my issue may exist. Looking forward to your reply ---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
