openinx commented on issue #2384: URL: https://github.com/apache/iceberg/issues/2384#issuecomment-808007746
Checked the code path ( https://github.com/apache/iceberg/blob/master/flink/src/test/java/org/apache/iceberg/flink/TestFlinkTableSink.java#L290), If the flink job did a failover, then it will produce some data files under the partition directories, which has not been committed to apache iceberg table, so actually those files should not be visible to iceberg users. I think we'd better to change the way to get the partition files by `Table.newScan().planFiles()` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
