beyond1920 opened a new issue, #9090: URL: https://github.com/apache/hudi/issues/9090
I cherry pick [HUDI-1517](https://issues.apache.org/jira/browse/HUDI-1517) into internal HUDI version. And find a FileNotFoundException during read latest snapshot of a MOR table.  The exception would happen if enable spark speculative feature, there exists concurrent writer and reader. For example: 1. Job1 is writing to a MOR table and not finished yet. It enables spark speculative feature. 2. Job2 is reading the latest snapshot from the MOR table, when it call getLatestMergedFileSlicesBeforeOrOn, it might list the log files which are written by speculative attempt task in Job1. 3. Job1 is finished, deletes the log files which are written by slow speculative tasks. 4. Job2 throws the FileNotFoundException when it read the log file which is already deleted in step3 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
