veenaypatil commented on pull request #3646:
URL: https://github.com/apache/hudi/pull/3646#issuecomment-967034819
any update on this , I faced an issue today where the ETL failed in Prod
because the file was deleted by Cleaner service
```
Caused by: java.io.FileNotFoundException: No such file or directory:
s3a://bucket/cdcv2/data/a02b9653-d715-43a7-8faf-950cbdafebc4-0_123-95450-42475479_20211111205930.parquet
[2021-11-12 00:40:15,491] {logging_mixin.py:112} INFO - [2021-11-12
00:40:15,491] {batch.py:321} INFO - at
org.apache.hadoop.fs.s3a.S3AFileSystem.s3GetFileStatus(S3AFileSystem.java:2269)
```
Having config based on time will help instead of relying on Number of commits
cc @vinothchandar
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
