[GitHub] [hudi] vinothchandar commented on issue #1764: [SUPPORT] Commits stays INFLIGHT forever after S3 consistency check fails when Hudi tries to delete duplicate datafiles

vinothchandar commented on issue #1764: URL: https://github.com/apache/hudi/issues/1764#issuecomment-653221883 > just skipping (and not failing the job) it should work better as an interim solution given the current design of using marker files may be we should discuss the scenario

[GitHub] [hudi] vinothchandar commented on issue #1764: [SUPPORT] Commits stays INFLIGHT forever after S3 consistency check fails when Hudi tries to delete duplicate datafiles

vinothchandar commented on issue #1764: URL: https://github.com/apache/hudi/issues/1764#issuecomment-650366410 >I am just thinking if we really need to wait for all files to appear here, or even if we need to wait, if at the end of the wait period the file is not present it should be safe

[GitHub] [hudi] vinothchandar commented on issue #1764: [SUPPORT] Commits stays INFLIGHT forever after S3 consistency check fails when Hudi tries to delete duplicate datafiles

vinothchandar commented on issue #1764: URL: https://github.com/apache/hudi/issues/1764#issuecomment-650243029 @umehrot2 if we reverse the order, then it might violate guarantee that if there was a file created in storage, then there is a marker file involved... the task can open a

[GitHub] [hudi] vinothchandar commented on issue #1764: [SUPPORT] Commits stays INFLIGHT forever after S3 consistency check fails when Hudi tries to delete duplicate datafiles

vinothchandar commented on issue #1764: URL: https://github.com/apache/hudi/issues/1764#issuecomment-648882567 @zuyanton thanks for reporting this.. let's work together to resolve this. can you please paste the `.hoodie/hoodie.properties` file? The .inflight file hanging around could be