bwu2 edited a comment on issue #1128: [HUDI-453] Fix throw failed to archive commits error when writing data to MOR/COW table URL: https://github.com/apache/incubator-hudi/pull/1128#issuecomment-594824906 It's not easily reproducible. It seems to be a sporadic thing that happens from time to time; from the 95 tables we ingest (every 4 hours) for the last couple of weeks, the error has occurred 4 or 5 times. The jobs that created these zero bytes files seem to complete successfully and without any error. In the Spark logs for the job, things seem ok (this is the only mention of `.clean` files): `20/02/21 21:11:13 INFO S3NativeFileSystem2: rename s3://bucket/path/to/table/.hoodie/20200221211054.clean.inflight s3://bucket/path/to/table/.hoodie/20200221211054.clean` The s3 `.hoodie` folder looks like (ignore the dates on the files, i copied and pasted from the original folder; also, the listing is truncated):  and the archived folder looks like:  We didn't actually notice the error until much later when the cleaning fails because of some old zero byte files.
---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected] With regards, Apache Git Services
