codope commented on pull request #4712: URL: https://github.com/apache/hudi/pull/4712#issuecomment-1026419828
> can you help understand this scenario: > > 1. lets say we take a backup. and delete original table config. start writing new table config w/ updates. but crashed mid -way. Until we restart the hudi writer, primary table config is in corrupted state and backup property file is in good state. So, during this, what does reader do? do they get routed to backup or original? > 2. If in above case, if crash happens mid-way, lets say out of 10 entries, only 5 got added, and if readers are using the primary copy to read table props, wouldn't they be reading partial table props? > Probably this has nothing to do w/ the checksum patch I guess. but wanted to understand in general. 1. Get routed to backup props. 2. You mean while writing props from backup file, some crash happened even before checksum match? In that case, yes partial files would be written to hdfs but the writer will fail due to IOException. Next time, when it comes up then props are recovered from the backup. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
