GeorgeJahad commented on PR #4680: URL: https://github.com/apache/ozone/pull/4680#issuecomment-1553512855
> it could still append new compaction log entries (and hardlink new SST files) in CompactionBegin/CompletedListener. I don't think the new sst files are a serious problem. The question is are the new compaction log entries a problem. That is what I need help determining. If they are not a problem, then we don't need to pause compactions. The issue is after the bootstrap process takes the checkpoint of the active fs, any compactions before the tarball is finished will create entries to the comaction logs. Those entries don't correspond the active fs in the tarball. Once the follower loads the tarball, it will start doing it's own compactions which may differ from the compaction log entries. It could cause conflicting entries in the compaction log, which might break the dag. That is the problem we may need to avoid by invoking pauseBackgroundWork(). For example, the leader could compact files a and b into file c while the follower could compact them into file d. What is the compaction dag supposed to do with conflicting entries like that? That is why I'm leaning towards stopping compactions. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
