JacksonYao287 commented on PR #3360: URL: https://github.com/apache/ozone/pull/3360#issuecomment-1302895601
my plan is to complete this work in two steps. first, in this patch, DELETED container is removed from scm by default(this can be configured). and for those unknown container , no matter it is empty or not, we will deleted it by default(this can be configured). in production environment，which use the default configuration, no matter an unknown container is empty, it will not be referenced by any key, we can just delete it. so although Deleting a non-empty unknown container by default is an aggressive operation, i think it makes sense here. in development environment, we can set the default operation to log WARN, so it give us a chance to investigate the problem. and for those old DELETED container reference in rocksDB, we can just left them there and do nothing and later newly deployed cluster will not have this problem. second, i will create a new patch to solve [this problem.](https://github.com/apache/ozone/pull/3360#issuecomment-1119204766). my idea here is after container is closed, only deleteblockLog can shrink the container size, not `calculateUsage ` in containerReport handler. the size of container is reduced only after the deleteTransaction is committed in deleteblockLog. so even if a container replica has some orphan blocks and a bigger size, scm has a definite size of the container. when the size of a container in scm is shrinked to zero, scm will delete all the replicas of the container . if a container is deleted , and scm receives a container report from a resurrected datanode, it will be treated as an unknown container , and will be deleted by scm by default. @errose28 @sodonnel what do you think? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
