errose28 opened a new pull request, #9862: URL: https://github.com/apache/ozone/pull/9862
## What changes were proposed in this pull request? [This check](https://github.com/apache/ozone/blob/1e785c599b7ccd7359125061eefe56e66feca9fa/hadoop-hdds/container-service/src/main/java/org/apache/hadoop/ozone/container/keyvalue/KeyValueHandler.java#L1700) is intended to skip blocks identified as missing in the merkle tree but are actually present in the container. However, it only checks for the existence of metadata. It is possible that the block file has been lost but the metadata is still in the DB. This will show up as a missing block in the merkle tree diff, but reconciliation should not skip the block in this case. This PR simply removes the check. `reconcileChunksPerBlock` has its own [block existence check](https://github.com/apache/ozone/blob/f8ab58835c358d2f9bec010cff08d51c7967b09c/hadoop-hdds/container-service/src/main/java/org/apache/hadoop/ozone/container/keyvalue/KeyValueHandler.java#L1844) to reconcile any metadata as needed, and is already equipped to handle chunk overwrites in the cases of corruption or truncation. ## What is the link to the Apache JIRA HDDS-14763 ## How was this patch tested? Existing tests pass. It is difficult to add a test case for this type of corruption in `TestContainerReconciliationWithMockDatanodes` where it belongs without HDDS-11942. The test suite currently does not support different types of missing block corruption like missing data vs metadata. It also uses the metadata block list in other unrelated checks, like those calling [getSortedBlocks](https://github.com/apache/ozone/blob/a534ac2f38891a088bfa8c821e8c228b16864a82/hadoop-hdds/container-service/src/test/java/org/apache/hadoop/ozone/container/keyvalue/TestContainerReconciliationWithMockDatanodes.java#L677). I was able to hack the test to remove only the block file in some cases and those executions passed, but there were unrelated failures due to later parts of the tests expecting all blocks to be in the metadata list. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
