ashishkumar50 commented on PR #11564: URL: https://github.com/apache/ozone/pull/11564#issuecomment-6347438227
> Does OM not get the deleted keys from the deleted keys table, then derive the blocks and send them to SCM. When SCM acks, it removes from the deleted key table? If so, then OM has no memory impact of backing off and clients should be unaffected. If SCM acknowledges the request and OM removes the corresponding entries from its deleted-keys table, that is the happy path. The problem is that SCM may have already processed and acknowledged a block-deletion request, but OM can still submit the same block again without removing it from the table. This can happen in several scenarios: 1. Request timeout: SCM processes the request, but OM times out waiting for the response and retries it. 2. Code bug: The issue I pointed out earlier causes OM to resubmit blocks even though SCM has already acknowledged them. 3. Restart or leader switch: An interruption during request processing or recovery can result in previously submitted blocks being sent again. Although duplicate block deletion is idempotent from a correctness perspective, it still introduces unnecessary work. SCM persist and process the duplicate block-deletion request again and send it to the DataNode, which also has to process and persist the duplicate request. And if we want to backoff from SCM, it means OM deleted key table size will grow. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
