xichen01 commented on PR #4939: URL: https://github.com/apache/ozone/pull/4939#issuecomment-1631074137
> @xichen01 IMO, the changes is not required, as having queue will not provide any benefits. As per SCM deleteBlock processing, it will retry same blocks for delete till response in HB is not received. SCM sends duplicate BlockIDs to the DN, we should avoid this problem because it affects the speed of DN marking blocks, in our Cluster, only about 1/3 of the blocks sent by SCM to the DN are valid, and the remaining 2/3 are duplicates. And if the DN's `DeleteBlocksCommandHandlerThread` thread is blocked for any reason, this DN will not be able to report the status of the delete transaction to the SCM for a long time, which will cause the SCM to keep sending delete transactions to the stuck DN, and if the DN is stuck for a long enough period of time, you'll find that the SCM will send a delete transaction only to the stuck DN (all the deletes are duplicated), and other normal DNs will not be able to receive new deletion transactions (because they will be skipped). Our business requires a deletion speed of at least 1000+ QPS to meet our business needs. The current Ozone deletion speed is not stable enough to reach this speed. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
