duongkame commented on PR #4962: URL: https://github.com/apache/ozone/pull/4962#issuecomment-1603727415
>> Just for my curiosity, why was the initial stripe size chosen to be 512? > Shameless copy from @duongkame's PR: With striped locks, collisions can happen. It is when different keys are resolved to the same lock instance. Note that the lock instance is resolved based on the key hashCode % striped_size. Collisions can slow things down because the works that can be done in parallel (with LockManager) now get to block each other. However, they don't result in deadlocks. So, it's ok to have collisions at a low rate. The size of the lock pool (Striped size) is a tradeoff between getting a low collision rate and memory footprint. Assuming we have 100 threads competing striped locks, the maximum number of unique lock keys to be requested concurrently is 100. Then, if the striped size is 100 and hashCode is perfect, the probability of collisions is ~0. Yet, because hashCode is not perfect, we can simply have Striped size as X times the number of threads. In OM, the default number of RPC handler threads is 100, and I put X=5, hence 512 (closest binary). I didn't test it thoroughly and that may be too generous. Think 2X is quite enough to keep the collision rate ~0. And of course, the right Stripe size also depends on the cardinality of the keys. @adoroszlai @Galsza -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
