rohangarg commented on PR #12817: URL: https://github.com/apache/druid/pull/12817#issuecomment-1195202133
Thanks for the review @abhishekagarwal87. I have added a test for verifying distribution in 100 request keys amongst 5 brokers. While testing, I found that as the number of request keys decrease the skewness increases in the new algorithm as well. I ran some tests with old and new rendezvous hashing algorithm along with consistent hashing. The test includes 8 brokers and computes the max skew for (100, 1000, 10_000) request keys. max skew is calculated as the difference in request between lightest and heaviest broker. Each request key count config is run 1000 times and the result is the maximum skew amongst all the runs. Please find below the results : | hashing algorithm | max skew with 100 request keys | max skew with 1000 request keys | max skew with 10_000 request keys| |-------------------|------------------|-------------------|----------------------| | rendezvous old | 20 (20%) | 67 (6.7%) | 193 (1.93%) | | rendezvous new | 34 (34%) | 238 (23.8%) | 2090 (21%) | | consistent hash | 22 (22%) | 72 (7.2%) | 395 (3.95%) | -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
