clintropolis edited a comment on issue #8578: parallel broker merges on fork join pool URL: https://github.com/apache/incubator-druid/pull/8578#issuecomment-549733336 ### simulated heavy load I collected another round of data using the same benchmarks as my 'more realistic worst case' comment, but this time plotting what happens when a large number of queries all start within a 500ms spread, which might be a more typical heavy load, rather than simulating a large concurrent spike of simultaneous queries like the last set of results. In this scenario, parallel merges outperform the same threaded merges until much higher concurrency than the concurrent spike model. This is at least partially driven by the fact that each individual thread can make a better estimate about utilization than is possible in the spike model. ##### 'small' sequences  ##### 'moderately large' sequences  ##### overall average  I think future work could focus on making the concurrent spike behavior a bit more chillax through a variety of means, but I find these results to be 'good enough' for now. Anyone want to see any other scenarios?
---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected] With regards, Apache Git Services --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
