himanshug commented on issue #263: Theta sketch - Concurrent union implementation URL: https://github.com/apache/incubator-datasketches-java/issues/263#issuecomment-509836845 > assume aggregation at query time are executed in memory, even when data already resides on disk. What I suggest here is to consider a way to expedite query processing by having multiple threads (can be a small number) executing the aggregation in parallel, instead of sequentially by a single thread. At one index/aggregator level , it might look like that we are not exploiting all available cores and adding concurrency might help. However, in reality there are many queries running concurrently, each individual query is further split into per segment scan/aggregation job thats runs concurrently. That does exhaust available cores.
---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected] With regards, Apache Git Services --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
