clintropolis commented on a change in pull request #7117: Improve doc for auto compaction URL: https://github.com/apache/incubator-druid/pull/7117#discussion_r260533674
########## File path: docs/content/operations/segment-optimization.md ########## @@ -32,15 +32,57 @@ In Druid, it's important to optimize the segment size because which hold the input segments of the query. Each node has a processing threads pool and use one thread per segment to process it. If the segment size is too large, data might not be well distributed over the whole cluster, thereby decreasing the degree of parallelism. If the segment size is too small, - each processing thread processes too small data. This might reduce the processing speed of other queries as well as - the input query itself because the processing threads are shared for executing all queries. + each processing thread might process too small data. This can reduce the overall processing speed because + parallel processing involves some overhead like thread scheduling. Review comment: Just a suggestion for this section, feel free to change or not: ``` If segment sizes are too large, data might not be well distributed between data servers, decreasing the degree of parallelism possible during query processing. At the other extreme where segment sizes are too small, the scheduling overhead of processing a larger number of segments per query can reduce performance, as the threads that process each segment compete for the fixed slots of the processing pool. ``` ---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services --------------------------------------------------------------------- To unsubscribe, e-mail: commits-unsubscr...@druid.apache.org For additional commands, e-mail: commits-h...@druid.apache.org
