abhishekagarwal87 commented on issue #15149: URL: https://github.com/apache/druid/issues/15149#issuecomment-1770256665
The pruning on the broker's side will require the real-time segment partitions to be generated with a particular shard spec. That is tricky to implement and would only work under strict conditions (Kafka topic has to be partitioned with the same key, and re-partitioning can lead to incorrect results). A HTTP call from broker to peon/indexer is not expensive but the query execution on the peons is not fast enough in the absence of indexes. Having a bitmap index is much more flexible from the operations point of view. The input topic can be re-partitioned without any issues, and having a bitmap would be useful in far more situations. - It would work with any streaming connector - It would help optimize the queries even further if you have a filter on a dimension that input data is not partitioned on. - It doesn't necessarily require the input data to be partitioned in a specific way. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
