m-ghazanfar commented on issue #15167: URL: https://github.com/apache/druid/issues/15167#issuecomment-1772221758
@abhishekagarwal87 doesn't matter if you use `maxRowsPerSegment` or `maxTotalRows` - at handoff time, all segments on the indexer get handed-off. This will again result in increased load on the historicals. Quoting from [here](https://druid.apache.org/docs/latest/development/extensions-core/kafka-supervisor-operations#deployment-notes-on-kafka-partitions-and-druid-segments), > When the task reaches one of maxRowsPerSegment, maxTotalRows, or intermediateHandoffPeriod, it hands off all the segments and creates a new new set of segments will be created for further events --- An increase in historical load is a problem because, I have to over-provision my historicals to be able to handle the additional load during the segment hand-off. If I can prevent this load on the historicals, I can move my historicals to cheaper instances or reduce their number. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
