techdocsmith opened a new issue, #12982: URL: https://github.com/apache/druid/issues/12982
From Apache Druid slack: As an administrator: - I want to is attempting to take a middle manager down for maintenance and wants end running indexing tasks and move them to other nodes. - I want to take a historical down for maintenance without rebalancing the cluster. Using `decommissioningNodes` with `decommissioningMaxPercentOfMaxSegmentsToMove=0` triggered a rebalance as segments became unavailable. `pauseCoordination` but it seemed a bit heavy-handed for the purpose. @gianm 's advice: you need `pauseCoordination` for this case. When coordination is running, if a historical goes away, we don't assume it's going to come back. so we need to start replicating its segments somewhere else. > I just thought segments assigned to nodes in the `decommissioningNodes` wouldn't get reassigned if segments to move was 0%. I would hope remaining nodes could continue to collect data from ingestion jobs. Makes a rolling restart kind of useless, entire cluster has to be marked down for maintenance rather being able to work on one node at a time. If the server is offline, it's considered a "replication" not a "move" since we're going from N to N+1 replicas rather than moving a segment from one server to another. there's a replication throttle setting in the dynamic Coordinator configs — generally when doing rolling updates you would set that low and then roll the servers one by one, lowering the replication throttle means that when a server is offline we won't blitz out a bunch of loads for the segments it had — assuming your replication factor is at least 2 for replication factor 1, we will still blitz out a bunch of loads, since the segments would be totally unavailable in that case once a server goes offline. and the system prioritizes making data available Suggested change: If `decommissioningMaxPercentOfMaxSegmentsToMove` is 0, segments will neither be moved from or to 'decommissioning' servers, effectively putting them in a sort of "maintenance" mode that will not participate in balancing or assignment by load rules. Probably better written as: If `decommissioningMaxPercentOfMaxSegmentsToMove` is 0, segments will not be moved to 'decommissioning' servers, effectively putting them in a sort of "maintenance" mode that will not participate in balancing or assignment by load rules. Note: the segments from the decommissioning servers will still be candidate for move or replication to non-decommissioning servers. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
