chungen0126 opened a new pull request, #10865: URL: https://github.com/apache/ozone/pull/10865
## What changes were proposed in this pull request? During rolling upgrades or in heterogeneous clusters, random node selection makes forming a 100% Ratis Streaming pipeline rare (e.g., (1/2)^3 = 12.5% chance when 50% nodes support it), leading to frequent client fallbacks. To maximize streaming pipeline creation without sacrificing availability, we introduce a two-phase node selection strategy for THREE-factor pipelines: 1. Strict Phase: Exclude nodes lacking Ratis Streaming support (along with over-utilized nodes) and attempt allocation. 2. Fallback Phase: If allocation fails due to insufficient streaming nodes, fall back to standard allocation with relaxed capability checks. This prioritizes Ratis Streaming nodes when available while ensuring full cluster fallback resilience. Key Implementation Details: - Refactored filterPipelineEngagement() to filterNodes(boolean filterRatisStreaming) to check for DatanodeDetails.Port.Name.RATIS_DATASTREAM. - Safe handling of currentExcluded lists to avoid polluting original excludedNodes during the fallback attempt. ## What is the link to the Apache JIRA https://issues.apache.org/jira/browse/HDDS-15969 ## How was this patch tested? Added new unit tests in TestRatisPipelineProvider: - testCreatePipelinePrioritizesRatisStreamingNodes: Verifies that SCM strictly selects DataNodes with RATIS_DATASTREAM when sufficient nodes exist. - testCreatePipelineFallsBackWhenNotEnoughRatisStreamingNodes: Verifies that SCM gracefully falls back and includes non-streaming DataNodes when streaming-capable nodes are fewer than 3. ci: https://github.com/chungen0126/ozone/actions/runs/30139473851 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
