zahed1994 opened a new pull request, #58126: URL: https://github.com/apache/spark/pull/58126
### What changes were proposed in this pull request? This PR fixes a resource profile ID collision where custom stage-level `ResourceProfile` instances were assigned `id = 0`, colliding with `ResourceProfile.DEFAULT_RESOURCE_PROFILE_ID` (`0`). Specifically, `nextProfileId` in `ResourceProfile.scala` is now initialized to `1` so that custom profiles start allocation at ID `1`. ### Why are the changes needed? Spark uses `DEFAULT_RESOURCE_PROFILE_ID = 0` for the default profile. Previously, `nextProfileId` was initialized to `0`, causing the first user-created `ResourceProfile` to also receive `id = 0`. This caused schedulers and cluster manager allocators (Yarn, K8s, Standalone) to misidentify custom stage-level resource profiles as the default profile. ### Does this PR introduce _any_ user-facing change? No API changes. Fixes incorrect profile ID assignment for custom resource profiles. ### How was this patch tested? Added unit test in `ResourceProfileSuite`. ### Was this patch authored or co-authored using generative AI tooling? No. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
