zhengruifeng opened a new pull request, #59633: URL: https://github.com/apache/spark/pull/59633
### What changes were proposed in this pull request? Mark `ShuffleLocationResponse`, `BarrierTaskContext.addPostStatusUpdateListener`, and `SparkEnv.streamingShuffleOutputTracker` as `private[spark]`. Remove the premature deprecation of `SparkEnv.shuffleManager` and document that it returns the manager for regular, materialized shuffle dependencies. ### Why are the changes needed? The new streaming shuffle declarations expose implementation types that are already `private[spark]`. The barrier listener override also widens the visibility of the package-private declaration in `TaskContext`. The public `SparkEnv.shuffleManager` accessor cannot usefully be deprecated in favor of `shuffleManagerFor` or `blockingShuffleManager`, because both suggested replacements are `private[spark]`. Retaining the accessor without that deprecation gives external callers a usable API without exposing additional implementation methods. Please also backport to `branch-4.x` and `branch-4.3` before the 4.3 release. ### Does this PR introduce _any_ user-facing change? Yes. The three implementation declarations introduced for 4.3 are hidden from the public Scala API, and `SparkEnv.shuffleManager` no longer carries the unusable deprecation advice. Internal shuffle and listener behavior is unchanged. ### How was this patch tested? Ran `core/testOnly org.apache.spark.scheduler.TaskContextSuite org.apache.spark.scheduler.BarrierTaskContextSuite org.apache.spark.StreamingShuffleOutputTrackerSuite`: all 62 tests passed. Also ran `git diff --check` and checked changed Scala lines for ASCII and the 100-column limit. ### Was this patch authored or co-authored using generative AI tooling? Generated-by: OpenAI Codex (GPT-6) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
