Ruifeng Zheng created SPARK-60383:
-------------------------------------
Summary: Keep streaming shuffle implementation details out of the
public API
Key: SPARK-60383
URL: https://issues.apache.org/jira/browse/SPARK-60383
Project: Spark
Issue Type: Bug
Components: Spark Core
Affects Versions: 5.0.0
Reporter: Ruifeng Zheng
Spark 4.3.0 RC2 exposes streaming shuffle implementation types through
ShuffleLocationResponse, BarrierTaskContext.addPostStatusUpdateListener, and
SparkEnv.streamingShuffleOutputTracker. Their types are private[spark], but the
declarations are public.
Mark these declarations private[spark]. Also remove the premature deprecation
of the public SparkEnv.shuffleManager accessor: both suggested replacements are
private[spark], so external callers cannot follow the migration advice. Retain
the existing accessor and describe its blocking-shuffle role.
The correction should be backported to branch-4.x and branch-4.3 before release.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
