ivoson commented on code in PR #54136:
URL: https://github.com/apache/spark/pull/54136#discussion_r2894330615
##########
core/src/main/scala/org/apache/spark/storage/BlockManager.scala:
##########
@@ -196,7 +196,11 @@ private[spark] class BlockManager(
// We initialize the ShuffleManager later in SparkContext and Executor, to
allow
// user jars to define custom ShuffleManagers, as such `_shuffleManager`
will be null here
// (except for tests) and we ask for the instance from the SparkEnv.
- private lazy val shuffleManager =
Option(_shuffleManager).getOrElse(SparkEnv.get.shuffleManager)
+ private lazy val shuffleManager = {
+ // Wait for ShuffleManager to be initialized before handling shuffle
operations.
+ waitForShuffleManagerInit()
Review Comment:
Move the `waitForShuffleManagerInit()` right before
`SparkEnv.get.shuffleManager` to avoid unnecessary calls.
I think this should be called when it's needed despite it's driver or
executor to avoid initializing the `_shuffleManager` as null, especially
considering the risk of future code ordering changes.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
