Andrew Or created SPARK-10653:
---------------------------------
Summary: Remove unnecessary things from SparkEnv
Key: SPARK-10653
URL: https://issues.apache.org/jira/browse/SPARK-10653
Project: Spark
Issue Type: Bug
Components: Spark Core
Affects Versions: 1.0.0
Reporter: Andrew Or
As of the writing of this message, there are at least two things that can be
removed from it:
{code}
@DeveloperApi
class SparkEnv (
val executorId: String,
private[spark] val rpcEnv: RpcEnv,
val serializer: Serializer,
val closureSerializer: Serializer,
val cacheManager: CacheManager,
val mapOutputTracker: MapOutputTracker,
val shuffleManager: ShuffleManager,
val broadcastManager: BroadcastManager,
val blockTransferService: BlockTransferService, // this one can go
val blockManager: BlockManager,
val securityManager: SecurityManager,
val httpFileServer: HttpFileServer,
val sparkFilesDir: String, // this one maybe? It's only used in 1 place.
val metricsSystem: MetricsSystem,
val shuffleMemoryManager: ShuffleMemoryManager,
val executorMemoryManager: ExecutorMemoryManager, // this can go
val outputCommitCoordinator: OutputCommitCoordinator,
val conf: SparkConf) extends Logging {
...
}
{code}
We should avoid adding to this infinite list of things in SparkEnv's
constructors if they're not needed.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
