Sandy Ryza created SPARK-8135:
---------------------------------
Summary: In SerializableWritable, don't load defaults when
instantiating Configuration
Key: SPARK-8135
URL: https://issues.apache.org/jira/browse/SPARK-8135
Project: Spark
Issue Type: Improvement
Components: Spark Core
Reporter: Sandy Ryza
Assignee: Sandy Ryza
Calling "new Configuration()" is an expensive operation because it loads any
Hadoop configuration XMLs from disk.
In SerializableWritable, we call new Configuration needlessly when
instantiating an ObjectWritable. The ObjectWritable only needs the
Configuration for its class cache, not for any Hadoop properties that might be
in XML files, so it should be ok to call new Configuration with loadDefaults =
false.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
