Patrick Wendell created SPARK-3288:
--------------------------------------
Summary: All fields in TaskMetrics should be private and use
getters/setters
Key: SPARK-3288
URL: https://issues.apache.org/jira/browse/SPARK-3288
Project: Spark
Issue Type: Improvement
Components: Spark Core
Reporter: Patrick Wendell
Assignee: Andrew Or
This is particularly bad because we expose this as a developer API. Technically
a library could create a TaskMetrics object and then change the values inside
of it and pass it onto someone else. It can be written pretty compactly like
below:
{code}
/**
* Number of bytes written for the shuffle by this task
*/
@volatile private var _shuffleBytesWritten: Long = _
def incrementShuffleBytesWritten(value: Long) = _shuffleBytesWritten += value
def decrementShuffleBytesWritten(value: Long) = _shuffleBytesWritten -= value
def shuffleBytesWritten = _shuffleBytesWritten
{code}
--
This message was sent by Atlassian JIRA
(v6.2#6252)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
