[
https://issues.apache.org/jira/browse/SPARK-9036?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14643601#comment-14643601
]
Ryan Williams commented on SPARK-9036:
--------------------------------------
Heh, [guilty as
charged|https://github.com/hammerlab/spark-json-relay/blob/1.0.0/src/main/scala/org/apache/spark/JsonRelay.scala#L1],
[~pwendell]! I put some more thoughts about this on my long email to dev@ just
now, but feel free to chime in here or there. The fact that
{{EventLoggingListener}} doesn't bother with this one kind of event (as an
optimization, I guess?) seems like a strange thing to encode in the
{{JsonProtocol}} itself, to me.
> SparkListenerExecutorMetricsUpdate messages not included in JsonProtocol
> ------------------------------------------------------------------------
>
> Key: SPARK-9036
> URL: https://issues.apache.org/jira/browse/SPARK-9036
> Project: Spark
> Issue Type: Improvement
> Components: Spark Core
> Affects Versions: 1.4.0, 1.4.1
> Reporter: Ryan Williams
> Assignee: Ryan Williams
> Priority: Minor
> Fix For: 1.5.0
>
>
> The JsonProtocol added in SPARK-3454 [doesn't
> include|https://github.com/apache/spark/blob/v1.4.1-rc4/core/src/main/scala/org/apache/spark/util/JsonProtocol.scala#L95-L96]
> code for ser/de of
> [{{SparkListenerExecutorMetricsUpdate}}|https://github.com/apache/spark/blob/v1.4.1-rc4/core/src/main/scala/org/apache/spark/scheduler/SparkListener.scala#L107-L110]
> messages.
> The comment notes that they are "not used", which presumably refers to the
> fact that the [{{EventLoggingListener}} doesn't write these
> events|https://github.com/apache/spark/blob/v1.4.1-rc4/core/src/main/scala/org/apache/spark/scheduler/EventLoggingListener.scala#L200-L201].
> However, individual listeners can and should make that determination for
> themselves; I have recently written custom listeners that would like to
> consume metrics-update messages as JSON, so it would be nice to round out the
> JsonProtocol implementation by supporting them.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
