[
https://issues.apache.org/jira/browse/SPARK-16859?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17071298#comment-17071298
]
Dan Ziemba commented on SPARK-16859:
------------------------------------
"spark.eventLog.logBlockUpdates.enabled=true" works for me on Spark 2.4.0
(Cloudera's build 2.4.0.cloudera2)
> History Server storage information is missing
> ---------------------------------------------
>
> Key: SPARK-16859
> URL: https://issues.apache.org/jira/browse/SPARK-16859
> Project: Spark
> Issue Type: Bug
> Components: Spark Core
> Affects Versions: 1.6.2, 2.0.0
> Reporter: Andrei Ivanov
> Priority: Major
> Labels: historyserver, newbie
>
> It looks like job history storage tab in history server is broken for
> completed jobs since *1.6.2*.
> More specifically it's broken since
> [SPARK-13845|https://issues.apache.org/jira/browse/SPARK-13845].
> I've fixed for my installation by effectively reverting the above patch
> ([see|https://github.com/EinsamHauer/spark/commit/3af62ea09af8bb350c8c8a9117149c09b8feba08]).
> IMHO, the most straightforward fix would be to implement
> _SparkListenerBlockUpdated_ serialization to JSON in _JsonProtocol_ making
> sure it works from _ReplayListenerBus_.
> The downside will be that it will still work incorrectly with pre patch job
> histories. But then, it doesn't work since *1.6.2* anyhow.
> PS: I'd really love to have this fixed eventually. But I'm pretty new to
> Apache Spark and missing hands on Scala experience. So I'd prefer that it be
> fixed by someone experienced with roadmap vision. If nobody volunteers I'll
> try to patch myself.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
