Gengliang Wang created SPARK-26363:
--------------------------------------
Summary: Remove redundant field `executorLogs` in TaskData
Key: SPARK-26363
URL: https://issues.apache.org/jira/browse/SPARK-26363
Project: Spark
Issue Type: Improvement
Components: Web UI
Affects Versions: 3.0.0
Reporter: Gengliang Wang
In https://github.com/apache/spark/pull/21688, a new filed `executorLogs` is
added to `TaskData` in `api.scala`:
1. The field should not belong to `TaskData` (from the meaning of wording).
2. This is redundant with ExecutorSummary.
3. For each row in the task table, the executor log value is lookup in KV store
every time, which can be avoided for better performance in large scale.
This PR propose to reuse the executor details of request "/allexecutors" , so
that we can have a cleaner api data structure, and redundant KV store queries
are avoided.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
