Github user uncleGen commented on the pull request:
https://github.com/apache/spark/pull/2076#issuecomment-52908740
@srowen yes! Not only in "StorageTab", "ExectutorTab" may also lose some
rdd-infos which have been overwritten by following rdd in a same task.
"StorageTab": when multiple stages run simultaneously, completed stage will
remove rdd-info which belong to other stages which are still running.
"ExectutorTab": In a dependency chain of rdds, task can only update the
latest rdd info. Like the following example:
val r1 = sc.paralize(..).cache()
val r2 = r1.map(...).cache()
val n = r2.count()
Currently, "ExecutorTab" show a wrong info about the "Memory Used", it only
shows the usage of r2, but not r1 and r2.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [email protected] or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
