Joseph K. Bradley created SPARK-15255:
-----------------------------------------
Summary: RDD name from DataFrame op should not include full local
relation data
Key: SPARK-15255
URL: https://issues.apache.org/jira/browse/SPARK-15255
Project: Spark
Issue Type: Improvement
Components: SQL
Affects Versions: 2.0.0
Reporter: Joseph K. Bradley
Priority: Minor
Currently, if you create a DataFrame from local data, do some operations with
it, and cache it, then the name of the RDD in the "Storage" tab in the Spark UI
will contain the entire local relation's data. This is not scalable and can
cause the browser to become unresponsive.
I'd propose there be a limit on the size of the data to display.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
