GitHub user LantaoJin opened a pull request:
https://github.com/apache/spark/pull/22353
[SPARK-25357][SQL] Abbreviated metadata in DataSourceScanExec results in
incomplete information in event log
## What changes were proposed in this pull request?
Field metadata removed from SparkPlanInfo in SPARK-17701. Corresponding,
this field was also removed from event SparkListenerSQLExecutionStart in Spark
event log. If we want to analyze event log to get some fields which wider than
100 (e.g the Location or ReadSchema of FileScan), they are abbreviated in
simpleString of SparkPlanInfo JSON or physicalPlanDescription JSON.
## How was this patch tested?
Manual test/Check event log
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/LantaoJin/spark SPARK-25357
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/spark/pull/22353.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #22353
----
commit 89fff2ac653fa05ae92ad09ba8307649d6972900
Author: LantaoJin <jinlantao@...>
Date: 2018-09-06T14:04:56Z
[SPARK-25357][SQL] Abbreviated metadata in DataSourceScanExec results in
incomplete information in event log
----
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
