GitHub user cloud-fan opened a pull request:
https://github.com/apache/spark/pull/20477
[SPARK-23303][SQL] improve the explain result for data source v2 relations
## What changes were proposed in this pull request?
The current explain result for data source v2 relation is unreadable:
```
== Parsed Logical Plan ==
'Filter ('i > 6)
+- AnalysisBarrier
+- Project [j#1]
+- DataSourceV2Relation [i#0, j#1],
org.apache.spark.sql.sources.v2.AdvancedDataSourceV2$Reader@3b415940
== Analyzed Logical Plan ==
j: int
Project [j#1]
+- Filter (i#0 > 6)
+- Project [j#1, i#0]
+- DataSourceV2Relation [i#0, j#1],
org.apache.spark.sql.sources.v2.AdvancedDataSourceV2$Reader@3b415940
== Optimized Logical Plan ==
Project [j#1]
+- Filter isnotnull(i#0)
+- DataSourceV2Relation [i#0, j#1],
org.apache.spark.sql.sources.v2.AdvancedDataSourceV2$Reader@3b415940
== Physical Plan ==
*(1) Project [j#1]
+- *(1) Filter isnotnull(i#0)
+- *(1) DataSourceV2Scan [i#0, j#1],
org.apache.spark.sql.sources.v2.AdvancedDataSourceV2$Reader@3b415940
```
after this PR
```
== Parsed Logical Plan ==
'Project [unresolvedalias('j, None)]
+- AnalysisBarrier
+- Relation SimpleDataSourceV2[i#0, j#1]
== Analyzed Logical Plan ==
j: int
Project [j#1]
+- Relation SimpleDataSourceV2[i#0, j#1]
== Optimized Logical Plan ==
Project [j#1]
+- Relation SimpleDataSourceV2[i#0, j#1]
== Physical Plan ==
*(1) Project [j#1]
+- *(1) Scan SimpleDataSourceV2[i#0, j#1]
```
-------
```
== Parsed Logical Plan ==
'Filter ('i > 3)
+- AnalysisBarrier
+- Relation AdvancedDataSourceV2[i#0, j#1]
== Analyzed Logical Plan ==
i: int, j: int
Filter (i#0 > 3)
+- Relation AdvancedDataSourceV2[i#0, j#1]
== Optimized Logical Plan ==
Relation AdvancedDataSourceV2[i#0, j#1]
== Physical Plan ==
*(1) Scan AdvancedDataSourceV2[i#0, j#1] (PushedFilter: [IsNotNull(i),
GreaterThan(i,3)])
```
## How was this patch tested?
N/A
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/cloud-fan/spark explain
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/spark/pull/20477.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #20477
----
----
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
