Github user heary-cao commented on the issue:
https://github.com/apache/spark/pull/20699
@hvanhovell , Iâm sorry to delayed answer your question. we have a task
to write with RDD, and do repartition on the memory table and then left join
another table.
Optimized Logical Plan look like:
```
Join LeftOuter, ((x004#854 = XXXX#2308) && (x005#855 = YYYY#2316))
: :- Repartition 108, true
: : +- Filter (isnotnull(c019#867) && (cast(c019#867 as int) = 4))
: : +- InMemoryRelation [ ... 109 more fields], true, 10000,
StorageLevel(disk, memory, deserialized, 1 replicas)
........
```
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
