GitHub user gatorsmile opened a pull request:
https://github.com/apache/spark/pull/20365
[SPARK-23192] [SQL] Keep the Hint after Using Cached Data
## What changes were proposed in this pull request?
The hint of the plan segment is lost, if the plan segment is replaced by
the cached data.
```Scala
val df1 = spark.createDataFrame(Seq((1, "4"), (2, "2"))).toDF("key",
"value")
val df2 = spark.createDataFrame(Seq((1, "1"), (2, "2"))).toDF("key",
"value")
df2.cache()
val df3 = df1.join(broadcast(df2), Seq("key"), "inner")
```
This PR is to fix it.
## How was this patch tested?
(Please explain how this patch was tested. E.g. unit tests, integration
tests, manual tests)
(If this patch involves UI changes, please attach a screenshot; otherwise,
remove this)
Please review http://spark.apache.org/contributing.html before opening a
pull request.
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/gatorsmile/spark fixBroadcastHintloss
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/spark/pull/20365.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #20365
----
commit 1186ef5e38a34ff77fa62521de0da73666b0de96
Author: gatorsmile <gatorsmile@...>
Date: 2018-01-23T16:41:24Z
fix
----
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
