GitHub user adrian555 opened a pull request:
https://github.com/apache/spark/pull/22455
[SPARK-24572][SparkR] "eager execution" for R shell, IDE
## What changes were proposed in this pull request?
Check the `spark.sql.repl.eagerEval.enabled` configuration property in
SparkDataFrame `show()` method. If the `SparkSession` has eager execution
enabled, the data will be returned to the R client when the data frame is
created. So instead of seeing this
```
> df <- createDataFrame(faithful)
> df
SparkDataFrame[eruptions:double, waiting:double]
```
you will see
```
> df <- createDataFrame(faithful)
> df
+---------+-------+
|eruptions|waiting|
+---------+-------+
| 3.6| 79.0|
| 1.8| 54.0|
| 3.333| 74.0|
| 2.283| 62.0|
| 4.533| 85.0|
| 2.883| 55.0|
| 4.7| 88.0|
| 3.6| 85.0|
| 1.95| 51.0|
| 4.35| 85.0|
| 1.833| 54.0|
| 3.917| 84.0|
| 4.2| 78.0|
| 1.75| 47.0|
| 4.7| 83.0|
| 2.167| 52.0|
| 1.75| 62.0|
| 4.8| 84.0|
| 1.6| 52.0|
| 4.25| 79.0|
+---------+-------+
only showing top 20 rows
```
## How was this patch tested?
Manual tests as well as unit tests (one new test case is added).
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/adrian555/spark eager_execution
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/spark/pull/22455.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #22455
----
commit b3e014c4e8050e8a0b3da190bb327347f9136b7e
Author: adrian555 <v2ave10p>
Date: 2018-09-18T20:48:56Z
support eager execution
commit d0be3a8582de548862a78cfa5ffb58df933efa8d
Author: adrian555 <v2ave10p>
Date: 2018-09-18T20:51:43Z
Merge remote-tracking branch 'remotes/upstream/master' into eager_execution
commit cd8a7041c6eecc59d22db72f4f2065ed9f06640a
Author: adrian555 <v2ave10p>
Date: 2018-09-18T21:09:48Z
add newline
----
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
