GitHub user jerryshao opened a pull request:
https://github.com/apache/spark/pull/20447
[SPARK-23279][SS] Avoid triggering distributed job for Console sink
## What changes were proposed in this pull request?
Console sink will redistribute collected local data and trigger a
distributed job in each batch, this is not necessary, so here change to local
job.
## How was this patch tested?
Existing UT and manual verification.
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/jerryshao/apache-spark console-minor
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/spark/pull/20447.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #20447
----
commit 4b2baeb8e00d575189036609c232a9c24f69e4a0
Author: jerryshao <sshao@...>
Date: 2018-01-31T02:29:01Z
Avoid triggering distributed job for Console sink
----
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
