GitHub user fuyufjh opened a pull request:
https://github.com/apache/spark/pull/21890
[SPARK-24932] Allow update mode for streaming queries with join
## What changes were proposed in this pull request?
In issue SPARK-19140 we supported update output mode for non-aggregation
streaming queries. This should also be applied to streaming join to keep
semantic consistent.
PS. Streaming join feature is added after SPARK-19140.
When using *update* output mode the join will works exactly as *append*
mode. However, for example, this will allow user to run an
aggregation-after-join query in *update* mode in order to get a more real-time
result output.
## How was this patch tested?
See changes in UnsupportedOperationsSuite.
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/fuyufjh/spark
SPARK-19140-allow-update-for-stream-join
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/spark/pull/21890.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #21890
----
commit f2daf62c4e5d9daf397fc804ed9365204933ddbd
Author: Eric Fu <fuyufjh@...>
Date: 2018-07-27T02:55:17Z
[SPARK-24932] Allow update mode for streaming queries with join
----
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
