GitHub user agsachin opened a pull request:
https://github.com/apache/spark/pull/12958
SPARK-15183
## What changes were proposed in this pull request?
while experimenting with structure streaming. I found that mode() is used
for non-continuous queries while outputMode() is used for continuous queries.
ouputMode is not defined, so I have written the some raw implementation and
test cases just to make sure the streaming app works
Note:-
/** Start a query */
private[sql] def startQuery(
name: String,
checkpointLocation: String,
df: DataFrame,
sink: Sink,
trigger: Trigger = ProcessingTime(0),
triggerClock: Clock = new SystemClock(),
outputMode: OutputMode = Append): ContinuousQuery = {
As per me outputMode should be defined before triggerClock, the constructor
with outputMode defined will be used more often then triggerClock.
I have added triggerClock() method also
## How was this patch tested?
using unit test locally
(If this patch involves UI changes, please attach a screenshot; otherwise,
remove this)
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/agsachin/spark streaming
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/spark/pull/12958.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #12958
----
commit b418b4526e57b1ef437b9dab7779c3be1a5fd497
Author: sachin aggarwal <[email protected]>
Date: 2016-05-06T15:47:16Z
SPARK-15183
----
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [email protected] or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
