GitHub user agsachin opened a pull request:
https://github.com/apache/spark/pull/12518
[SPARK-14745] [Streaming] [WIP] CEP support in Spark Streaming
## What changes were proposed in this pull request?
This JIRA is about how/what can we add in Spark Streaming to support CEP
out of the box.
The code in this PR needs to merge up with Structured Streaming changes in
2.0. The PR is issued for now to facilitate discussion and view by other users
interested in CEP and asked on spark-user group
## How was this patch tested?
Manually with KafkaDirect. WIP
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/agsachin/spark CEP
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/spark/pull/12518.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #12518
----
commit 87032d23a1ebfa909408edb193763165f351383a
Author: sachin aggarwal <[email protected]>
Date: 2016-03-23T11:57:53Z
adding initial code for CEP
commit cca266e84f123061d186ad8d9f8fdaafc6bff236
Author: sachin aggarwal <[email protected]>
Date: 2016-03-23T12:00:54Z
adding initial code for CEP
commit 633408de388db4decc38616327e588477e7147d8
Author: mariobriggs <[email protected]>
Date: 2016-03-30T14:51:38Z
sortedPrevMappedStream created incrementally
commit 087bac9bf8158ff1b556166c24a039fffc5b9f3d
Author: mariobriggs <[email protected]>
Date: 2016-03-31T13:03:57Z
add back dynamic window
commit 750c9716626b1bf0213625cc7b4365399c063d81
Author: mariobriggs <[email protected]>
Date: 2016-03-31T14:03:14Z
remove println & api Cleanup
commit 9a4d32dcc01bce30a3322fac95be3ec839dbcd59
Author: mariobriggs <[email protected]>
Date: 2016-03-31T14:09:54Z
changed WindowMetric to WindowState
commit e433602529dfb5b84eaf0390b8324c40a6e2d7a7
Author: mariobriggs <[email protected]>
Date: 2016-03-31T14:15:41Z
undo add to test suite
commit 5e67477f5dd187a371f1f6916fae2e75a652f4ac
Author: mariobriggs <[email protected]>
Date: 2016-03-31T14:17:47Z
more undo to test suite
commit 94410a75a279261e62a0bdd5aaf9bda74c08a022
Author: Mario Briggs <[email protected]>
Date: 2016-04-13T17:28:15Z
Per part commit
commit b16f1389f5c64964e12bb5d29b5f2ce09428cc61
Author: mariobriggs <[email protected]>
Date: 2016-04-19T18:34:24Z
Support on PairDStream to enable pattern matching by key
commit 8e5a5b28b3872f4f6620eb5bbf86aab84d83bfcd
Author: mariobriggs <[email protected]>
Date: 2016-04-19T18:43:21Z
missed file KeyedPatternMatchDStrean
commit 162ffbe39e41e958bbf31bc68e0f5cbe2d68ec78
Author: mariobriggs <[email protected]>
Date: 2016-04-20T04:41:19Z
CEP samples using DirectKafkaInputDStream
commit 1e1c6703c7b6dfa17b88c171029d9b689c584244
Author: sachin aggarwal <[email protected]>
Date: 2016-04-20T04:41:48Z
Merge pull request #2 from agsachin/cep-marioPerPar
Support for Pattern match on PairDStream
----
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [email protected] or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
