[
https://issues.apache.org/jira/browse/EAGLE-348?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15434070#comment-15434070
]
ASF GitHub Bot commented on EAGLE-348:
--------------------------------------
GitHub user r7raul1984 opened a pull request:
https://github.com/apache/incubator-eagle/pull/379
EAGLE-348 Alert engine base on spark streaming
- EAGLE-424
- Support kafka add/remove topic without requiring restart of the alert
engine
- Read spec metadata by restful web app
- Create UnitSparkTopologyRunner to demo spark streaming usage
- Update eagle-alert/README.md to describe how to start spark streaming
alert engine
- EAGLE-425
- Use Spark Accumulator to save and recover state
- Use Spark Accumulator to support Hybird Time Window
https://issues.apache.org/jira/browse/EAGLE-348
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/r7raul1984/incubator-eagle spark-EAGLE425
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/incubator-eagle/pull/379.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #379
----
commit 2588eb922f99b28d2df5f7239ca0d54c80f61ff1
Author: r7raul1984 <[email protected]>
Date: 2016-08-15T07:30:20Z
EAGLE-424 Kafka Integration without requiring restart of the alert engine
- Support kafka add/remove topic without requiring restart of the alert
engine
- Read spec metadata by restful web app
- Create UnitSparkTopologyRunner to demo spark streaming usage
- Update eagle-alert/README.md to describe how to start spark streaming
alert engine
https://issues.apache.org/jira/browse/EAGLE-424
commit 4458c02153977ea84271720ef9c1a8b7d12eb007
Author: r7raul1984 <[email protected]>
Date: 2016-08-15T09:17:58Z
Modify TestSiddhiEvaluator to sleep more time to wait the result.
commit 0849e03419946341d9158a4ac27d528cfac305d1
Author: r7raul1984 <[email protected]>
Date: 2016-08-24T01:38:45Z
EAGLE-425 Supporting Hybird Time Window
- Use Spark Accumulator to support Hybird Time Window
- Use Spark Accumulator to save and recover state
https://issues.apache.org/jira/browse/EAGLE-425
----
> Alert engine base on spark streaming
> ------------------------------------
>
> Key: EAGLE-348
> URL: https://issues.apache.org/jira/browse/EAGLE-348
> Project: Eagle
> Issue Type: New Feature
> Affects Versions: v0.6.0
> Reporter: JiJun Tang
> Assignee: Hao Chen
> Priority: Minor
> Fix For: v0.6.0
>
>
> {noformat}
> New alert engine architecture:
> 1)Metaservice: a) get meta(policies,topologies,streamdefinitions and so on)
> 2)Coordinator: a) pull policies and figure out if polices are changed by
> Metaservice and notify config consumer of config changes by ConfigBusService
> (powered by zk node cache)
> b) placePolicy depend on policy usage on alertbolt
> 3)UnitTopology: a) connect CorrelationSpout(read all topics data from kafka)
> StreamRouterBolt(find which alertbolt to execute) AlertBolt(execute policy on
> stream) AlertPublisherBolt (publish execute result to db,email and so on)
>
> Let the alert engine to work on spark streaming,I think :
> 1) #1 and #2 can be worked on both storm and spark platform, can use
> broadcast to distribute this service
> 2) we should rewirte
> CorrelationSpout,StreamRouterBolt,AlertBolt,AlertPublisherBolt to Dstream and
> connect them as DAG,then submit into spark platform.
> 3) spark can dynamic allocation executor without restart
> 4) in the future spark streaming can support add or delete kafka topic
> without restart
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
