[
https://issues.apache.org/jira/browse/EAGLE-348?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15434584#comment-15434584
]
ASF GitHub Bot commented on EAGLE-348:
--------------------------------------
Github user haoch commented on the issue:
https://github.com/apache/incubator-eagle/pull/379
One of the target is to make sure alert engine could be decoupled to run on
both Storm and Spark, while they should share the same metadata, I don't
understand why do we need spark specific metadata like: `SpecMetadataResource`?
> Alert engine base on spark streaming
> ------------------------------------
>
> Key: EAGLE-348
> URL: https://issues.apache.org/jira/browse/EAGLE-348
> Project: Eagle
> Issue Type: New Feature
> Affects Versions: v0.6.0
> Reporter: JiJun Tang
> Assignee: Hao Chen
> Priority: Minor
> Fix For: v0.6.0
>
>
> {noformat}
> New alert engine architecture:
> 1)Metaservice: a) get meta(policies,topologies,streamdefinitions and so on)
> 2)Coordinator: a) pull policies and figure out if polices are changed by
> Metaservice and notify config consumer of config changes by ConfigBusService
> (powered by zk node cache)
> b) placePolicy depend on policy usage on alertbolt
> 3)UnitTopology: a) connect CorrelationSpout(read all topics data from kafka)
> StreamRouterBolt(find which alertbolt to execute) AlertBolt(execute policy on
> stream) AlertPublisherBolt (publish execute result to db,email and so on)
>
> Let the alert engine to work on spark streaming,I think :
> 1) #1 and #2 can be worked on both storm and spark platform, can use
> broadcast to distribute this service
> 2) we should rewirte
> CorrelationSpout,StreamRouterBolt,AlertBolt,AlertPublisherBolt to Dstream and
> connect them as DAG,then submit into spark platform.
> 3) spark can dynamic allocation executor without restart
> 4) in the future spark streaming can support add or delete kafka topic
> without restart
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
