[jira] [Assigned] (EAGLE-771) AlertEngine: Make publishment kafka endpoint as optional

[Su Ralph (JIRA)]

 [ 
https://issues.apache.org/jira/browse/EAGLE-771?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Su Ralph reassigned EAGLE-771:
--

Assignee: Garrett Li  (was: Su Ralph)

> AlertEngine: Make publishment kafka endpoint as optional
> 
>
> Key: EAGLE-771
> URL: https://issues.apache.org/jira/browse/EAGLE-771
> Project: Eagle
>  Issue Type: Bug
>Affects Versions: v0.5.0
>Reporter: Su Ralph
>Assignee: Garrett Li
> Fix For: v0.5.0
>
>
> For alert engine runtime, current kafka publishment have explicitly endpoint 
> set, but in real deployment, these kafka endpoint are mostly the same the 
> spout kafka url. 
> So we could make it as optional and by default reuse the kafka endpoint in 
> the application.conf. This help reduce the redundancy of the metadata, and 
> have a benefit that metadata would be the same for different deployment 
> environments.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Created] (EAGLE-771) AlertEngine: Make publishment kafka endpoint as optional

[Su Ralph (JIRA)]

Su Ralph created EAGLE-771:
--

 Summary: AlertEngine: Make publishment kafka endpoint as optional
 Key: EAGLE-771
 URL: https://issues.apache.org/jira/browse/EAGLE-771
 Project: Eagle
  Issue Type: Bug
Affects Versions: v0.5.0
Reporter: Su Ralph
Assignee: Su Ralph
 Fix For: v0.5.0


For alert engine runtime, current kafka publishment have explicitly endpoint 
set, but in real deployment, these kafka endpoint are mostly the same the spout 
kafka url. 

So we could make it as optional and by default reuse the kafka endpoint in the 
application.conf. This help reduce the redundancy of the metadata, and have a 
benefit that metadata would be the same for different deployment environments.




--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Created] (EAGLE-768) AlertEngine: Policy Validation API doesn't evaluate the data source missing

[Su Ralph (JIRA)]

Su Ralph created EAGLE-768:
--

 Summary: AlertEngine: Policy Validation API doesn't evaluate the 
data source missing
 Key: EAGLE-768
 URL: https://issues.apache.org/jira/browse/EAGLE-768
 Project: Eagle
  Issue Type: Bug
Affects Versions: v0.5.0
Reporter: Su Ralph
Assignee: Garrett Li
 Fix For: v0.5.0


Below scenario escape the policy validation,

Policy Input Stream refers an invalid datasource.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Created] (EAGLE-765) AlertEngine: Coordinator return timeout instead of real error when metadata invalid

[Su Ralph (JIRA)]

Su Ralph created EAGLE-765:
--

 Summary: AlertEngine: Coordinator return timeout instead of real 
error when metadata invalid
 Key: EAGLE-765
 URL: https://issues.apache.org/jira/browse/EAGLE-765
 Project: Eagle
  Issue Type: Bug
Reporter: Su Ralph
Assignee: Su Ralph






--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Resolved] (EAGLE-746) PolicyValidation is insufficient, and will cause blocking for extended handler

[Su Ralph (JIRA)]

 [ 
https://issues.apache.org/jira/browse/EAGLE-746?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Su Ralph resolved EAGLE-746.

Resolution: Fixed

> PolicyValidation is insufficient, and will cause blocking for extended handler
> --
>
> Key: EAGLE-746
> URL: https://issues.apache.org/jira/browse/EAGLE-746
> Project: Eagle
>  Issue Type: Bug
>Affects Versions: v0.5.0
>Reporter: Su Ralph
>Assignee: Su Ralph
> Fix For: v0.5.0
>
>
> Policy validation is hard code to siddhi ql without checking the policy type. 
> Also for complex query, we need to make sure leftjoin query things also works



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Assigned] (EAGLE-750) Improve coordinator schedule strategy to reuse alert work slot

[Su Ralph (JIRA)]

 [ 
https://issues.apache.org/jira/browse/EAGLE-750?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Su Ralph reassigned EAGLE-750:
--

Assignee: Garrett Li  (was: Su Ralph)

> Improve coordinator schedule strategy to reuse alert work slot
> --
>
> Key: EAGLE-750
> URL: https://issues.apache.org/jira/browse/EAGLE-750
> Project: Eagle
>  Issue Type: Bug
>Affects Versions: v0.5.0
>Reporter: Hao Chen
>Assignee: Garrett Li
> Fix For: v0.5.0
>
>
> We catch with some critical problem about alert engine policy schedule 
> strategy: 
> For example for an alert topology with 20 alert bolts , and after boarding 4 
> policies and each assigning 5 slots, then the current alert coordinator will 
> treat the work slots as used up and be unable to schedule any more new 
> policies.
> But in fact some typical eagle deployment will have at least 20+ streams with 
> lots of different partition requirements, which means we could just define 
> very few policies under such scheduling strategy and may waste slot resources 
> of eagle’s deployment infrastructure.
> In original design, we should have designed to reused slot queue among 
> in-conflict monitored stream (stream-partition-sort), then eagle could try to 
> reuse alert slots if possible to improve resource utilization, and  it should 
> be time to implement it now.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Assigned] (EAGLE-675) AlertEngine: don't host long-live curator framework for schedule

[Su Ralph (JIRA)]

 [ 
https://issues.apache.org/jira/browse/EAGLE-675?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Su Ralph reassigned EAGLE-675:
--

Assignee: Su Ralph  (was: Zeng, Bryant)

> AlertEngine: don't host long-live curator framework for schedule
> 
>
> Key: EAGLE-675
> URL: https://issues.apache.org/jira/browse/EAGLE-675
> Project: Eagle
>  Issue Type: Bug
>Affects Versions: v0.5.0
>Reporter: Su Ralph
>Assignee: Su Ralph
> Fix For: v0.5.0
>
>
> In Coordinator, the config bus producer is hold as member. This is try to 
> reuse connection for continous operation. But this is buggy, since when zk 
> server in network issue, curator framework would generate a lot of logs that 
> flood the log file.
> Same for ExclusiveExecutor.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (EAGLE-746) PolicyValidation is insufficient, and will cause blocking for extended handler

[Su Ralph (JIRA)]

 [ 
https://issues.apache.org/jira/browse/EAGLE-746?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Su Ralph updated EAGLE-746:
---
Summary: PolicyValidation is insufficient, and will cause blocking for 
extended handler  (was: PolicyValidation is insufficient, and will cause 
blocking for extension handler)

> PolicyValidation is insufficient, and will cause blocking for extended handler
> --
>
> Key: EAGLE-746
> URL: https://issues.apache.org/jira/browse/EAGLE-746
> Project: Eagle
>  Issue Type: Bug
>Affects Versions: v0.5.0
>Reporter: Su Ralph
>Assignee: Su Ralph
> Fix For: v0.5.0
>
>
> Policy validation is hard code to siddhi ql without checking the policy type. 
> Also for complex query, we need to make sure leftjoin query things also works



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (EAGLE-746) PolicyValidation is insufficient, and will cause blocking for extension handler

[Su Ralph (JIRA)]

 [ 
https://issues.apache.org/jira/browse/EAGLE-746?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Su Ralph updated EAGLE-746:
---
Summary: PolicyValidation is insufficient, and will cause blocking for 
extension handler  (was: PolicyValidation is insufficient, and make blocking 
for extension.)

> PolicyValidation is insufficient, and will cause blocking for extension 
> handler
> ---
>
> Key: EAGLE-746
> URL: https://issues.apache.org/jira/browse/EAGLE-746
> Project: Eagle
>  Issue Type: Bug
>Affects Versions: v0.5.0
>Reporter: Su Ralph
>Assignee: Su Ralph
> Fix For: v0.5.0
>
>
> Policy validation is hard code to siddhi ql without checking the policy type. 
> Also for complex query, we need to make sure leftjoin query things also works



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Created] (EAGLE-746) PolicyValidation is insufficient, and make blocking for extension.

[Su Ralph (JIRA)]

Su Ralph created EAGLE-746:
--

 Summary: PolicyValidation is insufficient, and make blocking for 
extension.
 Key: EAGLE-746
 URL: https://issues.apache.org/jira/browse/EAGLE-746
 Project: Eagle
  Issue Type: Bug
Affects Versions: v0.5.0
Reporter: Su Ralph
Assignee: Su Ralph
 Fix For: v0.5.0


Policy validation is hard code to siddhi ql without checking the policy type. 
Also for complex query, we need to make sure leftjoin query things also works



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (EAGLE-249) Support Postgres as hive metadata

[Su Ralph (JIRA)]

 [ 
https://issues.apache.org/jira/browse/EAGLE-249?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Su Ralph updated EAGLE-249:
---
Priority: Minor  (was: Major)

change to minor, as we already support that.

> Support Postgres as hive metadata
> -
>
> Key: EAGLE-249
> URL: https://issues.apache.org/jira/browse/EAGLE-249
> Project: Eagle
>  Issue Type: Bug
>Affects Versions: v0.3.0
>Reporter: Su Ralph
>Assignee: Su Ralph
>Priority: Minor
> Fix For: v0.5.0
>
>
> From Capital One case
> 
> It looks very interesting that Postgres has this behavior. Is Postgres your 
> production setting for holding hive metadata?
> If that is the case, I think we need extend Eagle a little bit to support 
> Postgres. (Hive metadata tables use upper case for both table name and column 
> name)
> Thanks
> Edward
> On Mon, Apr 11, 2016 at 9:02 PM, Carnali, Matthew R. 
>  wrote:
> Here are the correct queries:
> 1.
> select "NAME" from "DBS”;
> 2. 
> select "t"."TBL_NAME" from "TBLS" t, "DBS" d where "t"."DB_ID"="d"."DB_ID" 
> and "d"."NAME"='%s’;
> 3.
> select "c"."COLUMN_NAME" from "DBS" d join "TBLS" t on 
> "d"."DB_ID"="t"."DB_ID" join "SDS" s on "t"."SD_ID"="s"."SD_ID" join 
> "COLUMNS_V2" c on "s"."CD_ID"="c"."CD_ID" where "d"."NAME"='%s' and 
> "t"."TBL_NAME"='%s';
> From: "Carnali, Matthew R." 
> Date: Monday, April 11, 2016 at 8:11 PM
> To: Edward Zhang 
> Cc: "qingwz...@ebay.com" , Hao Chen , 
> "li...@ebay.com" 
> Subject: Re: Fail to get WEB_CONFIG configurations for data classification
> So what I found is the following:
> http://stackoverflow.com/questions/6331504/omitting-the-double-quote-to-do-query-on-postgresql
> I modified the queries to have quotes like so (but it still does not seem 
> like it is working correctly, I am investigating further):
> 1. select ’NAME' from "DBS";
> 2. select 't.TBL_NAME' from "TBLS" t, "DBS" d where 't.db_id'='d.db_id' and 
> 'd.name'='%s’;
> 3. select 'c.column_name' from "DBS" d join "TBLS" t on 'd.db_id'='t.db_id' 
> join "SDS" s on 't.sd_id'='s.sd_id' join "COLUMNS_V2" c on 
> 's.cd_id'='c.cd_id' where 'd.name'='%s' and 't.tbl_name'='%s’;
> From: Edward Zhang 
> Date: Monday, April 11, 2016 at 3:52 PM
> To: "Carnali, Matthew R." 
> Cc: "qingwz...@ebay.com" , Hao Chen , 
> "li...@ebay.com" 
> Subject: Re: Fail to get WEB_CONFIG configurations for data classification
> We use the following queries, let me know if any of queries can't be run 
> directly against postgres?
> 1. to get databases
> String sql = "select name from DBS";
> 2. to get tables
> String sql_format = "select t.tbl_name from TBLS t, DBS d where 
> t.db_id=d.db_id and d.name='%s'";
> 3. to get columns
> String sql_format =
> "select c.column_name " +
> "from DBS d join TBLS t on d.db_id=t.db_id " +
> "join SDS s on t.sd_id=s.sd_id " +
> "join COLUMNS_V2 c on s.cd_id=c.cd_id " +
> "where d.name='%s' and t.tbl_name='%s';";



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Resolved] (EAGLE-670) AlertEngine: Make Kafka Publisher configurable to async, for throughput tuning

[Su Ralph (JIRA)]

 [ 
https://issues.apache.org/jira/browse/EAGLE-670?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Su Ralph resolved EAGLE-670.

Resolution: Fixed

> AlertEngine: Make Kafka Publisher configurable to async, for throughput tuning
> --
>
> Key: EAGLE-670
> URL: https://issues.apache.org/jira/browse/EAGLE-670
> Project: Eagle
>  Issue Type: Task
>Affects Versions: v0.5.0
>Reporter: Su Ralph
>Assignee: Garrett Li
> Fix For: v0.5.0
>
>
> Kafka send alert in sync would limit the throughput. Make this configurable, 
> and use async by default.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Resolved] (EAGLE-672) AlertEngine: MongoStateStore should be removed

[Su Ralph (JIRA)]

 [ 
https://issues.apache.org/jira/browse/EAGLE-672?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Su Ralph resolved EAGLE-672.

Resolution: Fixed

> AlertEngine: MongoStateStore should be removed
> --
>
> Key: EAGLE-672
> URL: https://issues.apache.org/jira/browse/EAGLE-672
> Project: Eagle
>  Issue Type: Task
>Affects Versions: v0.5.0
>Reporter: Su Ralph
>Assignee: Garrett Li
> Fix For: v0.5.0
>
>
> Below stack reported by Hao. We should not have mongo state store evolved in 
> deployment that doesn't have mongo enabled. And the class have too much 
> unreasonable hardcode field names. Please consider remove it.
> Stack:
> 2016-10-23 03:39:02 Thread-13 
> org.apache.eagle.alert.engine.publisher.dedup.MongoDedupEventsStore [ERROR] 
> find dedup state failed, but the state in memory is good, could be ingored.
> java.lang.NullPointerException: null
>at 
> org.apache.eagle.alert.engine.publisher.dedup.MongoDedupEventsStore.getEvents(MongoDedupEventsStore.java:102)
>  ~[stormjar.jar:na]
>at 
> org.apache.eagle.alert.engine.publisher.dedup.DedupCache.getEvents(DedupCache.java:103)
>  [stormjar.jar:na]
>at 
> org.apache.eagle.alert.engine.publisher.dedup.DedupCache$2.run(DedupCache.java:81)
>  [stormjar.jar:na]
>at 
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) 
> [na:1.8.0_91]
>at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308) 
> [na:1.8.0_91]
>at 
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180)
>  [na:1.8.0_91]
>at 
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294)
>  [na:1.8.0_91]
>at 
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
>  [na:1.8.0_91]
>at 
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
>  [na:1.8.0_91]
>at java.lang.Thread.run(Thread.java:745) [na:1.8.0_91]



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Resolved] (EAGLE-440) Alert mongodb storage refine

[Su Ralph (JIRA)]

 [ 
https://issues.apache.org/jira/browse/EAGLE-440?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Su Ralph resolved EAGLE-440.

Resolution: Fixed

> Alert mongodb storage refine
> 
>
> Key: EAGLE-440
> URL: https://issues.apache.org/jira/browse/EAGLE-440
> Project: Eagle
>  Issue Type: Improvement
>Reporter: Zeng, Bryant
>Assignee: Zeng, Bryant
>
> Currently, some Alert Mongo Collection is too large. Consider future 
> increase, need to split large collection to some small collections.
> To back compatible, need to keep the interface as the same as before. 



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Resolved] (EAGLE-602) Exception that Spec Version [xxx] of AlertBolt is newer than Stream Event Version [null]!

[Su Ralph (JIRA)]

 [ 
https://issues.apache.org/jira/browse/EAGLE-602?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Su Ralph resolved EAGLE-602.

Resolution: Fixed

> Exception that Spec Version [xxx] of AlertBolt is newer than Stream Event 
> Version [null]!
> -
>
> Key: EAGLE-602
> URL: https://issues.apache.org/jira/browse/EAGLE-602
> Project: Eagle
>  Issue Type: Bug
>Affects Versions: v0.5.0
>Reporter: Zeng, Bryant
>Assignee: Zeng, Bryant
> Fix For: v0.5.0
>
>
> Just find exception as below:
> 2016-10-06 22:51:41 org.apache.eagle.alert.engine.runner.AlertBolt [WARN] 
> Spec Version [spec_version_1475817732537] of AlertBolt is newer than Stream 
> Event Version [null]!
> Seems there's an error when get meta version from stream event or 
> initialising failed when stream event version is null.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Resolved] (EAGLE-684) AlertEngine : PolicyDefinition.Defintion should not be changed.

[Su Ralph (JIRA)]

 [ 
https://issues.apache.org/jira/browse/EAGLE-684?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Su Ralph resolved EAGLE-684.

Resolution: Fixed

> AlertEngine : PolicyDefinition.Defintion should not be changed.
> ---
>
> Key: EAGLE-684
> URL: https://issues.apache.org/jira/browse/EAGLE-684
> Project: Eagle
>  Issue Type: Bug
>Affects Versions: v0.5.0
>Reporter: Su Ralph
>Assignee: Su Ralph
>Priority: Critical
> Fix For: v0.5.0
>
>
> When introduced stated policy definition in one policy, there is a logic to 
> set the PolicyDefinition.Defintion's input and output stream while create the 
> policy handler.
> This behavior change the policy definition itself, thus cause the metadata 
> reload incorrectly close and recreate policy evaluation handler.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Created] (EAGLE-684) AlertEngine : PolicyDefinition.Defintion should not be changed.

[Su Ralph (JIRA)]

Su Ralph created EAGLE-684:
--

 Summary: AlertEngine : PolicyDefinition.Defintion should not be 
changed.
 Key: EAGLE-684
 URL: https://issues.apache.org/jira/browse/EAGLE-684
 Project: Eagle
  Issue Type: Bug
Affects Versions: v0.5.0
Reporter: Su Ralph
Assignee: Su Ralph
Priority: Critical
 Fix For: v0.5.0


When introduced stated policy definition in one policy, there is a logic to set 
the PolicyDefinition.Defintion's input and output stream while create the 
policy handler.

This behavior change the policy definition itself, thus cause the metadata 
reload incorrectly close and recreate policy evaluation handler.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Created] (EAGLE-683) AlertEngine : use capped collection for mongodb schedule specs, keep last 1000

[Su Ralph (JIRA)]

Su Ralph created EAGLE-683:
--

 Summary: AlertEngine : use capped collection for mongodb schedule 
specs, keep last 1000
 Key: EAGLE-683
 URL: https://issues.apache.org/jira/browse/EAGLE-683
 Project: Eagle
  Issue Type: Improvement
Affects Versions: v0.5.0
Reporter: Su Ralph
Assignee: Zeng, Bryant
 Fix For: v0.5.0


Currently, we enable periodically schedule in coordinator service itself. If 
this is enabled, the schedule spec will finally used up metadata storage.

We should
1. Disable periodically schedule by default (storage like mysql doesn't have 
capped feature)
2. For mongodb storage, use capped collection for schedule_specs, 
policy_snapshots, streamSnapshots, groupSpecs, alertSpecs, monitoredStreams, 
spoutSpecs (all schedule spec related collection).

Let use default config of 20,000 as capped collection size.




--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Created] (EAGLE-675) AlertEngine: don't host long-live curator framework for schedule

[Su Ralph (JIRA)]

Su Ralph created EAGLE-675:
--

 Summary: AlertEngine: don't host long-live curator framework for 
schedule
 Key: EAGLE-675
 URL: https://issues.apache.org/jira/browse/EAGLE-675
 Project: Eagle
  Issue Type: Bug
Affects Versions: v0.5.0
Reporter: Su Ralph
Assignee: Zeng, Bryant
 Fix For: v0.5.0


In Coordinator, the config bus producer is hold as member. This is try to reuse 
connection for continous operation. But this is buggy, since when zk server in 
network issue, curator framework would generate a lot of logs that flood the 
log file.

Same for ExclusiveExecutor.




--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Resolved] (EAGLE-671) AlertEngine: incorrect metric system use : too many metric object create and registered

[Su Ralph (JIRA)]

 [ 
https://issues.apache.org/jira/browse/EAGLE-671?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Su Ralph resolved EAGLE-671.

Resolution: Fixed

Resolve as merged

> AlertEngine: incorrect metric system use : too many metric object create and 
> registered
> ---
>
> Key: EAGLE-671
> URL: https://issues.apache.org/jira/browse/EAGLE-671
> Project: Eagle
>  Issue Type: Task
>Affects Versions: v0.5.0
>Reporter: Su Ralph
>Assignee: Zeng, Bryant
> Fix For: v0.5.0
>
>
> In AlertBolt: we should register one metric to metric system, then use the 
> metric object to send metric. Not  every-time register the metric to metric 
> systm.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Created] (EAGLE-672) AlertEngine: MongoStateStore should be removed

[Su Ralph (JIRA)]

Su Ralph created EAGLE-672:
--

 Summary: AlertEngine: MongoStateStore should be removed
 Key: EAGLE-672
 URL: https://issues.apache.org/jira/browse/EAGLE-672
 Project: Eagle
  Issue Type: Task
Affects Versions: v0.5.0
Reporter: Su Ralph
Assignee: Garrett Li
 Fix For: v0.5.0


Below stack reported by Hao. We should not have mongo state store evolved in 
deployment that doesn't have mongo enabled. And the class have too much 
unreasonable hardcode field names. Please consider remove it.

Stack:

2016-10-23 03:39:02 Thread-13 
org.apache.eagle.alert.engine.publisher.dedup.MongoDedupEventsStore [ERROR] 
find dedup state failed, but the state in memory is good, could be ingored.
java.lang.NullPointerException: null
   at 
org.apache.eagle.alert.engine.publisher.dedup.MongoDedupEventsStore.getEvents(MongoDedupEventsStore.java:102)
 ~[stormjar.jar:na]
   at 
org.apache.eagle.alert.engine.publisher.dedup.DedupCache.getEvents(DedupCache.java:103)
 [stormjar.jar:na]
   at 
org.apache.eagle.alert.engine.publisher.dedup.DedupCache$2.run(DedupCache.java:81)
 [stormjar.jar:na]
   at 
java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) 
[na:1.8.0_91]
   at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308) 
[na:1.8.0_91]
   at 
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180)
 [na:1.8.0_91]
   at 
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294)
 [na:1.8.0_91]
   at 
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) 
[na:1.8.0_91]
   at 
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) 
[na:1.8.0_91]
   at java.lang.Thread.run(Thread.java:745) [na:1.8.0_91]





--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Created] (EAGLE-671) AlertEngine: incorrect metric system use : too many metric object create and registered

[Su Ralph (JIRA)]

Su Ralph created EAGLE-671:
--

 Summary: AlertEngine: incorrect metric system use : too many 
metric object create and registered
 Key: EAGLE-671
 URL: https://issues.apache.org/jira/browse/EAGLE-671
 Project: Eagle
  Issue Type: Task
Affects Versions: v0.5.0
Reporter: Su Ralph
Assignee: Zeng, Bryant
 Fix For: v0.5.0


In AlertBolt: we should register one metric to metric system, then use the 
metric object to send metric. Not  every-time register the metric to metric 
systm.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Created] (EAGLE-670) AlertEngine: Make Kafka Publisher configurable to async, for throughput tuning

[Su Ralph (JIRA)]

Su Ralph created EAGLE-670:
--

 Summary: AlertEngine: Make Kafka Publisher configurable to async, 
for throughput tuning
 Key: EAGLE-670
 URL: https://issues.apache.org/jira/browse/EAGLE-670
 Project: Eagle
  Issue Type: Task
Affects Versions: v0.5.0
Reporter: Su Ralph
Assignee: Garrett Li
 Fix For: v0.5.0


Kafka send alert in sync would limit the throughput. Make this configurable, 
and use async by default.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Resolved] (EAGLE-622) AlertEngine : unit test failed caused by datasource name and topic name change

[Su Ralph (JIRA)]

 [ 
https://issues.apache.org/jira/browse/EAGLE-622?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Su Ralph resolved EAGLE-622.

Resolution: Fixed

> AlertEngine : unit test failed caused by datasource name and topic name change
> --
>
> Key: EAGLE-622
> URL: https://issues.apache.org/jira/browse/EAGLE-622
> Project: Eagle
>  Issue Type: Bug
>Affects Versions: v0.5.0
>Reporter: Su Ralph
>Assignee: Su Ralph
>Priority: Blocker
> Fix For: v0.5.0
>
>
> The commit 
> https://github.com/apache/incubator-eagle/commit/2e715e3e94a6b98e0f3073d5d384878a0a845b72
> change the datasource name to topic name in spout metadata. But the unit test 
> are not updated, thus cause failure.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Created] (EAGLE-622) AlertEngine : unit test failed caused by datasource name and topic name change

[Su Ralph (JIRA)]

Su Ralph created EAGLE-622:
--

 Summary: AlertEngine : unit test failed caused by datasource name 
and topic name change
 Key: EAGLE-622
 URL: https://issues.apache.org/jira/browse/EAGLE-622
 Project: Eagle
  Issue Type: Bug
Affects Versions: v0.5.0
Reporter: Su Ralph
Assignee: Su Ralph
Priority: Blocker
 Fix For: v0.5.0


The commit 
https://github.com/apache/incubator-eagle/commit/2e715e3e94a6b98e0f3073d5d384878a0a845b72

change the datasource name to topic name in spout metadata. But the unit test 
are not updated, thus cause failure.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Resolved] (EAGLE-620) AlertEngine: SpoutWrapper are sending duplicated message.

[Su Ralph (JIRA)]

 [ 
https://issues.apache.org/jira/browse/EAGLE-620?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Su Ralph resolved EAGLE-620.

Resolution: Fixed

> AlertEngine: SpoutWrapper are sending duplicated message.
> -
>
> Key: EAGLE-620
> URL: https://issues.apache.org/jira/browse/EAGLE-620
> Project: Eagle
>  Issue Type: Bug
>Affects Versions: v0.5.0
>Reporter: Su Ralph
>Assignee: Su Ralph
>Priority: Blocker
> Fix For: v0.5.0
>
>
> An event of stream id "ncAlertOutputStream" will be send three times given 
> below spec:
> process.network.alerts: [
> {
> topicName: "network_aggregate_alert_output_datasource",
> streamId: "ncAlertOutputStream",
> groupingStrategies: [
> {
> partition: {
> streamId: "ncAlertOutputStream",
> type: "GROUPBY",
> columns: [
> "entity"
> ]
> },
> numTotalParticipatingRouterBolts: 4,
> startSequence: 0,
> totalTargetBoltIds: [
> "streamRouterBolt0",
> "streamRouterBolt1",
> "streamRouterBolt2",
> "streamRouterBolt3"
> ]
> }
> ]
> },
> {
> topicName: "network_aggregate_alert_output_datasource",
> streamId: "sherlockAlertOutputStream",
> groupingStrategies: [
> {
> partition: {
> streamId: "sherlockAlertOutputStream",
> type: "GROUPBY",
> columns: [
> "entity"
> ]
> },
> numTotalParticipatingRouterBolts: 4,
> startSequence: 0,
> totalTargetBoltIds: [
> "streamRouterBolt0",
> "streamRouterBolt1",
> "streamRouterBolt2",
> "streamRouterBolt3"
> ]
> }
> ]
> },
> {
> topicName: "network_aggregate_alert_output_datasource",
> streamId: "correlatedAlertStream",
> groupingStrategies: [
> {
> partition: {
> streamId: "correlatedAlertStream",
> type: "GROUPBY",
> columns: [
> "linkedSwitch"
> ]
> },
> numTotalParticipatingRouterBolts: 4,
> startSequence: 0,
> totalTargetBoltIds: [
> "streamRouterBolt0",
> "streamRouterBolt1",
> "streamRouterBolt2",
> "streamRouterBolt3"
> ]
> }
> ]
> }
> ],
> The root cause is in SpoutOutputCollectorWrapper.emit() the streamId of group 
> strategy is not compared with the given stream event.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Created] (EAGLE-620) AlertEngine: SpoutWrapper are sending duplicated message.

[Su Ralph (JIRA)]

Su Ralph created EAGLE-620:
--

 Summary: AlertEngine: SpoutWrapper are sending duplicated message.
 Key: EAGLE-620
 URL: https://issues.apache.org/jira/browse/EAGLE-620
 Project: Eagle
  Issue Type: Bug
Affects Versions: v0.5.0
Reporter: Su Ralph
Assignee: Su Ralph
Priority: Blocker
 Fix For: v0.5.0


An event of stream id "ncAlertOutputStream" will be send three times given 
below spec:

process.network.alerts: [
{
topicName: "network_aggregate_alert_output_datasource",
streamId: "ncAlertOutputStream",
groupingStrategies: [
{
partition: {
streamId: "ncAlertOutputStream",
type: "GROUPBY",
columns: [
"entity"
]
},
numTotalParticipatingRouterBolts: 4,
startSequence: 0,
totalTargetBoltIds: [
"streamRouterBolt0",
"streamRouterBolt1",
"streamRouterBolt2",
"streamRouterBolt3"
]
}
]
},
{
topicName: "network_aggregate_alert_output_datasource",
streamId: "sherlockAlertOutputStream",
groupingStrategies: [
{
partition: {
streamId: "sherlockAlertOutputStream",
type: "GROUPBY",
columns: [
"entity"
]
},
numTotalParticipatingRouterBolts: 4,
startSequence: 0,
totalTargetBoltIds: [
"streamRouterBolt0",
"streamRouterBolt1",
"streamRouterBolt2",
"streamRouterBolt3"
]
}
]
},
{
topicName: "network_aggregate_alert_output_datasource",
streamId: "correlatedAlertStream",
groupingStrategies: [
{
partition: {
streamId: "correlatedAlertStream",
type: "GROUPBY",
columns: [
"linkedSwitch"
]
},
numTotalParticipatingRouterBolts: 4,
startSequence: 0,
totalTargetBoltIds: [
"streamRouterBolt0",
"streamRouterBolt1",
"streamRouterBolt2",
"streamRouterBolt3"
]
}
]
}
],

The root cause is in SpoutOutputCollectorWrapper.emit() the streamId of group 
strategy is not compared with the given stream event.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (EAGLE-259) Compare JDBC Schema and Alter if necessary when initializing

[Su Ralph (JIRA)]

[ 
https://issues.apache.org/jira/browse/EAGLE-259?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15567857#comment-15567857
 ] 

Su Ralph commented on EAGLE-259:


[~haoch] should we have more comments for this one? And it's for v0.3.0, a 
release months ago, if this is not need in eagle v0.5.0. Suggest to close it as 
fix version of v0.5.0.

> Compare JDBC Schema and Alter if necessary when initializing
> 
>
> Key: EAGLE-259
> URL: https://issues.apache.org/jira/browse/EAGLE-259
> Project: Eagle
>  Issue Type: Improvement
>Affects Versions: v0.3.0
>Reporter: Hao Chen
>Priority: Minor
>




--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (EAGLE-253) query=AlertService[@hostname=~".*"] returns different results under HBASE and JDBC DB

[Su Ralph (JIRA)]

 [ 
https://issues.apache.org/jira/browse/EAGLE-253?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Su Ralph updated EAGLE-253:
---
Description: when AlertAPIEntity#host is null, the expected result is a 
empty list. But the query on HBASE can ignore this condition, and gives the 
results satisfying other conditions

> query=AlertService[@hostname=~".*"] returns different results under HBASE and 
> JDBC DB
> -
>
> Key: EAGLE-253
> URL: https://issues.apache.org/jira/browse/EAGLE-253
> Project: Eagle
>  Issue Type: Bug
>Affects Versions: v0.3.0
> Environment: when AlertAPIEntity#host is null, the expected result is 
> a empty list. But the query on HBASE can ignore this condition, and gives the 
> results satisfying other conditions
>Reporter: Zhao, Qingwen
> Fix For: v0.5.0
>
>
> when AlertAPIEntity#host is null, the expected result is a empty list. But 
> the query on HBASE can ignore this condition, and gives the results 
> satisfying other conditions



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (EAGLE-250) Metadata on mongodb

[Su Ralph (JIRA)]

[ 
https://issues.apache.org/jira/browse/EAGLE-250?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15567850#comment-15567850
 ] 

Su Ralph commented on EAGLE-250:


In eagle v0.5.0, we use mysql, but also provide MongoMetadataDAOImpl. Resolve 
this one.

> Metadata on mongodb
> ---
>
> Key: EAGLE-250
> URL: https://issues.apache.org/jira/browse/EAGLE-250
> Project: Eagle
>  Issue Type: New Feature
>Affects Versions: v0.3.0
>Reporter: Su Ralph
>Priority: Minor
>  Labels: new
> Fix For: v0.5.0
>
>
> Currently, eagle support metadata on hbase/jdbc. One might want to extend to 
> mongdb which is widely used as configuration store.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Resolved] (EAGLE-250) Metadata on mongodb

[Su Ralph (JIRA)]

 [ 
https://issues.apache.org/jira/browse/EAGLE-250?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Su Ralph resolved EAGLE-250.

Resolution: Fixed
  Assignee: Su Ralph

> Metadata on mongodb
> ---
>
> Key: EAGLE-250
> URL: https://issues.apache.org/jira/browse/EAGLE-250
> Project: Eagle
>  Issue Type: New Feature
>Affects Versions: v0.3.0
>Reporter: Su Ralph
>Assignee: Su Ralph
>Priority: Minor
>  Labels: new
> Fix For: v0.5.0
>
>
> Currently, eagle support metadata on hbase/jdbc. One might want to extend to 
> mongdb which is widely used as configuration store.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (EAGLE-250) Metadata on mongodb

[Su Ralph (JIRA)]

 [ 
https://issues.apache.org/jira/browse/EAGLE-250?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Su Ralph updated EAGLE-250:
---
Fix Version/s: v0.5.0

> Metadata on mongodb
> ---
>
> Key: EAGLE-250
> URL: https://issues.apache.org/jira/browse/EAGLE-250
> Project: Eagle
>  Issue Type: New Feature
>Affects Versions: v0.3.0
>Reporter: Su Ralph
>Priority: Minor
>  Labels: new
> Fix For: v0.5.0
>
>
> Currently, eagle support metadata on hbase/jdbc. One might want to extend to 
> mongdb which is widely used as configuration store.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (EAGLE-227) java.lang.NoClassDefFoundError: org/apache/commons/pool/impl/CursorableLinkedList$ListIter

[Su Ralph (JIRA)]

[ 
https://issues.apache.org/jira/browse/EAGLE-227?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15567843#comment-15567843
 ] 

Su Ralph commented on EAGLE-227:


should we close this one? It's for v0.4.0, but the status is "reopened". i 
believe this is already fixed.

> java.lang.NoClassDefFoundError: 
> org/apache/commons/pool/impl/CursorableLinkedList$ListIter
> --
>
> Key: EAGLE-227
> URL: https://issues.apache.org/jira/browse/EAGLE-227
> Project: Eagle
>  Issue Type: Bug
>Affects Versions: v0.3.0
>Reporter: Hao Chen
>Assignee: Hao Chen
> Fix For: v0.4.1
>
>
> Exception in thread "Thread-4" java.lang.NoClassDefFoundError: 
> org/apache/commons/pool/impl/CursorableLinkedList$ListIter
> at 
> org.apache.commons.pool.impl.CursorableLinkedList.listIterator(CursorableLinkedList.java:502)
> at 
> org.apache.commons.pool.impl.CursorableLinkedList.iterator(CursorableLinkedList.java:449)
> at 
> org.apache.commons.pool.impl.CursorableLinkedList.clear(CursorableLinkedList.java:215)
> at 
> org.apache.commons.pool.impl.GenericKeyedObjectPool.clear(GenericKeyedObjectPool.java:1342)
> at 
> org.apache.commons.pool.impl.GenericKeyedObjectPool.close(GenericKeyedObjectPool.java:1725)
> at 
> org.apache.commons.dbcp.datasources.SharedPoolDataSource.close(SharedPoolDataSource.java:75)
> at 
> org.apache.torque.dsfactory.SharedPoolDataSourceFactory.close(SharedPoolDataSourceFactory.java:111)
> at org.apache.torque.TorqueInstance.shutdown(TorqueInstance.java:871)
> at org.apache.torque.Torque.shutdown(Torque.java:242)
> at 
> org.apache.eagle.storage.jdbc.conn.impl.TorqueConnectionManagerImpl.shutdown(TorqueConnectionManagerImpl.java:95)
> at 
> org.apache.eagle.storage.jdbc.conn.ConnectionManagerFactory$ConnectionManagerShutdownHook.run(ConnectionManagerFactory.java:69)



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (EAGLE-216) Added RM Policy and GC Policies in Resource

[Su Ralph (JIRA)]

[ 
https://issues.apache.org/jira/browse/EAGLE-216?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15567835#comment-15567835
 ] 

Su Ralph commented on EAGLE-216:


Eagle now contains the all the policies including the GC policies in source. 
This actually happens since v0.4.0. 

> Added RM Policy and GC Policies in Resource
> ---
>
> Key: EAGLE-216
> URL: https://issues.apache.org/jira/browse/EAGLE-216
> Project: Eagle
>  Issue Type: Improvement
>Affects Versions: v0.3.0
>Reporter: Hao Chen
>Assignee: Senthilkumar
>  Labels: Policy
> Fix For: v0.4.0
>
>
> Added RM Policy and GC Policies in Resource. 



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (EAGLE-216) Added RM Policy and GC Policies in Resource

[Su Ralph (JIRA)]

 [ 
https://issues.apache.org/jira/browse/EAGLE-216?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Su Ralph updated EAGLE-216:
---
Description: Added RM Policy and GC Policies in Resource. 

> Added RM Policy and GC Policies in Resource
> ---
>
> Key: EAGLE-216
> URL: https://issues.apache.org/jira/browse/EAGLE-216
> Project: Eagle
>  Issue Type: Improvement
>Affects Versions: v0.3.0
>Reporter: Hao Chen
>Assignee: Senthilkumar
>  Labels: Policy
> Fix For: v0.4.0
>
>
> Added RM Policy and GC Policies in Resource. 



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (EAGLE-209) Test Project

[Su Ralph (JIRA)]

[ 
https://issues.apache.org/jira/browse/EAGLE-209?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15567830#comment-15567830
 ] 

Su Ralph commented on EAGLE-209:


[~lakshmi.prashant] is this jira for test purpose? There is no description, 
close this one. Feel free to create a new one with description if necessary.

> Test Project
> 
>
> Key: EAGLE-209
> URL: https://issues.apache.org/jira/browse/EAGLE-209
> Project: Eagle
>  Issue Type: New Feature
>Reporter: prashant
>Priority: Trivial
>




--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Resolved] (EAGLE-209) Test Project

[Su Ralph (JIRA)]

 [ 
https://issues.apache.org/jira/browse/EAGLE-209?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Su Ralph resolved EAGLE-209.

Resolution: Won't Fix

> Test Project
> 
>
> Key: EAGLE-209
> URL: https://issues.apache.org/jira/browse/EAGLE-209
> Project: Eagle
>  Issue Type: New Feature
>Reporter: prashant
>Priority: Trivial
>




--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (EAGLE-199) gzip lib/EAGLE/package/patches/app.js lib/EAGLE/package/patches/app.js.gz avoiding binary file in source code

[Su Ralph (JIRA)]

[ 
https://issues.apache.org/jira/browse/EAGLE-199?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15567823#comment-15567823
 ] 

Su Ralph commented on EAGLE-199:


resolve this one as change already merged.

> gzip lib/EAGLE/package/patches/app.js lib/EAGLE/package/patches/app.js.gz 
> avoiding binary file in source code
> -
>
> Key: EAGLE-199
> URL: https://issues.apache.org/jira/browse/EAGLE-199
> Project: Eagle
>  Issue Type: New Feature
>Reporter: Hao Chen
>Assignee: Hao Chen
>




--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Resolved] (EAGLE-199) gzip lib/EAGLE/package/patches/app.js lib/EAGLE/package/patches/app.js.gz avoiding binary file in source code

[Su Ralph (JIRA)]

 [ 
https://issues.apache.org/jira/browse/EAGLE-199?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Su Ralph resolved EAGLE-199.

Resolution: Fixed

> gzip lib/EAGLE/package/patches/app.js lib/EAGLE/package/patches/app.js.gz 
> avoiding binary file in source code
> -
>
> Key: EAGLE-199
> URL: https://issues.apache.org/jira/browse/EAGLE-199
> Project: Eagle
>  Issue Type: New Feature
>Reporter: Hao Chen
>Assignee: Hao Chen
>




--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (EAGLE-199) gzip lib/EAGLE/package/patches/app.js lib/EAGLE/package/patches/app.js.gz avoiding binary file in source code

[Su Ralph (JIRA)]

 [ 
https://issues.apache.org/jira/browse/EAGLE-199?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Su Ralph updated EAGLE-199:
---
Description: gzip lib/EAGLE/package/patches/app.js 
lib/EAGLE/package/patches/app.js.gz avoiding binary file in source code

> gzip lib/EAGLE/package/patches/app.js lib/EAGLE/package/patches/app.js.gz 
> avoiding binary file in source code
> -
>
> Key: EAGLE-199
> URL: https://issues.apache.org/jira/browse/EAGLE-199
> Project: Eagle
>  Issue Type: New Feature
>Reporter: Hao Chen
>Assignee: Hao Chen
>
> gzip lib/EAGLE/package/patches/app.js lib/EAGLE/package/patches/app.js.gz 
> avoiding binary file in source code



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Resolved] (EAGLE-191) Fix Copyright message and update pom version

[Su Ralph (JIRA)]

 [ 
https://issues.apache.org/jira/browse/EAGLE-191?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Su Ralph resolved EAGLE-191.

Resolution: Fixed
  Assignee: hemanth dendukuri

> Fix Copyright message and update pom version
> 
>
> Key: EAGLE-191
> URL: https://issues.apache.org/jira/browse/EAGLE-191
> Project: Eagle
>  Issue Type: Bug
>Reporter: hemanth dendukuri
>Assignee: hemanth dendukuri
>




--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (EAGLE-191) Fix Copyright message and update pom version

[Su Ralph (JIRA)]

[ 
https://issues.apache.org/jira/browse/EAGLE-191?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15567796#comment-15567796
 ] 

Su Ralph commented on EAGLE-191:


resolve this as the change is now merged.

> Fix Copyright message and update pom version
> 
>
> Key: EAGLE-191
> URL: https://issues.apache.org/jira/browse/EAGLE-191
> Project: Eagle
>  Issue Type: Bug
>Reporter: hemanth dendukuri
>




--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Resolved] (EAGLE-186) Add capacityusage in script support

[Su Ralph (JIRA)]

 [ 
https://issues.apache.org/jira/browse/EAGLE-186?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Su Ralph resolved EAGLE-186.

Resolution: Fixed

> Add capacityusage in script support
> ---
>
> Key: EAGLE-186
> URL: https://issues.apache.org/jira/browse/EAGLE-186
> Project: Eagle
>  Issue Type: Improvement
>Reporter: Hao Chen
>Assignee: Hao Chen
>
> {code}
> hadoop.namenode.fsnamesystemstate.capacityusage=
> hadoop.namenode.fsnamesystemstate.capacityused / 
> hadoop.namenode.fsnamesystemstate.capacitytotal * 100
> {code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (EAGLE-183) Replace Legacy "dataSource" field with "application" in UI request

[Su Ralph (JIRA)]

[ 
https://issues.apache.org/jira/browse/EAGLE-183?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15567789#comment-15567789
 ] 

Su Ralph commented on EAGLE-183:


See eagle-182 as well.

> Replace Legacy "dataSource" field with "application" in UI request
> --
>
> Key: EAGLE-183
> URL: https://issues.apache.org/jira/browse/EAGLE-183
> Project: Eagle
>  Issue Type: Sub-task
>Affects Versions: v0.3.0
>Reporter: Zhao, Qingwen
>Assignee: Jilin, Jiang
> Fix For: v0.3.0
>
>




--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (EAGLE-182) Replace Legacy "dataSource" field with "application" in UI request

[Su Ralph (JIRA)]

 [ 
https://issues.apache.org/jira/browse/EAGLE-182?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Su Ralph updated EAGLE-182:
---
Description: We change the "dataSource" to "application" in service side. 
Replace Legacy "dataSource" field with "application" in UI request  (was: 
Replace Legacy "dataSource" field with "application" in UI request)

> Replace Legacy "dataSource" field with "application" in UI request
> --
>
> Key: EAGLE-182
> URL: https://issues.apache.org/jira/browse/EAGLE-182
> Project: Eagle
>  Issue Type: Improvement
>Affects Versions: v0.4.0
>Reporter: Zhao, Qingwen
>Assignee: Jilin, Jiang
> Fix For: v0.4.0
>
>
> We change the "dataSource" to "application" in service side. Replace Legacy 
> "dataSource" field with "application" in UI request



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (EAGLE-182) Replace Legacy "dataSource" field with "application" in UI request

[Su Ralph (JIRA)]

 [ 
https://issues.apache.org/jira/browse/EAGLE-182?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Su Ralph updated EAGLE-182:
---
Description: Replace Legacy "dataSource" field with "application" in UI 
request

> Replace Legacy "dataSource" field with "application" in UI request
> --
>
> Key: EAGLE-182
> URL: https://issues.apache.org/jira/browse/EAGLE-182
> Project: Eagle
>  Issue Type: Improvement
>Affects Versions: v0.4.0
>Reporter: Zhao, Qingwen
>Assignee: Jilin, Jiang
> Fix For: v0.4.0
>
>
> Replace Legacy "dataSource" field with "application" in UI request



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Reopened] (EAGLE-182) Replace Legacy "dataSource" field with "application" in UI request

[Su Ralph (JIRA)]

 [ 
https://issues.apache.org/jira/browse/EAGLE-182?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Su Ralph reopened EAGLE-182:


> Replace Legacy "dataSource" field with "application" in UI request
> --
>
> Key: EAGLE-182
> URL: https://issues.apache.org/jira/browse/EAGLE-182
> Project: Eagle
>  Issue Type: Improvement
>Affects Versions: v0.4.0
>Reporter: Zhao, Qingwen
>Assignee: Jilin, Jiang
> Fix For: v0.4.0
>
>




--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (EAGLE-602) Exception that Spec Version [xxx] of AlertBolt is newer than Stream Event Version [null]!

[Su Ralph (JIRA)]

 [ 
https://issues.apache.org/jira/browse/EAGLE-602?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Su Ralph updated EAGLE-602:
---
Affects Version/s: v0.5.0

> Exception that Spec Version [xxx] of AlertBolt is newer than Stream Event 
> Version [null]!
> -
>
> Key: EAGLE-602
> URL: https://issues.apache.org/jira/browse/EAGLE-602
> Project: Eagle
>  Issue Type: Bug
>Affects Versions: v0.5.0
>Reporter: Zeng, Bryant
>Assignee: Zeng, Bryant
> Fix For: v0.5.0
>
>
> Just find exception as below:
> 2016-10-06 22:51:41 org.apache.eagle.alert.engine.runner.AlertBolt [WARN] 
> Spec Version [spec_version_1475817732537] of AlertBolt is newer than Stream 
> Event Version [null]!
> Seems there's an error when get meta version from stream event or 
> initialising failed when stream event version is null.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (EAGLE-602) Exception that Spec Version [xxx] of AlertBolt is newer than Stream Event Version [null]!

[Su Ralph (JIRA)]

 [ 
https://issues.apache.org/jira/browse/EAGLE-602?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Su Ralph updated EAGLE-602:
---
Fix Version/s: v0.5.0

> Exception that Spec Version [xxx] of AlertBolt is newer than Stream Event 
> Version [null]!
> -
>
> Key: EAGLE-602
> URL: https://issues.apache.org/jira/browse/EAGLE-602
> Project: Eagle
>  Issue Type: Bug
>Affects Versions: v0.5.0
>Reporter: Zeng, Bryant
>Assignee: Zeng, Bryant
> Fix For: v0.5.0
>
>
> Just find exception as below:
> 2016-10-06 22:51:41 org.apache.eagle.alert.engine.runner.AlertBolt [WARN] 
> Spec Version [spec_version_1475817732537] of AlertBolt is newer than Stream 
> Event Version [null]!
> Seems there's an error when get meta version from stream event or 
> initialising failed when stream event version is null.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (EAGLE-142) Replace AlertDefinitionDAOImpl with PolicyDefinitionDAOImpl

[Su Ralph (JIRA)]

 [ 
https://issues.apache.org/jira/browse/EAGLE-142?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Su Ralph updated EAGLE-142:
---
Description: 
Replace AlertDefinitionDAOImpl with PolicyDefinitionDAOImpl

Refactor AlertDefinitionDAOImpl to PolicyDefinitionDAOImpl to dedup the policy 
evaluation from alert.


> Replace AlertDefinitionDAOImpl with PolicyDefinitionDAOImpl
> ---
>
> Key: EAGLE-142
> URL: https://issues.apache.org/jira/browse/EAGLE-142
> Project: Eagle
>  Issue Type: Bug
>Affects Versions: v0.3.0
>Reporter: Su Ralph
>Assignee: Su Ralph
>Priority: Minor
> Fix For: v0.3.0
>
>
> Replace AlertDefinitionDAOImpl with PolicyDefinitionDAOImpl
> Refactor AlertDefinitionDAOImpl to PolicyDefinitionDAOImpl to dedup the 
> policy evaluation from alert.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Reopened] (EAGLE-440) Alert mongodb storage refine

[Su Ralph (JIRA)]

 [ 
https://issues.apache.org/jira/browse/EAGLE-440?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Su Ralph reopened EAGLE-440:


> Alert mongodb storage refine
> 
>
> Key: EAGLE-440
> URL: https://issues.apache.org/jira/browse/EAGLE-440
> Project: Eagle
>  Issue Type: Improvement
>Reporter: Zeng, Bryant
>Assignee: Zeng, Bryant
>
> Currently, some Alert Mongo Collection is too large. Consider future 
> increase, need to split large collection to some small collections.
> To back compatible, need to keep the interface as the same as before. 



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (EAGLE-519) No data / gap alert enhancement

[Su Ralph (JIRA)]

 [ 
https://issues.apache.org/jira/browse/EAGLE-519?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Su Ralph updated EAGLE-519:
---
Description: 
1. no data for an entire stream won't trigger gap alert (use local time & batch 
window) 
2. no data policies definition is too complex, make the nodata offer optional 
settings in stream definition 
3. bolt parallellism will cause multiple alerts in the same time window (use 
loopback) 

The extension of no data alert design is posted at 
https://cwiki.apache.org/confluence/display/EAG/Alert+Engine+Extension.

  was:
1. no data for an entire stream won't trigger gap alert (use local time & batch 
window) 
2. no data policies definition is too complex, make the nodata offer optional 
settings in stream definition 
3. bolt parallellism will cause multiple alerts in the same time window (use 
loopback) 



> No data / gap alert enhancement
> ---
>
> Key: EAGLE-519
> URL: https://issues.apache.org/jira/browse/EAGLE-519
> Project: Eagle
>  Issue Type: Improvement
>Affects Versions: v0.5.0
>Reporter: Garrett Li
>Assignee: Garrett Li
> Fix For: v0.5.0
>
>
> 1. no data for an entire stream won't trigger gap alert (use local time & 
> batch window) 
> 2. no data policies definition is too complex, make the nodata offer optional 
> settings in stream definition 
> 3. bolt parallellism will cause multiple alerts in the same time window (use 
> loopback) 
> The extension of no data alert design is posted at 
> https://cwiki.apache.org/confluence/display/EAG/Alert+Engine+Extension.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (EAGLE-544) Enhance dedup to support extended deduplicator

[Su Ralph (JIRA)]

 [ 
https://issues.apache.org/jira/browse/EAGLE-544?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Su Ralph updated EAGLE-544:
---
Description: 
The deduplicator configuration in publishment metadata is not configurable.

To provide extensibility, need to make deduplicator configurable also provide 
base interface/class for service provide to plugin.
Add "overrideDeduplicator" in publishment.


  was:
The deduplicator configuration in publishment metadata is not configurable.

To provide extensibility, need to make deduplicator configurable also provide 
base interface/class for service provide to plugin.



> Enhance dedup to support extended deduplicator
> --
>
> Key: EAGLE-544
> URL: https://issues.apache.org/jira/browse/EAGLE-544
> Project: Eagle
>  Issue Type: Improvement
>Affects Versions: v0.5.0
>Reporter: Garrett Li
>Assignee: Garrett Li
> Fix For: v0.5.0
>
>
> The deduplicator configuration in publishment metadata is not configurable.
> To provide extensibility, need to make deduplicator configurable also provide 
> base interface/class for service provide to plugin.
> Add "overrideDeduplicator" in publishment.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Resolved] (EAGLE-544) Enhance dedup to support extended deduplicator

[Su Ralph (JIRA)]

 [ 
https://issues.apache.org/jira/browse/EAGLE-544?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Su Ralph resolved EAGLE-544.

Resolution: Fixed

> Enhance dedup to support extended deduplicator
> --
>
> Key: EAGLE-544
> URL: https://issues.apache.org/jira/browse/EAGLE-544
> Project: Eagle
>  Issue Type: Improvement
>Affects Versions: v0.5.0
>Reporter: Garrett Li
>Assignee: Garrett Li
> Fix For: v0.5.0
>
>
> The deduplicator configuration in publishment metadata is not configurable.
> To provide extensibility, need to make deduplicator configurable also provide 
> base interface/class for service provide to plugin.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (EAGLE-544) Enhance dedup to support extended deduplicator

[Su Ralph (JIRA)]

 [ 
https://issues.apache.org/jira/browse/EAGLE-544?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Su Ralph updated EAGLE-544:
---
Description: 
The deduplicator configuration in publishment metadata is not configurable.

To provide extensibility, need to make deduplicator configurable also provide 
base interface/class for service provide to plugin.


> Enhance dedup to support extended deduplicator
> --
>
> Key: EAGLE-544
> URL: https://issues.apache.org/jira/browse/EAGLE-544
> Project: Eagle
>  Issue Type: Improvement
>Affects Versions: v0.5.0
>Reporter: Garrett Li
>Assignee: Garrett Li
> Fix For: v0.5.0
>
>
> The deduplicator configuration in publishment metadata is not configurable.
> To provide extensibility, need to make deduplicator configurable also provide 
> base interface/class for service provide to plugin.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (EAGLE-439) Introduce policy enable/disable support

[Su Ralph (JIRA)]

 [ 
https://issues.apache.org/jira/browse/EAGLE-439?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Su Ralph updated EAGLE-439:
---
Description: 
Introduce policy enable/disable support.

Suggestion:
Add policyStatus field in policy definition. With value of "ENABLED" and 
"DISABLED".
For DISABLED policy, the coordinator will exclude the policy with DISABLED 
status.

  was:Introduce policy enable/disable support


> Introduce policy enable/disable support
> ---
>
> Key: EAGLE-439
> URL: https://issues.apache.org/jira/browse/EAGLE-439
> Project: Eagle
>  Issue Type: Improvement
>Affects Versions: v0.4.0
>Reporter: Garrett Li
>Assignee: Garrett Li
>  Labels: alert-engine
> Fix For: v0.5.0
>
>
> Introduce policy enable/disable support.
> Suggestion:
> Add policyStatus field in policy definition. With value of "ENABLED" and 
> "DISABLED".
> For DISABLED policy, the coordinator will exclude the policy with DISABLED 
> status.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (EAGLE-435) Coordiantor schedule operation must be exclusive in distributed deployment

[Su Ralph (JIRA)]

 [ 
https://issues.apache.org/jira/browse/EAGLE-435?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Su Ralph updated EAGLE-435:
---
Description: 
Coordiantor schedule operation must be exclusive in distributed deployment

This is critical for ha-mode deploy of coordinator. We don't want to have 
coordinator concurrently build the specs.

  was:Coordiantor schedule operation must be exclusive in distributed deployment


> Coordiantor schedule operation must be exclusive in distributed deployment
> --
>
> Key: EAGLE-435
> URL: https://issues.apache.org/jira/browse/EAGLE-435
> Project: Eagle
>  Issue Type: Improvement
>Affects Versions: v0.5.0
>Reporter: Garrett Li
>Assignee: Garrett Li
>  Labels: alert_engine
> Fix For: v0.5.0
>
>   Original Estimate: 24h
>  Remaining Estimate: 24h
>
> Coordiantor schedule operation must be exclusive in distributed deployment
> This is critical for ha-mode deploy of coordinator. We don't want to have 
> coordinator concurrently build the specs.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Resolved] (EAGLE-567) Enhance Slack publish format

[Su Ralph (JIRA)]

 [ 
https://issues.apache.org/jira/browse/EAGLE-567?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Su Ralph resolved EAGLE-567.

Resolution: Fixed

> Enhance Slack publish format
> 
>
> Key: EAGLE-567
> URL: https://issues.apache.org/jira/browse/EAGLE-567
> Project: Eagle
>  Issue Type: Improvement
>Reporter: Zeng, Bryant
>Assignee: Zeng, Bryant
>
> Currently message publish to Slack is raw String, and this is not very clear 
> for user.
> So need to add format for it for better viewing.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (EAGLE-567) Enhance Slack publish format

[Su Ralph (JIRA)]

[ 
https://issues.apache.org/jira/browse/EAGLE-567?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15559701#comment-15559701
 ] 

Su Ralph commented on EAGLE-567:


Link to https://issues.apache.org/jira/browse/EAGLE-570 which is the further 
enhancement on the publishment change.

> Enhance Slack publish format
> 
>
> Key: EAGLE-567
> URL: https://issues.apache.org/jira/browse/EAGLE-567
> Project: Eagle
>  Issue Type: Improvement
>Reporter: Zeng, Bryant
>Assignee: Zeng, Bryant
>
> Currently message publish to Slack is raw String, and this is not very clear 
> for user.
> So need to add format for it for better viewing.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (EAGLE-567) Enhance Slack publish format

[Su Ralph (JIRA)]

[ 
https://issues.apache.org/jira/browse/EAGLE-567?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15559697#comment-15559697
 ] 

Su Ralph commented on EAGLE-567:


```from pr

The ticket is aim to enhance the format of slack publishment of Alert. The 
original message was just a raw string which is not clear for user. Add more 
format such as color for better viewing.


> Enhance Slack publish format
> 
>
> Key: EAGLE-567
> URL: https://issues.apache.org/jira/browse/EAGLE-567
> Project: Eagle
>  Issue Type: Improvement
>Reporter: Zeng, Bryant
>Assignee: Zeng, Bryant
>
> Currently message publish to Slack is raw String, and this is not very clear 
> for user.
> So need to add format for it for better viewing.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Resolved] (EAGLE-121) predefined/generic topology for adhoc data sources

[Su Ralph (JIRA)]

 [ 
https://issues.apache.org/jira/browse/EAGLE-121?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Su Ralph resolved EAGLE-121.

Resolution: Fixed

> predefined/generic topology for adhoc data sources
> --
>
> Key: EAGLE-121
> URL: https://issues.apache.org/jira/browse/EAGLE-121
> Project: Eagle
>  Issue Type: New Feature
>Reporter: Aroop Maliakkal
>Assignee: Su Ralph
>
> Eagle should provide one or more predefined topology where user can send any 
> metrics to the predefined topology. This predefined/generic topology should 
> be exposed with well documented contract/grammer details so that any process 
> which talks to this generic topology can send metrics on that contract



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Resolved] (EAGLE-335) Show alert definition so that user could see the trigger condition of alert

[Su Ralph (JIRA)]

 [ 
https://issues.apache.org/jira/browse/EAGLE-335?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Su Ralph resolved EAGLE-335.

Resolution: Won't Fix

Alert UI is redesigned. This is PR is now deprecated. closed.

> Show alert definition so that user could see the trigger condition of alert
> ---
>
> Key: EAGLE-335
> URL: https://issues.apache.org/jira/browse/EAGLE-335
> Project: Eagle
>  Issue Type: Improvement
>Affects Versions: v0.3.0
>Reporter: Su Ralph
>Assignee: Su Ralph
> Fix For: v0.5.0
>
>
> When user see a alert happens from a policy's evaluation. It would be better 
> to see the trigger condition in alert. Etc. in jmx metric cases, there would 
> alert with contents of
> The Policy "NameNodeLagPolicy" has been detected with the below information: 
> transactIdA="1.2623229E7" transactIdB="1.2623228E7" 
> showing the threshold would make it more intuitive for alert reader.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Resolved] (EAGLE-580) MongodbMetadata: kafka have dot in topic name cause spec store failed

[Su Ralph (JIRA)]

 [ 
https://issues.apache.org/jira/browse/EAGLE-580?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Su Ralph resolved EAGLE-580.

Resolution: Fixed
  Assignee: Zeng, Bryant  (was: Su Ralph)

> MongodbMetadata: kafka have dot in topic name cause spec store failed
> -
>
> Key: EAGLE-580
> URL: https://issues.apache.org/jira/browse/EAGLE-580
> Project: Eagle
>  Issue Type: Bug
>Affects Versions: v0.5.0
>Reporter: Su Ralph
>Assignee: Zeng, Bryant
> Fix For: v0.5.0
>
>
> It's possible that user have DOT in topic names. In this case, mongodb 
> metadata store of spoutspec would use the topic name as key in mongodb, which 
> rejected by mongodb since it doens't allow dot in key.
> Solution is to do covnert of SpoutSpec in mongdb storage, and do appropriate 
> transform when read from db.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Resolved] (EAGLE-586) AlertEngine: multiple routing of alerts is broken because of event not copied

[Su Ralph (JIRA)]

 [ 
https://issues.apache.org/jira/browse/EAGLE-586?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Su Ralph resolved EAGLE-586.

Resolution: Fixed

> AlertEngine: multiple routing of alerts is broken because of event not copied
> -
>
> Key: EAGLE-586
> URL: https://issues.apache.org/jira/browse/EAGLE-586
> Project: Eagle
>  Issue Type: Bug
>Affects Versions: v0.5.0
>Reporter: Su Ralph
>Assignee: Su Ralph
> Fix For: v0.5.0
>
>
> AlertEngine: multiple routing of alerts is broken because of event not copied
> The default deduplicator contains a shallow copy of the events 
> (StreamEvent.data) is not copied, thus following change impact both two 
> event. 



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (EAGLE-586) AlertEngine: multiple routing of alerts is broken because of event not copied

[Su Ralph (JIRA)]

 [ 
https://issues.apache.org/jira/browse/EAGLE-586?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Su Ralph updated EAGLE-586:
---
Description: 
AlertEngine: multiple routing of alerts is broken because of event not copied

The default deduplicator contains a shallow copy of the events 
(StreamEvent.data) is not copied, thus following change impact both two event. 

> AlertEngine: multiple routing of alerts is broken because of event not copied
> -
>
> Key: EAGLE-586
> URL: https://issues.apache.org/jira/browse/EAGLE-586
> Project: Eagle
>  Issue Type: Bug
>Affects Versions: v0.5.0
>Reporter: Su Ralph
>Assignee: Su Ralph
> Fix For: v0.5.0
>
>
> AlertEngine: multiple routing of alerts is broken because of event not copied
> The default deduplicator contains a shallow copy of the events 
> (StreamEvent.data) is not copied, thus following change impact both two 
> event. 



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Resolved] (EAGLE-590) AlertEngine: the kafka_spout might be dropped by metadata update when system ack a tupe

[Su Ralph (JIRA)]

 [ 
https://issues.apache.org/jira/browse/EAGLE-590?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Su Ralph resolved EAGLE-590.

Resolution: Fixed

> AlertEngine: the kafka_spout might be dropped by metadata update when system 
> ack a tupe
> ---
>
> Key: EAGLE-590
> URL: https://issues.apache.org/jira/browse/EAGLE-590
> Project: Eagle
>  Issue Type: Bug
>Affects Versions: v0.5.0
>Reporter: Su Ralph
>Assignee: Su Ralph
> Fix For: v0.5.0
>
>
> In correlationSpout's ack/failure method, given a messageId, a spout might be 
> not available when there is a metadata update. Thus cause an NPE in spout 
> ack/failure, this would cause spout error.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Resolved] (EAGLE-589) PR 472 incorrect merge of pr 471

[Su Ralph (JIRA)]

 [ 
https://issues.apache.org/jira/browse/EAGLE-589?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Su Ralph resolved EAGLE-589.

Resolution: Fixed

> PR 472 incorrect merge of pr 471
> 
>
> Key: EAGLE-589
> URL: https://issues.apache.org/jira/browse/EAGLE-589
> Project: Eagle
>  Issue Type: Bug
>Affects Versions: v0.5.0
>Reporter: Su Ralph
>Assignee: Su Ralph
> Fix For: v0.5.0
>
>
> An incorrect hurry merge of pr when work in mid night mode..
> Fix to reapply...



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Created] (EAGLE-589) PR 472 incorrect merge of pr 471

[Su Ralph (JIRA)]

Su Ralph created EAGLE-589:
--

 Summary: PR 472 incorrect merge of pr 471
 Key: EAGLE-589
 URL: https://issues.apache.org/jira/browse/EAGLE-589
 Project: Eagle
  Issue Type: Bug
Affects Versions: v0.5.0
Reporter: Su Ralph
Assignee: Su Ralph
 Fix For: v0.5.0


An incorrect hurry merge of pr when work in mid night mode..

Fix to reapply...



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Resolved] (EAGLE-588) AlertEngine : Not to Use commons3 stringutil.isnoneblank

[Su Ralph (JIRA)]

 [ 
https://issues.apache.org/jira/browse/EAGLE-588?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Su Ralph resolved EAGLE-588.

Resolution: Fixed

> AlertEngine : Not to Use commons3 stringutil.isnoneblank
> 
>
> Key: EAGLE-588
> URL: https://issues.apache.org/jira/browse/EAGLE-588
> Project: Eagle
>  Issue Type: Bug
>Affects Versions: v0.5.0
>Reporter: Su Ralph
>Assignee: Su Ralph
> Fix For: v0.5.0
>
>
> commons3 strngutils introduce a isnoneblank, don't use this one, since when 
> integrate with other module, this api is not there, use isNotBlank.
> This is a introduced regression, get fixed.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Created] (EAGLE-588) AlertEngine : Not to Use commons3 stringutil.isnoneblank

[Su Ralph (JIRA)]

Su Ralph created EAGLE-588:
--

 Summary: AlertEngine : Not to Use commons3 stringutil.isnoneblank
 Key: EAGLE-588
 URL: https://issues.apache.org/jira/browse/EAGLE-588
 Project: Eagle
  Issue Type: Bug
Affects Versions: v0.5.0
Reporter: Su Ralph
Assignee: Su Ralph
 Fix For: v0.5.0


commons3 strngutils introduce a isnoneblank, don't use this one, since when 
integrate with other module, this api is not there, use isNotBlank.

This is a introduced regression, get fixed.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Resolved] (EAGLE-587) AlertEngine : simplify state-based dedup to have only deupvalue for given dedup key

[Su Ralph (JIRA)]

 [ 
https://issues.apache.org/jira/browse/EAGLE-587?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Su Ralph resolved EAGLE-587.

Resolution: Fixed

> AlertEngine : simplify state-based dedup to have only deupvalue for given 
> dedup key
> ---
>
> Key: EAGLE-587
> URL: https://issues.apache.org/jira/browse/EAGLE-587
> Project: Eagle
>  Issue Type: Bug
>Affects Versions: v0.5.0
>Reporter: Su Ralph
>Assignee: Su Ralph
> Fix For: v0.5.0
>
>
> For deuplicator, the expected behavior for state-based is that
> for given dedupKey, should have only one record, maintain state transition 
> based on state field, close record when stateClose value is met.
> Currently behavior will generate multiple record when state changed.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Created] (EAGLE-587) AlertEngine : simplify state-based dedup to have only deupvalue for given dedup key

[Su Ralph (JIRA)]

Su Ralph created EAGLE-587:
--

 Summary: AlertEngine : simplify state-based dedup to have only 
deupvalue for given dedup key
 Key: EAGLE-587
 URL: https://issues.apache.org/jira/browse/EAGLE-587
 Project: Eagle
  Issue Type: Bug
Affects Versions: v0.5.0
Reporter: Su Ralph
Assignee: Su Ralph
 Fix For: v0.5.0


For deuplicator, the expected behavior for state-based is that

for given dedupKey, should have only one record, maintain state transition 
based on state field, close record when stateClose value is met.

Currently behavior will generate multiple record when state changed.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Created] (EAGLE-586) AlertEngine: multiple routing of alerts is broken because of event not copied

[Su Ralph (JIRA)]

Su Ralph created EAGLE-586:
--

 Summary: AlertEngine: multiple routing of alerts is broken because 
of event not copied
 Key: EAGLE-586
 URL: https://issues.apache.org/jira/browse/EAGLE-586
 Project: Eagle
  Issue Type: Bug
Affects Versions: v0.5.0
Reporter: Su Ralph
Assignee: Su Ralph
 Fix For: v0.5.0






--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Resolved] (EAGLE-585) AlertEngine: create metadata validation api

[Su Ralph (JIRA)]

 [ 
https://issues.apache.org/jira/browse/EAGLE-585?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Su Ralph resolved EAGLE-585.

Resolution: Fixed

> AlertEngine: create metadata validation api
> ---
>
> Key: EAGLE-585
> URL: https://issues.apache.org/jira/browse/EAGLE-585
> Project: Eagle
>  Issue Type: Bug
>Affects Versions: v0.5.0
>Reporter: Su Ralph
>Assignee: Su Ralph
> Fix For: v0.5.0
>
>
> Currently alert engine metadata are inter connected but without strong 
> validation. User usually confused by coordinator build failure or even when 
> coordinator build succeed but runtime failure. In deployment with traffic 
> runtime exception/error often lost in flood of logs.
> Step1: Provide metadata validation api so user is able to verify the metadata 
> before push data.
> Step2: do more validation during coordinator build.
> This issue to track step1.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Created] (EAGLE-585) AlertEngine: create metadata validation api

[Su Ralph (JIRA)]

Su Ralph created EAGLE-585:
--

 Summary: AlertEngine: create metadata validation api
 Key: EAGLE-585
 URL: https://issues.apache.org/jira/browse/EAGLE-585
 Project: Eagle
  Issue Type: Bug
Affects Versions: v0.5.0
Reporter: Su Ralph
Assignee: Su Ralph
 Fix For: v0.5.0


Currently alert engine metadata are inter connected but without strong 
validation. User usually confused by coordinator build failure or even when 
coordinator build succeed but runtime failure. In deployment with traffic 
runtime exception/error often lost in flood of logs.

Step1: Provide metadata validation api so user is able to verify the metadata 
before push data.

Step2: do more validation during coordinator build.

This issue to track step1.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (EAGLE-580) MongodbMetadata: kafka have dot in topic name cause spec store failed

[Su Ralph (JIRA)]

 [ 
https://issues.apache.org/jira/browse/EAGLE-580?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Su Ralph updated EAGLE-580:
---
Description: 
It's possible that user have DOT in topic names. In this case, mongodb metadata 
store of spoutspec would use the topic name as key in mongodb, which rejected 
by mongodb since it doens't allow dot in key.

Solution is to do covnert of SpoutSpec in mongdb storage, and do appropriate 
transform when read from db.

> MongodbMetadata: kafka have dot in topic name cause spec store failed
> -
>
> Key: EAGLE-580
> URL: https://issues.apache.org/jira/browse/EAGLE-580
> Project: Eagle
>  Issue Type: Bug
>Affects Versions: v0.5.0
>Reporter: Su Ralph
>Assignee: Zeng, Bryant
> Fix For: v0.5.0
>
>
> It's possible that user have DOT in topic names. In this case, mongodb 
> metadata store of spoutspec would use the topic name as key in mongodb, which 
> rejected by mongodb since it doens't allow dot in key.
> Solution is to do covnert of SpoutSpec in mongdb storage, and do appropriate 
> transform when read from db.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Created] (EAGLE-580) MongodbMetadata: kafka have dot in topic name cause spec store failed

[Su Ralph (JIRA)]

Su Ralph created EAGLE-580:
--

 Summary: MongodbMetadata: kafka have dot in topic name cause spec 
store failed
 Key: EAGLE-580
 URL: https://issues.apache.org/jira/browse/EAGLE-580
 Project: Eagle
  Issue Type: Bug
Affects Versions: v0.5.0
Reporter: Su Ralph
Assignee: Zeng, Bryant
 Fix For: v0.5.0






--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Resolved] (EAGLE-573) NPE when apply change of 569

[Su Ralph (JIRA)]

 [ 
https://issues.apache.org/jira/browse/EAGLE-573?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Su Ralph resolved EAGLE-573.

Resolution: Fixed

> NPE when apply change of 569
> 
>
> Key: EAGLE-573
> URL: https://issues.apache.org/jira/browse/EAGLE-573
> Project: Eagle
>  Issue Type: Sub-task
>Affects Versions: v0.5.0
>Reporter: Su Ralph
>Assignee: Su Ralph
> Fix For: v0.5.0
>
>
> There is an NPE when policyId not in publishment plugin mapping. This often 
> happen metadata refresh happens.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Created] (EAGLE-573) NPE when apply change of 569

[Su Ralph (JIRA)]

Su Ralph created EAGLE-573:
--

 Summary: NPE when apply change of 569
 Key: EAGLE-573
 URL: https://issues.apache.org/jira/browse/EAGLE-573
 Project: Eagle
  Issue Type: Sub-task
Affects Versions: v0.5.0
Reporter: Su Ralph
Assignee: Su Ralph
 Fix For: v0.5.0


There is an NPE when policyId not in publishment plugin mapping. This often 
happen metadata refresh happens.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Created] (EAGLE-572) AlertEngine: Metadata API doesn't do validation of metadata

[Su Ralph (JIRA)]

Su Ralph created EAGLE-572:
--

 Summary: AlertEngine: Metadata API doesn't do validation of 
metadata
 Key: EAGLE-572
 URL: https://issues.apache.org/jira/browse/EAGLE-572
 Project: Eagle
  Issue Type: Improvement
Affects Versions: v0.5.0
Reporter: Su Ralph
Assignee: Su Ralph
 Fix For: v0.5.0


Current metadata api only do minor validation of the given metadata. User is 
able to submit stream definition with duplicated field name.

Ideally, Metadata API should be first place to avoid user post invalid metadata.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Created] (EAGLE-571) AlertEngine: Topology should support loopback topic created after policy deployed

[Su Ralph (JIRA)]

Su Ralph created EAGLE-571:
--

 Summary: AlertEngine: Topology should support loopback topic 
created after policy deployed
 Key: EAGLE-571
 URL: https://issues.apache.org/jira/browse/EAGLE-571
 Project: Eagle
  Issue Type: Improvement
Affects Versions: v0.5.0
Reporter: Su Ralph
Assignee: Su Ralph
 Fix For: v0.5.0


when processing havetopic1 -> policy1 -> topic2; and  topic2 -> policy2 -> 
topic2 case, the topology is not able to start. Since the topic2 initialization 
failure (topic2 not create yet) would make topic1 not able to be consumed.  
Thus chicken-egg problem.





--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Resolved] (EAGLE-569) AlertPublishImpl: Concurrency : Inplace change metadata cause concurrent modification issue

[Su Ralph (JIRA)]

 [ 
https://issues.apache.org/jira/browse/EAGLE-569?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Su Ralph resolved EAGLE-569.

Resolution: Fixed

> AlertPublishImpl: Concurrency : Inplace change metadata cause concurrent 
> modification issue
> ---
>
> Key: EAGLE-569
> URL: https://issues.apache.org/jira/browse/EAGLE-569
> Project: Eagle
>  Issue Type: Bug
>Affects Versions: v0.5.0
>Reporter: Su Ralph
>Assignee: Su Ralph
>Priority: Critical
> Fix For: v0.5.0
>
>
> We are try to re-use eagle alert module as part our streaming processing 
> core, during test, we found below error.
> 6700java.util.ConcurrentModificationException at 
> java.util.ArrayList$Itr.checkForComodification(ArrayList.java:901) at 
> java.util.ArrayList$Itr.next(ArrayList.java:851) at 
> org.apache.eagle.alert.engine.publisher.impl.AlertPublisherImpl.notifyAlert(AlertPublisherImpl.java:74)
> It happens to be in AlertPublisherImpl, the metadata change is doing inplace 
> update, which cause the concurrency issue when the bolt is continuing ingest 
> data.
> This exception should cause following data stream not able to be published. 
> Hence be blocker



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Assigned] (EAGLE-569) AlertPublishImpl: Concurrency : Inplace change metadata cause concurrent modification issue

[Su Ralph (JIRA)]

 [ 
https://issues.apache.org/jira/browse/EAGLE-569?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Su Ralph reassigned EAGLE-569:
--

Assignee: Su Ralph  (was: Garrett Li)

> AlertPublishImpl: Concurrency : Inplace change metadata cause concurrent 
> modification issue
> ---
>
> Key: EAGLE-569
> URL: https://issues.apache.org/jira/browse/EAGLE-569
> Project: Eagle
>  Issue Type: Bug
>Affects Versions: v0.5.0
>Reporter: Su Ralph
>Assignee: Su Ralph
>Priority: Critical
> Fix For: v0.5.0
>
>
> We are try to re-use eagle alert module as part our streaming processing 
> core, during test, we found below error.
> 6700java.util.ConcurrentModificationException at 
> java.util.ArrayList$Itr.checkForComodification(ArrayList.java:901) at 
> java.util.ArrayList$Itr.next(ArrayList.java:851) at 
> org.apache.eagle.alert.engine.publisher.impl.AlertPublisherImpl.notifyAlert(AlertPublisherImpl.java:74)
> It happens to be in AlertPublisherImpl, the metadata change is doing inplace 
> update, which cause the concurrency issue when the bolt is continuing ingest 
> data.
> This exception should cause following data stream not able to be published. 
> Hence be blocker



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Created] (EAGLE-570) Refactor: Make sure engine publisher only have general feature.

[Su Ralph (JIRA)]

Su Ralph created EAGLE-570:
--

 Summary: Refactor: Make sure engine publisher only have general 
feature.
 Key: EAGLE-570
 URL: https://issues.apache.org/jira/browse/EAGLE-570
 Project: Eagle
  Issue Type: Sub-task
Affects Versions: v0.5.0
Reporter: Su Ralph
Assignee: Zeng, Bryant
 Fix For: v0.5.0


Eagle-567 intend to provide better formatting of slack channel message.  But 
the implementation introduce too much specific logic that beyond a common 
alerting engine should have.

Need to refactor to move the specific logic to either vendor specific extension 
or have configurable formatting.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (EAGLE-567) Enhance Slack publish format

[Su Ralph (JIRA)]

[ 
https://issues.apache.org/jira/browse/EAGLE-567?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15526488#comment-15526488
 ] 

Su Ralph commented on EAGLE-567:


This make the slack message much more pretty. Thanks.

> Enhance Slack publish format
> 
>
> Key: EAGLE-567
> URL: https://issues.apache.org/jira/browse/EAGLE-567
> Project: Eagle
>  Issue Type: Improvement
>Reporter: Zeng, Bryant
>Assignee: Zeng, Bryant
>
> Currently message publish to Slack is raw String, and this is not very clear 
> for user.
> So need to add format for it for better viewing.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Created] (EAGLE-569) AlertPublishImpl: Concurrency : Inplace change metadata cause concurrent modification issue

[Su Ralph (JIRA)]

Su Ralph created EAGLE-569:
--

 Summary: AlertPublishImpl: Concurrency : Inplace change metadata 
cause concurrent modification issue
 Key: EAGLE-569
 URL: https://issues.apache.org/jira/browse/EAGLE-569
 Project: Eagle
  Issue Type: Bug
Affects Versions: v0.5.0
Reporter: Su Ralph
Assignee: Garrett Li
Priority: Critical
 Fix For: v0.5.0


We are try to re-use eagle alert module as part our streaming processing core, 
during test, we found below error.

6700java.util.ConcurrentModificationException at 
java.util.ArrayList$Itr.checkForComodification(ArrayList.java:901) at 
java.util.ArrayList$Itr.next(ArrayList.java:851) at 
org.apache.eagle.alert.engine.publisher.impl.AlertPublisherImpl.notifyAlert(AlertPublisherImpl.java:74)


It happens to be in AlertPublisherImpl, the metadata change is doing inplace 
update, which cause the concurrency issue when the bolt is continuing ingest 
data.

This exception should cause following data stream not able to be published. 
Hence be blocker



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Resolved] (EAGLE-506) AlertEngine : Make policy definition handler extensible

[Su Ralph (JIRA)]

 [ 
https://issues.apache.org/jira/browse/EAGLE-506?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Su Ralph resolved EAGLE-506.

Resolution: Fixed

> AlertEngine : Make policy definition handler extensible
> ---
>
> Key: EAGLE-506
> URL: https://issues.apache.org/jira/browse/EAGLE-506
> Project: Eagle
>  Issue Type: Improvement
>Affects Versions: v0.5.0
>Reporter: Su Ralph
>Assignee: Su Ralph
> Fix For: v0.5.0
>
>
> {
> “name”: “”,
> “inputStream”: [“”, “” ],
> “definition” : {
> “type”: “Extended”,
> “handlerClass”: “io.sherlock.networkmonitoring.AlertJoinHandler”,
> },
> “outputStreams”: [“”, “”]
> }



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (EAGLE-506) AlertEngine : Make policy definition handler extensible

[Su Ralph (JIRA)]

 [ 
https://issues.apache.org/jira/browse/EAGLE-506?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Su Ralph updated EAGLE-506:
---
Description: 
{
“name”: “”,
“inputStream”: [“”, “” ],
“definition” : {
“type”: “Extended”,
“handlerClass”: “io.sherlock.networkmonitoring.AlertJoinHandler”,
},
“outputStreams”: [“”, “”]
}

> AlertEngine : Make policy definition handler extensible
> ---
>
> Key: EAGLE-506
> URL: https://issues.apache.org/jira/browse/EAGLE-506
> Project: Eagle
>  Issue Type: Improvement
>Affects Versions: v0.5.0
>Reporter: Su Ralph
>Assignee: Su Ralph
> Fix For: v0.5.0
>
>
> {
> “name”: “”,
> “inputStream”: [“”, “” ],
> “definition” : {
> “type”: “Extended”,
> “handlerClass”: “io.sherlock.networkmonitoring.AlertJoinHandler”,
> },
> “outputStreams”: [“”, “”]
> }



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Created] (EAGLE-506) AlertEngine : Make policy definition handler extensible

[Su Ralph (JIRA)]

Su Ralph created EAGLE-506:
--

 Summary: AlertEngine : Make policy definition handler extensible
 Key: EAGLE-506
 URL: https://issues.apache.org/jira/browse/EAGLE-506
 Project: Eagle
  Issue Type: Improvement
Affects Versions: v0.5.0
Reporter: Su Ralph
Assignee: Su Ralph
 Fix For: v0.5.0






--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Resolved] (EAGLE-429) CoordinatorTest.test_01 sometimes fail

[Su Ralph (JIRA)]

 [ 
https://issues.apache.org/jira/browse/EAGLE-429?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Su Ralph resolved EAGLE-429.

Resolution: Fixed

> CoordinatorTest.test_01 sometimes fail
> --
>
> Key: EAGLE-429
> URL: https://issues.apache.org/jira/browse/EAGLE-429
> Project: Eagle
>  Issue Type: Bug
>Affects Versions: v0.5.0
>Reporter: Edward Zhang
>Assignee: Su Ralph
>Priority: Minor
> Fix For: v0.5.0
>
>
> This happens from time to time that test_01 use case fails.
> 2016-08-08 17:47:48,957 INFO [main-SendThread(localhost:2181)] 
> zookeeper.ClientCnxn[852]: Socket connection established to 
> localhost/127.0.0.1:2181, initiating session
> 2016-08-08 17:47:48,960 INFO [main-SendThread(localhost:2181)] 
> zookeeper.ClientCnxn[1235]: Session establishment complete on server 
> localhost/127.0.0.1:2181, sessionid = 0x156629bb321002f, negotiated timeout = 
> 1
> 2016-08-08 17:47:48,960 INFO [main-EventThread] 
> state.ConnectionStateManager[228]: State change: CONNECTED
> 2016-08-08 17:47:49,003 ERROR [main-EventThread] 
> listen.ListenerContainer[96]: Listener 
> (org.apache.eagle.alert.config.ConfigBusConsumer$$Lambda$3/1403704789@1fb700ee)
>  threw an exception
> org.junit.ComparisonFailure: expected: but 
> was:
>   at org.junit.Assert.assertEquals(Assert.java:115)
>   at org.junit.Assert.assertEquals(Assert.java:144)
>   at 
> org.apache.alert.coordinator.CoordinatorTest$2.onNewConfig(CoordinatorTest.java:117)
>   at 
> org.apache.eagle.alert.config.ConfigBusConsumer.lambda$new$0(ConfigBusConsumer.java:46)
>   at 
> org.apache.curator.framework.recipes.cache.NodeCache$4.apply(NodeCache.java:294)
>   at 
> org.apache.curator.framework.recipes.cache.NodeCache$4.apply(NodeCache.java:288)
>   at 
> org.apache.curator.framework.listen.ListenerContainer$1.run(ListenerContainer.java:92)
>   at 
> com.google.common.util.concurrent.MoreExecutors$DirectExecutorService.execute(MoreExecutors.java:310)
>   at 
> org.apache.curator.framework.listen.ListenerContainer.forEach(ListenerContainer.java:84)
>   at 
> org.apache.curator.framework.recipes.cache.NodeCache.setNewData(NodeCache.java:286)
>   at 
> org.apache.curator.framework.recipes.cache.NodeCache.processBackgroundResult(NodeCache.java:253)
>   at 
> org.apache.curator.framework.recipes.cache.NodeCache.access$300(NodeCache.java:54)
>   at 
> org.apache.curator.framework.recipes.cache.NodeCache$3.processResult(NodeCache.java:112)
>   at 
> org.apache.curator.framework.imps.CuratorFrameworkImpl.sendToBackgroundCallback(CuratorFrameworkImpl.java:728)
>   at 
> org.apache.curator.framework.imps.CuratorFrameworkImpl.processBackgroundOperation(CuratorFrameworkImpl.java:505)
>   at 
> org.apache.curator.framework.imps.GetDataBuilderImpl$3.processResult(GetDataBuilderImpl.java:254)
>   at 
> org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:561)
>   at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:498)
> java.lang.AssertionError
>   at org.junit.Assert.fail(Assert.java:86)
>   at org.junit.Assert.assertTrue(Assert.java:41)
>   at org.junit.Assert.assertTrue(Assert.java:52)
>   at 
> org.apache.alert.coordinator.CoordinatorTest.test_01(CoordinatorTest.java:124)
>   at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>   at 
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
>   at 
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
>   at java.lang.reflect.Method.invoke(Method.java:498)
>   at 
> org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:50)
>   at 
> org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
>   at 
> org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:47)
>   at 
> org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
>   at 
> org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:26)
>   at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:325)
>   at 
> org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:78)
>   at 
> org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:57)
>   at org.junit.runners.ParentRunner$3.run(ParentRunner.java:290)
>   at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:71)
>   at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:288)
>   at org.junit.runners.ParentRunner.access$000(ParentRunner.java:58)
>   at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:268)
>  

[jira] [Assigned] (EAGLE-491) AlertPublisherImpl dedup throws exception when dedup column is not configured

[Su Ralph (JIRA)]

 [ 
https://issues.apache.org/jira/browse/EAGLE-491?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Su Ralph reassigned EAGLE-491:
--

Assignee: Zeng, Bryant  (was: Edward Zhang)

> AlertPublisherImpl dedup throws exception when dedup column is not configured
> -
>
> Key: EAGLE-491
> URL: https://issues.apache.org/jira/browse/EAGLE-491
> Project: Eagle
>  Issue Type: Bug
>Affects Versions: v0.5.0
>Reporter: Edward Zhang
>Assignee: Zeng, Bryant
>Priority: Trivial
> Fix For: v0.5.0
>
>
> 2016-08-22 14:51:49,728 WARN [alertBolt1-evaluator_stage1] 
> runner.AlertBolt[127]: Spec Version [spec_version_1471902692329] of AlertBolt 
> is newer than Stream Event Version [null]!
> 2016-08-22 14:51:49,729 ERROR [Thread-45-alertPublishBolt] 
> impl.AlertPublisherImpl[80]: Fail invoking publisher's onAlert, continue 
> java.lang.NullPointerException
>   at 
> org.apache.eagle.alert.engine.publisher.impl.DefaultDeduplicator.dedup(DefaultDeduplicator.java:114)
>   at 
> org.apache.eagle.alert.engine.publisher.impl.AbstractPublishPlugin.dedup(AbstractPublishPlugin.java:68)
>   at 
> org.apache.eagle.alert.engine.publisher.impl.AlertEmailPublisher.onAlert(AlertEmailPublisher.java:67)
>   at 
> org.apache.eagle.alert.engine.publisher.impl.AlertPublisherImpl.notifyAlert(AlertPublisherImpl.java:78)
>   at 
> org.apache.eagle.alert.engine.publisher.impl.AlertPublisherImpl.nextEvent(AlertPublisherImpl.java:65)
>   at 
> org.apache.eagle.alert.engine.runner.AlertPublisherBolt.execute(AlertPublisherBolt.java:68)
>   at 
> backtype.storm.daemon.executor$fn__3441$tuple_action_fn__3443.invoke(executor.clj:633)
>   at 
> backtype.storm.daemon.executor$mk_task_receiver$fn__3364.invoke(executor.clj:401)
>   at 
> backtype.storm.disruptor$clojure_handler$reify__1447.onEvent(disruptor.clj:58)
>   at 
> backtype.storm.utils.DisruptorQueue.consumeBatchToCursor(DisruptorQueue.java:125)
>   at 
> backtype.storm.utils.DisruptorQueue.consumeBatchWhenAvailable(DisruptorQueue.java:99)
>   at 
> backtype.storm.disruptor$consume_batch_when_available.invoke(disruptor.clj:80)
>   at 
> backtype.storm.daemon.executor$fn__3441$fn__3453$fn__3500.invoke(executor.clj:748)
>   at backtype.storm.util$async_loop$fn__464.invoke(util.clj:463)
>   at clojure.lang.AFn.run(AFn.java:24)
>   at java.lang.Thread.run(Thread.java:745)



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Created] (EAGLE-471) AlertBolt introduce un-serializble member

[Su Ralph (JIRA)]

Su Ralph created EAGLE-471:
--

 Summary: AlertBolt introduce un-serializble member 
 Key: EAGLE-471
 URL: https://issues.apache.org/jira/browse/EAGLE-471
 Project: Eagle
  Issue Type: Bug
Affects Versions: v0.5.0
Reporter: Su Ralph
Assignee: Zeng, Bryant
Priority: Blocker


AlerBolt.executors  

2016-08-16 16:13:32,994 INFO [main] runner.UnitTopologyRunner[160]: Set 
topology.message.timeout.secs as 3600
Exception in thread "main" java.lang.RuntimeException: 
java.io.NotSerializableException: java.util.concurrent.ThreadPoolExecutor
   at 
backtype.storm.serialization.DefaultSerializationDelegate.serialize(DefaultSerializationDelegate.java:43)
at backtype.storm.utils.Utils.serialize(Utils.java:85)
at 
backtype.storm.topology.TopologyBuilder.createTopology(TopologyBuilder.java:106)
at 
org.apache.eagle.alert.engine.runner.UnitTopologyRunner.buildTopology(UnitTopologyRunner.java:144)
at 
org.apache.eagle.alert.engine.runner.UnitTopologyRunner.run(UnitTopologyRunner.java:168)
at 
org.apache.eagle.alert.engine.runner.UnitTopologyRunner.run(UnitTopologyRunner.java:193)
at 
org.apache.eagle.alert.engine.UnitTopologyMain.main(UnitTopologyMain.java:65)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at 
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at 
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.intellij.rt.execution.application.AppMain.main(AppMain.java:144)
Caused by: java.io.NotSerializableException: 
java.util.concurrent.ThreadPoolExecutor
at java.io.ObjectOutputStream.writeObject0(ObjectOutputStream.java:1184)
at 
java.io.ObjectOutputStream.defaultWriteFields(ObjectOutputStream.java:1548)
at java.io.ObjectOutputStream.writeSerialData(ObjectOutputStream.java:1509)
at 
java.io.ObjectOutputStream.writeOrdinaryObject(ObjectOutputStream.java:1432)
at java.io.ObjectOutputStream.writeObject0(ObjectOutputStream.java:1178)
at java.io.ObjectOutputStream.writeObject(ObjectOutputStream.java:348)
at 
backtype.storm.serialization.DefaultSerializationDelegate.serialize(DefaultSerializationDelegate.java:39)[4:15]
  
idea里跑UnitTopologyMain



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Created] (EAGLE-464) StateCheck: multiple stage of definition in single policy

[Su Ralph (JIRA)]

Su Ralph created EAGLE-464:
--

 Summary: StateCheck: multiple stage of definition in single policy
 Key: EAGLE-464
 URL: https://issues.apache.org/jira/browse/EAGLE-464
 Project: Eagle
  Issue Type: New Feature
Affects Versions: v0.5.0
Reporter: Su Ralph
Assignee: Su Ralph
 Fix For: v0.5.0


The requirement of alert state and transition comes from two real customer 
needs.
Alert de-duplication
"IMO, eagle should do state checks for all services. Eagle should not alert in 
the first attempt itself. Instead it should change the state to SOFT for 2 
tries and then if it is the same state, change the state to HARD and then send 
the alert." - Aroop
Currently, eagle's alert engine(and also that of UMP) use a simple 
deduplication spec of time based redundancy check(dedupIntervalMin of 
Publishment). This deduplication is not flexible to reflect the need of alerts. 
There are common requests like to hold a alert/policy state (basically a alert 
state is policy state on given partition value, more in latter), and trigger 
alert when the state changed. This state change manner could be 
> Same alert trigger again in M time interval
> N alerts in given M time interval.
NOTE: on here, in this de-duplication mode, there is no required change of the 
policy itself.
Alert policy define on transition
One example of the missingblock policy we met(only alert when missingblock 
number changes). There is more general case with minor difference, given a 
metric (or a field of a given stream), define value range, where each range 
indicate different state. Etc. for perfmon.latency.avg.perpool, define value 
range state as
metric
value range
state
alert trigger
perfmon.latency.avg.perpool.5min3000 - UnlimitedFATAL   always 
(every 5min until FATAL fixed or alert muted explictly)
1000 - 3000 CRITICALon dual transition
100 - 1000  WARNon dual transition
10 - 50 NORMAL  on worse transition
0-10GOODon worse transition
Then the alert should be trigger during the state changed expect for FATAL.




--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Created] (EAGLE-421) JMX Metric APP use new application framework

[Su Ralph (JIRA)]

Su Ralph created EAGLE-421:
--

 Summary: JMX Metric APP use new application framework
 Key: EAGLE-421
 URL: https://issues.apache.org/jira/browse/EAGLE-421
 Project: Eagle
  Issue Type: Bug
Affects Versions: v0.5.0
Reporter: Su Ralph
Assignee: Su Ralph
 Fix For: v0.5.0






--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Resolved] (EAGLE-401) StreamRouterBolt and PublishBolt also have NPE

[Su Ralph (JIRA)]

 [ 
https://issues.apache.org/jira/browse/EAGLE-401?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Su Ralph resolved EAGLE-401.

Resolution: Fixed

> StreamRouterBolt and PublishBolt also have NPE 
> ---
>
> Key: EAGLE-401
> URL: https://issues.apache.org/jira/browse/EAGLE-401
> Project: Eagle
>  Issue Type: Bug
>Affects Versions: v0.5.0
>Reporter: Su Ralph
>Assignee: Su Ralph
> Fix For: v0.5.0
>
>




--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Resolved] (EAGLE-394) Fix NPE bug while metadata inconsistent; Fix unit test

[Su Ralph (JIRA)]

 [ 
https://issues.apache.org/jira/browse/EAGLE-394?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Su Ralph resolved EAGLE-394.

Resolution: Fixed

> Fix NPE bug while metadata inconsistent; Fix unit test
> --
>
> Key: EAGLE-394
> URL: https://issues.apache.org/jira/browse/EAGLE-394
> Project: Eagle
>  Issue Type: Bug
>Affects Versions: v0.5.0
>Reporter: Su Ralph
>Assignee: Su Ralph
> Fix For: v0.5.0
>
>




--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Resolved] (EAGLE-384) Alert Engine BugFix and Improvements

[Su Ralph (JIRA)]

 [ 
https://issues.apache.org/jira/browse/EAGLE-384?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Su Ralph resolved EAGLE-384.

Resolution: Fixed

> Alert Engine BugFix and Improvements
> 
>
> Key: EAGLE-384
> URL: https://issues.apache.org/jira/browse/EAGLE-384
> Project: Eagle
>  Issue Type: Task
>Affects Versions: v0.5.0
>Reporter: Su Ralph
>Assignee: Su Ralph
> Fix For: v0.5.0
>
>




--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (EAGLE-384) Alert Engine BugFix and Improvements

[Su Ralph (JIRA)]

 [ 
https://issues.apache.org/jira/browse/EAGLE-384?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Su Ralph updated EAGLE-384:
---
Fix Version/s: v0.5.0

> Alert Engine BugFix and Improvements
> 
>
> Key: EAGLE-384
> URL: https://issues.apache.org/jira/browse/EAGLE-384
> Project: Eagle
>  Issue Type: Task
>Affects Versions: v0.5.0
>Reporter: Su Ralph
>Assignee: Su Ralph
> Fix For: v0.5.0
>
>




--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Created] (EAGLE-369) Improve eagle alert email template to provide more useful information and user experience

[Su Ralph (JIRA)]

Su Ralph created EAGLE-369:
--

 Summary: Improve eagle alert email template to provide more useful 
information and user experience
 Key: EAGLE-369
 URL: https://issues.apache.org/jira/browse/EAGLE-369
 Project: Eagle
  Issue Type: Improvement
Affects Versions: v0.3.1
Reporter: Su Ralph
Assignee: Su Ralph
 Fix For: v0.5.0


>From feedback:

(1) Time Stamp needs to be human readable. 
(2) Consider capitalize wordings.
(3) What does lowNum and highNum mean here?  Do they mean number of live 
data nodes?  How does this alert come out?  Does that mean the current number 
of live data nodes go above the highNum or below the lowNum?  The lowNum should 
display as “2,884” instead of “2884.0”.  
(4) The Description field should always be there.  If description is not 
available, it should display as “Not provided by Policy Owner”.
(5) Should we consider displaying the name of the policy owner?  That way, 
people know whom to contact if they have question about the policy.
(6) The information in the Actions Required field is too generic and it is 
not actionable.  Can we do something about it?
(7) The text alignment (now always align center) may consider to align 
left.  Use want to make it look more professional.
(8) Text font is not consistent across the message.  Should align to one 
single font.  Consider use Calibri as this is now the eBay official font type.  
(9) Lack of policy definition
The definition is 
from every (e1 = hadoopJmxMetricEventStream[metric == 
\"hadoop.namenode.fsnamesystemstate.numlivedatanodes\" ]) -> e2 = 
hadoopJmxMetricEventStream[metric == e1.metric and host == e1.host and 
(convert(e1.value, \"long\") - 2) >= convert(value, \"long\") ] within 5 min 
select e1.metric, e1.host, e1.value as highNum, e1.timestamp as start, e2.value 
as lowNum, e2.timestamp as end insert into tmp;
 






--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (EAGLE-343) missing blocks alerting

[Su Ralph (JIRA)]

[ 
https://issues.apache.org/jira/browse/EAGLE-343?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15333171#comment-15333171
 ] 

Su Ralph commented on EAGLE-343:


Above i give a example of do the check, using a window (i'm using time windows, 
but length window could be better here as edward mentioned), then any match 
func like max - min should be able to detect the edge.



> missing blocks alerting
> ---
>
> Key: EAGLE-343
> URL: https://issues.apache.org/jira/browse/EAGLE-343
> Project: Eagle
>  Issue Type: New Feature
>Reporter: Jing Ge
>  Labels: JPM
> Fix For: v0.6.0
>
>
> It will normally take a long time to fix missing blocks issue. As a Hadoop 
> cluster administrator, I only want to receive alerting when the the number of 
> missing blocks is changed.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)