[jira] [Updated] (EAGLE-531) Dedup alerts according to state change

[Edward Zhang (JIRA)]

 [ 
https://issues.apache.org/jira/browse/EAGLE-531?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Edward Zhang updated EAGLE-531:
---
Description: This is for stateful and stateless logs  (was: 
https://wiki.vip.corp.ebay.com/pages/viewpage.action?pageId=361488851)

> Dedup alerts according to state change
> --
>
> Key: EAGLE-531
> URL: https://issues.apache.org/jira/browse/EAGLE-531
> Project: Eagle
>  Issue Type: Improvement
>Affects Versions: v0.5.0
>Reporter: Garrett Li
>Assignee: Garrett Li
> Fix For: v0.5.0
>
>
> This is for stateful and stateless logs



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (EAGLE-541) Application policy template framework

[Edward Zhang (JIRA)]

 [ 
https://issues.apache.org/jira/browse/EAGLE-541?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Edward Zhang updated EAGLE-541:
---
Description: 
policy is pretty complex for example it should contain
1) siddhi expression (filter, timed window, etc.)
2) data partition keys
3) dedupe configuration
4) publishment configuration

Eagle should provide a wizard for user to easily onboard polices whatever it is 
a simple filtering policy or complicated timed windows

> Application policy template framework
> -
>
> Key: EAGLE-541
> URL: https://issues.apache.org/jira/browse/EAGLE-541
> Project: Eagle
>  Issue Type: New Feature
>Affects Versions: v0.5.0
>Reporter: Hao Chen
>Assignee: Hao Chen
> Fix For: v0.5.0
>
>
> policy is pretty complex for example it should contain
> 1) siddhi expression (filter, timed window, etc.)
> 2) data partition keys
> 3) dedupe configuration
> 4) publishment configuration
> Eagle should provide a wizard for user to easily onboard polices whatever it 
> is a simple filtering policy or complicated timed windows



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Closed] (EAGLE-187) JDK 1.8 support for Eagle Build

[Edward Zhang (JIRA)]

 [ 
https://issues.apache.org/jira/browse/EAGLE-187?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Edward Zhang closed EAGLE-187.
--
Resolution: Fixed

JDK 1.8 is already supported

> JDK 1.8 support for Eagle Build
> ---
>
> Key: EAGLE-187
> URL: https://issues.apache.org/jira/browse/EAGLE-187
> Project: Eagle
>  Issue Type: Bug
> Environment: Eagle should build with JDK1.8 and support it.
>Reporter: hemanth dendukuri
>Assignee: Edward Zhang
>




--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Closed] (EAGLE-116) StormTopologyCompiler's boltCache looks not working

[Edward Zhang (JIRA)]

 [ 
https://issues.apache.org/jira/browse/EAGLE-116?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Edward Zhang closed EAGLE-116.
--
Resolution: Won't Fix

This feature won't be useful with the new alert engine

> StormTopologyCompiler's boltCache looks not working
> ---
>
> Key: EAGLE-116
> URL: https://issues.apache.org/jira/browse/EAGLE-116
> Project: Eagle
>  Issue Type: Bug
>Reporter: Edward Zhang
>Assignee: Edward Zhang
>Priority: Minor
>
> I saw boltCache is not used anymore, need investigate and clean that if 
> necessary.
>   val boltCache = scala.collection.mutable.Map[StreamProducer[Any], 
> StormBoltWrapper]()
> def createBoltIfAbsent(graph: StreamProducerGraph, producer : 
> StreamProducer[Any]): BaseRichBolt ={
> boltCache.get(producer) match{
>   case Some(bolt) => bolt
>   case None => {
> StormBoltFactory.getBoltWrapper(graph, producer, config)
>   }
> }
>   }



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Closed] (EAGLE-105) add test cases for state recovery in stream join, partitions, event table etc.

[Edward Zhang (JIRA)]

 [ 
https://issues.apache.org/jira/browse/EAGLE-105?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Edward Zhang closed EAGLE-105.
--
Resolution: Won't Fix

This is no longer a feature because new alert engine can do stream join 
automatically

> add test cases for state recovery in stream join, partitions, event table etc.
> --
>
> Key: EAGLE-105
> URL: https://issues.apache.org/jira/browse/EAGLE-105
> Project: Eagle
>  Issue Type: Improvement
>Reporter: Edward Zhang
>Assignee: Edward Zhang
>Priority: Minor
>
> stream join, partitions, and event tables are features Eagle will use, we 
> need test if fault tolerance(state store and recover) can be successful under 
> those cases.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (EAGLE-43) Eagle Job Running Spout should support HTTPS

[Edward Zhang (JIRA)]

[ 
https://issues.apache.org/jira/browse/EAGLE-43?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15501842#comment-15501842
 ] 

Edward Zhang commented on EAGLE-43:
---

Jinhu, does job spout already support HTTPS?

> Eagle Job Running Spout should support HTTPS
> 
>
> Key: EAGLE-43
> URL: https://issues.apache.org/jira/browse/EAGLE-43
> Project: Eagle
>  Issue Type: Improvement
>Affects Versions: v0.3.0
>Reporter: Libin, Sun
>Assignee: wujinhu
>
> Job Running Spout is a data source for hive security monitoring, it get data 
> from RM yarn API and RM web page, currently we only support HTTP, should add 
> HTTPS supprot



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (EAGLE-43) Eagle Job Running Spout should support HTTPS

[Edward Zhang (JIRA)]

 [ 
https://issues.apache.org/jira/browse/EAGLE-43?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Edward Zhang updated EAGLE-43:
--
Assignee: wujinhu  (was: Edward Zhang)

> Eagle Job Running Spout should support HTTPS
> 
>
> Key: EAGLE-43
> URL: https://issues.apache.org/jira/browse/EAGLE-43
> Project: Eagle
>  Issue Type: Improvement
>Affects Versions: v0.3.0
>Reporter: Libin, Sun
>Assignee: wujinhu
>
> Job Running Spout is a data source for hive security monitoring, it get data 
> from RM yarn API and RM web page, currently we only support HTTP, should add 
> HTTPS supprot



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Closed] (EAGLE-39) Policy State Management for robustness - replay events upon node crash

[Edward Zhang (JIRA)]

 [ 
https://issues.apache.org/jira/browse/EAGLE-39?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Edward Zhang closed EAGLE-39.
-
Resolution: Won't Fix

This ticket will be considered in long time. We can revisit this feature when 
it is really necessary for that.

> Policy State Management for robustness - replay events upon node crash
> --
>
> Key: EAGLE-39
> URL: https://issues.apache.org/jira/browse/EAGLE-39
> Project: Eagle
>  Issue Type: New Feature
>Reporter: Edward Zhang
>Assignee: Edward Zhang
>
> Most useful policies are stateful, for example slide-window based policy is 
> evaluated not only for current event but also for a sequential set of events 
> happening before current event. 
> When the node which runs the stateful policy crashes, the policy would be 
> built up in another node by Eagle, but problem is we never see the events 
> which were populated for this policy before node crash.
> The problem sometimes will cause serious issues as it may miss some alerts. 
> Think about a policy which takes one month's data is gone because of the node 
> crash, how do we recover the one-month data for continuous evaluation.
> Unfortunately neither Storm as stream infrastructure nor Siddhi as CEP engine 
> provides state management of intermediate data generated in the program.
> Storm can bring up another bolt when one bolt goes down, but it is not 
> responsible for replay the events which application were previously received. 
> Siddhi does not provide any state recover when CEP runtime goes down.
> Eagle should provide some API for easy state management between monitoring 
> application and CEP engine. That would make a lot of sense when we define 
> complex policies.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Closed] (EAGLE-26) Generic solution for Eagle topology merge

[Edward Zhang (JIRA)]

 [ 
https://issues.apache.org/jira/browse/EAGLE-26?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Edward Zhang closed EAGLE-26.
-

> Generic solution for Eagle topology merge
> -
>
> Key: EAGLE-26
> URL: https://issues.apache.org/jira/browse/EAGLE-26
> Project: Eagle
>  Issue Type: New Feature
>Reporter: Edward Zhang
>Assignee: Edward Zhang
> Fix For: 0.5
>
>
> One Eagle topology can have multiple data sources, but in practice, some 
> policies need to join streams from multiple different topologies on the fly. 
> For example user security logging stream and user audit logging stream are 2 
> different physical topologies, if we want a policy like "user tries logon xxx 
> times and then access sensitive information", then we have develop a new 
> topology to join the 2 data sources. That is not good. We should provide a 
> way of easily merge the streams from the 2 existing topologies in runtime 
> instead of in development time.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (EAGLE-509) make streaming application's sink type to be java bean

[Edward Zhang (JIRA)]

[ 
https://issues.apache.org/jira/browse/EAGLE-509?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15447189#comment-15447189
 ] 

Edward Zhang commented on EAGLE-509:


I am still not convinced by myself for how to do this correctly. We should 
think through some problems before we try to fix

1) is that possible we have multiple output streams from one application?

2) is that possible for one output stream to have multiple sink type e.g. kafka 
and log file?

3) is the binding between output stream and sink type configurable or fixed in 
code?

4) how to configure specific settings for one sink type per output stream

> make streaming application's sink type to be java bean
> --
>
> Key: EAGLE-509
> URL: https://issues.apache.org/jira/browse/EAGLE-509
> Project: Eagle
>  Issue Type: Improvement
>Affects Versions: v0.5.0
>Reporter: Edward Zhang
>Assignee: Edward Zhang
> Fix For: v0.5.0
>
>
> Application's sink is used by alert engine, i.e. alert engine needs know each 
> field type for one sink.
> Sink type now is a Map type, which means developer has to configure each 
> field type in application descriptor file, that is boring and error-prone. If 
> we make sink type to be a java bean, then framework can automatically 
> generate field type to be used for defining alert metadata.
> Also with java bean, we can use annotation to customize more metadata, for 
> example use annotation to specify which field is timestamp column



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (EAGLE-505) oozie auditlog parser dontn`t deal with NullPointerException

[Edward Zhang (JIRA)]

[ 
https://issues.apache.org/jira/browse/EAGLE-505?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15446997#comment-15446997
 ] 

Edward Zhang commented on EAGLE-505:


oozie audit log monitoring needs to be converted to use app framework, are you 
going to do that?

> oozie auditlog parser dontn`t deal with NullPointerException
> 
>
> Key: EAGLE-505
> URL: https://issues.apache.org/jira/browse/EAGLE-505
> Project: Eagle
>  Issue Type: Improvement
>Reporter: Lingang Deng
>Assignee: Lingang Deng
>Priority: Minor
>
> entity = parser.parse(logLine); object entity maybe is null.
> It will cause NullPointerException if there is no processing.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Created] (EAGLE-509) make streaming application's sink type to be java bean

[Edward Zhang (JIRA)]

Edward Zhang created EAGLE-509:
--

 Summary: make streaming application's sink type to be java bean
 Key: EAGLE-509
 URL: https://issues.apache.org/jira/browse/EAGLE-509
 Project: Eagle
  Issue Type: Improvement
Affects Versions: v0.5.0
Reporter: Edward Zhang
Assignee: Edward Zhang
 Fix For: v0.5.0


Application's sink is used by alert engine, i.e. alert engine needs know each 
field type for one sink.

Sink type now is a Map type, which means developer has to configure each field 
type in application descriptor file, that is boring and error-prone. If we make 
sink type to be a java bean, then framework can automatically generate field 
type to be used for defining alert metadata.

Also with java bean, we can use annotation to customize more metadata, for 
example use annotation to specify which field is timestamp column



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (EAGLE-493) Create alert metadata based on application stream sink configuration

[Edward Zhang (JIRA)]

 [ 
https://issues.apache.org/jira/browse/EAGLE-493?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Edward Zhang updated EAGLE-493:
---
Description: 
When I integrate hdfs audit log application with alert engine, it looks streams 
defined in applicationProvider.xml is not useful, because anyway we have to 
define data source, stream, policy and publishment using alert API. It is very 
difficult to convert streams in applicationProvider.xml to metadata used by 
alert engine. 

To achieve better user experience, we should create alert metadata 
automatically when application is installed based on application stream sink.

  was:
When I integrate hdfs audit log application with alert engine, it looks streams 
defined in applicationProvider.xml is not useful, because anyway we have to 
define data source, stream, policy and publishment using alert API. It is very 
difficult to convert streams in applicationProvider.xml to metadata used by 
alert engine. 

So my suggestion is to remove streams definition in applicationProvider.xml, 
and put related metadata scripts into docs in applicationProvider.xml.

By this way, stream definition becomes optional and application developer can 
write script for integrating application with alert engine.


> Create alert metadata based on application stream sink configuration
> 
>
> Key: EAGLE-493
> URL: https://issues.apache.org/jira/browse/EAGLE-493
> Project: Eagle
>  Issue Type: Improvement
>Affects Versions: v0.5.0
>Reporter: Edward Zhang
>Assignee: Edward Zhang
>Priority: Minor
> Fix For: v0.5.0
>
>
> When I integrate hdfs audit log application with alert engine, it looks 
> streams defined in applicationProvider.xml is not useful, because anyway we 
> have to define data source, stream, policy and publishment using alert API. 
> It is very difficult to convert streams in applicationProvider.xml to 
> metadata used by alert engine. 
> To achieve better user experience, we should create alert metadata 
> automatically when application is installed based on application stream sink.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (EAGLE-493) Create alert metadata based on application stream sink configuration

[Edward Zhang (JIRA)]

 [ 
https://issues.apache.org/jira/browse/EAGLE-493?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Edward Zhang updated EAGLE-493:
---
Summary: Create alert metadata based on application stream sink 
configuration  (was: Move stream definition in streams of provider.xml to docs)

> Create alert metadata based on application stream sink configuration
> 
>
> Key: EAGLE-493
> URL: https://issues.apache.org/jira/browse/EAGLE-493
> Project: Eagle
>  Issue Type: Improvement
>Affects Versions: v0.5.0
>Reporter: Edward Zhang
>Assignee: Edward Zhang
>Priority: Minor
> Fix For: v0.5.0
>
>
> When I integrate hdfs audit log application with alert engine, it looks 
> streams defined in applicationProvider.xml is not useful, because anyway we 
> have to define data source, stream, policy and publishment using alert API. 
> It is very difficult to convert streams in applicationProvider.xml to 
> metadata used by alert engine. 
> So my suggestion is to remove streams definition in applicationProvider.xml, 
> and put related metadata scripts into docs in applicationProvider.xml.
> By this way, stream definition becomes optional and application developer can 
> write script for integrating application with alert engine.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Created] (EAGLE-493) Move stream definition in streams of provider.xml to docs

[Edward Zhang (JIRA)]

Edward Zhang created EAGLE-493:
--

 Summary: Move stream definition in streams of provider.xml to docs
 Key: EAGLE-493
 URL: https://issues.apache.org/jira/browse/EAGLE-493
 Project: Eagle
  Issue Type: Improvement
Affects Versions: v0.5.0
Reporter: Edward Zhang
Assignee: Edward Zhang
Priority: Minor
 Fix For: v0.5.0


When I integrate hdfs audit log application with alert engine, it looks streams 
defined in applicationProvider.xml is not useful, because anyway we have to 
define data source, stream, policy and publishment using alert API. It is very 
difficult to convert streams in applicationProvider.xml to metadata used by 
alert engine. 

So my suggestion is to remove streams definition in applicationProvider.xml, 
and put related metadata scripts into docs in applicationProvider.xml.

By this way, stream definition becomes optional and application developer can 
write script for integrating application with alert engine.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Created] (EAGLE-491) AlertPublisherImpl dedup throws exception when dedup column is not configured

[Edward Zhang (JIRA)]

Edward Zhang created EAGLE-491:
--

 Summary: AlertPublisherImpl dedup throws exception when dedup 
column is not configured
 Key: EAGLE-491
 URL: https://issues.apache.org/jira/browse/EAGLE-491
 Project: Eagle
  Issue Type: Bug
Affects Versions: v0.5.0
Reporter: Edward Zhang
Assignee: Edward Zhang
Priority: Trivial
 Fix For: v0.5.0


2016-08-22 14:51:49,728 WARN [alertBolt1-evaluator_stage1] 
runner.AlertBolt[127]: Spec Version [spec_version_1471902692329] of AlertBolt 
is newer than Stream Event Version [null]!
2016-08-22 14:51:49,729 ERROR [Thread-45-alertPublishBolt] 
impl.AlertPublisherImpl[80]: Fail invoking publisher's onAlert, continue 
java.lang.NullPointerException
at 
org.apache.eagle.alert.engine.publisher.impl.DefaultDeduplicator.dedup(DefaultDeduplicator.java:114)
at 
org.apache.eagle.alert.engine.publisher.impl.AbstractPublishPlugin.dedup(AbstractPublishPlugin.java:68)
at 
org.apache.eagle.alert.engine.publisher.impl.AlertEmailPublisher.onAlert(AlertEmailPublisher.java:67)
at 
org.apache.eagle.alert.engine.publisher.impl.AlertPublisherImpl.notifyAlert(AlertPublisherImpl.java:78)
at 
org.apache.eagle.alert.engine.publisher.impl.AlertPublisherImpl.nextEvent(AlertPublisherImpl.java:65)
at 
org.apache.eagle.alert.engine.runner.AlertPublisherBolt.execute(AlertPublisherBolt.java:68)
at 
backtype.storm.daemon.executor$fn__3441$tuple_action_fn__3443.invoke(executor.clj:633)
at 
backtype.storm.daemon.executor$mk_task_receiver$fn__3364.invoke(executor.clj:401)
at 
backtype.storm.disruptor$clojure_handler$reify__1447.onEvent(disruptor.clj:58)
at 
backtype.storm.utils.DisruptorQueue.consumeBatchToCursor(DisruptorQueue.java:125)
at 
backtype.storm.utils.DisruptorQueue.consumeBatchWhenAvailable(DisruptorQueue.java:99)
at 
backtype.storm.disruptor$consume_batch_when_available.invoke(disruptor.clj:80)
at 
backtype.storm.daemon.executor$fn__3441$fn__3453$fn__3500.invoke(executor.clj:748)
at backtype.storm.util$async_loop$fn__464.invoke(util.clj:463)
at clojure.lang.AFn.run(AFn.java:24)
at java.lang.Thread.run(Thread.java:745)



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Created] (EAGLE-490) KafkaStreamSink should not assume fixed field user to do partition

[Edward Zhang (JIRA)]

Edward Zhang created EAGLE-490:
--

 Summary: KafkaStreamSink should not assume fixed field user to do 
partition
 Key: EAGLE-490
 URL: https://issues.apache.org/jira/browse/EAGLE-490
 Project: Eagle
  Issue Type: Improvement
Affects Versions: v0.5.0
Reporter: Edward Zhang
Assignee: Edward Zhang
 Fix For: v0.5.0


KafkaStreamSink assumes user as the field to do partition. This should be 
configurable in application metadata file.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (EAGLE-451) oozie ingestion+alert engine preview

[Edward Zhang (JIRA)]

[ 
https://issues.apache.org/jira/browse/EAGLE-451?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15431226#comment-15431226
 ] 

Edward Zhang commented on EAGLE-451:


yes, they are the same, I have close #448

> oozie ingestion+alert engine preview
> 
>
> Key: EAGLE-451
> URL: https://issues.apache.org/jira/browse/EAGLE-451
> Project: Eagle
>  Issue Type: Improvement
>Reporter: Lingang Deng
>Assignee: Lingang Deng
>Priority: Trivial
>




--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Closed] (EAGLE-448) convert eagle-oozie app to use new app framework

[Edward Zhang (JIRA)]

 [ 
https://issues.apache.org/jira/browse/EAGLE-448?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Edward Zhang closed EAGLE-448.
--
Resolution: Duplicate

same to #451

> convert eagle-oozie app to use new app framework
> 
>
> Key: EAGLE-448
> URL: https://issues.apache.org/jira/browse/EAGLE-448
> Project: Eagle
>  Issue Type: Improvement
>Affects Versions: v0.5.0
>Reporter: Edward Zhang
>Assignee: Edward Zhang
> Fix For: v0.5.0
>
>
> eagle-oozie app should use new app framework to manage its lifecycle



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (EAGLE-487) use guice module better in app framework

[Edward Zhang (JIRA)]

 [ 
https://issues.apache.org/jira/browse/EAGLE-487?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Edward Zhang updated EAGLE-487:
---
Description: 
There are multiple issues with using guice module in app framework.

1. ApplicationProviderServiceImpl is instantiated twice

2. the interface between application and eagle server is not clear. It is not 
correct to only let application to register submodules,, instead it should 
includes some state inherited from eagle server, for example metadata storage 
type.

3. not necessary to use module scope, it is pretty over-engineering when 
application only want to register some module.

4. MemoryMetadataStore should use singleton

  was:
There are multiple issues with using guice module in app framework.

1. ApplicationProviderServiceImpl is instantiated twice

2. the interface between application and eagle server is not clear. It is not 
correct to only let application to register submodules,, instead it should 
includes some state inherited from eagle server, for example metadata storage 
type.

3. not necessary to use module scope, it is pretty over-engineering when 
application only want to register some module.


> use guice module better in app framework
> 
>
> Key: EAGLE-487
> URL: https://issues.apache.org/jira/browse/EAGLE-487
> Project: Eagle
>  Issue Type: Improvement
>Affects Versions: v0.5.0
>Reporter: Edward Zhang
>Assignee: Edward Zhang
> Fix For: v0.5.0
>
>
> There are multiple issues with using guice module in app framework.
> 1. ApplicationProviderServiceImpl is instantiated twice
> 2. the interface between application and eagle server is not clear. It is not 
> correct to only let application to register submodules,, instead it should 
> includes some state inherited from eagle server, for example metadata storage 
> type.
> 3. not necessary to use module scope, it is pretty over-engineering when 
> application only want to register some module.
> 4. MemoryMetadataStore should use singleton



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (EAGLE-487) use guice module better in app framework

[Edward Zhang (JIRA)]

[ 
https://issues.apache.org/jira/browse/EAGLE-487?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15429520#comment-15429520
 ] 

Edward Zhang commented on EAGLE-487:


Proposed solution:
We have several level of modules:
1. eagle server module, for example config 
2. alert service module, for example alertMetadata service
3. app service module, for example appProviderService, appEntityService etc.
4. app specific module, for example security monitoring service module



> use guice module better in app framework
> 
>
> Key: EAGLE-487
> URL: https://issues.apache.org/jira/browse/EAGLE-487
> Project: Eagle
>  Issue Type: Improvement
>Affects Versions: v0.5.0
>Reporter: Edward Zhang
>Assignee: Edward Zhang
> Fix For: v0.5.0
>
>
> There are multiple issues with using guice module in app framework.
> 1. ApplicationProviderServiceImpl is instantiated twice
> 2. the interface between application and eagle server is not clear. It is not 
> correct to only let application to register submodules,, instead it should 
> includes some state inherited from eagle server, for example metadata storage 
> type.
> 3. not necessary to use module scope, it is pretty over-engineering when 
> application only want to register some module.
> 4. MemoryMetadataStore should use singleton



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (EAGLE-487) use guice module better in app framework

[Edward Zhang (JIRA)]

[ 
https://issues.apache.org/jira/browse/EAGLE-487?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15428850#comment-15428850
 ] 

Edward Zhang commented on EAGLE-487:


different createInjector will separate different implementations for one 
interface. This would cause multiple instances.

> use guice module better in app framework
> 
>
> Key: EAGLE-487
> URL: https://issues.apache.org/jira/browse/EAGLE-487
> Project: Eagle
>  Issue Type: Improvement
>Affects Versions: v0.5.0
>Reporter: Edward Zhang
>Assignee: Edward Zhang
> Fix For: v0.5.0
>
>
> There are multiple issues with using guice module in app framework.
> 1. ApplicationProviderServiceImpl is instantiated twice
> 2. the interface between application and eagle server is not clear. It is not 
> correct to only let application to register submodules,, instead it should 
> includes some state inherited from eagle server, for example metadata storage 
> type.
> 3. not necessary to use module scope, it is pretty over-engineering when 
> application only want to register some module.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Created] (EAGLE-487) use guice module better in app framework

[Edward Zhang (JIRA)]

Edward Zhang created EAGLE-487:
--

 Summary: use guice module better in app framework
 Key: EAGLE-487
 URL: https://issues.apache.org/jira/browse/EAGLE-487
 Project: Eagle
  Issue Type: Improvement
Affects Versions: v0.5.0
Reporter: Edward Zhang
Assignee: Edward Zhang
 Fix For: v0.5.0


There are multiple issues with using guice module in app framework.

1. ApplicationProviderServiceImpl is instantiated twice

2. the interface between application and eagle server is not clear. It is not 
correct to only let application to register submodules,, instead it should 
includes some state inherited from eagle server, for example metadata storage 
type.

3. not necessary to use module scope, it is pretty over-engineering when 
application only want to register some module.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Created] (EAGLE-481) Make alert engine topology to be one application so to be managed by Eagle API

[Edward Zhang (JIRA)]

Edward Zhang created EAGLE-481:
--

 Summary: Make alert engine topology to be one application so to be 
managed by Eagle API
 Key: EAGLE-481
 URL: https://issues.apache.org/jira/browse/EAGLE-481
 Project: Eagle
  Issue Type: Improvement
Affects Versions: v0.5.0
Reporter: Edward Zhang
Assignee: Edward Zhang
 Fix For: v0.5.0


Eagle alert engine as of 0.5 is multiple separate storm topologies. Instead of 
running alert engine manually, we can leverage Eagle 0.5 application API to 
manage alert engine. 

Options:
1. Wrapper alert engine topology to be one Eagle application

2. Define a new service API to directly start/stop alert engine



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Created] (EAGLE-480) Make Hadoop jmx monitoring working in Eagle 0.5

[Edward Zhang (JIRA)]

Edward Zhang created EAGLE-480:
--

 Summary: Make Hadoop jmx monitoring working in Eagle 0.5
 Key: EAGLE-480
 URL: https://issues.apache.org/jira/browse/EAGLE-480
 Project: Eagle
  Issue Type: Improvement
Affects Versions: v0.5.0
Reporter: Edward Zhang
Assignee: Edward Zhang
 Fix For: v0.5.0


Hadoop jmx metrics data will go to alert engine directly, we need verify how to 
create data source, stream and policies in Eagle 0.5



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Created] (EAGLE-478) Eagle README in Eagle 0.5 version

[Edward Zhang (JIRA)]

Edward Zhang created EAGLE-478:
--

 Summary: Eagle README in Eagle 0.5 version
 Key: EAGLE-478
 URL: https://issues.apache.org/jira/browse/EAGLE-478
 Project: Eagle
  Issue Type: Improvement
Affects Versions: v0.5.0
Reporter: Edward Zhang
Assignee: Edward Zhang
Priority: Minor
 Fix For: v0.5.0


Readme for starting Eagle 0.5 services:
1) start eagle-server
2) start one application
3) start alert engine topology



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Created] (EAGLE-477) eagle-data-process project clean up to contain only common processing classes

[Edward Zhang (JIRA)]

Edward Zhang created EAGLE-477:
--

 Summary: eagle-data-process project clean up to contain only 
common processing classes
 Key: EAGLE-477
 URL: https://issues.apache.org/jira/browse/EAGLE-477
 Project: Eagle
  Issue Type: Improvement
Affects Versions: v0.5.0
Reporter: Edward Zhang
Assignee: Edward Zhang
Priority: Minor
 Fix For: v0.5.0


eagle-data-process was used for orchestrating application logic, but now it is 
not useful any more, we only need keep some common classes for example spout 
factory and some custom partitioning code



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (EAGLE-474) some improvements of app framework: appId naming, jarFile field, InstallOperation etc.

[Edward Zhang (JIRA)]

[ 
https://issues.apache.org/jira/browse/EAGLE-474?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15425625#comment-15425625
 ] 

Edward Zhang commented on EAGLE-474:


appId in ApplicationEntity should be renamed to appType. It is not necessary to 
let appId to contain site.

Application instance should be identified by app type and uuid (uuid can 
contain site)

> some improvements of app framework: appId naming, jarFile field, 
> InstallOperation etc.
> --
>
> Key: EAGLE-474
> URL: https://issues.apache.org/jira/browse/EAGLE-474
> Project: Eagle
>  Issue Type: Improvement
>Affects Versions: v0.5.0
>Reporter: Edward Zhang
>Assignee: Edward Zhang
>Priority: Minor
> Fix For: v0.5.0
>
>
> App framework has some bugs and also needs some improvement
> 1. appId in ApplicationEntity should not contain "[,]" chars, which will make 
> storm UI not show the topology details
> 2. jarPath is not necessary to be part of ApplicationDesc or metadata xml file
> 3. InstallOperation should contain jarPath which should be in the same level 
> of appType etc.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Created] (EAGLE-469) make dataSource configuration extremely simple

[Edward Zhang (JIRA)]

Edward Zhang created EAGLE-469:
--

 Summary: make dataSource configuration extremely simple
 Key: EAGLE-469
 URL: https://issues.apache.org/jira/browse/EAGLE-469
 Project: Eagle
  Issue Type: Improvement
Affects Versions: v0.5.0
Reporter: Edward Zhang
Assignee: Edward Zhang
Priority: Minor
 Fix For: v0.5.0


Today dataSource configuration is very cumbersome, we actually can reduce 
significantly the size of config entries as most of configurations use the 
default values.
"dataSourceConfig": {
"topic" : "sandbox_hbase_security_log",
"zkConnection" : "sandbox.hortonworks.com:2181",
"zkConnectionTimeoutMS" : 15000,
"consumerGroupId" : "EagleConsumer",
"fetchSize" : 1048586,
"deserializerClass" : 
"org.apache.eagle.security.hbase.HbaseAuditLogKafkaDeserializer",
"transactionZKServers" : "sandbox.hortonworks.com",
"transactionZKPort" : 2181,
"transactionZKRoot" : "/consumers",
"consumerGroupId" : "eagle.hbasesecurity.consumer",
"transactionStateUpdateMS" : 2000
  }

The new data source config would be like:
"dataSourceConfig": {
"topic" : "hbase_audit_log",
"zkConnection" : "server.eagle.apache.org:2181",
"txZkServers" : "server.eagle.apache.org:2181",
"schemeCls" : "storm.kafka.StringScheme"
  }



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Created] (EAGLE-468) clean up eagle-application-service to be replaced by new app framework

[Edward Zhang (JIRA)]

Edward Zhang created EAGLE-468:
--

 Summary: clean up eagle-application-service to be replaced by new 
app framework
 Key: EAGLE-468
 URL: https://issues.apache.org/jira/browse/EAGLE-468
 Project: Eagle
  Issue Type: Improvement
Affects Versions: v0.5.0
Reporter: Edward Zhang
Assignee: Edward Zhang
Priority: Minor
 Fix For: v0.5.0


application service provides web resources for managing storm topology 
remotely, this is replaced by new app framework.
But in new app framework, we still need investigate how to monitor application 
status.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Created] (EAGLE-463) One App type may be run as multiple instances in one site

[Edward Zhang (JIRA)]

Edward Zhang created EAGLE-463:
--

 Summary: One App type may be run as multiple instances in one site
 Key: EAGLE-463
 URL: https://issues.apache.org/jira/browse/EAGLE-463
 Project: Eagle
  Issue Type: Improvement
Affects Versions: v0.5.0
Reporter: Edward Zhang
Assignee: Edward Zhang
 Fix For: v0.5.0


Today in one site, one app type can be run only as one instance. In some 
scenarios, one application code can be run with multiple instances, for example 
to deal with different topics.

One typical example is JMX metric application can be for namenode, datanode , 
resource manager etc. They are all the same type of applications, i.e. code 
logic with different configurations for example topic for data sources



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (EAGLE-432) Application status monitoring

[Edward Zhang (JIRA)]

 [ 
https://issues.apache.org/jira/browse/EAGLE-432?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Edward Zhang updated EAGLE-432:
---
Assignee: DanielZhou  (was: Edward Zhang)

> Application status monitoring
> -
>
> Key: EAGLE-432
> URL: https://issues.apache.org/jira/browse/EAGLE-432
> Project: Eagle
>  Issue Type: Improvement
>Affects Versions: v0.5.0
>Reporter: Edward Zhang
>Assignee: DanielZhou
> Fix For: v0.5.0
>
>
> App framework needs to probe application status periodically or on demand.
> ApplicationEntity contains status field, and this field's value should come 
> from physical storm applications.
> We can start background threads to check application status or on-demand.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (EAGLE-447) convert eagle-maprfs app to use new app framework

[Edward Zhang (JIRA)]

 [ 
https://issues.apache.org/jira/browse/EAGLE-447?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Edward Zhang updated EAGLE-447:
---
Assignee: DanielZhou  (was: Edward Zhang)

> convert eagle-maprfs app to use new app framework
> -
>
> Key: EAGLE-447
> URL: https://issues.apache.org/jira/browse/EAGLE-447
> Project: Eagle
>  Issue Type: Improvement
>Affects Versions: v0.5.0
>Reporter: Edward Zhang
>Assignee: DanielZhou
> Fix For: v0.5.0
>
>
>  eagle-maprfs app should use new app framework to manage its lifecycle



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Created] (EAGLE-449) clean inner process alert engine code

[Edward Zhang (JIRA)]

Edward Zhang created EAGLE-449:
--

 Summary: clean inner process alert engine code
 Key: EAGLE-449
 URL: https://issues.apache.org/jira/browse/EAGLE-449
 Project: Eagle
  Issue Type: Improvement
Affects Versions: v0.5.0
Reporter: Edward Zhang
Assignee: Edward Zhang
 Fix For: v0.5.0


alert engine now is separated into a standalone storm topology, it is not 
necessary to keep inner process alert engine



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Created] (EAGLE-448) convert eagle-oozie app to use new app framework

[Edward Zhang (JIRA)]

Edward Zhang created EAGLE-448:
--

 Summary: convert eagle-oozie app to use new app framework
 Key: EAGLE-448
 URL: https://issues.apache.org/jira/browse/EAGLE-448
 Project: Eagle
  Issue Type: Improvement
Affects Versions: v0.5.0
Reporter: Edward Zhang
Assignee: Edward Zhang
 Fix For: v0.5.0


eagle-oozie app should use new app framework to manage its lifecycle



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Created] (EAGLE-447) convert eagle-maprfs app to use new app framework

[Edward Zhang (JIRA)]

Edward Zhang created EAGLE-447:
--

 Summary: convert eagle-maprfs app to use new app framework
 Key: EAGLE-447
 URL: https://issues.apache.org/jira/browse/EAGLE-447
 Project: Eagle
  Issue Type: Improvement
Affects Versions: v0.5.0
Reporter: Edward Zhang
Assignee: Edward Zhang
 Fix For: v0.5.0


 eagle-maprfs app should use new app framework to manage its lifecycle



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Created] (EAGLE-446) convert eagle-hive app to use new app framework

[Edward Zhang (JIRA)]

Edward Zhang created EAGLE-446:
--

 Summary: convert eagle-hive app to use new app framework
 Key: EAGLE-446
 URL: https://issues.apache.org/jira/browse/EAGLE-446
 Project: Eagle
  Issue Type: Improvement
Affects Versions: v0.5.0
Reporter: Edward Zhang
Assignee: Edward Zhang
 Fix For: v0.5.0


eagle-hive should use new application framework



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Created] (EAGLE-444) convert eagle-gc app to use new app framework

[Edward Zhang (JIRA)]

Edward Zhang created EAGLE-444:
--

 Summary: convert eagle-gc app to use new app framework
 Key: EAGLE-444
 URL: https://issues.apache.org/jira/browse/EAGLE-444
 Project: Eagle
  Issue Type: Improvement
Affects Versions: v0.5.0
Reporter: Edward Zhang
Assignee: Edward Zhang
 Fix For: v0.5.0






--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Closed] (EAGLE-426) Load app web classes and app topology classes using configured lib dir

[Edward Zhang (JIRA)]

 [ 
https://issues.apache.org/jira/browse/EAGLE-426?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Edward Zhang closed EAGLE-426.
--

> Load app web classes and app topology classes using configured lib dir
> --
>
> Key: EAGLE-426
> URL: https://issues.apache.org/jira/browse/EAGLE-426
> Project: Eagle
>  Issue Type: Improvement
>Affects Versions: v0.5.0
>Reporter: Edward Zhang
>Assignee: Edward Zhang
>Priority: Minor
> Fix For: v0.5.0
>
>
> App framework has the following issues after remove hard dependency from 
> eagle server to individual applications
> 1. load web app class through jersey package scanner by providing correct 
> context classloader
> 2. load web topology class through correct classloader
> Basically we use SPI and application.provider.dir to specify libraries and 
> load them



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Resolved] (EAGLE-420) HBase audit log monitoring using new app framework

[Edward Zhang (JIRA)]

 [ 
https://issues.apache.org/jira/browse/EAGLE-420?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Edward Zhang resolved EAGLE-420.

Resolution: Fixed

> HBase audit log monitoring using new app framework
> --
>
> Key: EAGLE-420
> URL: https://issues.apache.org/jira/browse/EAGLE-420
> Project: Eagle
>  Issue Type: Improvement
>Affects Versions: v0.5.0
>Reporter: Edward Zhang
>Assignee: Edward Zhang
> Fix For: v0.5.0
>
>
> This ticket is used for converting HBase audit log monitoring to use new 
> monitoring application framework
> New monitoring application supports the following 2 features
> 1) in development time,  developer can write topology and test topology 
> without knowing this topology will be managed by Eagle service API
> 2) in runtime, operator can manage topology lifecycle using API



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Closed] (EAGLE-420) HBase audit log monitoring using new app framework

[Edward Zhang (JIRA)]

 [ 
https://issues.apache.org/jira/browse/EAGLE-420?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Edward Zhang closed EAGLE-420.
--

> HBase audit log monitoring using new app framework
> --
>
> Key: EAGLE-420
> URL: https://issues.apache.org/jira/browse/EAGLE-420
> Project: Eagle
>  Issue Type: Improvement
>Affects Versions: v0.5.0
>Reporter: Edward Zhang
>Assignee: Edward Zhang
> Fix For: v0.5.0
>
>
> This ticket is used for converting HBase audit log monitoring to use new 
> monitoring application framework
> New monitoring application supports the following 2 features
> 1) in development time,  developer can write topology and test topology 
> without knowing this topology will be managed by Eagle service API
> 2) in runtime, operator can manage topology lifecycle using API



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Resolved] (EAGLE-426) Load app web classes and app topology classes using configured lib dir

[Edward Zhang (JIRA)]

 [ 
https://issues.apache.org/jira/browse/EAGLE-426?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Edward Zhang resolved EAGLE-426.

Resolution: Fixed

> Load app web classes and app topology classes using configured lib dir
> --
>
> Key: EAGLE-426
> URL: https://issues.apache.org/jira/browse/EAGLE-426
> Project: Eagle
>  Issue Type: Improvement
>Affects Versions: v0.5.0
>Reporter: Edward Zhang
>Assignee: Edward Zhang
>Priority: Minor
> Fix For: v0.5.0
>
>
> App framework has the following issues after remove hard dependency from 
> eagle server to individual applications
> 1. load web app class through jersey package scanner by providing correct 
> context classloader
> 2. load web topology class through correct classloader
> Basically we use SPI and application.provider.dir to specify libraries and 
> load them



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Resolved] (EAGLE-430) explore new packaging method for eagle using new app framework

[Edward Zhang (JIRA)]

 [ 
https://issues.apache.org/jira/browse/EAGLE-430?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Edward Zhang resolved EAGLE-430.

Resolution: Fixed

> explore new packaging method for eagle using new app framework
> --
>
> Key: EAGLE-430
> URL: https://issues.apache.org/jira/browse/EAGLE-430
> Project: Eagle
>  Issue Type: Improvement
>Affects Versions: v0.5.0
>Reporter: Edward Zhang
>Assignee: Edward Zhang
> Fix For: v0.5.0
>
>
> New app framework will manage application jars in central place, while 
> application jars include topology jars and web jars. 
> We need figure out Eagle 0.5 binary layout.
> The goal is with only Eagle 0.5 server we can start any topology remotely.
> /eagle-server
> /conf  (application.conf)
> /bin(add class path)
> /lib
>/eagle-server.jar
>/apps
> app1.jar
> app1-web.jar
> app2.jar
> /web
> index.html
> /js



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Closed] (EAGLE-430) explore new packaging method for eagle using new app framework

[Edward Zhang (JIRA)]

 [ 
https://issues.apache.org/jira/browse/EAGLE-430?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Edward Zhang closed EAGLE-430.
--

> explore new packaging method for eagle using new app framework
> --
>
> Key: EAGLE-430
> URL: https://issues.apache.org/jira/browse/EAGLE-430
> Project: Eagle
>  Issue Type: Improvement
>Affects Versions: v0.5.0
>Reporter: Edward Zhang
>Assignee: Edward Zhang
> Fix For: v0.5.0
>
>
> New app framework will manage application jars in central place, while 
> application jars include topology jars and web jars. 
> We need figure out Eagle 0.5 binary layout.
> The goal is with only Eagle 0.5 server we can start any topology remotely.
> /eagle-server
> /conf  (application.conf)
> /bin(add class path)
> /lib
>/eagle-server.jar
>/apps
> app1.jar
> app1-web.jar
> app2.jar
> /web
> index.html
> /js



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Closed] (EAGLE-437) create workable fat jar for starting application remotely

[Edward Zhang (JIRA)]

 [ 
https://issues.apache.org/jira/browse/EAGLE-437?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Edward Zhang closed EAGLE-437.
--

> create workable fat jar for starting application remotely
> -
>
> Key: EAGLE-437
> URL: https://issues.apache.org/jira/browse/EAGLE-437
> Project: Eagle
>  Issue Type: Improvement
>Affects Versions: v0.5.0
>Reporter: Edward Zhang
>Assignee: Edward Zhang
> Fix For: v0.5.0
>
>
> Explore ways to package fat jar for starting applications
> 1) package all topology jars together into one single fat jar. Need fix
>a. metadata.xml should be differentiated by different directory, for 
> example metadata.xml is under directory of 
> /META-INF/apps//metadata.xml
>b. SPI configuration file should be removed and 
> ApplicationProviderSPILoader should be redesign to get all providers
> 2) package fat jar on demand.
> No far jar after packaging.
> Need remember all dependent jars and write code to package fat jar on 
> demand



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Resolved] (EAGLE-437) create workable fat jar for starting application remotely

[Edward Zhang (JIRA)]

 [ 
https://issues.apache.org/jira/browse/EAGLE-437?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Edward Zhang resolved EAGLE-437.

Resolution: Fixed

> create workable fat jar for starting application remotely
> -
>
> Key: EAGLE-437
> URL: https://issues.apache.org/jira/browse/EAGLE-437
> Project: Eagle
>  Issue Type: Improvement
>Affects Versions: v0.5.0
>Reporter: Edward Zhang
>Assignee: Edward Zhang
> Fix For: v0.5.0
>
>
> Explore ways to package fat jar for starting applications
> 1) package all topology jars together into one single fat jar. Need fix
>a. metadata.xml should be differentiated by different directory, for 
> example metadata.xml is under directory of 
> /META-INF/apps//metadata.xml
>b. SPI configuration file should be removed and 
> ApplicationProviderSPILoader should be redesign to get all providers
> 2) package fat jar on demand.
> No far jar after packaging.
> Need remember all dependent jars and write code to package fat jar on 
> demand



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Closed] (EAGLE-428) Hbase monitoring's metadata browse API in new app framework

[Edward Zhang (JIRA)]

 [ 
https://issues.apache.org/jira/browse/EAGLE-428?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Edward Zhang closed EAGLE-428.
--

> Hbase monitoring's metadata browse API in new app framework
> ---
>
> Key: EAGLE-428
> URL: https://issues.apache.org/jira/browse/EAGLE-428
> Project: Eagle
>  Issue Type: Improvement
>Affects Versions: v0.5.0
>Reporter: Edward Zhang
>Assignee: Edward Zhang
> Fix For: v0.5.0
>
>
> Hbase monitoring's metadata browse API is used for retrieve hbase's table, 
> column etc, and check against sensitivity type.
> We should use the configuration from ApplicationEntity to manage metadata 
> query



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Resolved] (EAGLE-428) Hbase monitoring's metadata browse API in new app framework

[Edward Zhang (JIRA)]

 [ 
https://issues.apache.org/jira/browse/EAGLE-428?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Edward Zhang resolved EAGLE-428.

Resolution: Fixed

> Hbase monitoring's metadata browse API in new app framework
> ---
>
> Key: EAGLE-428
> URL: https://issues.apache.org/jira/browse/EAGLE-428
> Project: Eagle
>  Issue Type: Improvement
>Affects Versions: v0.5.0
>Reporter: Edward Zhang
>Assignee: Edward Zhang
> Fix For: v0.5.0
>
>
> Hbase monitoring's metadata browse API is used for retrieve hbase's table, 
> column etc, and check against sensitivity type.
> We should use the configuration from ApplicationEntity to manage metadata 
> query



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (EAGLE-438) Multiple Inputs for Hadoop JMX Collector Python Script

[Edward Zhang (JIRA)]

[ 
https://issues.apache.org/jira/browse/EAGLE-438?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15414305#comment-15414305
 ] 

Edward Zhang commented on EAGLE-438:


[~senthilec566] I think we already have resource manager related script?
[~petkim] are you going to have all components share the same script? but they 
are using different topic right?

> Multiple Inputs for Hadoop JMX Collector Python Script
> --
>
> Key: EAGLE-438
> URL: https://issues.apache.org/jira/browse/EAGLE-438
> Project: Eagle
>  Issue Type: New Feature
>Reporter: Peter Kim
>Priority: Trivial
>  Labels: features
>
> It would be very useful for the hadoop jmx collector to be able to collect 
> jmx metrics for multiple components at once. So here, I wish to extend the 
> default python script that collects Hadoop JMX Metrics to support multiple 
> inputs as oppose to only one. So, with this, one can collect all jmx metrics 
> for multiple components such as namenode, datanode, resource manager, 
> hmaster, etc. all at once. 



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Created] (EAGLE-437) create workable fat jar for starting application remotely

[Edward Zhang (JIRA)]

Edward Zhang created EAGLE-437:
--

 Summary: create workable fat jar for starting application remotely
 Key: EAGLE-437
 URL: https://issues.apache.org/jira/browse/EAGLE-437
 Project: Eagle
  Issue Type: Improvement
Affects Versions: v0.5.0
Reporter: Edward Zhang
Assignee: Edward Zhang
 Fix For: v0.5.0


Explore ways to package fat jar for starting applications
1) package all topology jars together into one single fat jar. Need fix
   a. metadata.xml should be differentiated by different directory, for example 
metadata.xml is under directory of /META-INF/apps//metadata.xml
   b. SPI configuration file should be removed and ApplicationProviderSPILoader 
should be redesign to get all providers

2) package fat jar on demand.
No far jar after packaging.
Need remember all dependent jars and write code to package fat jar on demand



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (EAGLE-430) explore new packaging method for eagle using new app framework

[Edward Zhang (JIRA)]

[ 
https://issues.apache.org/jira/browse/EAGLE-430?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15414218#comment-15414218
 ] 

Edward Zhang commented on EAGLE-430:


We should think about fat jar packaging process.
When we package, metadata.xml and SPI file are overridden with each other.
One solution is to build fat jar on-demand when we start an application in 
remote.
When we do package on-demand, we should know all names of dependent jars and 
try to package them together.
This is not scope for this ticket, will have another ticket for this problem

> explore new packaging method for eagle using new app framework
> --
>
> Key: EAGLE-430
> URL: https://issues.apache.org/jira/browse/EAGLE-430
> Project: Eagle
>  Issue Type: Improvement
>Affects Versions: v0.5.0
>Reporter: Edward Zhang
>Assignee: Edward Zhang
> Fix For: v0.5.0
>
>
> New app framework will manage application jars in central place, while 
> application jars include topology jars and web jars. 
> We need figure out Eagle 0.5 binary layout.
> The goal is with only Eagle 0.5 server we can start any topology remotely.
> /eagle-server
> /conf  (application.conf)
> /bin(add class path)
> /lib
>/eagle-server.jar
>/apps
> app1.jar
> app1-web.jar
> app2.jar
> /web
> index.html
> /js



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Created] (EAGLE-432) Application status monitoring

[Edward Zhang (JIRA)]

Edward Zhang created EAGLE-432:
--

 Summary: Application status monitoring
 Key: EAGLE-432
 URL: https://issues.apache.org/jira/browse/EAGLE-432
 Project: Eagle
  Issue Type: Improvement
Affects Versions: v0.5.0
Reporter: Edward Zhang
Assignee: Edward Zhang
 Fix For: v0.5.0


App framework needs to probe application status periodically or on demand.

ApplicationEntity contains status field, and this field's value should come 
from physical storm applications.

We can start background threads to check application status or on-demand.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Created] (EAGLE-431) MetadataDaoFactory should use config from eagle server

[Edward Zhang (JIRA)]

Edward Zhang created EAGLE-431:
--

 Summary: MetadataDaoFactory should use config from eagle server
 Key: EAGLE-431
 URL: https://issues.apache.org/jira/browse/EAGLE-431
 Project: Eagle
  Issue Type: Improvement
Affects Versions: v0.5.0
Reporter: Edward Zhang
Assignee: Edward Zhang
 Fix For: v0.5.0


org.apache.eagle.alert.metadata.impl.MetadataDaoFactory uses  
ConfigFactory.load() to get configuration. We should use config object from 
eagle server.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Created] (EAGLE-430) explore new packaging method for eagle using new app framework

[Edward Zhang (JIRA)]

Edward Zhang created EAGLE-430:
--

 Summary: explore new packaging method for eagle using new app 
framework
 Key: EAGLE-430
 URL: https://issues.apache.org/jira/browse/EAGLE-430
 Project: Eagle
  Issue Type: Improvement
Affects Versions: v0.5.0
Reporter: Edward Zhang
Assignee: Edward Zhang
 Fix For: v0.5.0


New app framework will manage application jars in central place, while 
application jars include topology jars and web jars. 
We need figure out Eagle 0.5 binary layout.
The goal is with only Eagle 0.5 server we can start any topology remotely.

/eagle-server
/conf  (application.conf)
/bin(add class path)
/lib
   /eagle-server.jar
   /apps
app1.jar
app1-web.jar
app2.jar
/web
index.html
/js




--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (EAGLE-429) CoordinatorTest.test_01 sometimes fail

[Edward Zhang (JIRA)]

[ 
https://issues.apache.org/jira/browse/EAGLE-429?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15412737#comment-15412737
 ] 

Edward Zhang commented on EAGLE-429:


Maybe that is because of some temporary failure of zk

2016-08-08 17:47:42,786 INFO [main] impl.GreedyPolicyScheduler[150]: work set 
calculation: []
2016-08-08 17:47:47,818 ERROR [main] curator.ConnectionState[200]: Connection 
timed out for connection string (localhost:2181) and timeout (5000) / elapsed 
(5046)
org.apache.curator.CuratorConnectionLossException: KeeperErrorCode = 
ConnectionLoss
at 
org.apache.curator.ConnectionState.checkTimeouts(ConnectionState.java:197)
at 
org.apache.curator.ConnectionState.getZooKeeper(ConnectionState.java:87)
at 
org.apache.curator.CuratorZookeeperClient.getZooKeeper(CuratorZookeeperClient.java:115)
at 
org.apache.curator.framework.imps.CuratorFrameworkImpl.getZooKeeper(CuratorFrameworkImpl.java:477)
at 
org.apache.curator.framework.imps.ExistsBuilderImpl$2.call(ExistsBuilderImpl.java:172)
at 
org.apache.curator.framework.imps.ExistsBuilderImpl$2.call(ExistsBuilderImpl.java:161)
at org.apache.curator.RetryLoop.callWithRetry(RetryLoop.java:107)
at 
org.apache.curator.framework.imps.ExistsBuilderImpl.pathInForeground(ExistsBuilderImpl.java:158)
at 
org.apache.curator.framework.imps.ExistsBuilderImpl.forPath(ExistsBuilderImpl.java:148)
at 
org.apache.curator.framework.imps.ExistsBuilderImpl.forPath(ExistsBuilderImpl.java:36)
at 
org.apache.eagle.alert.config.ConfigBusProducer.send(ConfigBusProducer.java:39)
at 
org.apache.eagle.alert.coordinator.Coordinator.postSchedule(Coordinator.java:137)
at 
org.apache.eagle.alert.coordinator.Coordinator.schedule(Coordinator.java:115)
at 
org.apache.alert.coordinator.CoordinatorTest.test_01(CoordinatorTest.java:106)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at 
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at 
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at 
org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:50)
at 
org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
at 
org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:47)
at 
org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
at 
org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:26)
at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:325)
at 
org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:78)
at 
org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:57)
at org.junit.runners.ParentRunner$3.run(ParentRunner.java:290)
at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:71)
at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:288)
at org.junit.runners.ParentRunner.access$000(ParentRunner.java:58)
at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:268)
at 
org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:26)
at 
org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27)
at org.junit.runners.ParentRunner.run(ParentRunner.java:363)
at org.junit.runner.JUnitCore.run(JUnitCore.java:137)
at 
com.intellij.junit4.JUnit4IdeaTestRunner.startRunnerWithArgs(JUnit4IdeaTestRunner.java:69)
at 
com.intellij.rt.execution.junit.JUnitStarter.prepareStreamsAndStart(JUnitStarter.java:234)
at 
com.intellij.rt.execution.junit.JUnitStarter.main(JUnitStarter.java:74)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at 
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at 
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.intellij.rt.execution.application.AppMain.main(AppMain.java:144)
2016-08-08 17:47:47,843 INFO [main-SendThread(fe80:0:0:0:0:0:0:1%1:2181)] 
zookeeper.ClientCnxn[975]: Opening socket connection to server 
fe80:0:0:0:0:0:0:1%1/fe80:0:0:0:0:0:0:1%1:2181. Will not attempt to 
authenticate using SASL (unknown error)
2016-08-08 17:47:47,843 INFO [main-SendThread(fe80:0:0:0:0:0:0:1%1:2181)] 
zookeeper.ClientCnxn[852]: Socket

> CoordinatorTest.test_01 sometimes fail
> --
>
> Key: EAGLE-429
> URL: https://issues.apache.org/jira/browse/EAGLE-429
> Project: Eagle
>  Issue Type: Bug
>

[jira] [Created] (EAGLE-429) CoordinatorTest.test_01 sometimes fail

[Edward Zhang (JIRA)]

Edward Zhang created EAGLE-429:
--

 Summary: CoordinatorTest.test_01 sometimes fail
 Key: EAGLE-429
 URL: https://issues.apache.org/jira/browse/EAGLE-429
 Project: Eagle
  Issue Type: Bug
Affects Versions: v0.5.0
Reporter: Edward Zhang
Assignee: Su Ralph
Priority: Minor
 Fix For: v0.5.0


This happens from time to time that test_01 use case fails.

2016-08-08 17:47:48,957 INFO [main-SendThread(localhost:2181)] 
zookeeper.ClientCnxn[852]: Socket connection established to 
localhost/127.0.0.1:2181, initiating session
2016-08-08 17:47:48,960 INFO [main-SendThread(localhost:2181)] 
zookeeper.ClientCnxn[1235]: Session establishment complete on server 
localhost/127.0.0.1:2181, sessionid = 0x156629bb321002f, negotiated timeout = 
1
2016-08-08 17:47:48,960 INFO [main-EventThread] 
state.ConnectionStateManager[228]: State change: CONNECTED
2016-08-08 17:47:49,003 ERROR [main-EventThread] listen.ListenerContainer[96]: 
Listener 
(org.apache.eagle.alert.config.ConfigBusConsumer$$Lambda$3/1403704789@1fb700ee) 
threw an exception
org.junit.ComparisonFailure: expected: but 
was:
at org.junit.Assert.assertEquals(Assert.java:115)
at org.junit.Assert.assertEquals(Assert.java:144)
at 
org.apache.alert.coordinator.CoordinatorTest$2.onNewConfig(CoordinatorTest.java:117)
at 
org.apache.eagle.alert.config.ConfigBusConsumer.lambda$new$0(ConfigBusConsumer.java:46)
at 
org.apache.curator.framework.recipes.cache.NodeCache$4.apply(NodeCache.java:294)
at 
org.apache.curator.framework.recipes.cache.NodeCache$4.apply(NodeCache.java:288)
at 
org.apache.curator.framework.listen.ListenerContainer$1.run(ListenerContainer.java:92)
at 
com.google.common.util.concurrent.MoreExecutors$DirectExecutorService.execute(MoreExecutors.java:310)
at 
org.apache.curator.framework.listen.ListenerContainer.forEach(ListenerContainer.java:84)
at 
org.apache.curator.framework.recipes.cache.NodeCache.setNewData(NodeCache.java:286)
at 
org.apache.curator.framework.recipes.cache.NodeCache.processBackgroundResult(NodeCache.java:253)
at 
org.apache.curator.framework.recipes.cache.NodeCache.access$300(NodeCache.java:54)
at 
org.apache.curator.framework.recipes.cache.NodeCache$3.processResult(NodeCache.java:112)
at 
org.apache.curator.framework.imps.CuratorFrameworkImpl.sendToBackgroundCallback(CuratorFrameworkImpl.java:728)
at 
org.apache.curator.framework.imps.CuratorFrameworkImpl.processBackgroundOperation(CuratorFrameworkImpl.java:505)
at 
org.apache.curator.framework.imps.GetDataBuilderImpl$3.processResult(GetDataBuilderImpl.java:254)
at 
org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:561)
at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:498)

java.lang.AssertionError
at org.junit.Assert.fail(Assert.java:86)
at org.junit.Assert.assertTrue(Assert.java:41)
at org.junit.Assert.assertTrue(Assert.java:52)
at 
org.apache.alert.coordinator.CoordinatorTest.test_01(CoordinatorTest.java:124)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at 
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at 
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at 
org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:50)
at 
org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
at 
org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:47)
at 
org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
at 
org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:26)
at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:325)
at 
org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:78)
at 
org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:57)
at org.junit.runners.ParentRunner$3.run(ParentRunner.java:290)
at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:71)
at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:288)
at org.junit.runners.ParentRunner.access$000(ParentRunner.java:58)
at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:268)
at 
org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:26)
at 
org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27)
at org.junit.runners.ParentRunner.run(ParentRunner.java:363)
at 

[jira] [Created] (EAGLE-428) Hbase monitoring's metadata browse API in new app framework

[Edward Zhang (JIRA)]

Edward Zhang created EAGLE-428:
--

 Summary: Hbase monitoring's metadata browse API in new app 
framework
 Key: EAGLE-428
 URL: https://issues.apache.org/jira/browse/EAGLE-428
 Project: Eagle
  Issue Type: Improvement
Affects Versions: v0.5.0
Reporter: Edward Zhang
Assignee: Edward Zhang
 Fix For: v0.5.0


Hbase monitoring's metadata browse API is used for retrieve hbase's table, 
column etc, and check against sensitivity type.

We should use the configuration from ApplicationEntity to manage metadata query



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Created] (EAGLE-410) stream tumbling window aggregate library

[Edward Zhang (JIRA)]

Edward Zhang created EAGLE-410:
--

 Summary: stream tumbling window aggregate library
 Key: EAGLE-410
 URL: https://issues.apache.org/jira/browse/EAGLE-410
 Project: Eagle
  Issue Type: Task
Affects Versions: v0.5.0
Reporter: Edward Zhang
Assignee: Edward Zhang
 Fix For: v0.5.0


Many use cases need aggregate data in streaming process, but Storm does not 
provide high level aggregation API. The most frequently used example is to 
aggregate data per minute while data is streamlined.

After investigation, we found Siddhi CEP engine is good for this task.

https://docs.wso2.com/display/CEP420/Inbuilt+Windows#InbuiltWindows-externalTimeBatchexternalTimeBatch

This ticket will develop a preview of simple aggregation API for Eagle use case.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (EAGLE-59) Add support for Audit Logs from Apache Ranger

[Edward Zhang (JIRA)]

 [ 
https://issues.apache.org/jira/browse/EAGLE-59?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Edward Zhang updated EAGLE-59:
--
Assignee: Don Bosco Durai

> Add support for Audit Logs from Apache Ranger
> -
>
> Key: EAGLE-59
> URL: https://issues.apache.org/jira/browse/EAGLE-59
> Project: Eagle
>  Issue Type: New Feature
>Reporter: Don Bosco Durai
>Assignee: Don Bosco Durai
>
> Apache Ranger centralizes the audit logs for most of the Hadoop components. 
> The logs are also standardized in common format. It might be good if we can 
> ensure the audit format is supported by Apache Eagle.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (EAGLE-251) Run eagle on flink/samza

[Edward Zhang (JIRA)]

[ 
https://issues.apache.org/jira/browse/EAGLE-251?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15378425#comment-15378425
 ] 

Edward Zhang commented on EAGLE-251:


[~milap.wadhwa] looks this jira ticket is pretty long and probably we need 
revisit the request for running eagle on flink/samza.

In current development branch, Eagle now evolves to have applications and alert 
engine separated. And application can be on top of apache storm, spark etc. and 
alert engine is designed to run on top of apache storm.

So when we look at this ticket, we need make sure what we are going to do, is 
that application's runtime or alert engine runtime.

> Run eagle on flink/samza
> 
>
> Key: EAGLE-251
> URL: https://issues.apache.org/jira/browse/EAGLE-251
> Project: Eagle
>  Issue Type: New Feature
>Affects Versions: v0.3.0
>Reporter: Su Ralph
>Priority: Minor
>  Labels: new
>
> Eagle is running on storm. One might want to do some interesting port to 
> other stream framework like flink/samza.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (EAGLE-371) remove eagle-docs from develop branch

[Edward Zhang (JIRA)]

[ 
https://issues.apache.org/jira/browse/EAGLE-371?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15374047#comment-15374047
 ] 

Edward Zhang commented on EAGLE-371:


We tried with Eagle branch 
https://github.com/apache/incubator-eagle/tree/document long time ago, and will 
officially move all documents to this branch in Eagle 0.5. Use 
https://issues.apache.org/jira/browse/EAGLE-373 to track

> remove eagle-docs from develop branch
> -
>
> Key: EAGLE-371
> URL: https://issues.apache.org/jira/browse/EAGLE-371
> Project: Eagle
>  Issue Type: Improvement
>Affects Versions: 0.5
>Reporter: Edward Zhang
>Assignee: Edward Zhang
>Priority: Trivial
> Fix For: 0.5
>
>
> eagle-docs will be maintained in 
> https://github.com/eaglemonitoring/eaglemonitoring.github.io, so we don't 
> need maintain duplicated documentation



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Created] (EAGLE-373) Move Eagle document to Eagle source code control

[Edward Zhang (JIRA)]

Edward Zhang created EAGLE-373:
--

 Summary: Move Eagle document to Eagle source code control
 Key: EAGLE-373
 URL: https://issues.apache.org/jira/browse/EAGLE-373
 Project: Eagle
  Issue Type: Improvement
Affects Versions: 0.5
Reporter: Edward Zhang
Assignee: Edward Zhang
Priority: Minor
 Fix For: 0.5


Today Eagle's document is maintained in 
https://github.com/eaglemonitoring/eaglemonitoring.github.io, it does not 
conform to Apache policy. We should move this into Apache git source control. 
The approach is to create a branch called document in 
https://github.com/apache/incubator-eagle/tree/document and put document into 
it.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (EAGLE-371) remove eagle-docs from develop branch

[Edward Zhang (JIRA)]

[ 
https://issues.apache.org/jira/browse/EAGLE-371?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15372026#comment-15372026
 ] 

Edward Zhang commented on EAGLE-371:


[~julianhyde]maybe your concern is we should maintain eagle documents in apache 
git. If that is the concern, today eagle does not maintain document in apache 
git. We may need do that in eagle 0.5 when we have document maintained in one 
branch of apache git

> remove eagle-docs from develop branch
> -
>
> Key: EAGLE-371
> URL: https://issues.apache.org/jira/browse/EAGLE-371
> Project: Eagle
>  Issue Type: Improvement
>Affects Versions: 0.5
>Reporter: Edward Zhang
>Assignee: Edward Zhang
>Priority: Trivial
> Fix For: 0.5
>
>
> eagle-docs will be maintained in 
> https://github.com/eaglemonitoring/eaglemonitoring.github.io, so we don't 
> need maintain duplicated documentation



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (EAGLE-371) remove eagle-docs from develop branch

[Edward Zhang (JIRA)]

[ 
https://issues.apache.org/jira/browse/EAGLE-371?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15372017#comment-15372017
 ] 

Edward Zhang commented on EAGLE-371:


[~julianhyde] we have eagle document well maintained in 
https://github.com/eaglemonitoring/eaglemonitoring.github.io, and the URL 
http://eagle.incubator.apache.org/docs still remains the same.

This is a temporary solution, and we want to move all documents into one branch 
of eagle github but not embedded together with source code.

We look at many projects in terms of document lifecycle management, for Eagle 
we want to use branch to manage eagle doc. And document differences between 
different eagle releases are maintained in eagle eagle release if we want to 
keep old document.



> remove eagle-docs from develop branch
> -
>
> Key: EAGLE-371
> URL: https://issues.apache.org/jira/browse/EAGLE-371
> Project: Eagle
>  Issue Type: Improvement
>Affects Versions: 0.5
>Reporter: Edward Zhang
>Assignee: Edward Zhang
>Priority: Trivial
> Fix For: 0.5
>
>
> eagle-docs will be maintained in 
> https://github.com/eaglemonitoring/eaglemonitoring.github.io, so we don't 
> need maintain duplicated documentation



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Resolved] (EAGLE-371) remove eagle-docs from develop branch

[Edward Zhang (JIRA)]

 [ 
https://issues.apache.org/jira/browse/EAGLE-371?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Edward Zhang resolved EAGLE-371.

Resolution: Fixed

merged

> remove eagle-docs from develop branch
> -
>
> Key: EAGLE-371
> URL: https://issues.apache.org/jira/browse/EAGLE-371
> Project: Eagle
>  Issue Type: Improvement
>Affects Versions: 0.5
>Reporter: Edward Zhang
>Assignee: Edward Zhang
>Priority: Trivial
> Fix For: 0.5
>
>
> eagle-docs will be maintained in 
> https://github.com/eaglemonitoring/eaglemonitoring.github.io, so we don't 
> need maintain duplicated documentation



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Closed] (EAGLE-371) remove eagle-docs from develop branch

[Edward Zhang (JIRA)]

 [ 
https://issues.apache.org/jira/browse/EAGLE-371?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Edward Zhang closed EAGLE-371.
--

> remove eagle-docs from develop branch
> -
>
> Key: EAGLE-371
> URL: https://issues.apache.org/jira/browse/EAGLE-371
> Project: Eagle
>  Issue Type: Improvement
>Affects Versions: 0.5
>Reporter: Edward Zhang
>Assignee: Edward Zhang
>Priority: Trivial
> Fix For: 0.5
>
>
> eagle-docs will be maintained in 
> https://github.com/eaglemonitoring/eaglemonitoring.github.io, so we don't 
> need maintain duplicated documentation



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Created] (EAGLE-371) remove eagle-docs from develop branch

[Edward Zhang (JIRA)]

Edward Zhang created EAGLE-371:
--

 Summary: remove eagle-docs from develop branch
 Key: EAGLE-371
 URL: https://issues.apache.org/jira/browse/EAGLE-371
 Project: Eagle
  Issue Type: Improvement
Affects Versions: 0.5
Reporter: Edward Zhang
Assignee: Edward Zhang
Priority: Trivial
 Fix For: 0.5


eagle-docs will be maintained in 
https://github.com/eaglemonitoring/eaglemonitoring.github.io, so we don't need 
maintain duplicated documentation



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Closed] (EAGLE-370) Absence alert engine

[Edward Zhang (JIRA)]

 [ 
https://issues.apache.org/jira/browse/EAGLE-370?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Edward Zhang closed EAGLE-370.
--
Resolution: Fixed

merge with pr 262

> Absence alert engine
> 
>
> Key: EAGLE-370
> URL: https://issues.apache.org/jira/browse/EAGLE-370
> Project: Eagle
>  Issue Type: Improvement
>Affects Versions: 0.5
>Reporter: Edward Zhang
>Assignee: Edward Zhang
> Fix For: 0.5
>
>   Original Estimate: 168h
>  Remaining Estimate: 168h
>
> Absence alert is used for triggering alert when something does not happen 
> within a time period. For example one job is expected to run in 2pm - 3pm 
> every day, if it does not appear, then alert.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Created] (EAGLE-370) Absence alert engine

[Edward Zhang (JIRA)]

Edward Zhang created EAGLE-370:
--

 Summary: Absence alert engine
 Key: EAGLE-370
 URL: https://issues.apache.org/jira/browse/EAGLE-370
 Project: Eagle
  Issue Type: Improvement
Affects Versions: 0.5
Reporter: Edward Zhang
Assignee: Edward Zhang
 Fix For: 0.5


Absence alert is used for triggering alert when something does not happen 
within a time period. For example one job is expected to run in 2pm - 3pm every 
day, if it does not appear, then alert.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (EAGLE-366) Create HBase tables if not exists in Eagle 0.5

[Edward Zhang (JIRA)]

[ 
https://issues.apache.org/jira/browse/EAGLE-366?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15371232#comment-15371232
 ] 

Edward Zhang commented on EAGLE-366:


[~qingwzhao][~haoch] in 0.5 how do we handle table creation for those built-in 
applications? Looks it is not very wise to create all the tables.
Also another question is metadata should be moved into RDBMS instead of HBase. 
In HBase we only store Hadoop monitoring related data.

> Create HBase tables if not exists in Eagle 0.5
> --
>
> Key: EAGLE-366
> URL: https://issues.apache.org/jira/browse/EAGLE-366
> Project: Eagle
>  Issue Type: Improvement
>Reporter: Zhao, Qingwen
>Assignee: Zhao, Qingwen
>
> 1. Migrate create hbase tables in Eagle 0.4 to Eagle 0.5
> 2. Add config value 'env=test' checking when creating tables



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (EAGLE-368) JSlint failed in common/controller.js

[Edward Zhang (JIRA)]

[ 
https://issues.apache.org/jira/browse/EAGLE-368?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15371205#comment-15371205
 ] 

Edward Zhang commented on EAGLE-368:


In which version is this happening and in which version do we want to fix tthat

> JSlint failed in common/controller.js
> -
>
> Key: EAGLE-368
> URL: https://issues.apache.org/jira/browse/EAGLE-368
> Project: Eagle
>  Issue Type: Bug
>Reporter: Jilin, Jiang
>Assignee: Jilin, Jiang
>




--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (EAGLE-369) Improve eagle alert email template to provide more useful information and user experience

[Edward Zhang (JIRA)]

[ 
https://issues.apache.org/jira/browse/EAGLE-369?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15371201#comment-15371201
 ] 

Edward Zhang commented on EAGLE-369:


I would suggest to let user to edit template as they like, but Eagle provides 
some predefined template. Because for those customized text information, we 
have to custom template.

> Improve eagle alert email template to provide more useful information and 
> user experience
> -
>
> Key: EAGLE-369
> URL: https://issues.apache.org/jira/browse/EAGLE-369
> Project: Eagle
>  Issue Type: Improvement
>Affects Versions: v0.3.1
>Reporter: Su Ralph
>Assignee: Su Ralph
> Fix For: v0.5.0
>
>
> From feedback:
> (1)   Time Stamp needs to be human readable. 
> (2)   Consider capitalize wordings.
> (3)   What does lowNum and highNum mean here?  Do they mean number of live 
> data nodes?  How does this alert come out?  Does that mean the current number 
> of live data nodes go above the highNum or below the lowNum?  The lowNum 
> should display as “2,884” instead of “2884.0”.  
> (4)   The Description field should always be there.  If description is not 
> available, it should display as “Not provided by Policy Owner”.
> (5)   Should we consider displaying the name of the policy owner?  That way, 
> people know whom to contact if they have question about the policy.
> (6)   The information in the Actions Required field is too generic and it is 
> not actionable.  Can we do something about it?
> (7)   The text alignment (now always align center) may consider to align 
> left.  Use want to make it look more professional.
> (8)   Text font is not consistent across the message.  Should align to one 
> single font.  Consider use Calibri as this is now the eBay official font 
> type.  
> (9)   Lack of policy definition
> The definition is 
> from every (e1 = hadoopJmxMetricEventStream[metric == 
> \"hadoop.namenode.fsnamesystemstate.numlivedatanodes\" ]) -> e2 = 
> hadoopJmxMetricEventStream[metric == e1.metric and host == e1.host and 
> (convert(e1.value, \"long\") - 2) >= convert(value, \"long\") ] within 5 min 
> select e1.metric, e1.host, e1.value as highNum, e1.timestamp as start, 
> e2.value as lowNum, e2.timestamp as end insert into tmp;
>  



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Closed] (EAGLE-340) refactor bin/eagle-sandbox-starter.sh to make it easier to use

[Edward Zhang (JIRA)]

 [ 
https://issues.apache.org/jira/browse/EAGLE-340?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Edward Zhang closed EAGLE-340.
--

> refactor bin/eagle-sandbox-starter.sh to make it easier to use
> --
>
> Key: EAGLE-340
> URL: https://issues.apache.org/jira/browse/EAGLE-340
> Project: Eagle
>  Issue Type: Improvement
>Reporter: Zhao, Qingwen
>Assignee: Zhao, Qingwen
> Fix For: v0.4.0
>
>
> 1. setup Eagle in sandbox with minimum effort
> 2. start a sample application so users can trigger an alert



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Resolved] (EAGLE-219) Use PUT method for updating request when possible in front-end.

[Edward Zhang (JIRA)]

 [ 
https://issues.apache.org/jira/browse/EAGLE-219?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Edward Zhang resolved EAGLE-219.

Resolution: Fixed

> Use PUT method for updating request when possible in front-end.
> ---
>
> Key: EAGLE-219
> URL: https://issues.apache.org/jira/browse/EAGLE-219
> Project: Eagle
>  Issue Type: Sub-task
>Affects Versions: v0.3.0
>Reporter: Hao Chen
>Assignee: Jilin, Jiang
> Fix For: v0.4.0
>
>




--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Closed] (EAGLE-219) Use PUT method for updating request when possible in front-end.

[Edward Zhang (JIRA)]

 [ 
https://issues.apache.org/jira/browse/EAGLE-219?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Edward Zhang closed EAGLE-219.
--

> Use PUT method for updating request when possible in front-end.
> ---
>
> Key: EAGLE-219
> URL: https://issues.apache.org/jira/browse/EAGLE-219
> Project: Eagle
>  Issue Type: Sub-task
>Affects Versions: v0.3.0
>Reporter: Hao Chen
>Assignee: Jilin, Jiang
> Fix For: v0.4.0
>
>




--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (EAGLE-145) Send mail failed for javax.mail.internet.AddressException: Illegal semicolon, not in group

[Edward Zhang (JIRA)]

 [ 
https://issues.apache.org/jira/browse/EAGLE-145?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Edward Zhang updated EAGLE-145:
---
Fix Version/s: (was: v0.4.0)
   v0.5.0

> Send mail failed for javax.mail.internet.AddressException: Illegal semicolon, 
> not in group
> --
>
> Key: EAGLE-145
> URL: https://issues.apache.org/jira/browse/EAGLE-145
> Project: Eagle
>  Issue Type: Bug
>Affects Versions: v0.3.0
>Reporter: Hao Chen
>Assignee: Hao Chen
> Fix For: v0.5.0
>
>
> {code}
> 2016-02-02 02:52:03 o.a.e.p.e.PolicyProcessExecutor [INFO] Detected 1 alerts 
> for policy missingBlockPolicy
> 2016-02-02 02:52:03 o.a.e.m.r.EagleServiceReporterMetricListener [INFO] Wrote 
> 1 entities to service
> 2016-02-02 02:52:03 o.a.e.a.p.EaglePersist [INFO] Going to persist entities, 
> type:  AlertAPIEntity, list size: 1
> 2016-02-02 02:52:03 o.a.e.a.c.AlertEmailSender [INFO] Initialized 
> Thread-4-AlertNotificationExecutor_10: origin is : 
> druid-test-host1-556191.slc01.dev.ebayc3.com(pid:14052), recipient of the 
> email: li...@ebay.com;hch...@ebay.com, velocity TPL file: ALERT_DEFAULT.vm
> 2016-02-02 02:52:03 o.a.e.a.n.AlertEmailGenerator [INFO] Sending email  in 
> asynchronous to: h...@apache.org, cc: null
> 2016-02-02 02:52:03 o.a.e.a.c.AlertEmailSender [INFO] Sending email, tried: 
> 1, max: 3
> 2016-02-02 02:52:03 STDIO [INFO] DEBUG: setDebug: JavaMail version 1.4ea
> 2016-02-02 02:52:03 o.a.e.a.c.AlertEmailSender [INFO] Env is: prod
> 2016-02-02 02:52:03 o.a.e.a.c.AlertEmailSender [INFO] After calling 
> generateCommonContext...
> 2016-02-02 02:52:03 o.a.velocity [ERROR] ResourceManager : unable to find 
> resource 'templates/ALERT_DEFAULT.vm' in any resource loader.
> 2016-02-02 02:52:03 o.a.e.c.e.EagleMailClient [INFO] Send mail failed, got an 
> AddressException: Illegal semicolon, not in group
> javax.mail.internet.AddressException: Illegal semicolon, not in group
>   at javax.mail.internet.InternetAddress.parse(InternetAddress.java:780) 
> ~[stormjar.jar:na]
>   at javax.mail.internet.InternetAddress.parse(InternetAddress.java:555) 
> ~[stormjar.jar:na]
>   at javax.mail.internet.InternetAddress.parse(InternetAddress.java:532) 
> ~[stormjar.jar:na]
>   at 
> org.apache.eagle.common.email.EagleMailClient._send(EagleMailClient.java:104) 
> [stormjar.jar:na]
>   at 
> org.apache.eagle.common.email.EagleMailClient.send(EagleMailClient.java:167) 
> [stormjar.jar:na]
>   at 
> org.apache.eagle.common.email.EagleMailClient.send(EagleMailClient.java:187) 
> [stormjar.jar:na]
>   at 
> org.apache.eagle.common.email.EagleMailClient.send(EagleMailClient.java:193) 
> [stormjar.jar:na]
>   at 
> org.apache.eagle.alert.common.AlertEmailSender.run(AlertEmailSender.java:129) 
> [stormjar.jar:na]
>   at 
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) 
> [na:1.7.0_60]
>   at java.util.concurrent.FutureTask.run(FutureTask.java:262) 
> [na:1.7.0_60]
>   at 
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
>  [na:1.7.0_60]
>   at 
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
>  [na:1.7.0_60]
>   at java.lang.Thread.run(Thread.java:745) [na:1.7.0_60]
> 2016-02-02 02:52:03 o.a.e.a.c.AlertEmailSender [INFO] Success of sending 
> email: false
> 2016-02-02 02:52:03 o.a.e.a.c.AlertEmailSender [INFO] Sleep for a while 
> before retrying
> {code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Resolved] (EAGLE-291) JDBC: Update transactions fail in PostgreSQL

[Edward Zhang (JIRA)]

 [ 
https://issues.apache.org/jira/browse/EAGLE-291?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Edward Zhang resolved EAGLE-291.

Resolution: Fixed

> JDBC: Update transactions fail in PostgreSQL
> 
>
> Key: EAGLE-291
> URL: https://issues.apache.org/jira/browse/EAGLE-291
> Project: Eagle
>  Issue Type: Bug
>Affects Versions: v0.3.0
> Environment: Linux, PostgreSQL 9.1  as metadata store.
>Reporter: Jaspaul Chahal
> Fix For: v0.4.0
>
>
> When it tries to update some entity, throws the following exception: 
> org.apache.torque.TorqueException: org.postgresql.util.PSQLException: ERROR: 
> current transaction is aborted, commands ignored until end of transaction 
> block.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Closed] (EAGLE-291) JDBC: Update transactions fail in PostgreSQL

[Edward Zhang (JIRA)]

 [ 
https://issues.apache.org/jira/browse/EAGLE-291?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Edward Zhang closed EAGLE-291.
--

> JDBC: Update transactions fail in PostgreSQL
> 
>
> Key: EAGLE-291
> URL: https://issues.apache.org/jira/browse/EAGLE-291
> Project: Eagle
>  Issue Type: Bug
>Affects Versions: v0.3.0
> Environment: Linux, PostgreSQL 9.1  as metadata store.
>Reporter: Jaspaul Chahal
> Fix For: v0.4.0
>
>
> When it tries to update some entity, throws the following exception: 
> org.apache.torque.TorqueException: org.postgresql.util.PSQLException: ERROR: 
> current transaction is aborted, commands ignored until end of transaction 
> block.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (EAGLE-256) Replace HDP version based hadoop dependencies with community version instead.

[Edward Zhang (JIRA)]

 [ 
https://issues.apache.org/jira/browse/EAGLE-256?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Edward Zhang updated EAGLE-256:
---
Fix Version/s: (was: v0.4.0)
   v0.5.0

> Replace HDP version based hadoop dependencies with community version instead.
> -
>
> Key: EAGLE-256
> URL: https://issues.apache.org/jira/browse/EAGLE-256
> Project: Eagle
>  Issue Type: Improvement
>Affects Versions: v0.3.0
>Reporter: Hao Chen
> Fix For: v0.5.0
>
>




--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (EAGLE-267) node_modules is packaged into eagle tar package

[Edward Zhang (JIRA)]

 [ 
https://issues.apache.org/jira/browse/EAGLE-267?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Edward Zhang updated EAGLE-267:
---
Fix Version/s: (was: v0.4.0)
   v0.5.0

> node_modules is packaged into eagle tar package
> ---
>
> Key: EAGLE-267
> URL: https://issues.apache.org/jira/browse/EAGLE-267
> Project: Eagle
>  Issue Type: Bug
>Affects Versions: v0.3.0
>Reporter: Hao Chen
>Assignee: Hao Chen
> Fix For: v0.5.0
>
>




--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (EAGLE-260) Support to fetch audit log from http://localhost:50070/logs/hdfs-audit.log for quick preview

[Edward Zhang (JIRA)]

 [ 
https://issues.apache.org/jira/browse/EAGLE-260?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Edward Zhang updated EAGLE-260:
---
Fix Version/s: (was: v0.4.0)
   v0.5.0

> Support to fetch audit log from http://localhost:50070/logs/hdfs-audit.log 
> for quick preview
> 
>
> Key: EAGLE-260
> URL: https://issues.apache.org/jira/browse/EAGLE-260
> Project: Eagle
>  Issue Type: Improvement
>Affects Versions: v0.3.0
>Reporter: Hao Chen
>Assignee: Hao Chen
> Fix For: v0.5.0
>
>




--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (EAGLE-265) AlertExecutor Support to load all policies if AlertExecutorId is not defined

[Edward Zhang (JIRA)]

 [ 
https://issues.apache.org/jira/browse/EAGLE-265?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Edward Zhang updated EAGLE-265:
---
Fix Version/s: (was: v0.4.0)
   v0.5.0

> AlertExecutor Support to load all policies if AlertExecutorId is not defined
> 
>
> Key: EAGLE-265
> URL: https://issues.apache.org/jira/browse/EAGLE-265
> Project: Eagle
>  Issue Type: Improvement
>Affects Versions: v0.3.0
>Reporter: Hao Chen
>Assignee: Hao Chen
> Fix For: v0.5.0
>
>




--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (EAGLE-233) Integrate [maven-release-plugin] for eagle release

[Edward Zhang (JIRA)]

 [ 
https://issues.apache.org/jira/browse/EAGLE-233?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Edward Zhang updated EAGLE-233:
---
Fix Version/s: (was: v0.4.0)
   v0.5.0

> Integrate [maven-release-plugin] for eagle release
> --
>
> Key: EAGLE-233
> URL: https://issues.apache.org/jira/browse/EAGLE-233
> Project: Eagle
>  Issue Type: New Feature
>Affects Versions: v0.3.0
>Reporter: Hao Chen
>Assignee: Hao Chen
>  Labels: development, tools
> Fix For: v0.5.0
>
>




--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Closed] (EAGLE-73) Put docker steps to site tutorial

[Edward Zhang (JIRA)]

 [ 
https://issues.apache.org/jira/browse/EAGLE-73?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Edward Zhang closed EAGLE-73.
-
Resolution: Fixed

this is in new eagle site

> Put docker steps to site tutorial
> -
>
> Key: EAGLE-73
> URL: https://issues.apache.org/jira/browse/EAGLE-73
> Project: Eagle
>  Issue Type: Task
>Affects Versions: v0.3.0
>Reporter: Su Ralph
>Assignee: Zhaokun Qin
> Fix For: v0.4.0
>
>
> Dockerize code already checked, please help to push the dockerize steps to 
> eagle site documentation under installation topic.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (EAGLE-146) SiddhiExtensionLoader Error: viewing zip file for jar:./lib/ojdbc6.jar

[Edward Zhang (JIRA)]

 [ 
https://issues.apache.org/jira/browse/EAGLE-146?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Edward Zhang updated EAGLE-146:
---
Fix Version/s: (was: v0.4.0)
   v0.5.0

> SiddhiExtensionLoader Error: viewing zip file for jar:./lib/ojdbc6.jar
> --
>
> Key: EAGLE-146
> URL: https://issues.apache.org/jira/browse/EAGLE-146
> Project: Eagle
>  Issue Type: Bug
>Affects Versions: v0.3.0
>Reporter: Hao Chen
>Assignee: Hao Chen
> Fix For: v0.5.0
>
>
> {code}
> 2016-02-02 02:46:23 o.a.e.p.PolicyManager [INFO] Supported policy type : 
> siddhiCEPEngine
> 2016-02-02 02:46:23 o.a.e.p.PolicyManager [INFO] Supported policy type : 
> MachineLearning
> 2016-02-02 02:46:23 o.w.s.c.u.SiddhiExtensionLoader [ERROR] Error viewing zip 
> file for jar:./lib/ojdbc6.jar
> java.io.FileNotFoundException: ./lib/ojdbc6.jar (No such file or directory)
> at java.util.zip.ZipFile.open(Native Method) ~[na:1.7.0_60]
> at java.util.zip.ZipFile.(ZipFile.java:215) ~[na:1.7.0_60]
> at java.util.zip.ZipFile.(ZipFile.java:145) ~[na:1.7.0_60]
> at java.util.zip.ZipFile.(ZipFile.java:159) ~[na:1.7.0_60]
> at 
> org.wso2.siddhi.core.util.SiddhiExtensionLoader.getContentFromJarFile(SiddhiExtensionLoader.java:111)
>  [stormjar.jar:na]
> at 
> org.wso2.siddhi.core.util.SiddhiExtensionLoader.getContent(SiddhiExtensionLoader.java:82)
>  [stormjar.jar:na]
> at 
> org.wso2.siddhi.core.util.SiddhiExtensionLoader.getContentFromDirectory(SiddhiExtensionLoader.java:101)
>  [stormjar.jar:na]
> at 
> org.wso2.siddhi.core.util.SiddhiExtensionLoader.getContent(SiddhiExtensionLoader.java:77)
>  [stormjar.jar:na]
> at 
> org.wso2.siddhi.core.util.SiddhiExtensionLoader.getContentFromDirectory(SiddhiExtensionLoader.java:101)
>  [stormjar.jar:na]
> at 
> org.wso2.siddhi.core.util.SiddhiExtensionLoader.getContent(SiddhiExtensionLoader.java:77)
>  [stormjar.jar:na]
> at 
> org.wso2.siddhi.core.util.SiddhiExtensionLoader.getResources(SiddhiExtensionLoader.java:71)
>  [stormjar.jar:na]
> at 
> org.wso2.siddhi.core.util.SiddhiExtensionLoader.loadSiddhiExtensions(SiddhiExtensionLoader.java:51)
>  [stormjar.jar:na]
> at 
> org.wso2.siddhi.core.config.SiddhiContext.(SiddhiContext.java:35) 
> [stormjar.jar:na]
> at org.wso2.siddhi.core.SiddhiManager.(SiddhiManager.java:40) 
> [stormjar.jar:na]
> at 
> org.apache.eagle.policy.siddhi.SiddhiPolicyEvaluator.createSiddhiRuntime(SiddhiPolicyEvaluator.java:108)
>  [stormjar.jar:na]
> at 
> org.apache.eagle.policy.siddhi.SiddhiPolicyEvaluator.init(SiddhiPolicyEvaluator.java:83)
>  [stormjar.jar:na]
> at 
> org.apache.eagle.policy.siddhi.SiddhiPolicyEvaluator.(SiddhiPolicyEvaluator.java:79)
>  [stormjar.jar:na]
> at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native 
> Method) [na:1.7.0_60]
> at 
> sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:57)
>  [na:1.7.0_60]
> at 
> sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
>  [na:1.7.0_60]
> at java.lang.reflect.Constructor.newInstance(Constructor.java:526) 
> [na:1.7.0_60]
> at 
> org.apache.eagle.policy.executor.PolicyProcessExecutor.createPolicyEvaluator(PolicyProcessExecutor.java:243)
>  [stormjar.jar:na]
> at 
> org.apache.eagle.policy.executor.PolicyProcessExecutor.init(PolicyProcessExecutor.java:188)
>  [stormjar.jar:na]
> at 
> org.apache.eagle.datastream.storm.JavaStormBoltWrapper.prepare(JavaStormBoltWrapper.scala:36)
>  [stormjar.jar:na]
> at 
> backtype.storm.daemon.executor$fn__5697$fn__5710.invoke(executor.clj:732) 
> [storm-core-0.9.3.2.2.0.0-2041.jar:0.9.3.2.2.0.0-2041]
> at backtype.storm.util$async_loop$fn__452.invoke(util.clj:463) 
> [storm-core-0.9.3.2.2.0.0-2041.jar:0.9.3.2.2.0.0-2041]
> at clojure.lang.AFn.run(AFn.java:24) [clojure-1.5.1.jar:na]
> at java.lang.Thread.run(Thread.java:745) [na:1.7.0_60]
> 2016-02-02 02:46:23 o.w.s.c.u.SiddhiExtensionLoader [ERROR] Error viewing zip 
> file for jar:./lib/ojdbc6.jar
> java.io.FileNotFoundException: ./lib/ojdbc6.jar (No such file or directory)
> {code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (EAGLE-220) Add mvn script to handle maven dependency

[Edward Zhang (JIRA)]

 [ 
https://issues.apache.org/jira/browse/EAGLE-220?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Edward Zhang updated EAGLE-220:
---
Fix Version/s: (was: v0.4.0)
   v0.5.0

> Add mvn script to handle maven dependency
> -
>
> Key: EAGLE-220
> URL: https://issues.apache.org/jira/browse/EAGLE-220
> Project: Eagle
>  Issue Type: Improvement
>Affects Versions: v0.3.0
>Reporter: Hao Chen
>Assignee: Hao Chen
>  Labels: Development
> Fix For: v0.5.0
>
>
> Refer to https://github.com/apache/spark/blob/master/build/mvn



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Closed] (EAGLE-218) convert /list to /entities

[Edward Zhang (JIRA)]

 [ 
https://issues.apache.org/jira/browse/EAGLE-218?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Edward Zhang closed EAGLE-218.
--
   Resolution: Won't Fix
Fix Version/s: (was: v0.4.0)
   v0.5.0

move this to 0.5 when we do job performance monitoring

> convert /list to /entities
> --
>
> Key: EAGLE-218
> URL: https://issues.apache.org/jira/browse/EAGLE-218
> Project: Eagle
>  Issue Type: Improvement
>Affects Versions: v0.3.0
>Reporter: Edward Zhang
>Assignee: Hao Chen
> Fix For: v0.5.0
>
>
> Today still many places in UI uses /list. The problem is with /list, 
> DataStorage interface is actually not used and HBASE is always used, we 
> should convert all /list into /entities



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Resolved] (EAGLE-185) UI create cache after building

[Edward Zhang (JIRA)]

 [ 
https://issues.apache.org/jira/browse/EAGLE-185?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Edward Zhang resolved EAGLE-185.

Resolution: Fixed

> UI create cache after building
> --
>
> Key: EAGLE-185
> URL: https://issues.apache.org/jira/browse/EAGLE-185
> Project: Eagle
>  Issue Type: Improvement
>Affects Versions: v0.3.0
>Reporter: Jilin, Jiang
>Assignee: Jilin, Jiang
> Fix For: v0.4.0
>
>
> Current UI load html template & feature module using dynamic suffix which 
> keeping getting latest data. But for built ui folder, the data should be in 
> cache instead of updating. Which will fast the page loading speed.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Closed] (EAGLE-185) UI create cache after building

[Edward Zhang (JIRA)]

 [ 
https://issues.apache.org/jira/browse/EAGLE-185?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Edward Zhang closed EAGLE-185.
--

> UI create cache after building
> --
>
> Key: EAGLE-185
> URL: https://issues.apache.org/jira/browse/EAGLE-185
> Project: Eagle
>  Issue Type: Improvement
>Affects Versions: v0.3.0
>Reporter: Jilin, Jiang
>Assignee: Jilin, Jiang
> Fix For: v0.4.0
>
>
> Current UI load html template & feature module using dynamic suffix which 
> keeping getting latest data. But for built ui folder, the data should be in 
> cache instead of updating. Which will fast the page loading speed.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Closed] (EAGLE-203) Metrics feature support merge chart

[Edward Zhang (JIRA)]

 [ 
https://issues.apache.org/jira/browse/EAGLE-203?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Edward Zhang closed EAGLE-203.
--

> Metrics feature support merge chart
> ---
>
> Key: EAGLE-203
> URL: https://issues.apache.org/jira/browse/EAGLE-203
> Project: Eagle
>  Issue Type: New Feature
>Affects Versions: v0.3.0
>Reporter: Jilin, Jiang
>Assignee: Jilin, Jiang
> Fix For: v0.4.0
>
>
> Sometime user want to see the trend in one chart.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (EAGLE-302) Add sample main app that read config for alert-executor to make onboard easier with topo-mgmt

[Edward Zhang (JIRA)]

[ 
https://issues.apache.org/jira/browse/EAGLE-302?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15334583#comment-15334583
 ] 

Edward Zhang commented on EAGLE-302:


[~qingwzhao][~ralphsu] I would suggest we put this issue to 0.5 because in 0.4 
we use topo-mgmt as an experiment. Topo-mgmt will be really used seamlessly in 
0.5.

In 0.4, user still need manually start KafkaStreamMonitorApp for simple use 
case, do you agree?

> Add sample main app that read config for alert-executor to make onboard 
> easier with topo-mgmt
> -
>
> Key: EAGLE-302
> URL: https://issues.apache.org/jira/browse/EAGLE-302
> Project: Eagle
>  Issue Type: Improvement
>Affects Versions: v0.4.0
>Reporter: Su Ralph
>Assignee: Zhao, Qingwen
> Fix For: v0.5.0
>
>   Original Estimate: 48h
>  Remaining Estimate: 48h
>
> Currently, if user have a simple request of do some alerting where the data 
> source is well matched json to stream schema definition. 
> He has to write a code like KafkaStreamMonitorApp, specify the alert executor 
> id, like what we do for HadoopJmxMetricMonitor.
> This steps are good before v0.4, since at that time user need to manually 
> start the topology. In v0.4, we have topo-mgmt, which we could encourage user 
> to start/stop topology remotely through api/ui. We could provide/enhance to 
> provide an sample app that consume for one policy executor id(from the 
> config). So that for typical use case, user only need to specify the 
> application.conf, then a topology with alert function could be started though 
> UI without any code/jar submit.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (EAGLE-302) Add sample main app that read config for alert-executor to make onboard easier with topo-mgmt

[Edward Zhang (JIRA)]

 [ 
https://issues.apache.org/jira/browse/EAGLE-302?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Edward Zhang updated EAGLE-302:
---
Fix Version/s: (was: v0.4.0)
   v0.5.0

> Add sample main app that read config for alert-executor to make onboard 
> easier with topo-mgmt
> -
>
> Key: EAGLE-302
> URL: https://issues.apache.org/jira/browse/EAGLE-302
> Project: Eagle
>  Issue Type: Improvement
>Affects Versions: v0.4.0
>Reporter: Su Ralph
>Assignee: Zhao, Qingwen
> Fix For: v0.5.0
>
>   Original Estimate: 48h
>  Remaining Estimate: 48h
>
> Currently, if user have a simple request of do some alerting where the data 
> source is well matched json to stream schema definition. 
> He has to write a code like KafkaStreamMonitorApp, specify the alert executor 
> id, like what we do for HadoopJmxMetricMonitor.
> This steps are good before v0.4, since at that time user need to manually 
> start the topology. In v0.4, we have topo-mgmt, which we could encourage user 
> to start/stop topology remotely through api/ui. We could provide/enhance to 
> provide an sample app that consume for one policy executor id(from the 
> config). So that for typical use case, user only need to specify the 
> application.conf, then a topology with alert function could be started though 
> UI without any code/jar submit.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (EAGLE-340) refactor bin/eagle-sandbox-starter.sh to make it easier to use

[Edward Zhang (JIRA)]

 [ 
https://issues.apache.org/jira/browse/EAGLE-340?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Edward Zhang updated EAGLE-340:
---
Fix Version/s: v0.4.0

> refactor bin/eagle-sandbox-starter.sh to make it easier to use
> --
>
> Key: EAGLE-340
> URL: https://issues.apache.org/jira/browse/EAGLE-340
> Project: Eagle
>  Issue Type: Improvement
>Reporter: Zhao, Qingwen
>Assignee: Zhao, Qingwen
> Fix For: v0.4.0
>
>
> 1. setup Eagle in sandbox with minimum effort
> 2. start a sample application so users can trigger an alert



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (EAGLE-280) Update logstash-kafka-conf.md

[Edward Zhang (JIRA)]

 [ 
https://issues.apache.org/jira/browse/EAGLE-280?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Edward Zhang updated EAGLE-280:
---
Fix Version/s: v0.4.0

> Update logstash-kafka-conf.md
> -
>
> Key: EAGLE-280
> URL: https://issues.apache.org/jira/browse/EAGLE-280
> Project: Eagle
>  Issue Type: Task
>Affects Versions: v0.3.0
>Reporter: DanielZhou
>Assignee: DanielZhou
> Fix For: v0.4.0
>
>
> The 2.0 release of Logstash includes a new version of the Kafka output plugin 
> with significant configuration changes. One might need a new sample of 
> logstash config file for logstash2.X version.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Reopened] (EAGLE-225) Create eagle bootstrap scripts for examples

[Edward Zhang (JIRA)]

 [ 
https://issues.apache.org/jira/browse/EAGLE-225?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Edward Zhang reopened EAGLE-225:


> Create eagle bootstrap scripts for examples 
> 
>
> Key: EAGLE-225
> URL: https://issues.apache.org/jira/browse/EAGLE-225
> Project: Eagle
>  Issue Type: New Feature
>Affects Versions: v0.3.0
>Reporter: Hao Chen
>Assignee: Hao Chen
>




--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (EAGLE-157) policy metric should be refreshed every minute

[Edward Zhang (JIRA)]

 [ 
https://issues.apache.org/jira/browse/EAGLE-157?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Edward Zhang updated EAGLE-157:
---
Fix Version/s: v0.4.0

> policy metric should be refreshed every minute
> --
>
> Key: EAGLE-157
> URL: https://issues.apache.org/jira/browse/EAGLE-157
> Project: Eagle
>  Issue Type: Bug
>Affects Versions: v0.3.0
>Reporter: Edward Zhang
>Assignee: Jilin, Jiang
>Priority: Minor
> Fix For: v0.4.0
>
>
> Today policy metric like evaluation count, alert count etc, is not refreshed 
> in UI in time. We need some frequent refresh for example per minute



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Reopened] (EAGLE-280) Update logstash-kafka-conf.md

[Edward Zhang (JIRA)]

 [ 
https://issues.apache.org/jira/browse/EAGLE-280?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Edward Zhang reopened EAGLE-280:


> Update logstash-kafka-conf.md
> -
>
> Key: EAGLE-280
> URL: https://issues.apache.org/jira/browse/EAGLE-280
> Project: Eagle
>  Issue Type: Task
>Affects Versions: v0.3.0
>Reporter: DanielZhou
>Assignee: DanielZhou
>
> The 2.0 release of Logstash includes a new version of the Kafka output plugin 
> with significant configuration changes. One might need a new sample of 
> logstash config file for logstash2.X version.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (EAGLE-157) policy metric should be refreshed every minute

[Edward Zhang (JIRA)]

 [ 
https://issues.apache.org/jira/browse/EAGLE-157?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Edward Zhang updated EAGLE-157:
---
Affects Version/s: v0.3.0

> policy metric should be refreshed every minute
> --
>
> Key: EAGLE-157
> URL: https://issues.apache.org/jira/browse/EAGLE-157
> Project: Eagle
>  Issue Type: Bug
>Affects Versions: v0.3.0
>Reporter: Edward Zhang
>Assignee: Jilin, Jiang
>Priority: Minor
> Fix For: v0.4.0
>
>
> Today policy metric like evaluation count, alert count etc, is not refreshed 
> in UI in time. We need some frequent refresh for example per minute



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (EAGLE-176) Metric dashboard UI keep api refresh after page switch

[Edward Zhang (JIRA)]

 [ 
https://issues.apache.org/jira/browse/EAGLE-176?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Edward Zhang updated EAGLE-176:
---
Affects Version/s: v0.3.0

> Metric dashboard UI keep api refresh after page switch
> --
>
> Key: EAGLE-176
> URL: https://issues.apache.org/jira/browse/EAGLE-176
> Project: Eagle
>  Issue Type: Bug
>Affects Versions: v0.3.0
>Reporter: Jilin, Jiang
>Assignee: Jilin, Jiang
>Priority: Trivial
> Fix For: v0.4.0
>
>
> setInterval(function() {
>   if(!$scope.dashboardReady) return;
>   $scope.chartRefresh(true);
> }, 1000 * 30);
> Interval not clean



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Closed] (EAGLE-18) Follow up with infra about website creation

[Edward Zhang (JIRA)]

 [ 
https://issues.apache.org/jira/browse/EAGLE-18?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Edward Zhang closed EAGLE-18.
-
Resolution: Fixed

> Follow up with infra about website creation
> ---
>
> Key: EAGLE-18
> URL: https://issues.apache.org/jira/browse/EAGLE-18
> Project: Eagle
>  Issue Type: Bug
>Affects Versions: v0.3.0
>Reporter: Henry Saputra
> Fix For: v0.4.0
>
>
> This is Eagle side for issue filed for INFRA:
>   https://issues.apache.org/jira/browse/INFRA-10747



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)