[jira] [Closed] (EAGLE-491) AlertPublisherImpl dedup throws exception when dedup column is not configured

[Zeng, Bryant (JIRA)]

 [ 
https://issues.apache.org/jira/browse/EAGLE-491?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Zeng, Bryant closed EAGLE-491.
--
Resolution: Fixed

> AlertPublisherImpl dedup throws exception when dedup column is not configured
> -
>
> Key: EAGLE-491
> URL: https://issues.apache.org/jira/browse/EAGLE-491
> Project: Eagle
>  Issue Type: Bug
>Affects Versions: v0.5.0
>Reporter: Edward Zhang
>Assignee: Zeng, Bryant
>Priority: Trivial
> Fix For: v0.5.0
>
>
> 2016-08-22 14:51:49,728 WARN [alertBolt1-evaluator_stage1] 
> runner.AlertBolt[127]: Spec Version [spec_version_1471902692329] of AlertBolt 
> is newer than Stream Event Version [null]!
> 2016-08-22 14:51:49,729 ERROR [Thread-45-alertPublishBolt] 
> impl.AlertPublisherImpl[80]: Fail invoking publisher's onAlert, continue 
> java.lang.NullPointerException
>   at 
> org.apache.eagle.alert.engine.publisher.impl.DefaultDeduplicator.dedup(DefaultDeduplicator.java:114)
>   at 
> org.apache.eagle.alert.engine.publisher.impl.AbstractPublishPlugin.dedup(AbstractPublishPlugin.java:68)
>   at 
> org.apache.eagle.alert.engine.publisher.impl.AlertEmailPublisher.onAlert(AlertEmailPublisher.java:67)
>   at 
> org.apache.eagle.alert.engine.publisher.impl.AlertPublisherImpl.notifyAlert(AlertPublisherImpl.java:78)
>   at 
> org.apache.eagle.alert.engine.publisher.impl.AlertPublisherImpl.nextEvent(AlertPublisherImpl.java:65)
>   at 
> org.apache.eagle.alert.engine.runner.AlertPublisherBolt.execute(AlertPublisherBolt.java:68)
>   at 
> backtype.storm.daemon.executor$fn__3441$tuple_action_fn__3443.invoke(executor.clj:633)
>   at 
> backtype.storm.daemon.executor$mk_task_receiver$fn__3364.invoke(executor.clj:401)
>   at 
> backtype.storm.disruptor$clojure_handler$reify__1447.onEvent(disruptor.clj:58)
>   at 
> backtype.storm.utils.DisruptorQueue.consumeBatchToCursor(DisruptorQueue.java:125)
>   at 
> backtype.storm.utils.DisruptorQueue.consumeBatchWhenAvailable(DisruptorQueue.java:99)
>   at 
> backtype.storm.disruptor$consume_batch_when_available.invoke(disruptor.clj:80)
>   at 
> backtype.storm.daemon.executor$fn__3441$fn__3453$fn__3500.invoke(executor.clj:748)
>   at backtype.storm.util$async_loop$fn__464.invoke(util.clj:463)
>   at clojure.lang.AFn.run(AFn.java:24)
>   at java.lang.Thread.run(Thread.java:745)



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (EAGLE-491) AlertPublisherImpl dedup throws exception when dedup column is not configured

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/EAGLE-491?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15432182#comment-15432182
 ] 

ASF GitHub Bot commented on EAGLE-491:
--

Github user mizeng closed the pull request at:

https://github.com/apache/incubator-eagle/pull/376


> AlertPublisherImpl dedup throws exception when dedup column is not configured
> -
>
> Key: EAGLE-491
> URL: https://issues.apache.org/jira/browse/EAGLE-491
> Project: Eagle
>  Issue Type: Bug
>Affects Versions: v0.5.0
>Reporter: Edward Zhang
>Assignee: Zeng, Bryant
>Priority: Trivial
> Fix For: v0.5.0
>
>
> 2016-08-22 14:51:49,728 WARN [alertBolt1-evaluator_stage1] 
> runner.AlertBolt[127]: Spec Version [spec_version_1471902692329] of AlertBolt 
> is newer than Stream Event Version [null]!
> 2016-08-22 14:51:49,729 ERROR [Thread-45-alertPublishBolt] 
> impl.AlertPublisherImpl[80]: Fail invoking publisher's onAlert, continue 
> java.lang.NullPointerException
>   at 
> org.apache.eagle.alert.engine.publisher.impl.DefaultDeduplicator.dedup(DefaultDeduplicator.java:114)
>   at 
> org.apache.eagle.alert.engine.publisher.impl.AbstractPublishPlugin.dedup(AbstractPublishPlugin.java:68)
>   at 
> org.apache.eagle.alert.engine.publisher.impl.AlertEmailPublisher.onAlert(AlertEmailPublisher.java:67)
>   at 
> org.apache.eagle.alert.engine.publisher.impl.AlertPublisherImpl.notifyAlert(AlertPublisherImpl.java:78)
>   at 
> org.apache.eagle.alert.engine.publisher.impl.AlertPublisherImpl.nextEvent(AlertPublisherImpl.java:65)
>   at 
> org.apache.eagle.alert.engine.runner.AlertPublisherBolt.execute(AlertPublisherBolt.java:68)
>   at 
> backtype.storm.daemon.executor$fn__3441$tuple_action_fn__3443.invoke(executor.clj:633)
>   at 
> backtype.storm.daemon.executor$mk_task_receiver$fn__3364.invoke(executor.clj:401)
>   at 
> backtype.storm.disruptor$clojure_handler$reify__1447.onEvent(disruptor.clj:58)
>   at 
> backtype.storm.utils.DisruptorQueue.consumeBatchToCursor(DisruptorQueue.java:125)
>   at 
> backtype.storm.utils.DisruptorQueue.consumeBatchWhenAvailable(DisruptorQueue.java:99)
>   at 
> backtype.storm.disruptor$consume_batch_when_available.invoke(disruptor.clj:80)
>   at 
> backtype.storm.daemon.executor$fn__3441$fn__3453$fn__3500.invoke(executor.clj:748)
>   at backtype.storm.util$async_loop$fn__464.invoke(util.clj:463)
>   at clojure.lang.AFn.run(AFn.java:24)
>   at java.lang.Thread.run(Thread.java:745)



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[GitHub] incubator-eagle pull request #376: EAGLE-491: AlertPublisherImpl dedup throw...

[mizeng]

Github user mizeng closed the pull request at:

https://github.com/apache/incubator-eagle/pull/376


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[jira] [Commented] (EAGLE-492) Fix negative total execution time and make some code optimization for spark history job.

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/EAGLE-492?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15432172#comment-15432172
 ] 

ASF GitHub Bot commented on EAGLE-492:
--

Github user wujinhu commented on the issue:

https://github.com/apache/incubator-eagle/pull/375
  
Looks good to me. Please do more testing and merge then.


> Fix negative total execution time and make some code optimization for spark 
> history job.
> 
>
> Key: EAGLE-492
> URL: https://issues.apache.org/jira/browse/EAGLE-492
> Project: Eagle
>  Issue Type: Bug
>Affects Versions: v0.5.0
>Reporter: Huizhi Lu
>Assignee: Huizhi Lu
> Fix For: v0.5.0
>
>
> 1) negative total execution time
> It is because the app was killed or terminated unexpectedly without 
> completion. In the log file, it has only SparkListenerBlockManagerAdded 
> timestamp. "Timestamp":1470768362299
> And the code initiated executor starttime with this timestamp.
> No executor or app endtime available, so executor endtime = 0L.
> Finally, totalExecutionTime = -1470768362299
> The code logic does not consider the situation when executor or app Endtime = 
> 0L.
> Solution: If totalExecutionTime < 0L, set it = 0L. Stating the app is killed 
> and not completed.
> 2) Url builder bug for completed jobs



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[GitHub] incubator-eagle issue #375: [EAGLE-492] Fix negative total execution time an...

[wujinhu]

Github user wujinhu commented on the issue:

https://github.com/apache/incubator-eagle/pull/375
  
Looks good to me. Please do more testing and merge then.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[jira] [Updated] (EAGLE-493) Create alert metadata based on application stream sink configuration

[Edward Zhang (JIRA)]

 [ 
https://issues.apache.org/jira/browse/EAGLE-493?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Edward Zhang updated EAGLE-493:
---
Description: 
When I integrate hdfs audit log application with alert engine, it looks streams 
defined in applicationProvider.xml is not useful, because anyway we have to 
define data source, stream, policy and publishment using alert API. It is very 
difficult to convert streams in applicationProvider.xml to metadata used by 
alert engine. 

To achieve better user experience, we should create alert metadata 
automatically when application is installed based on application stream sink.

  was:
When I integrate hdfs audit log application with alert engine, it looks streams 
defined in applicationProvider.xml is not useful, because anyway we have to 
define data source, stream, policy and publishment using alert API. It is very 
difficult to convert streams in applicationProvider.xml to metadata used by 
alert engine. 

So my suggestion is to remove streams definition in applicationProvider.xml, 
and put related metadata scripts into docs in applicationProvider.xml.

By this way, stream definition becomes optional and application developer can 
write script for integrating application with alert engine.


> Create alert metadata based on application stream sink configuration
> 
>
> Key: EAGLE-493
> URL: https://issues.apache.org/jira/browse/EAGLE-493
> Project: Eagle
>  Issue Type: Improvement
>Affects Versions: v0.5.0
>Reporter: Edward Zhang
>Assignee: Edward Zhang
>Priority: Minor
> Fix For: v0.5.0
>
>
> When I integrate hdfs audit log application with alert engine, it looks 
> streams defined in applicationProvider.xml is not useful, because anyway we 
> have to define data source, stream, policy and publishment using alert API. 
> It is very difficult to convert streams in applicationProvider.xml to 
> metadata used by alert engine. 
> To achieve better user experience, we should create alert metadata 
> automatically when application is installed based on application stream sink.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (EAGLE-493) Create alert metadata based on application stream sink configuration

[Edward Zhang (JIRA)]

 [ 
https://issues.apache.org/jira/browse/EAGLE-493?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Edward Zhang updated EAGLE-493:
---
Summary: Create alert metadata based on application stream sink 
configuration  (was: Move stream definition in streams of provider.xml to docs)

> Create alert metadata based on application stream sink configuration
> 
>
> Key: EAGLE-493
> URL: https://issues.apache.org/jira/browse/EAGLE-493
> Project: Eagle
>  Issue Type: Improvement
>Affects Versions: v0.5.0
>Reporter: Edward Zhang
>Assignee: Edward Zhang
>Priority: Minor
> Fix For: v0.5.0
>
>
> When I integrate hdfs audit log application with alert engine, it looks 
> streams defined in applicationProvider.xml is not useful, because anyway we 
> have to define data source, stream, policy and publishment using alert API. 
> It is very difficult to convert streams in applicationProvider.xml to 
> metadata used by alert engine. 
> So my suggestion is to remove streams definition in applicationProvider.xml, 
> and put related metadata scripts into docs in applicationProvider.xml.
> By this way, stream definition becomes optional and application developer can 
> write script for integrating application with alert engine.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[GitHub] incubator-eagle pull request #376: EAGLE-491: AlertPublisherImpl dedup throw...

[mizeng]

GitHub user mizeng opened a pull request:

https://github.com/apache/incubator-eagle/pull/376

EAGLE-491: AlertPublisherImpl dedup throws exception when dedup colum…

…n is not configured

You can merge this pull request into a Git repository by running:

$ git pull https://github.com/mizeng/incubator-eagle EAGLE-491

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/incubator-eagle/pull/376.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #376


commit 065f1c230249555f9eb4dbdf595200da4e863831
Author: mizeng 
Date:   2016-08-23T01:52:24Z

EAGLE-491: AlertPublisherImpl dedup throws exception when dedup column is 
not configured




---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[jira] [Commented] (EAGLE-491) AlertPublisherImpl dedup throws exception when dedup column is not configured

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/EAGLE-491?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15431979#comment-15431979
 ] 

ASF GitHub Bot commented on EAGLE-491:
--

GitHub user mizeng opened a pull request:

https://github.com/apache/incubator-eagle/pull/376

EAGLE-491: AlertPublisherImpl dedup throws exception when dedup colum…

…n is not configured

You can merge this pull request into a Git repository by running:

$ git pull https://github.com/mizeng/incubator-eagle EAGLE-491

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/incubator-eagle/pull/376.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #376


commit 065f1c230249555f9eb4dbdf595200da4e863831
Author: mizeng 
Date:   2016-08-23T01:52:24Z

EAGLE-491: AlertPublisherImpl dedup throws exception when dedup column is 
not configured




> AlertPublisherImpl dedup throws exception when dedup column is not configured
> -
>
> Key: EAGLE-491
> URL: https://issues.apache.org/jira/browse/EAGLE-491
> Project: Eagle
>  Issue Type: Bug
>Affects Versions: v0.5.0
>Reporter: Edward Zhang
>Assignee: Zeng, Bryant
>Priority: Trivial
> Fix For: v0.5.0
>
>
> 2016-08-22 14:51:49,728 WARN [alertBolt1-evaluator_stage1] 
> runner.AlertBolt[127]: Spec Version [spec_version_1471902692329] of AlertBolt 
> is newer than Stream Event Version [null]!
> 2016-08-22 14:51:49,729 ERROR [Thread-45-alertPublishBolt] 
> impl.AlertPublisherImpl[80]: Fail invoking publisher's onAlert, continue 
> java.lang.NullPointerException
>   at 
> org.apache.eagle.alert.engine.publisher.impl.DefaultDeduplicator.dedup(DefaultDeduplicator.java:114)
>   at 
> org.apache.eagle.alert.engine.publisher.impl.AbstractPublishPlugin.dedup(AbstractPublishPlugin.java:68)
>   at 
> org.apache.eagle.alert.engine.publisher.impl.AlertEmailPublisher.onAlert(AlertEmailPublisher.java:67)
>   at 
> org.apache.eagle.alert.engine.publisher.impl.AlertPublisherImpl.notifyAlert(AlertPublisherImpl.java:78)
>   at 
> org.apache.eagle.alert.engine.publisher.impl.AlertPublisherImpl.nextEvent(AlertPublisherImpl.java:65)
>   at 
> org.apache.eagle.alert.engine.runner.AlertPublisherBolt.execute(AlertPublisherBolt.java:68)
>   at 
> backtype.storm.daemon.executor$fn__3441$tuple_action_fn__3443.invoke(executor.clj:633)
>   at 
> backtype.storm.daemon.executor$mk_task_receiver$fn__3364.invoke(executor.clj:401)
>   at 
> backtype.storm.disruptor$clojure_handler$reify__1447.onEvent(disruptor.clj:58)
>   at 
> backtype.storm.utils.DisruptorQueue.consumeBatchToCursor(DisruptorQueue.java:125)
>   at 
> backtype.storm.utils.DisruptorQueue.consumeBatchWhenAvailable(DisruptorQueue.java:99)
>   at 
> backtype.storm.disruptor$consume_batch_when_available.invoke(disruptor.clj:80)
>   at 
> backtype.storm.daemon.executor$fn__3441$fn__3453$fn__3500.invoke(executor.clj:748)
>   at backtype.storm.util$async_loop$fn__464.invoke(util.clj:463)
>   at clojure.lang.AFn.run(AFn.java:24)
>   at java.lang.Thread.run(Thread.java:745)



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (EAGLE-492) Fix negative total execution time and make some code optimization for spark history job.

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/EAGLE-492?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15431937#comment-15431937
 ] 

ASF GitHub Bot commented on EAGLE-492:
--

GitHub user pkuwm opened a pull request:

https://github.com/apache/incubator-eagle/pull/375

[EAGLE-492] Fix negative total execution time and make some code opti…

…mization for spark history job.

You can merge this pull request into a Git repository by running:

$ git pull https://github.com/pkuwm/incubator-eagle EAGLE-492

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/incubator-eagle/pull/375.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #375


commit cbf4cf7585f66d9cdb5cbd26b9832b529d2852fa
Author: pkuwm 
Date:   2016-08-23T01:21:31Z

[EAGLE-492] Fix negative total execution time and make some code 
optimization for spark history job.




> Fix negative total execution time and make some code optimization for spark 
> history job.
> 
>
> Key: EAGLE-492
> URL: https://issues.apache.org/jira/browse/EAGLE-492
> Project: Eagle
>  Issue Type: Bug
>Affects Versions: v0.5.0
>Reporter: Huizhi Lu
>Assignee: Huizhi Lu
> Fix For: v0.5.0
>
>
> 1) negative total execution time
> It is because the app was killed or terminated unexpectedly without 
> completion. In the log file, it has only SparkListenerBlockManagerAdded 
> timestamp. "Timestamp":1470768362299
> And the code initiated executor starttime with this timestamp.
> No executor or app endtime available, so executor endtime = 0L.
> Finally, totalExecutionTime = -1470768362299
> The code logic does not consider the situation when executor or app Endtime = 
> 0L.
> Solution: If totalExecutionTime < 0L, set it = 0L. Stating the app is killed 
> and not completed.
> 2) Url builder bug for completed jobs



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (EAGLE-492) Fix negative total execution time and make some code optimization for spark history job.

[Huizhi Lu (JIRA)]

 [ 
https://issues.apache.org/jira/browse/EAGLE-492?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Huizhi Lu updated EAGLE-492:

Description: 
1) negative total execution time

It is because the app was killed or terminated unexpectedly without completion. 
In the log file, it has only SparkListenerBlockManagerAdded timestamp. 
"Timestamp":1470768362299

And the code initiated executor starttime with this timestamp.
No executor or app endtime available, so executor endtime = 0L.
Finally, totalExecutionTime = -1470768362299

The code logic does not consider the situation when executor or app Endtime = 
0L.

Solution: If totalExecutionTime < 0L, set it = 0L. Stating the app is killed 
and not completed.

2) Url builder bug for completed jobs


  was:
1) 
It is because the app was killed or terminated unexpectedly without completion. 
In the log file, it has only SparkListenerBlockManagerAdded timestamp. 
"Timestamp":1470768362299

And the code initiated executor starttime with this timestamp.
No executor or app endtime available, so executor endtime = 0L.
Finally, totalExecutionTime = -1470768362299

The code logic does not consider the situation when executor or app Endtime = 
0L.

Solution: If totalExecutionTime < 0L, set it = 0L. Stating the app is killed 
and not completed.


> Fix negative total execution time and make some code optimization for spark 
> history job.
> 
>
> Key: EAGLE-492
> URL: https://issues.apache.org/jira/browse/EAGLE-492
> Project: Eagle
>  Issue Type: Bug
>Affects Versions: v0.5.0
>Reporter: Huizhi Lu
>Assignee: Huizhi Lu
> Fix For: v0.5.0
>
>
> 1) negative total execution time
> It is because the app was killed or terminated unexpectedly without 
> completion. In the log file, it has only SparkListenerBlockManagerAdded 
> timestamp. "Timestamp":1470768362299
> And the code initiated executor starttime with this timestamp.
> No executor or app endtime available, so executor endtime = 0L.
> Finally, totalExecutionTime = -1470768362299
> The code logic does not consider the situation when executor or app Endtime = 
> 0L.
> Solution: If totalExecutionTime < 0L, set it = 0L. Stating the app is killed 
> and not completed.
> 2) Url builder bug for completed jobs



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (EAGLE-492) Fix negative total execution time and make some code optimization for spark history job.

[Huizhi Lu (JIRA)]

 [ 
https://issues.apache.org/jira/browse/EAGLE-492?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Huizhi Lu updated EAGLE-492:

Affects Version/s: v0.5.0
Fix Version/s: v0.5.0

> Fix negative total execution time and make some code optimization for spark 
> history job.
> 
>
> Key: EAGLE-492
> URL: https://issues.apache.org/jira/browse/EAGLE-492
> Project: Eagle
>  Issue Type: Bug
>Affects Versions: v0.5.0
>Reporter: Huizhi Lu
>Assignee: Huizhi Lu
> Fix For: v0.5.0
>
>
> 1) negative total execution time
> It is because the app was killed or terminated unexpectedly without 
> completion. In the log file, it has only SparkListenerBlockManagerAdded 
> timestamp. "Timestamp":1470768362299
> And the code initiated executor starttime with this timestamp.
> No executor or app endtime available, so executor endtime = 0L.
> Finally, totalExecutionTime = -1470768362299
> The code logic does not consider the situation when executor or app Endtime = 
> 0L.
> Solution: If totalExecutionTime < 0L, set it = 0L. Stating the app is killed 
> and not completed.
> 2) Url builder bug for completed jobs



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Created] (EAGLE-493) Move stream definition in streams of provider.xml to docs

[Edward Zhang (JIRA)]

Edward Zhang created EAGLE-493:
--

 Summary: Move stream definition in streams of provider.xml to docs
 Key: EAGLE-493
 URL: https://issues.apache.org/jira/browse/EAGLE-493
 Project: Eagle
  Issue Type: Improvement
Affects Versions: v0.5.0
Reporter: Edward Zhang
Assignee: Edward Zhang
Priority: Minor
 Fix For: v0.5.0


When I integrate hdfs audit log application with alert engine, it looks streams 
defined in applicationProvider.xml is not useful, because anyway we have to 
define data source, stream, policy and publishment using alert API. It is very 
difficult to convert streams in applicationProvider.xml to metadata used by 
alert engine. 

So my suggestion is to remove streams definition in applicationProvider.xml, 
and put related metadata scripts into docs in applicationProvider.xml.

By this way, stream definition becomes optional and application developer can 
write script for integrating application with alert engine.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Created] (EAGLE-492) Fix negative total execution time and make some code optimization for spark history job.

[Huizhi Lu (JIRA)]

Huizhi Lu created EAGLE-492:
---

 Summary: Fix negative total execution time and make some code 
optimization for spark history job.
 Key: EAGLE-492
 URL: https://issues.apache.org/jira/browse/EAGLE-492
 Project: Eagle
  Issue Type: Bug
Reporter: Huizhi Lu
Assignee: Huizhi Lu


It is because the app was killed or terminated unexpectedly without completion. 
In the log file, it has only SparkListenerBlockManagerAdded timestamp. 
"Timestamp":1470768362299

And the code initiated executor starttime with this timestamp.
No executor or app endtime available, so executor endtime = 0L.
Finally, totalExecutionTime = -1470768362299

The code logic does not consider the situation when executor or app Endtime = 
0L.

Solution: If totalExecutionTime < 0L, set it = 0L. Stating the app is killed 
and not completed.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Assigned] (EAGLE-491) AlertPublisherImpl dedup throws exception when dedup column is not configured

[Su Ralph (JIRA)]

 [ 
https://issues.apache.org/jira/browse/EAGLE-491?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Su Ralph reassigned EAGLE-491:
--

Assignee: Zeng, Bryant  (was: Edward Zhang)

> AlertPublisherImpl dedup throws exception when dedup column is not configured
> -
>
> Key: EAGLE-491
> URL: https://issues.apache.org/jira/browse/EAGLE-491
> Project: Eagle
>  Issue Type: Bug
>Affects Versions: v0.5.0
>Reporter: Edward Zhang
>Assignee: Zeng, Bryant
>Priority: Trivial
> Fix For: v0.5.0
>
>
> 2016-08-22 14:51:49,728 WARN [alertBolt1-evaluator_stage1] 
> runner.AlertBolt[127]: Spec Version [spec_version_1471902692329] of AlertBolt 
> is newer than Stream Event Version [null]!
> 2016-08-22 14:51:49,729 ERROR [Thread-45-alertPublishBolt] 
> impl.AlertPublisherImpl[80]: Fail invoking publisher's onAlert, continue 
> java.lang.NullPointerException
>   at 
> org.apache.eagle.alert.engine.publisher.impl.DefaultDeduplicator.dedup(DefaultDeduplicator.java:114)
>   at 
> org.apache.eagle.alert.engine.publisher.impl.AbstractPublishPlugin.dedup(AbstractPublishPlugin.java:68)
>   at 
> org.apache.eagle.alert.engine.publisher.impl.AlertEmailPublisher.onAlert(AlertEmailPublisher.java:67)
>   at 
> org.apache.eagle.alert.engine.publisher.impl.AlertPublisherImpl.notifyAlert(AlertPublisherImpl.java:78)
>   at 
> org.apache.eagle.alert.engine.publisher.impl.AlertPublisherImpl.nextEvent(AlertPublisherImpl.java:65)
>   at 
> org.apache.eagle.alert.engine.runner.AlertPublisherBolt.execute(AlertPublisherBolt.java:68)
>   at 
> backtype.storm.daemon.executor$fn__3441$tuple_action_fn__3443.invoke(executor.clj:633)
>   at 
> backtype.storm.daemon.executor$mk_task_receiver$fn__3364.invoke(executor.clj:401)
>   at 
> backtype.storm.disruptor$clojure_handler$reify__1447.onEvent(disruptor.clj:58)
>   at 
> backtype.storm.utils.DisruptorQueue.consumeBatchToCursor(DisruptorQueue.java:125)
>   at 
> backtype.storm.utils.DisruptorQueue.consumeBatchWhenAvailable(DisruptorQueue.java:99)
>   at 
> backtype.storm.disruptor$consume_batch_when_available.invoke(disruptor.clj:80)
>   at 
> backtype.storm.daemon.executor$fn__3441$fn__3453$fn__3500.invoke(executor.clj:748)
>   at backtype.storm.util$async_loop$fn__464.invoke(util.clj:463)
>   at clojure.lang.AFn.run(AFn.java:24)
>   at java.lang.Thread.run(Thread.java:745)



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (EAGLE-476) Outdated HBase audit log parser

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/EAGLE-476?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15431793#comment-15431793
 ] 

ASF GitHub Bot commented on EAGLE-476:
--

Github user peterkim95 closed the pull request at:

https://github.com/apache/incubator-eagle/pull/363


> Outdated HBase audit log parser
> ---
>
> Key: EAGLE-476
> URL: https://issues.apache.org/jira/browse/EAGLE-476
> Project: Eagle
>  Issue Type: Bug
>Reporter: Peter Kim
>
> The parsing logic for HBase audit logs (security logs) fails for some of the 
> newly formatted hbase audit logs. Obviously, this can cause the eagle service 
> to overlook these log lines, and fail to generate alerts, which can have a 
> severe outcome in terms of security. For example:
> 2016-08-17 14:09:52,232 TRACE 
> SecurityLogger.org.apache.hadoop.hbase.security.access.AccessController: 
> Access allowed for user petkim; reason: Table permission granted; remote 
> address: /127.0.0.1; request: flush; context: (user=petkim, scope=hbase:meta, 
> params=[table=hbase:meta],action=ADMIN)
> 2016-08-17 14:04:27,042 TRACE 
> SecurityLogger.org.apache.hadoop.hbase.security.access.AccessController: 
> Access allowed for user petkim; reason: All users allowed; remote address: 
> /111.1.1.1; request: scan; context: (user=petkim, scope=hbase:meta, 
> family=info, params=[table=hbase:meta,family=info],action=READ)
> These log lines are not parsed correctly as the fields that the current regex 
> matches are static. The first log does not have the field "family" and the 
> second one has a new field named "params". So, the parsing logic fails here.
> To fix this and ensure scalability (reliable no matter how many fields are 
> omitted or added), I will extend the current parsing logic to more reliable.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (EAGLE-476) Outdated HBase audit log parser

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/EAGLE-476?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15431766#comment-15431766
 ] 

ASF GitHub Bot commented on EAGLE-476:
--

Github user yonzhang commented on the issue:

https://github.com/apache/incubator-eagle/pull/363
  
great fix and merged


> Outdated HBase audit log parser
> ---
>
> Key: EAGLE-476
> URL: https://issues.apache.org/jira/browse/EAGLE-476
> Project: Eagle
>  Issue Type: Bug
>Reporter: Peter Kim
>
> The parsing logic for HBase audit logs (security logs) fails for some of the 
> newly formatted hbase audit logs. Obviously, this can cause the eagle service 
> to overlook these log lines, and fail to generate alerts, which can have a 
> severe outcome in terms of security. For example:
> 2016-08-17 14:09:52,232 TRACE 
> SecurityLogger.org.apache.hadoop.hbase.security.access.AccessController: 
> Access allowed for user petkim; reason: Table permission granted; remote 
> address: /127.0.0.1; request: flush; context: (user=petkim, scope=hbase:meta, 
> params=[table=hbase:meta],action=ADMIN)
> 2016-08-17 14:04:27,042 TRACE 
> SecurityLogger.org.apache.hadoop.hbase.security.access.AccessController: 
> Access allowed for user petkim; reason: All users allowed; remote address: 
> /111.1.1.1; request: scan; context: (user=petkim, scope=hbase:meta, 
> family=info, params=[table=hbase:meta,family=info],action=READ)
> These log lines are not parsed correctly as the fields that the current regex 
> matches are static. The first log does not have the field "family" and the 
> second one has a new field named "params". So, the parsing logic fails here.
> To fix this and ensure scalability (reliable no matter how many fields are 
> omitted or added), I will extend the current parsing logic to more reliable.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[GitHub] incubator-eagle issue #363: EAGLE-476: outdated hbase audit log parser

[yonzhang]

Github user yonzhang commented on the issue:

https://github.com/apache/incubator-eagle/pull/363
  
great fix and merged


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[jira] [Created] (EAGLE-491) AlertPublisherImpl dedup throws exception when dedup column is not configured

[Edward Zhang (JIRA)]

Edward Zhang created EAGLE-491:
--

 Summary: AlertPublisherImpl dedup throws exception when dedup 
column is not configured
 Key: EAGLE-491
 URL: https://issues.apache.org/jira/browse/EAGLE-491
 Project: Eagle
  Issue Type: Bug
Affects Versions: v0.5.0
Reporter: Edward Zhang
Assignee: Edward Zhang
Priority: Trivial
 Fix For: v0.5.0


2016-08-22 14:51:49,728 WARN [alertBolt1-evaluator_stage1] 
runner.AlertBolt[127]: Spec Version [spec_version_1471902692329] of AlertBolt 
is newer than Stream Event Version [null]!
2016-08-22 14:51:49,729 ERROR [Thread-45-alertPublishBolt] 
impl.AlertPublisherImpl[80]: Fail invoking publisher's onAlert, continue 
java.lang.NullPointerException
at 
org.apache.eagle.alert.engine.publisher.impl.DefaultDeduplicator.dedup(DefaultDeduplicator.java:114)
at 
org.apache.eagle.alert.engine.publisher.impl.AbstractPublishPlugin.dedup(AbstractPublishPlugin.java:68)
at 
org.apache.eagle.alert.engine.publisher.impl.AlertEmailPublisher.onAlert(AlertEmailPublisher.java:67)
at 
org.apache.eagle.alert.engine.publisher.impl.AlertPublisherImpl.notifyAlert(AlertPublisherImpl.java:78)
at 
org.apache.eagle.alert.engine.publisher.impl.AlertPublisherImpl.nextEvent(AlertPublisherImpl.java:65)
at 
org.apache.eagle.alert.engine.runner.AlertPublisherBolt.execute(AlertPublisherBolt.java:68)
at 
backtype.storm.daemon.executor$fn__3441$tuple_action_fn__3443.invoke(executor.clj:633)
at 
backtype.storm.daemon.executor$mk_task_receiver$fn__3364.invoke(executor.clj:401)
at 
backtype.storm.disruptor$clojure_handler$reify__1447.onEvent(disruptor.clj:58)
at 
backtype.storm.utils.DisruptorQueue.consumeBatchToCursor(DisruptorQueue.java:125)
at 
backtype.storm.utils.DisruptorQueue.consumeBatchWhenAvailable(DisruptorQueue.java:99)
at 
backtype.storm.disruptor$consume_batch_when_available.invoke(disruptor.clj:80)
at 
backtype.storm.daemon.executor$fn__3441$fn__3453$fn__3500.invoke(executor.clj:748)
at backtype.storm.util$async_loop$fn__464.invoke(util.clj:463)
at clojure.lang.AFn.run(AFn.java:24)
at java.lang.Thread.run(Thread.java:745)



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (EAGLE-476) Outdated HBase audit log parser

[Peter Kim (JIRA)]

[ 
https://issues.apache.org/jira/browse/EAGLE-476?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15431416#comment-15431416
 ] 

Peter Kim commented on EAGLE-476:
-

Yes it is. I've made sure to test it on the previous log format, and also 
re-designed it to be more scalable so that it can parse through future log 
formats that might have different fields

> Outdated HBase audit log parser
> ---
>
> Key: EAGLE-476
> URL: https://issues.apache.org/jira/browse/EAGLE-476
> Project: Eagle
>  Issue Type: Bug
>Reporter: Peter Kim
>
> The parsing logic for HBase audit logs (security logs) fails for some of the 
> newly formatted hbase audit logs. Obviously, this can cause the eagle service 
> to overlook these log lines, and fail to generate alerts, which can have a 
> severe outcome in terms of security. For example:
> 2016-08-17 14:09:52,232 TRACE 
> SecurityLogger.org.apache.hadoop.hbase.security.access.AccessController: 
> Access allowed for user petkim; reason: Table permission granted; remote 
> address: /127.0.0.1; request: flush; context: (user=petkim, scope=hbase:meta, 
> params=[table=hbase:meta],action=ADMIN)
> 2016-08-17 14:04:27,042 TRACE 
> SecurityLogger.org.apache.hadoop.hbase.security.access.AccessController: 
> Access allowed for user petkim; reason: All users allowed; remote address: 
> /111.1.1.1; request: scan; context: (user=petkim, scope=hbase:meta, 
> family=info, params=[table=hbase:meta,family=info],action=READ)
> These log lines are not parsed correctly as the fields that the current regex 
> matches are static. The first log does not have the field "family" and the 
> second one has a new field named "params". So, the parsing logic fails here.
> To fix this and ensure scalability (reliable no matter how many fields are 
> omitted or added), I will extend the current parsing logic to more reliable.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Created] (EAGLE-490) KafkaStreamSink should not assume fixed field user to do partition

[Edward Zhang (JIRA)]

Edward Zhang created EAGLE-490:
--

 Summary: KafkaStreamSink should not assume fixed field user to do 
partition
 Key: EAGLE-490
 URL: https://issues.apache.org/jira/browse/EAGLE-490
 Project: Eagle
  Issue Type: Improvement
Affects Versions: v0.5.0
Reporter: Edward Zhang
Assignee: Edward Zhang
 Fix For: v0.5.0


KafkaStreamSink assumes user as the field to do partition. This should be 
configurable in application metadata file.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (EAGLE-487) use guice module better in app framework

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/EAGLE-487?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15431233#comment-15431233
 ] 

ASF GitHub Bot commented on EAGLE-487:
--

Github user yonzhang closed the pull request at:

https://github.com/apache/incubator-eagle/pull/368


> use guice module better in app framework
> 
>
> Key: EAGLE-487
> URL: https://issues.apache.org/jira/browse/EAGLE-487
> Project: Eagle
>  Issue Type: Improvement
>Affects Versions: v0.5.0
>Reporter: Edward Zhang
>Assignee: Edward Zhang
> Fix For: v0.5.0
>
>
> There are multiple issues with using guice module in app framework.
> 1. ApplicationProviderServiceImpl is instantiated twice
> 2. the interface between application and eagle server is not clear. It is not 
> correct to only let application to register submodules,, instead it should 
> includes some state inherited from eagle server, for example metadata storage 
> type.
> 3. not necessary to use module scope, it is pretty over-engineering when 
> application only want to register some module.
> 4. MemoryMetadataStore should use singleton



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[GitHub] incubator-eagle issue #368: use guice module better in app framework

[yonzhang]

Github user yonzhang commented on the issue:

https://github.com/apache/incubator-eagle/pull/368
  
merged 


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] incubator-eagle pull request #368: use guice module better in app framework

[yonzhang]

Github user yonzhang closed the pull request at:

https://github.com/apache/incubator-eagle/pull/368


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[jira] [Commented] (EAGLE-451) oozie ingestion+alert engine preview

[Edward Zhang (JIRA)]

[ 
https://issues.apache.org/jira/browse/EAGLE-451?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15431226#comment-15431226
 ] 

Edward Zhang commented on EAGLE-451:


yes, they are the same, I have close #448

> oozie ingestion+alert engine preview
> 
>
> Key: EAGLE-451
> URL: https://issues.apache.org/jira/browse/EAGLE-451
> Project: Eagle
>  Issue Type: Improvement
>Reporter: Lingang Deng
>Assignee: Lingang Deng
>Priority: Trivial
>




--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Closed] (EAGLE-448) convert eagle-oozie app to use new app framework

[Edward Zhang (JIRA)]

 [ 
https://issues.apache.org/jira/browse/EAGLE-448?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Edward Zhang closed EAGLE-448.
--
Resolution: Duplicate

same to #451

> convert eagle-oozie app to use new app framework
> 
>
> Key: EAGLE-448
> URL: https://issues.apache.org/jira/browse/EAGLE-448
> Project: Eagle
>  Issue Type: Improvement
>Affects Versions: v0.5.0
>Reporter: Edward Zhang
>Assignee: Edward Zhang
> Fix For: v0.5.0
>
>
> eagle-oozie app should use new app framework to manage its lifecycle



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Re: [Discussion] How to refactor EagleConfigFactory

[Edward Zhang]

EagleConfigFactory, LogReader, LogWriter are developed for reading/writing
data to HBase only at the beginning of Eagle open source. At that time,
HBase acts as both metadata storage and log/metric data storage. But some
feedback from community is that Eagle sometimes is used for monitoring
where HBase is not ready. There is a requirement that Eagle can be run
without HBase dependency. That is why in Eagle 0.5, all metadata are stored
in relational database or mongodb which is common to most users.

But for application data, for example metrics or logs, it is still not
decided yet which storage is final choice. But for job related performance
monitoring, all data including metrics and logs are stored in HBase.  Some
other storage is also evaluated for example druid for metrics and
elasticsearch for log. But it is not finalized.

Difference between log and metric is that metric stores numeric data with
dimensions, while log stores any type of data. Some storage like TSDB,
Druid they made optimization for metric storage.

Thanks
Edward

On Mon, Aug 22, 2016 at 2:08 AM, Chang Chen  wrote:

> What make me confused is EagleConfigFactory directly depends on HBase. But
> there are 2 interfaces for persisting data, one is DataStorage, the other
> are LogReader and LogWriter.  Which one will be removed?
>
> I also noticed that eagle 0.5 architecture (
> https://cwiki.apache.org/confluence/display/EAG/Architecture+of+Eagle+0.5)
> was updated. What are the difference between "log" and "metric"?
>
> Thanks
> Chang
>
> On Wednesday, August 17, 2016, Liangfei.Su  wrote:
>
> > On test scenario with different config need for differest, i simply do
> > ConfigFactory.invalidateCaches() in setup().
> >
> >
> > On Wed, Aug 17, 2016 at 9:43 AM, Edward Zhang  > >
> > wrote:
> >
> > > Sorry I mean "remove" :-)
> > >
> > > Thanks
> > > Edward
> > >
> > > On Tue, Aug 16, 2016 at 6:35 PM, Chang Chen  > > wrote:
> > >
> > > > Hi Edward
> > > >
> > > > Does remote mean run TestEagleConfig in different JVM?
> > > >
> > > > Thanks
> > > > Chang
> > > >
> > > > On Wednesday, August 17, 2016, Edward Zhang  > >
> > > > wrote:
> > > >
> > > > > TestEagleConfig is not a valid test case because it assumes that it
> > can
> > > > > initialize EagleConfigFactory, but when we execute multiple test
> > cases,
> > > > > this may not work as EagleConfigFactory may already be initialized
> > > before
> > > > > this test case is executed. So setting "config.resource" will not
> > have
> > > > any
> > > > > effect.
> > > > >
> > > > > I suggest we remote TestEagleConfig, and also going forward we can
> > need
> > > > > refactor EagleConfigFactor to be not a singleton but use
> > > > > com.typesafe.config.Config as constructor parameter.
> > > > >
> > > > > Thanks
> > > > > Edward
> > > > >
> > > > > On Tue, Aug 16, 2016 at 4:26 AM, Chang Chen  > 
> > > > > > wrote:
> > > > >
> > > > > > Hi Guys
> > > > > >
> > > > > > In the middle of writing UT for Eagle, we found it is difficult
> to
> > > mock
> > > > > >  EagleConfigFactory, since it has static method. The current
> > > workaround
> > > > > is
> > > > > > introducing powermock, so that we can set manager to mock object
> ,
> > > i.e.
> > > > > >
> > > > > >Whitebox.setInternalState(EagleConfigFactory.class,
> "manager",
> > > > > > eagleConfigFactory);
> > > > > >
> > > > > > This is OK in most cases, but TestEagleConfig.testInit could be
> > > failed,
> > > > > > because manager only initialized at first time and
> > > >  EagleConfigFactory's
> > > > > > Constructor is private:
> > > > > >
> > > > > >private static EagleConfigFactory manager = new
> > > > EagleConfigFactory();
> > > > > >
> > > > > > EagleConfigFactory actually has two roles, one is factory and
> > another
> > > > is
> > > > > > configuration. I propose refactoring EagleConfigFactory and
> > > distinguish
> > > > > > these two roles:
> > > > > >
> > > > > > public class EagleConfigImpl implements EagleConfig{
> > > > > > }
> > > > > >
> > > > > > public class EagleConfigFactory {
> > > > > >   public static EagleConfig load(){}
> > > > > >   public static void reset(EagleConfig config){
> > > > > > if (config == null) {
> > > > > >   //default
> > > > > > }else{
> > > > > > }
> > > > > >   }
> > > > > > }
> > > > > >
> > > > > > However doing so, we need call EagleConfigFactory.reset at the
> > > > beginning
> > > > > of
> > > > > >  eagle start.
> > > > > >
> > > > > > Any ideas?
> > > > > >
> > > > > > Thanks
> > > > > > Chang
> > > > > >
> > > > >
> > > >
> > >
> >
>

[jira] [Commented] (EAGLE-411) Refactor hbase related unit test cases with mockito and mini-hbase

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/EAGLE-411?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15430817#comment-15430817
 ] 

ASF GitHub Bot commented on EAGLE-411:
--

Github user baibaichen commented on the issue:

https://github.com/apache/incubator-eagle/pull/374
  
@haoch , I think we need write a new test class instead of modifying 
TestHBaseLogReader2.


> Refactor hbase related unit test cases with mockito and mini-hbase
> --
>
> Key: EAGLE-411
> URL: https://issues.apache.org/jira/browse/EAGLE-411
> Project: Eagle
>  Issue Type: Improvement
>Affects Versions: v0.3.0, v0.4.0
>Reporter: Hao Chen
>Assignee: Chang chen
>  Labels: unit-test
> Fix For: v0.5.0
>
>
> Current implementation is really not stable which cause unit test like 
> TestHBaseLogReader2 unable to pass in meanful time, we need a better version.
> https://github.com/apache/incubator-eagle/blob/master/eagle-core/eagle-embed/eagle-embed-hbase/src/main/java/org/apache/eagle/service/hbase/EmbeddedHbase.java
> http://blog.cloudera.com/blog/2013/09/how-to-test-hbase-applications-using-popular-tools/



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (EAGLE-411) Refactor hbase related unit test cases with mockito and mini-hbase

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/EAGLE-411?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15430506#comment-15430506
 ] 

ASF GitHub Bot commented on EAGLE-411:
--

Github user chitin closed the pull request at:

https://github.com/apache/incubator-eagle/pull/373


> Refactor hbase related unit test cases with mockito and mini-hbase
> --
>
> Key: EAGLE-411
> URL: https://issues.apache.org/jira/browse/EAGLE-411
> Project: Eagle
>  Issue Type: Improvement
>Affects Versions: v0.3.0, v0.4.0
>Reporter: Hao Chen
>Assignee: Chang chen
>  Labels: unit-test
> Fix For: v0.5.0
>
>
> Current implementation is really not stable which cause unit test like 
> TestHBaseLogReader2 unable to pass in meanful time, we need a better version.
> https://github.com/apache/incubator-eagle/blob/master/eagle-core/eagle-embed/eagle-embed-hbase/src/main/java/org/apache/eagle/service/hbase/EmbeddedHbase.java
> http://blog.cloudera.com/blog/2013/09/how-to-test-hbase-applications-using-popular-tools/



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[GitHub] incubator-eagle pull request #373: EAGLE-411 Refactor hbase related unit tes...

[chitin]

Github user chitin closed the pull request at:

https://github.com/apache/incubator-eagle/pull/373


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[jira] [Resolved] (EAGLE-484) StreamRouterSpec class should override equals method

[JiJun Tang (JIRA)]

 [ 
https://issues.apache.org/jira/browse/EAGLE-484?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

JiJun Tang resolved EAGLE-484.
--
Resolution: Fixed

> StreamRouterSpec class should override equals method 
> -
>
> Key: EAGLE-484
> URL: https://issues.apache.org/jira/browse/EAGLE-484
> Project: Eagle
>  Issue Type: Bug
>Affects Versions: v0.5.0
>Reporter: JiJun Tang
>Assignee: JiJun Tang
> Fix For: v0.5.0
>
>
> {code:title=StreamRouterBolt.java|borderStyle=solid}
> if(!newSRS.get(s).equals(cachedSRS.get(s))){ 
> // this compare will get wrong result ,so will return wrong 
> modifiedRouterSpecs
>  modifiedRouterSpecs.add(newSRS.get(s));
>  }
> {code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (EAGLE-411) Refactor hbase related unit test cases with mockito and mini-hbase

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/EAGLE-411?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15430409#comment-15430409
 ] 

ASF GitHub Bot commented on EAGLE-411:
--

GitHub user chitin opened a pull request:

https://github.com/apache/incubator-eagle/pull/373

EAGLE-411 Refactor hbase related unit test cases with mockito

EAGLE-411 Refactor hbase related unit test cases with mockito
- Mock the action of HBase reading, mock the meanful result of 
scanner.next(), unit test would not be enslaved to HBase.

https://issues.apache.org/jira/browse/EAGLE-411.

You can merge this pull request into a Git repository by running:

$ git pull https://github.com/chitin/incubator-eagle EAGLE411-2

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/incubator-eagle/pull/373.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #373


commit ad536489eda8adc694e9d27087d971ba8949296d
Author: chitin 
Date:   2016-08-22T09:23:00Z

EAGLE-411 Refactor hbase related unit test cases with mockito

- Mock the action of HBase reading, mock the meanful result of 
scanner.next(), unit test would not be enslaved to HBase.
https://issues.apache.org/jira/browse/EAGLE-411.




> Refactor hbase related unit test cases with mockito and mini-hbase
> --
>
> Key: EAGLE-411
> URL: https://issues.apache.org/jira/browse/EAGLE-411
> Project: Eagle
>  Issue Type: Improvement
>Affects Versions: v0.3.0, v0.4.0
>Reporter: Hao Chen
>Assignee: Chang chen
>  Labels: unit-test
> Fix For: v0.5.0
>
>
> Current implementation is really not stable which cause unit test like 
> TestHBaseLogReader2 unable to pass in meanful time, we need a better version.
> https://github.com/apache/incubator-eagle/blob/master/eagle-core/eagle-embed/eagle-embed-hbase/src/main/java/org/apache/eagle/service/hbase/EmbeddedHbase.java
> http://blog.cloudera.com/blog/2013/09/how-to-test-hbase-applications-using-popular-tools/



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[GitHub] incubator-eagle pull request #373: EAGLE-411 Refactor hbase related unit tes...

[chitin]

GitHub user chitin opened a pull request:

https://github.com/apache/incubator-eagle/pull/373

EAGLE-411 Refactor hbase related unit test cases with mockito

EAGLE-411 Refactor hbase related unit test cases with mockito
- Mock the action of HBase reading, mock the meanful result of 
scanner.next(), unit test would not be enslaved to HBase.

https://issues.apache.org/jira/browse/EAGLE-411.

You can merge this pull request into a Git repository by running:

$ git pull https://github.com/chitin/incubator-eagle EAGLE411-2

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/incubator-eagle/pull/373.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #373


commit ad536489eda8adc694e9d27087d971ba8949296d
Author: chitin 
Date:   2016-08-22T09:23:00Z

EAGLE-411 Refactor hbase related unit test cases with mockito

- Mock the action of HBase reading, mock the meanful result of 
scanner.next(), unit test would not be enslaved to HBase.
https://issues.apache.org/jira/browse/EAGLE-411.




---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

Re: [Discussion] How to refactor EagleConfigFactory

[Chang Chen]

What make me confused is EagleConfigFactory directly depends on HBase. But
there are 2 interfaces for persisting data, one is DataStorage, the other
are LogReader and LogWriter.  Which one will be removed?

I also noticed that eagle 0.5 architecture (
https://cwiki.apache.org/confluence/display/EAG/Architecture+of+Eagle+0.5)
was updated. What are the difference between "log" and "metric"?

Thanks
Chang

On Wednesday, August 17, 2016, Liangfei.Su  wrote:

> On test scenario with different config need for differest, i simply do
> ConfigFactory.invalidateCaches() in setup().
>
>
> On Wed, Aug 17, 2016 at 9:43 AM, Edward Zhang  >
> wrote:
>
> > Sorry I mean "remove" :-)
> >
> > Thanks
> > Edward
> >
> > On Tue, Aug 16, 2016 at 6:35 PM, Chang Chen  > wrote:
> >
> > > Hi Edward
> > >
> > > Does remote mean run TestEagleConfig in different JVM?
> > >
> > > Thanks
> > > Chang
> > >
> > > On Wednesday, August 17, 2016, Edward Zhang  >
> > > wrote:
> > >
> > > > TestEagleConfig is not a valid test case because it assumes that it
> can
> > > > initialize EagleConfigFactory, but when we execute multiple test
> cases,
> > > > this may not work as EagleConfigFactory may already be initialized
> > before
> > > > this test case is executed. So setting "config.resource" will not
> have
> > > any
> > > > effect.
> > > >
> > > > I suggest we remote TestEagleConfig, and also going forward we can
> need
> > > > refactor EagleConfigFactor to be not a singleton but use
> > > > com.typesafe.config.Config as constructor parameter.
> > > >
> > > > Thanks
> > > > Edward
> > > >
> > > > On Tue, Aug 16, 2016 at 4:26 AM, Chang Chen  
> > > > > wrote:
> > > >
> > > > > Hi Guys
> > > > >
> > > > > In the middle of writing UT for Eagle, we found it is difficult to
> > mock
> > > > >  EagleConfigFactory, since it has static method. The current
> > workaround
> > > > is
> > > > > introducing powermock, so that we can set manager to mock object ,
> > i.e.
> > > > >
> > > > >Whitebox.setInternalState(EagleConfigFactory.class, "manager",
> > > > > eagleConfigFactory);
> > > > >
> > > > > This is OK in most cases, but TestEagleConfig.testInit could be
> > failed,
> > > > > because manager only initialized at first time and
> > >  EagleConfigFactory's
> > > > > Constructor is private:
> > > > >
> > > > >private static EagleConfigFactory manager = new
> > > EagleConfigFactory();
> > > > >
> > > > > EagleConfigFactory actually has two roles, one is factory and
> another
> > > is
> > > > > configuration. I propose refactoring EagleConfigFactory and
> > distinguish
> > > > > these two roles:
> > > > >
> > > > > public class EagleConfigImpl implements EagleConfig{
> > > > > }
> > > > >
> > > > > public class EagleConfigFactory {
> > > > >   public static EagleConfig load(){}
> > > > >   public static void reset(EagleConfig config){
> > > > > if (config == null) {
> > > > >   //default
> > > > > }else{
> > > > > }
> > > > >   }
> > > > > }
> > > > >
> > > > > However doing so, we need call EagleConfigFactory.reset at the
> > > beginning
> > > > of
> > > > >  eagle start.
> > > > >
> > > > > Any ideas?
> > > > >
> > > > > Thanks
> > > > > Chang
> > > > >
> > > >
> > >
> >
>

[GitHub] incubator-eagle pull request #365: Test email alert publisher: configuration...

[pkuwm]

Github user pkuwm commented on a diff in the pull request:

https://github.com/apache/incubator-eagle/pull/365#discussion_r75637810
  
--- Diff: 
eagle-core/eagle-alert-parent/eagle-alert/alert-engine/src/test/resources/publishments1.json
 ---
@@ -1,20 +1,25 @@
 [
-{
-   "name":"test-stream-output",
-   
"type":"org.apache.eagle.alert.engine.publisher.impl.AlertEmailPublisher",
-   "policyIds": [
-   "perfmon_cpu_host_check", "perfmon_cpu_pool_check"
-   ],
-   "properties": {
- "subject":"UMP Test Alert",
- "template":"",
- "sender": "sen...@corp.com",
- "recipients": "servi...@corp.com",
- "smtp.server":"sender.mailhost.com",
- "connection": "plaintext",
- "smtp.port": "25"
-   },
-   "dedupIntervalMin" : "PT1M",
-   "serializer" : 
"org.apache.eagle.alert.engine.publisher.impl.StringEventSerializer"
-}
+  {
+"name": "test-stream-output",
+"type": 
"org.apache.eagle.alert.engine.publisher.impl.AlertEmailPublisher",
+"policyIds": [
+  "perfmon_cpu_host_check",
+  "perfmon_cpu_pool_check"
+],
+"properties": {
+  "subject": "UMP Test Alert",
+  "template": "",
+  "sender": "sen...@corp.com",
+  "recipients": "servi...@corp.com",
+  "mail.smtp.host": "smtp.office365.com",
--- End diff --

@haoch Removed. Thanks.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[jira] [Commented] (EAGLE-411) Refactor hbase related unit test cases with mockito and mini-hbase

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/EAGLE-411?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15430309#comment-15430309
 ] 

ASF GitHub Bot commented on EAGLE-411:
--

Github user chitin closed the pull request at:

https://github.com/apache/incubator-eagle/pull/372


> Refactor hbase related unit test cases with mockito and mini-hbase
> --
>
> Key: EAGLE-411
> URL: https://issues.apache.org/jira/browse/EAGLE-411
> Project: Eagle
>  Issue Type: Improvement
>Affects Versions: v0.3.0, v0.4.0
>Reporter: Hao Chen
>Assignee: Chang chen
>  Labels: unit-test
> Fix For: v0.5.0
>
>
> Current implementation is really not stable which cause unit test like 
> TestHBaseLogReader2 unable to pass in meanful time, we need a better version.
> https://github.com/apache/incubator-eagle/blob/master/eagle-core/eagle-embed/eagle-embed-hbase/src/main/java/org/apache/eagle/service/hbase/EmbeddedHbase.java
> http://blog.cloudera.com/blog/2013/09/how-to-test-hbase-applications-using-popular-tools/



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (EAGLE-411) Refactor hbase related unit test cases with mockito and mini-hbase

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/EAGLE-411?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15430308#comment-15430308
 ] 

ASF GitHub Bot commented on EAGLE-411:
--

Github user haoch commented on the issue:

https://github.com/apache/incubator-eagle/pull/372
  
Pls. resolve conflicts firstly.


> Refactor hbase related unit test cases with mockito and mini-hbase
> --
>
> Key: EAGLE-411
> URL: https://issues.apache.org/jira/browse/EAGLE-411
> Project: Eagle
>  Issue Type: Improvement
>Affects Versions: v0.3.0, v0.4.0
>Reporter: Hao Chen
>Assignee: Chang chen
>  Labels: unit-test
> Fix For: v0.5.0
>
>
> Current implementation is really not stable which cause unit test like 
> TestHBaseLogReader2 unable to pass in meanful time, we need a better version.
> https://github.com/apache/incubator-eagle/blob/master/eagle-core/eagle-embed/eagle-embed-hbase/src/main/java/org/apache/eagle/service/hbase/EmbeddedHbase.java
> http://blog.cloudera.com/blog/2013/09/how-to-test-hbase-applications-using-popular-tools/



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[GitHub] incubator-eagle pull request #372: EAGLE-411 Refactor hbase related unit tes...

[chitin]

Github user chitin closed the pull request at:

https://github.com/apache/incubator-eagle/pull/372


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] incubator-eagle issue #372: EAGLE-411 Refactor hbase related unit test cases...

[haoch]

Github user haoch commented on the issue:

https://github.com/apache/incubator-eagle/pull/372
  
Pls. resolve conflicts firstly.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[jira] [Commented] (EAGLE-411) Refactor hbase related unit test cases with mockito and mini-hbase

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/EAGLE-411?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15430305#comment-15430305
 ] 

ASF GitHub Bot commented on EAGLE-411:
--

Github user haoch commented on a diff in the pull request:

https://github.com/apache/incubator-eagle/pull/372#discussion_r75636528
  
--- Diff: eagle-core/eagle-query/eagle-service-base/pom.xml ---
@@ -28,6 +28,10 @@
eagle-service-base
jar
eagle-service-base
+
+1.6.5
+
1.6.5
+
--- End diff --

Should put all version declaration only in root pom.


> Refactor hbase related unit test cases with mockito and mini-hbase
> --
>
> Key: EAGLE-411
> URL: https://issues.apache.org/jira/browse/EAGLE-411
> Project: Eagle
>  Issue Type: Improvement
>Affects Versions: v0.3.0, v0.4.0
>Reporter: Hao Chen
>Assignee: Chang chen
>  Labels: unit-test
> Fix For: v0.5.0
>
>
> Current implementation is really not stable which cause unit test like 
> TestHBaseLogReader2 unable to pass in meanful time, we need a better version.
> https://github.com/apache/incubator-eagle/blob/master/eagle-core/eagle-embed/eagle-embed-hbase/src/main/java/org/apache/eagle/service/hbase/EmbeddedHbase.java
> http://blog.cloudera.com/blog/2013/09/how-to-test-hbase-applications-using-popular-tools/



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[GitHub] incubator-eagle pull request #372: EAGLE-411 Refactor hbase related unit tes...

[haoch]

Github user haoch commented on a diff in the pull request:

https://github.com/apache/incubator-eagle/pull/372#discussion_r75636528
  
--- Diff: eagle-core/eagle-query/eagle-service-base/pom.xml ---
@@ -28,6 +28,10 @@
eagle-service-base
jar
eagle-service-base
+
+1.6.5
+
1.6.5
+
--- End diff --

Should put all version declaration only in root pom.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[jira] [Commented] (EAGLE-411) Refactor hbase related unit test cases with mockito and mini-hbase

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/EAGLE-411?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15430301#comment-15430301
 ] 

ASF GitHub Bot commented on EAGLE-411:
--

GitHub user chitin opened a pull request:

https://github.com/apache/incubator-eagle/pull/372

EAGLE-411 Refactor hbase related unit test cases with mockito

EAGLE-411 Refactor hbase related unit test cases with mockito

- Mock the action of HBase reading, mock the meanful result of 
scanner.next(), unit test would not be enslaved to HBase.

https://issues.apache.org/jira/browse/EAGLE-411.

You can merge this pull request into a Git repository by running:

$ git pull https://github.com/chitin/incubator-eagle EAGLE411

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/incubator-eagle/pull/372.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #372


commit b8ac17fa76ea97768dab6dc929209439a5860c07
Author: chitin 
Date:   2016-08-22T08:29:44Z

EAGLE-411 Refactor hbase related unit test cases with mockito and mini-hbase
--Mock the action of HBase reading, mock the meanful result of 
scanner.next(), unit test would not be enslaved to HBase.

https://issues.apache.org/jira/browse/EAGLE-411.




> Refactor hbase related unit test cases with mockito and mini-hbase
> --
>
> Key: EAGLE-411
> URL: https://issues.apache.org/jira/browse/EAGLE-411
> Project: Eagle
>  Issue Type: Improvement
>Affects Versions: v0.3.0, v0.4.0
>Reporter: Hao Chen
>Assignee: Chang chen
>  Labels: unit-test
> Fix For: v0.5.0
>
>
> Current implementation is really not stable which cause unit test like 
> TestHBaseLogReader2 unable to pass in meanful time, we need a better version.
> https://github.com/apache/incubator-eagle/blob/master/eagle-core/eagle-embed/eagle-embed-hbase/src/main/java/org/apache/eagle/service/hbase/EmbeddedHbase.java
> http://blog.cloudera.com/blog/2013/09/how-to-test-hbase-applications-using-popular-tools/



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[GitHub] incubator-eagle pull request #372: EAGLE-411 Refactor hbase related unit tes...

[chitin]

GitHub user chitin opened a pull request:

https://github.com/apache/incubator-eagle/pull/372

EAGLE-411 Refactor hbase related unit test cases with mockito

EAGLE-411 Refactor hbase related unit test cases with mockito

- Mock the action of HBase reading, mock the meanful result of 
scanner.next(), unit test would not be enslaved to HBase.

https://issues.apache.org/jira/browse/EAGLE-411.

You can merge this pull request into a Git repository by running:

$ git pull https://github.com/chitin/incubator-eagle EAGLE411

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/incubator-eagle/pull/372.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #372


commit b8ac17fa76ea97768dab6dc929209439a5860c07
Author: chitin 
Date:   2016-08-22T08:29:44Z

EAGLE-411 Refactor hbase related unit test cases with mockito and mini-hbase
--Mock the action of HBase reading, mock the meanful result of 
scanner.next(), unit test would not be enslaved to HBase.

https://issues.apache.org/jira/browse/EAGLE-411.




---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[HELP] subscribe for private mailing list

[Michael Wu]

Hi Henry,

I've subscribed for "private" mailing list, and i'm informed that it may
require mentor's approval. Could you please help add my email address "
mchl...@gmail.com" to priv...@eagle.incubator.apache.org? Thanks!

Michael

[jira] [Commented] (EAGLE-451) oozie ingestion+alert engine preview

[Lingang Deng (JIRA)]

[ 
https://issues.apache.org/jira/browse/EAGLE-451?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15430283#comment-15430283
 ] 

Lingang Deng commented on EAGLE-451:


This Improvement is aimed at parsing kafka log,then writing to kafka for oozie 
auditlog, it seem to be similar with EAGLE-448.

> oozie ingestion+alert engine preview
> 
>
> Key: EAGLE-451
> URL: https://issues.apache.org/jira/browse/EAGLE-451
> Project: Eagle
>  Issue Type: Improvement
>Reporter: Lingang Deng
>Assignee: Lingang Deng
>Priority: Trivial
>




--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (EAGLE-484) StreamRouterSpec class should override equals method

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/EAGLE-484?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15430264#comment-15430264
 ] 

ASF GitHub Bot commented on EAGLE-484:
--

Github user haoch commented on the issue:

https://github.com/apache/incubator-eagle/pull/369
  
LGTM.


> StreamRouterSpec class should override equals method 
> -
>
> Key: EAGLE-484
> URL: https://issues.apache.org/jira/browse/EAGLE-484
> Project: Eagle
>  Issue Type: Bug
>Affects Versions: v0.5.0
>Reporter: JiJun Tang
>Assignee: JiJun Tang
> Fix For: v0.5.0
>
>
> {code:title=StreamRouterBolt.java|borderStyle=solid}
> if(!newSRS.get(s).equals(cachedSRS.get(s))){ 
> // this compare will get wrong result ,so will return wrong 
> modifiedRouterSpecs
>  modifiedRouterSpecs.add(newSRS.get(s));
>  }
> {code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[GitHub] incubator-eagle issue #369: [EAGLE-484] StreamRouterSpec class should overri...

[haoch]

Github user haoch commented on the issue:

https://github.com/apache/incubator-eagle/pull/369
  
LGTM.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[jira] [Updated] (EAGLE-488) fix not delete finished application path in zk and jobDefId issues

[wujinhu (JIRA)]

 [ 
https://issues.apache.org/jira/browse/EAGLE-488?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

wujinhu updated EAGLE-488:
--
Summary: fix not delete finished application path in zk and jobDefId issues 
 (was: fix RunningJobManager delete finished application path issue)

> fix not delete finished application path in zk and jobDefId issues
> --
>
> Key: EAGLE-488
> URL: https://issues.apache.org/jira/browse/EAGLE-488
> Project: Eagle
>  Issue Type: Bug
>Reporter: wujinhu
>Assignee: wujinhu
>




--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Closed] (EAGLE-443) refactor ProcessedTimeStamp

[wujinhu (JIRA)]

 [ 
https://issues.apache.org/jira/browse/EAGLE-443?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

wujinhu closed EAGLE-443.
-

> refactor ProcessedTimeStamp
> ---
>
> Key: EAGLE-443
> URL: https://issues.apache.org/jira/browse/EAGLE-443
> Project: Eagle
>  Issue Type: Improvement
>Reporter: wujinhu
>Assignee: wujinhu
>
> In the previous implementation, we use a blot to update processed job time 
> stamp. But we can do it in a simple way. The spout can update itself.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Closed] (EAGLE-422) eagle support for mr & spark running job monitoring

[wujinhu (JIRA)]

 [ 
https://issues.apache.org/jira/browse/EAGLE-422?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

wujinhu closed EAGLE-422.
-

> eagle support for mr & spark running job monitoring
> ---
>
> Key: EAGLE-422
> URL: https://issues.apache.org/jira/browse/EAGLE-422
> Project: Eagle
>  Issue Type: New Feature
>Affects Versions: v0.5.0
>Reporter: wujinhu
>Assignee: wujinhu
>Priority: Blocker
> Fix For: v0.5.0
>
>
> We have implemented MR & Spark running job performance monitoring(JPM) in the 
> past few weeks.
> In MR/Spark JPMs, we fetch running job list from the yarn resource manager, 
> get job details from rest api that supported by yarn and save the parsed 
> results to hbase.
> For the architecture, We build storm topologies to implement these 
> functionalities.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Closed] (EAGLE-465) Hive security monitoring refactor

[wujinhu (JIRA)]

 [ 
https://issues.apache.org/jira/browse/EAGLE-465?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

wujinhu closed EAGLE-465.
-

> Hive security monitoring refactor
> -
>
> Key: EAGLE-465
> URL: https://issues.apache.org/jira/browse/EAGLE-465
> Project: Eagle
>  Issue Type: Improvement
>Reporter: wujinhu
>Assignee: wujinhu
>
> refactor hive security monitoring with new application framework and delete 
> old mr running spout



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[GitHub] incubator-eagle pull request #369: [EAGLE-484] StreamRouterSpec class should...

[haoch]

Github user haoch commented on a diff in the pull request:

https://github.com/apache/incubator-eagle/pull/369#discussion_r75628734
  
--- Diff: 
eagle-core/eagle-alert-parent/eagle-alert/alert-common/src/main/java/org/apache/eagle/alert/coordination/model/PolicyWorkerQueue.java
 ---
@@ -72,6 +76,24 @@ public void setWorkers(List workers) {
 this.workers = workers;
 }
 
+@Override
+public boolean equals(Object other) {
+if (this == other) {
--- End diff --

Should check NPE?


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[jira] [Commented] (EAGLE-484) StreamRouterSpec class should override equals method

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/EAGLE-484?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15430246#comment-15430246
 ] 

ASF GitHub Bot commented on EAGLE-484:
--

Github user haoch commented on a diff in the pull request:

https://github.com/apache/incubator-eagle/pull/369#discussion_r75628734
  
--- Diff: 
eagle-core/eagle-alert-parent/eagle-alert/alert-common/src/main/java/org/apache/eagle/alert/coordination/model/PolicyWorkerQueue.java
 ---
@@ -72,6 +76,24 @@ public void setWorkers(List workers) {
 this.workers = workers;
 }
 
+@Override
+public boolean equals(Object other) {
+if (this == other) {
--- End diff --

Should check NPE?


> StreamRouterSpec class should override equals method 
> -
>
> Key: EAGLE-484
> URL: https://issues.apache.org/jira/browse/EAGLE-484
> Project: Eagle
>  Issue Type: Bug
>Affects Versions: v0.5.0
>Reporter: JiJun Tang
>Assignee: JiJun Tang
> Fix For: v0.5.0
>
>
> {code:title=StreamRouterBolt.java|borderStyle=solid}
> if(!newSRS.get(s).equals(cachedSRS.get(s))){ 
> // this compare will get wrong result ,so will return wrong 
> modifiedRouterSpecs
>  modifiedRouterSpecs.add(newSRS.get(s));
>  }
> {code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Resolved] (EAGLE-482) Fix checkstyle configuration

[Hao Chen (JIRA)]

 [ 
https://issues.apache.org/jira/browse/EAGLE-482?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Hao Chen resolved EAGLE-482.

Resolution: Fixed

> Fix checkstyle configuration
> 
>
> Key: EAGLE-482
> URL: https://issues.apache.org/jira/browse/EAGLE-482
> Project: Eagle
>  Issue Type: Bug
>Reporter: Hao Chen
>Assignee: Hao Chen
>
> Recently community contributed the checkstyle for eagle which is good but not 
> well defined, so we need to 
> 1. Refine checkstyle configuration following Google Java Code Style
> 2. Basically reformat code following the checkstyle.
> 3. (Later) continue reformatting left code to make sure all code clean
> 4. Turn on "failOnViolation" to force all code following the same style 
> otherwise fail to build.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (EAGLE-482) Fix checkstyle configuration

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/EAGLE-482?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15430241#comment-15430241
 ] 

ASF GitHub Bot commented on EAGLE-482:
--

Github user haoch closed the pull request at:

https://github.com/apache/incubator-eagle/pull/371


> Fix checkstyle configuration
> 
>
> Key: EAGLE-482
> URL: https://issues.apache.org/jira/browse/EAGLE-482
> Project: Eagle
>  Issue Type: Bug
>Reporter: Hao Chen
>Assignee: Hao Chen
>
> Recently community contributed the checkstyle for eagle which is good but not 
> well defined, so we need to 
> 1. Refine checkstyle configuration following Google Java Code Style
> 2. Basically reformat code following the checkstyle.
> 3. (Later) continue reformatting left code to make sure all code clean
> 4. Turn on "failOnViolation" to force all code following the same style 
> otherwise fail to build.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[GitHub] incubator-eagle pull request #371: EAGLE-482 Fix checkstyle configuration

[haoch]

Github user haoch closed the pull request at:

https://github.com/apache/incubator-eagle/pull/371


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---