[jira] [Commented] (NIFI-3196) Provide MiNiFi FlowStatus Insights processor that integrates MiNiFi Flow Status Report with Azure Application Insights

2017-08-03 Thread ASF GitHub Bot (JIRA) 

[ 
https://issues.apache.org/jira/browse/NIFI-3196?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16113436#comment-16113436
 ] 

ASF GitHub Bot commented on NIFI-3196:
--

Github user jzonthemtn commented on the issue:

https://github.com/apache/nifi/pull/1331
  
@apsaltis Just found this and I'm glad I did. It looks very nice. I'm 
facing the same type of task except with AWS CloudWatch. I wanted to ask if you 
are still continuing development toward what was discussed earlier in this 
conversation? Thanks!


> Provide MiNiFi FlowStatus Insights processor that integrates MiNiFi Flow 
> Status Report with Azure Application Insights
> --
>
> Key: NIFI-3196
> URL: https://issues.apache.org/jira/browse/NIFI-3196
> Project: Apache NiFi
>  Issue Type: New Feature
>Affects Versions: 1.1.0
>Reporter: Andrew Psaltis
>Assignee: Andrew Psaltis
>
> As a user of both NiFi and MiNiFi there are many times I would like to have 
> an operational dashboard for MiNiFi. Although, there is a discussion and 
> desire for [Centralized management | 
> https://cwiki.apache.org/confluence/display/MINIFI/MiNiFi+Community+Driven+Requirements]
>  there are many times where as a user I just want to be able to use the 
> operational tools I already have. In many cases those may be products such 
> as:  Nagios, Zenoos, Zabix, Graphite, Grafana and in cloud environments 
> perhaps Azure Application Insights and AWS Cloudwatch. 
> This JIRA and the related P/R provides a processor that ingests the MiNiFi 
> Flow Status report and sends it to Azure Application Insights as custom 
> metrics. This then allows a user to set alerts in Azure Application Insights, 
> thus providing the ability to monitor MiNiFi agents. As such, this has a 
> dependency on the following MiNiFi JIRA: [Change FlowStatusReport to be JSON 
> | https://issues.apache.org/jira/browse/MINIFI-170]



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (NIFI-3196) Provide MiNiFi FlowStatus Insights processor that integrates MiNiFi Flow Status Report with Azure Application Insights

2017-02-19 Thread ASF GitHub Bot (JIRA) 

[ 
https://issues.apache.org/jira/browse/NIFI-3196?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15873688#comment-15873688
 ] 

ASF GitHub Bot commented on NIFI-3196:
--

Github user apsaltis commented on the issue:

https://github.com/apache/nifi/pull/1331
  
@trixpan I agree that having something much more generic would be great. I 
am totally open to closing this PR, rebasing my branch and taking this 
processor in that direction. Honestly, after having time to think about it It 
being tied to MiNiFi and talking with others I totally agree. 


> Provide MiNiFi FlowStatus Insights processor that integrates MiNiFi Flow 
> Status Report with Azure Application Insights
> --
>
> Key: NIFI-3196
> URL: https://issues.apache.org/jira/browse/NIFI-3196
> Project: Apache NiFi
>  Issue Type: New Feature
>Affects Versions: 1.1.0
>Reporter: Andrew Psaltis
>Assignee: Andrew Psaltis
>
> As a user of both NiFi and MiNiFi there are many times I would like to have 
> an operational dashboard for MiNiFi. Although, there is a discussion and 
> desire for [Centralized management | 
> https://cwiki.apache.org/confluence/display/MINIFI/MiNiFi+Community+Driven+Requirements]
>  there are many times where as a user I just want to be able to use the 
> operational tools I already have. In many cases those may be products such 
> as:  Nagios, Zenoos, Zabix, Graphite, Grafana and in cloud environments 
> perhaps Azure Application Insights and AWS Cloudwatch. 
> This JIRA and the related P/R provides a processor that ingests the MiNiFi 
> Flow Status report and sends it to Azure Application Insights as custom 
> metrics. This then allows a user to set alerts in Azure Application Insights, 
> thus providing the ability to monitor MiNiFi agents. As such, this has a 
> dependency on the following MiNiFi JIRA: [Change FlowStatusReport to be JSON 
> | https://issues.apache.org/jira/browse/MINIFI-170]



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Commented] (NIFI-3196) Provide MiNiFi FlowStatus Insights processor that integrates MiNiFi Flow Status Report with Azure Application Insights

2017-02-19 Thread ASF GitHub Bot (JIRA) 

[ 
https://issues.apache.org/jira/browse/NIFI-3196?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15873633#comment-15873633
 ] 

ASF GitHub Bot commented on NIFI-3196:
--

Github user trixpan commented on the issue:

https://github.com/apache/nifi/pull/1331
  
@apsaltis - loved the idea behind this contribution - I'm also very much in 
favor of fitting MiNiFi with existing systems - and perhaps that's the reason 
for my first question:

Is there a reason why to bundle the Azure Insights as a bundle tied to this 
NAR? Looking at the metrics collected it seems fairly useful to NiFi as well 
and suspect people running NiFi in the cloud would rave about being able to 
send reporting data directly into insights instead of having to rely solely on 
Ambari and other existing reporting services.

It may be just me but I suspect AzureApplicationInsights if empowered by 
Expression Language enabled Dynamic Properties would be a killing processor...

What do you think?


> Provide MiNiFi FlowStatus Insights processor that integrates MiNiFi Flow 
> Status Report with Azure Application Insights
> --
>
> Key: NIFI-3196
> URL: https://issues.apache.org/jira/browse/NIFI-3196
> Project: Apache NiFi
>  Issue Type: New Feature
>Affects Versions: 1.1.0
>Reporter: Andrew Psaltis
>Assignee: Andrew Psaltis
>
> As a user of both NiFi and MiNiFi there are many times I would like to have 
> an operational dashboard for MiNiFi. Although, there is a discussion and 
> desire for [Centralized management | 
> https://cwiki.apache.org/confluence/display/MINIFI/MiNiFi+Community+Driven+Requirements]
>  there are many times where as a user I just want to be able to use the 
> operational tools I already have. In many cases those may be products such 
> as:  Nagios, Zenoos, Zabix, Graphite, Grafana and in cloud environments 
> perhaps Azure Application Insights and AWS Cloudwatch. 
> This JIRA and the related P/R provides a processor that ingests the MiNiFi 
> Flow Status report and sends it to Azure Application Insights as custom 
> metrics. This then allows a user to set alerts in Azure Application Insights, 
> thus providing the ability to monitor MiNiFi agents. As such, this has a 
> dependency on the following MiNiFi JIRA: [Change FlowStatusReport to be JSON 
> | https://issues.apache.org/jira/browse/MINIFI-170]



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Commented] (NIFI-3196) Provide MiNiFi FlowStatus Insights processor that integrates MiNiFi Flow Status Report with Azure Application Insights

2016-12-14 Thread Andrew Psaltis (JIRA) 

[ 
https://issues.apache.org/jira/browse/NIFI-3196?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15750168#comment-15750168
 ] 

Andrew Psaltis commented on NIFI-3196:
--

It would certainly be possible to put other processors in-front of this to do 
transformation of the data. The only thing this knows about the input is that 
it is a MiNiFi FlowStatus report serialized as JSON and from that it 
deserializes it and sends it along as metrics to AAI. 

> Provide MiNiFi FlowStatus Insights processor that integrates MiNiFi Flow 
> Status Report with Azure Application Insights
> --
>
> Key: NIFI-3196
> URL: https://issues.apache.org/jira/browse/NIFI-3196
> Project: Apache NiFi
>  Issue Type: New Feature
>Affects Versions: 1.1.0
>Reporter: Andrew Psaltis
>Assignee: Andrew Psaltis
>
> As a user of both NiFi and MiNiFi there are many times I would like to have 
> an operational dashboard for MiNiFi. Although, there is a discussion and 
> desire for [Centralized management | 
> https://cwiki.apache.org/confluence/display/MINIFI/MiNiFi+Community+Driven+Requirements]
>  there are many times where as a user I just want to be able to use the 
> operational tools I already have. In many cases those may be products such 
> as:  Nagios, Zenoos, Zabix, Graphite, Grafana and in cloud environments 
> perhaps Azure Application Insights and AWS Cloudwatch. 
> This JIRA and the related P/R provides a processor that ingests the MiNiFi 
> Flow Status report and sends it to Azure Application Insights as custom 
> metrics. This then allows a user to set alerts in Azure Application Insights, 
> thus providing the ability to monitor MiNiFi agents. As such, this has a 
> dependency on the following MiNiFi JIRA: [Change FlowStatusReport to be JSON 
> | https://issues.apache.org/jira/browse/MINIFI-170]



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (NIFI-3196) Provide MiNiFi FlowStatus Insights processor that integrates MiNiFi Flow Status Report with Azure Application Insights

2016-12-14 Thread ASF GitHub Bot (JIRA) 

[ 
https://issues.apache.org/jira/browse/NIFI-3196?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15750039#comment-15750039
 ] 

ASF GitHub Bot commented on NIFI-3196:
--

GitHub user apsaltis opened a pull request:

https://github.com/apache/nifi/pull/1331

NIFI-3196 Provide MiNiFi FlowStatus Insights processor that integrates 
MiNiFi Flow Status Report with Azure Application Insights

As a user of both NiFi and MiNiFi there are many times I would like to have 
an operational dashboard for MiNiFi. Although, there is a discussion and desire 
for Centralized management there are many times where as a user I just want to 
be able to use the operational tools I already have. In many cases those may be 
products such as: Nagios, Zenoos, Zabix, Graphite, Grafana and in cloud 
environments perhaps Azure Application Insights and AWS Cloudwatch.

This  P/R provides a processor that ingests the MiNiFi Flow Status report 
and sends it to Azure Application Insights as custom metrics. This then allows 
a user to set alerts in Azure Application Insights, thus providing the ability 
to monitor MiNiFi agents. As such, this has a dependency on the following 
MiNiFi JIRA: Change FlowStatusReport to be JSON 
([https://github.com/apache/nifi-minifi/pull/65](url))

An example flow of how to set this up for the MiNiFi side is:


![image](https://cloud.githubusercontent.com/assets/1100275/21208171/61891462-c23a-11e6-9075-e7657653f40f.png)

And the NiFi Side looks like:

![image](https://cloud.githubusercontent.com/assets/1100275/21208215/bc114cec-c23a-11e6-8393-ee2ec50e61e7.png)

Then once this is all set in Azure Application Insights I can build a 
dashboard that may look like this:

![image](https://cloud.githubusercontent.com/assets/1100275/21208329/67804e02-c23b-11e6-8e7d-933c56f33d3b.png)

And explore all the various metrics, that I may want to build a dashboard 
from:

![image](https://cloud.githubusercontent.com/assets/1100275/21208347/913e5b76-c23b-11e6-89b6-b8895210.png)

I can then also go and set alerts 

![image](https://cloud.githubusercontent.com/assets/1100275/21208458/128b653e-c23c-11e6-8737-d8665c280325.png)

In the end this type of integration can also be done with other enterprise 
tools.


You can merge this pull request into a Git repository by running:

$ git pull https://github.com/apsaltis/nifi NIFI-3196

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/nifi/pull/1331.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #1331


commit 9d3fa636a4f0feb9c7055488c5f686dd3f237e36
Author: Andrew Psaltis 
Date:   2016-12-14T16:40:27Z

NIFI-3196 Provide MiNiFi FlowStatus Insights processor that integrates 
MiNiFi Flow Status Report with Azure Application Insights




> Provide MiNiFi FlowStatus Insights processor that integrates MiNiFi Flow 
> Status Report with Azure Application Insights
> --
>
> Key: NIFI-3196
> URL: https://issues.apache.org/jira/browse/NIFI-3196
> Project: Apache NiFi
>  Issue Type: New Feature
>Affects Versions: 1.1.0
>Reporter: Andrew Psaltis
>Assignee: Andrew Psaltis
>
> As a user of both NiFi and MiNiFi there are many times I would like to have 
> an operational dashboard for MiNiFi. Although, there is a discussion and 
> desire for [Centralized management | 
> https://cwiki.apache.org/confluence/display/MINIFI/MiNiFi+Community+Driven+Requirements]
>  there are many times where as a user I just want to be able to use the 
> operational tools I already have. In many cases those may be products such 
> as:  Nagios, Zenoos, Zabix, Graphite, Grafana and in cloud environments 
> perhaps Azure Application Insights and AWS Cloudwatch. 
> This JIRA and the related P/R provides a processor that ingests the MiNiFi 
> Flow Status report and sends it to Azure Application Insights as custom 
> metrics. This then allows a user to set alerts in Azure Application Insights, 
> thus providing the ability to monitor MiNiFi agents. As such, this has a 
> dependency on the following MiNiFi JIRA: [Change FlowStatusReport to be JSON 
> | https://issues.apache.org/jira/browse/MINIFI-170]



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (NIFI-3196) Provide MiNiFi FlowStatus Insights processor that integrates MiNiFi Flow Status Report with Azure Application Insights

2016-12-14 Thread Aldrin Piri (JIRA) 

[ 
https://issues.apache.org/jira/browse/NIFI-3196?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15748789#comment-15748789
 ] 

Aldrin Piri commented on NIFI-3196:
---

Sounds like a good concept.  Curious if it makes sense to deconstruct this a 
bit more but do not have great familiarity with the Azure App Insights (AAI).  
Could see a series of processors that receive the data, then transform 
(transform JSON) and then a PutAAI or similar.  That last bit of functionality 
seems like it might get some nice reuse for a lot of other uses.


> Provide MiNiFi FlowStatus Insights processor that integrates MiNiFi Flow 
> Status Report with Azure Application Insights
> --
>
> Key: NIFI-3196
> URL: https://issues.apache.org/jira/browse/NIFI-3196
> Project: Apache NiFi
>  Issue Type: New Feature
>Affects Versions: 1.1.0
>Reporter: Andrew Psaltis
>Assignee: Andrew Psaltis
>
> As a user of both NiFi and MiNiFi there are many times I would like to have 
> an operational dashboard for MiNiFi. Although, there is a discussion and 
> desire for [Centralized management | 
> https://cwiki.apache.org/confluence/display/MINIFI/MiNiFi+Community+Driven+Requirements]
>  there are many times where as a user I just want to be able to use the 
> operational tools I already have. In many cases those may be products such 
> as:  Nagios, Zenoos, Zabix, Graphite, Grafana and in cloud environments 
> perhaps Azure Application Insights and AWS Cloudwatch. 
> This JIRA and the related P/R provides a processor that ingests the MiNiFi 
> Flow Status report and sends it to Azure Application Insights as custom 
> metrics. This then allows a user to set alerts in Azure Application Insights, 
> thus providing the ability to monitor MiNiFi agents. As such, this has a 
> dependency on the following MiNiFi JIRA: [Change FlowStatusReport to be JSON 
> | https://issues.apache.org/jira/browse/MINIFI-170]



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)