[
https://issues.apache.org/jira/browse/NIFI-4809?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16408613#comment-16408613
]
ASF GitHub Bot commented on NIFI-4809:
--------------------------------------
GitHub user pvillard31 opened a pull request:
https://github.com/apache/nifi/pull/2575
NIFI-4809 - Implement a SiteToSiteMetricsReportingTask
This PR is an update of the previous one (#2430) that has been reverted to
fix a dependency issue.
I've manually added a JSON Record reader in the abstract class using the
code of the existing reader. For now, it's only used by the Metrics reporting
task, but I plan to use the same approach for the other S2S reporting tasks.
**Do not merge this PR as-is**.
I currently see two issues:
- the one already reported by @mattyb149: the record writer is not showing
the reporting task in the "referencing components". I think it's probably a
framework issue that could be solved in a separate JIRA. Will try to have a
look later.
- more importantly, I see an issue when using a writer with an access
schema strategy using the Avro Schema registry CS. In that case, I get a Schema
not found exception "Failed to write metrics using record writer: Unable to
find schema with name 'metrics'". Not sure what is causing it, will have a look
asap. If using "inherit record schema" strategy, it works as expected.
Wanted to open this PR, to have feedbacks on the approach regarding the
JSON record reader.
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/pvillard31/nifi NIFI-4809
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/nifi/pull/2575.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #2575
----
commit db16a1617836dd02826605604b212d05190b5267
Author: Pierre Villard <pierre.villard.fr@...>
Date: 2018-01-23T22:15:18Z
NIFI-4809 - Implement a SiteToSiteMetricsReportingTask
commit 227b410f1edb9198a8ebd2b4b5335fd34e558623
Author: Pierre Villard <pierre.villard.fr@...>
Date: 2018-03-21T20:35:29Z
Fixed dependency issue by providing a local JSON reader
----
> Implement a SiteToSiteMetricsReportingTask
> ------------------------------------------
>
> Key: NIFI-4809
> URL: https://issues.apache.org/jira/browse/NIFI-4809
> Project: Apache NiFi
> Issue Type: New Feature
> Components: Extensions
> Reporter: Pierre Villard
> Assignee: Pierre Villard
> Priority: Major
>
> At the moment there is an AmbariReportingTask to send the NiFi-related
> metrics of the host to the Ambari Metrics Service. In a multi-cluster
> configuration, or when working with MiNiFi (Java) agents, it might not be
> possible for all the NiFi instances (NiFi and/or MiNiFi) to access the AMS
> REST API.
> To solve this problem, a solution would be to implement a
> SiteToSiteMetricsReportingTask to send the data via S2S to the "main" NiFi
> instance/cluster that will be able to publish the metrics into AMS (using
> InvokeHTTP). This way, it is possible to have the metrics of all the
> instances exposed in one AMS instance.
> I propose to send the data formatted as we are doing right now in the Ambari
> reporting task. If needed, it can be easily converted into another schema
> using the record processors once received via S2S.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
