[GitHub] nifi pull request #2430: NIFI-4809 - Implement a SiteToSiteMetricsReportingT...

[asfgit]

Github user asfgit closed the pull request at:

https://github.com/apache/nifi/pull/2430


---

[GitHub] nifi pull request #2430: NIFI-4809 - Implement a SiteToSiteMetricsReportingT...

[pvillard31]

Github user pvillard31 commented on a diff in the pull request:

https://github.com/apache/nifi/pull/2430#discussion_r174315953
  
--- Diff: 
nifi-nar-bundles/nifi-site-to-site-reporting-bundle/nifi-site-to-site-reporting-task/src/main/java/org/apache/nifi/reporting/SiteToSiteMetricsReportingTask.java
 ---
@@ -0,0 +1,212 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements.  See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License.  You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.nifi.reporting;
+
+import java.io.ByteArrayInputStream;
+import java.io.IOException;
+import java.io.InputStream;
+import java.lang.management.ManagementFactory;
+import java.lang.management.OperatingSystemMXBean;
+import java.nio.charset.StandardCharsets;
+import java.util.ArrayList;
+import java.util.Collection;
+import java.util.Collections;
+import java.util.HashMap;
+import java.util.List;
+import java.util.Map;
+import java.util.UUID;
+import java.util.concurrent.TimeUnit;
+
+import javax.json.Json;
+import javax.json.JsonBuilderFactory;
+import javax.json.JsonObject;
+
+import org.apache.avro.Schema;
+import org.apache.nifi.annotation.documentation.CapabilityDescription;
+import org.apache.nifi.annotation.documentation.Tags;
+import org.apache.nifi.avro.AvroTypeUtil;
+import org.apache.nifi.components.AllowableValue;
+import org.apache.nifi.components.PropertyDescriptor;
+import org.apache.nifi.components.ValidationContext;
+import org.apache.nifi.components.ValidationResult;
+import org.apache.nifi.components.Validator;
+import org.apache.nifi.controller.status.ProcessGroupStatus;
+import org.apache.nifi.flowfile.attributes.CoreAttributes;
+import org.apache.nifi.processor.exception.ProcessException;
+import org.apache.nifi.processor.util.StandardValidators;
+import org.apache.nifi.remote.Transaction;
+import org.apache.nifi.remote.TransferDirection;
+import org.apache.nifi.reporting.util.metrics.MetricNames;
+import org.apache.nifi.reporting.util.metrics.MetricsService;
+import org.apache.nifi.reporting.util.metrics.api.MetricsBuilder;
+
+import com.yammer.metrics.core.VirtualMachineMetrics;
+
+@Tags({"status", "metrics", "site", "site to site"})
+@CapabilityDescription("Publishes same metrics as the Ambari Reporting 
task using the Site To Site protocol.")
+public class SiteToSiteMetricsReportingTask extends 
AbstractSiteToSiteReportingTask {
+
+static final AllowableValue AMBARI_FORMAT = new 
AllowableValue("ambari-format", "Ambari Format", "Metrics will be formatted"
++ " according to the Ambari Metrics API. See Additional 
Details in Usage documentation.");
+static final AllowableValue RECORD_FORMAT = new 
AllowableValue("record-format", "Record Format", "Metrics will be formatted"
++ " using the Record Writer property of this reporting task. 
See Additional Details in Usage documentation to"
++ " have the description of the default schema.");
+
+static final PropertyDescriptor APPLICATION_ID = new 
PropertyDescriptor.Builder()
+.name("s2s-metrics-application-id")
+.displayName("Application ID")
+.description("The Application ID to be included in the 
metrics")
+.required(true)
+.expressionLanguageSupported(true)
+.defaultValue("nifi")
+.addValidator(StandardValidators.NON_EMPTY_VALIDATOR)
+.build();
+
+static final PropertyDescriptor HOSTNAME = new 
PropertyDescriptor.Builder()
+.name("s2s-metrics-hostname")
+.displayName("Hostname")
+.description("The Hostname of this NiFi instance to be 
included in the metrics")
+.required(true)
+.expressionLanguageSupported(true)
+.defaultValue("${hostname(true)}")
+.addValidator(StandardValidators.NON_EMPTY_VALIDATOR)
+.build();
+
+static final PropertyDescriptor FORMAT = new 

[GitHub] nifi pull request #2430: NIFI-4809 - Implement a SiteToSiteMetricsReportingT...

[pvillard31]

Github user pvillard31 commented on a diff in the pull request:

https://github.com/apache/nifi/pull/2430#discussion_r174316057
  
--- Diff: 
nifi-nar-bundles/nifi-site-to-site-reporting-bundle/nifi-site-to-site-reporting-task/src/main/java/org/apache/nifi/reporting/AbstractSiteToSiteReportingTask.java
 ---
@@ -140,8 +164,16 @@
 .sensitive(true)
 .addValidator(StandardValidators.NON_BLANK_VALIDATOR)
 .build();
+static final PropertyDescriptor RECORD_WRITER = new 
PropertyDescriptor.Builder()
+.name("record-writer")
+.displayName("Record Writer")
+.description("Specifies the Controller Service to use for 
writing out the records.")
--- End diff --

Left it as-is since I added the property Record Writer only in the task 
reporting task for now.


---

[GitHub] nifi pull request #2430: NIFI-4809 - Implement a SiteToSiteMetricsReportingT...

[pvillard31]

Github user pvillard31 commented on a diff in the pull request:

https://github.com/apache/nifi/pull/2430#discussion_r174315907
  
--- Diff: 
nifi-nar-bundles/nifi-site-to-site-reporting-bundle/nifi-site-to-site-reporting-task/src/main/java/org/apache/nifi/reporting/SiteToSiteMetricsReportingTask.java
 ---
@@ -0,0 +1,212 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements.  See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License.  You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.nifi.reporting;
+
+import java.io.ByteArrayInputStream;
+import java.io.IOException;
+import java.io.InputStream;
+import java.lang.management.ManagementFactory;
+import java.lang.management.OperatingSystemMXBean;
+import java.nio.charset.StandardCharsets;
+import java.util.ArrayList;
+import java.util.Collection;
+import java.util.Collections;
+import java.util.HashMap;
+import java.util.List;
+import java.util.Map;
+import java.util.UUID;
+import java.util.concurrent.TimeUnit;
+
+import javax.json.Json;
+import javax.json.JsonBuilderFactory;
+import javax.json.JsonObject;
+
+import org.apache.avro.Schema;
+import org.apache.nifi.annotation.documentation.CapabilityDescription;
+import org.apache.nifi.annotation.documentation.Tags;
+import org.apache.nifi.avro.AvroTypeUtil;
+import org.apache.nifi.components.AllowableValue;
+import org.apache.nifi.components.PropertyDescriptor;
+import org.apache.nifi.components.ValidationContext;
+import org.apache.nifi.components.ValidationResult;
+import org.apache.nifi.components.Validator;
+import org.apache.nifi.controller.status.ProcessGroupStatus;
+import org.apache.nifi.flowfile.attributes.CoreAttributes;
+import org.apache.nifi.processor.exception.ProcessException;
+import org.apache.nifi.processor.util.StandardValidators;
+import org.apache.nifi.remote.Transaction;
+import org.apache.nifi.remote.TransferDirection;
+import org.apache.nifi.reporting.util.metrics.MetricNames;
+import org.apache.nifi.reporting.util.metrics.MetricsService;
+import org.apache.nifi.reporting.util.metrics.api.MetricsBuilder;
+
+import com.yammer.metrics.core.VirtualMachineMetrics;
+
+@Tags({"status", "metrics", "site", "site to site"})
+@CapabilityDescription("Publishes same metrics as the Ambari Reporting 
task using the Site To Site protocol.")
+public class SiteToSiteMetricsReportingTask extends 
AbstractSiteToSiteReportingTask {
+
+static final AllowableValue AMBARI_FORMAT = new 
AllowableValue("ambari-format", "Ambari Format", "Metrics will be formatted"
++ " according to the Ambari Metrics API. See Additional 
Details in Usage documentation.");
+static final AllowableValue RECORD_FORMAT = new 
AllowableValue("record-format", "Record Format", "Metrics will be formatted"
++ " using the Record Writer property of this reporting task. 
See Additional Details in Usage documentation to"
++ " have the description of the default schema.");
+
+static final PropertyDescriptor APPLICATION_ID = new 
PropertyDescriptor.Builder()
+.name("s2s-metrics-application-id")
+.displayName("Application ID")
+.description("The Application ID to be included in the 
metrics")
+.required(true)
+.expressionLanguageSupported(true)
+.defaultValue("nifi")
+.addValidator(StandardValidators.NON_EMPTY_VALIDATOR)
+.build();
+
+static final PropertyDescriptor HOSTNAME = new 
PropertyDescriptor.Builder()
+.name("s2s-metrics-hostname")
+.displayName("Hostname")
+.description("The Hostname of this NiFi instance to be 
included in the metrics")
+.required(true)
+.expressionLanguageSupported(true)
+.defaultValue("${hostname(true)}")
+.addValidator(StandardValidators.NON_EMPTY_VALIDATOR)
+.build();
+
+static final PropertyDescriptor FORMAT = new 

[GitHub] nifi pull request #2430: NIFI-4809 - Implement a SiteToSiteMetricsReportingT...

[mattyb149]

Github user mattyb149 commented on a diff in the pull request:

https://github.com/apache/nifi/pull/2430#discussion_r174294363
  
--- Diff: 
nifi-nar-bundles/nifi-site-to-site-reporting-bundle/nifi-site-to-site-reporting-task/src/main/resources/docs/org.apache.nifi.reporting.SiteToSiteMetricsReportingTask/additionalDetails.html
 ---
@@ -0,0 +1,178 @@
+
+
+
+
+
+SiteToSiteMetricsReportingTask
+
+
+
+
+
--- End diff --

This is excellent documentation, thank you!


---

[GitHub] nifi pull request #2430: NIFI-4809 - Implement a SiteToSiteMetricsReportingT...

[mattyb149]

Github user mattyb149 commented on a diff in the pull request:

https://github.com/apache/nifi/pull/2430#discussion_r174293625
  
--- Diff: 
nifi-nar-bundles/nifi-site-to-site-reporting-bundle/nifi-site-to-site-reporting-task/src/main/java/org/apache/nifi/reporting/SiteToSiteMetricsReportingTask.java
 ---
@@ -0,0 +1,212 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements.  See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License.  You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.nifi.reporting;
+
+import java.io.ByteArrayInputStream;
+import java.io.IOException;
+import java.io.InputStream;
+import java.lang.management.ManagementFactory;
+import java.lang.management.OperatingSystemMXBean;
+import java.nio.charset.StandardCharsets;
+import java.util.ArrayList;
+import java.util.Collection;
+import java.util.Collections;
+import java.util.HashMap;
+import java.util.List;
+import java.util.Map;
+import java.util.UUID;
+import java.util.concurrent.TimeUnit;
+
+import javax.json.Json;
+import javax.json.JsonBuilderFactory;
+import javax.json.JsonObject;
+
+import org.apache.avro.Schema;
+import org.apache.nifi.annotation.documentation.CapabilityDescription;
+import org.apache.nifi.annotation.documentation.Tags;
+import org.apache.nifi.avro.AvroTypeUtil;
+import org.apache.nifi.components.AllowableValue;
+import org.apache.nifi.components.PropertyDescriptor;
+import org.apache.nifi.components.ValidationContext;
+import org.apache.nifi.components.ValidationResult;
+import org.apache.nifi.components.Validator;
+import org.apache.nifi.controller.status.ProcessGroupStatus;
+import org.apache.nifi.flowfile.attributes.CoreAttributes;
+import org.apache.nifi.processor.exception.ProcessException;
+import org.apache.nifi.processor.util.StandardValidators;
+import org.apache.nifi.remote.Transaction;
+import org.apache.nifi.remote.TransferDirection;
+import org.apache.nifi.reporting.util.metrics.MetricNames;
+import org.apache.nifi.reporting.util.metrics.MetricsService;
+import org.apache.nifi.reporting.util.metrics.api.MetricsBuilder;
+
+import com.yammer.metrics.core.VirtualMachineMetrics;
+
+@Tags({"status", "metrics", "site", "site to site"})
+@CapabilityDescription("Publishes same metrics as the Ambari Reporting 
task using the Site To Site protocol.")
+public class SiteToSiteMetricsReportingTask extends 
AbstractSiteToSiteReportingTask {
+
+static final AllowableValue AMBARI_FORMAT = new 
AllowableValue("ambari-format", "Ambari Format", "Metrics will be formatted"
++ " according to the Ambari Metrics API. See Additional 
Details in Usage documentation.");
+static final AllowableValue RECORD_FORMAT = new 
AllowableValue("record-format", "Record Format", "Metrics will be formatted"
++ " using the Record Writer property of this reporting task. 
See Additional Details in Usage documentation to"
++ " have the description of the default schema.");
+
+static final PropertyDescriptor APPLICATION_ID = new 
PropertyDescriptor.Builder()
+.name("s2s-metrics-application-id")
+.displayName("Application ID")
+.description("The Application ID to be included in the 
metrics")
+.required(true)
+.expressionLanguageSupported(true)
+.defaultValue("nifi")
+.addValidator(StandardValidators.NON_EMPTY_VALIDATOR)
+.build();
+
+static final PropertyDescriptor HOSTNAME = new 
PropertyDescriptor.Builder()
+.name("s2s-metrics-hostname")
+.displayName("Hostname")
+.description("The Hostname of this NiFi instance to be 
included in the metrics")
+.required(true)
+.expressionLanguageSupported(true)
+.defaultValue("${hostname(true)}")
+.addValidator(StandardValidators.NON_EMPTY_VALIDATOR)
+.build();
+
+static final PropertyDescriptor FORMAT = new 

[GitHub] nifi pull request #2430: NIFI-4809 - Implement a SiteToSiteMetricsReportingT...

[mattyb149]

Github user mattyb149 commented on a diff in the pull request:

https://github.com/apache/nifi/pull/2430#discussion_r174293350
  
--- Diff: 
nifi-nar-bundles/nifi-site-to-site-reporting-bundle/nifi-site-to-site-reporting-task/src/main/java/org/apache/nifi/reporting/AbstractSiteToSiteReportingTask.java
 ---
@@ -140,8 +164,16 @@
 .sensitive(true)
 .addValidator(StandardValidators.NON_BLANK_VALIDATOR)
 .build();
+static final PropertyDescriptor RECORD_WRITER = new 
PropertyDescriptor.Builder()
+.name("record-writer")
+.displayName("Record Writer")
+.description("Specifies the Controller Service to use for 
writing out the records.")
--- End diff --

The description here should mention that it is only used when the reporting 
task is configured to use a Record Writer. For the one you added, it has two 
modes, but future reporting tasks need not offer an alternative.


---

[GitHub] nifi pull request #2430: NIFI-4809 - Implement a SiteToSiteMetricsReportingT...

[mattyb149]

Github user mattyb149 commented on a diff in the pull request:

https://github.com/apache/nifi/pull/2430#discussion_r174294045
  
--- Diff: 
nifi-nar-bundles/nifi-site-to-site-reporting-bundle/nifi-site-to-site-reporting-task/src/main/java/org/apache/nifi/reporting/SiteToSiteMetricsReportingTask.java
 ---
@@ -0,0 +1,212 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements.  See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License.  You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.nifi.reporting;
+
+import java.io.ByteArrayInputStream;
+import java.io.IOException;
+import java.io.InputStream;
+import java.lang.management.ManagementFactory;
+import java.lang.management.OperatingSystemMXBean;
+import java.nio.charset.StandardCharsets;
+import java.util.ArrayList;
+import java.util.Collection;
+import java.util.Collections;
+import java.util.HashMap;
+import java.util.List;
+import java.util.Map;
+import java.util.UUID;
+import java.util.concurrent.TimeUnit;
+
+import javax.json.Json;
+import javax.json.JsonBuilderFactory;
+import javax.json.JsonObject;
+
+import org.apache.avro.Schema;
+import org.apache.nifi.annotation.documentation.CapabilityDescription;
+import org.apache.nifi.annotation.documentation.Tags;
+import org.apache.nifi.avro.AvroTypeUtil;
+import org.apache.nifi.components.AllowableValue;
+import org.apache.nifi.components.PropertyDescriptor;
+import org.apache.nifi.components.ValidationContext;
+import org.apache.nifi.components.ValidationResult;
+import org.apache.nifi.components.Validator;
+import org.apache.nifi.controller.status.ProcessGroupStatus;
+import org.apache.nifi.flowfile.attributes.CoreAttributes;
+import org.apache.nifi.processor.exception.ProcessException;
+import org.apache.nifi.processor.util.StandardValidators;
+import org.apache.nifi.remote.Transaction;
+import org.apache.nifi.remote.TransferDirection;
+import org.apache.nifi.reporting.util.metrics.MetricNames;
+import org.apache.nifi.reporting.util.metrics.MetricsService;
+import org.apache.nifi.reporting.util.metrics.api.MetricsBuilder;
+
+import com.yammer.metrics.core.VirtualMachineMetrics;
+
+@Tags({"status", "metrics", "site", "site to site"})
+@CapabilityDescription("Publishes same metrics as the Ambari Reporting 
task using the Site To Site protocol.")
+public class SiteToSiteMetricsReportingTask extends 
AbstractSiteToSiteReportingTask {
+
+static final AllowableValue AMBARI_FORMAT = new 
AllowableValue("ambari-format", "Ambari Format", "Metrics will be formatted"
++ " according to the Ambari Metrics API. See Additional 
Details in Usage documentation.");
+static final AllowableValue RECORD_FORMAT = new 
AllowableValue("record-format", "Record Format", "Metrics will be formatted"
++ " using the Record Writer property of this reporting task. 
See Additional Details in Usage documentation to"
++ " have the description of the default schema.");
+
+static final PropertyDescriptor APPLICATION_ID = new 
PropertyDescriptor.Builder()
+.name("s2s-metrics-application-id")
+.displayName("Application ID")
+.description("The Application ID to be included in the 
metrics")
+.required(true)
+.expressionLanguageSupported(true)
+.defaultValue("nifi")
+.addValidator(StandardValidators.NON_EMPTY_VALIDATOR)
+.build();
+
+static final PropertyDescriptor HOSTNAME = new 
PropertyDescriptor.Builder()
+.name("s2s-metrics-hostname")
+.displayName("Hostname")
+.description("The Hostname of this NiFi instance to be 
included in the metrics")
+.required(true)
+.expressionLanguageSupported(true)
+.defaultValue("${hostname(true)}")
+.addValidator(StandardValidators.NON_EMPTY_VALIDATOR)
+.build();
+
+static final PropertyDescriptor FORMAT = new 

[GitHub] nifi pull request #2430: NIFI-4809 - Implement a SiteToSiteMetricsReportingT...

[mattyb149]

Github user mattyb149 commented on a diff in the pull request:

https://github.com/apache/nifi/pull/2430#discussion_r170728915
  
--- Diff: 
nifi-nar-bundles/nifi-site-to-site-reporting-bundle/nifi-site-to-site-reporting-task/src/main/java/org/apache/nifi/reporting/SiteToSiteMetricsReportingTask.java
 ---
@@ -0,0 +1,151 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements.  See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License.  You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.nifi.reporting;
+
+import java.lang.management.ManagementFactory;
+import java.lang.management.OperatingSystemMXBean;
+import java.nio.charset.StandardCharsets;
+import java.util.ArrayList;
+import java.util.Collections;
+import java.util.HashMap;
+import java.util.List;
+import java.util.Map;
+import java.util.UUID;
+import java.util.concurrent.TimeUnit;
+
+import javax.json.Json;
+import javax.json.JsonBuilderFactory;
+import javax.json.JsonObject;
+
+import org.apache.nifi.annotation.documentation.CapabilityDescription;
+import org.apache.nifi.annotation.documentation.Tags;
+import org.apache.nifi.components.PropertyDescriptor;
+import org.apache.nifi.controller.status.ProcessGroupStatus;
+import org.apache.nifi.processor.exception.ProcessException;
+import org.apache.nifi.processor.util.StandardValidators;
+import org.apache.nifi.remote.Transaction;
+import org.apache.nifi.remote.TransferDirection;
+import org.apache.nifi.reporting.util.metrics.MetricsService;
+import org.apache.nifi.reporting.util.metrics.api.MetricsBuilder;
+
+import com.yammer.metrics.core.VirtualMachineMetrics;
+
+@Tags({"status", "metrics", "site", "site to site"})
+@CapabilityDescription("Publishes same metrics as the Ambari Reporting 
task using the Site To Site protocol. "
++ "Metrics are formatted according to the Ambari Metrics API.")
+public class SiteToSiteMetricsReportingTask extends 
AbstractSiteToSiteReportingTask {
+
+static final String TIMESTAMP_FORMAT = "-MM-dd'T'HH:mm:ss.SSS'Z'";
+
+static final PropertyDescriptor APPLICATION_ID = new 
PropertyDescriptor.Builder()
+.name("Application ID")
--- End diff --

Please set this(and the other properties) to a machine-friendly name, you 
can use the current value for .displayName()


---

[GitHub] nifi pull request #2430: NIFI-4809 - Implement a SiteToSiteMetricsReportingT...

[mattyb149]

Github user mattyb149 commented on a diff in the pull request:

https://github.com/apache/nifi/pull/2430#discussion_r170729017
  
--- Diff: 
nifi-nar-bundles/nifi-site-to-site-reporting-bundle/nifi-site-to-site-reporting-task/src/main/java/org/apache/nifi/reporting/SiteToSiteMetricsReportingTask.java
 ---
@@ -0,0 +1,151 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements.  See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License.  You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.nifi.reporting;
+
+import java.lang.management.ManagementFactory;
+import java.lang.management.OperatingSystemMXBean;
+import java.nio.charset.StandardCharsets;
+import java.util.ArrayList;
+import java.util.Collections;
+import java.util.HashMap;
+import java.util.List;
+import java.util.Map;
+import java.util.UUID;
+import java.util.concurrent.TimeUnit;
+
+import javax.json.Json;
+import javax.json.JsonBuilderFactory;
+import javax.json.JsonObject;
+
+import org.apache.nifi.annotation.documentation.CapabilityDescription;
+import org.apache.nifi.annotation.documentation.Tags;
+import org.apache.nifi.components.PropertyDescriptor;
+import org.apache.nifi.controller.status.ProcessGroupStatus;
+import org.apache.nifi.processor.exception.ProcessException;
+import org.apache.nifi.processor.util.StandardValidators;
+import org.apache.nifi.remote.Transaction;
+import org.apache.nifi.remote.TransferDirection;
+import org.apache.nifi.reporting.util.metrics.MetricsService;
+import org.apache.nifi.reporting.util.metrics.api.MetricsBuilder;
+
+import com.yammer.metrics.core.VirtualMachineMetrics;
+
+@Tags({"status", "metrics", "site", "site to site"})
+@CapabilityDescription("Publishes same metrics as the Ambari Reporting 
task using the Site To Site protocol. "
++ "Metrics are formatted according to the Ambari Metrics API.")
+public class SiteToSiteMetricsReportingTask extends 
AbstractSiteToSiteReportingTask {
+
+static final String TIMESTAMP_FORMAT = "-MM-dd'T'HH:mm:ss.SSS'Z'";
+
+static final PropertyDescriptor APPLICATION_ID = new 
PropertyDescriptor.Builder()
+.name("Application ID")
+.description("The Application ID to be included in the metrics 
sent to Ambari")
--- End diff --

The phrase "sent to Ambari" shows up a couple times, but should be 
replaced/removed


---

[GitHub] nifi pull request #2430: NIFI-4809 - Implement a SiteToSiteMetricsReportingT...

[mattyb149]

Github user mattyb149 commented on a diff in the pull request:

https://github.com/apache/nifi/pull/2430#discussion_r170730176
  
--- Diff: 
nifi-nar-bundles/nifi-site-to-site-reporting-bundle/nifi-site-to-site-reporting-task/src/main/java/org/apache/nifi/reporting/SiteToSiteMetricsReportingTask.java
 ---
@@ -0,0 +1,151 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements.  See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License.  You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.nifi.reporting;
+
+import java.lang.management.ManagementFactory;
+import java.lang.management.OperatingSystemMXBean;
+import java.nio.charset.StandardCharsets;
+import java.util.ArrayList;
+import java.util.Collections;
+import java.util.HashMap;
+import java.util.List;
+import java.util.Map;
+import java.util.UUID;
+import java.util.concurrent.TimeUnit;
+
+import javax.json.Json;
+import javax.json.JsonBuilderFactory;
+import javax.json.JsonObject;
+
+import org.apache.nifi.annotation.documentation.CapabilityDescription;
+import org.apache.nifi.annotation.documentation.Tags;
+import org.apache.nifi.components.PropertyDescriptor;
+import org.apache.nifi.controller.status.ProcessGroupStatus;
+import org.apache.nifi.processor.exception.ProcessException;
+import org.apache.nifi.processor.util.StandardValidators;
+import org.apache.nifi.remote.Transaction;
+import org.apache.nifi.remote.TransferDirection;
+import org.apache.nifi.reporting.util.metrics.MetricsService;
+import org.apache.nifi.reporting.util.metrics.api.MetricsBuilder;
+
+import com.yammer.metrics.core.VirtualMachineMetrics;
+
+@Tags({"status", "metrics", "site", "site to site"})
+@CapabilityDescription("Publishes same metrics as the Ambari Reporting 
task using the Site To Site protocol. "
++ "Metrics are formatted according to the Ambari Metrics API.")
+public class SiteToSiteMetricsReportingTask extends 
AbstractSiteToSiteReportingTask {
+
+static final String TIMESTAMP_FORMAT = "-MM-dd'T'HH:mm:ss.SSS'Z'";
+
+static final PropertyDescriptor APPLICATION_ID = new 
PropertyDescriptor.Builder()
+.name("Application ID")
+.description("The Application ID to be included in the metrics 
sent to Ambari")
+.required(true)
+.expressionLanguageSupported(true)
+.defaultValue("nifi")
+.addValidator(StandardValidators.NON_EMPTY_VALIDATOR)
+.build();
+
+static final PropertyDescriptor HOSTNAME = new 
PropertyDescriptor.Builder()
+.name("Hostname")
+.description("The Hostname of this NiFi instance to be 
included in the metrics sent to Ambari")
+.required(true)
+.expressionLanguageSupported(true)
+.defaultValue("${hostname(true)}")
+.addValidator(StandardValidators.NON_EMPTY_VALIDATOR)
+.build();
+
+private final MetricsService metricsService = new MetricsService();
+
+@Override
+protected List getSupportedPropertyDescriptors() {
+final List properties = new 
ArrayList<>(super.getSupportedPropertyDescriptors());
+properties.add(HOSTNAME);
+properties.add(APPLICATION_ID);
+properties.remove(BATCH_SIZE);
+return properties;
+}
+
+@Override
+public void onTrigger(final ReportingContext context) {
+final boolean isClustered = context.isClustered();
+final String nodeId = context.getClusterNodeIdentifier();
+if (nodeId == null && isClustered) {
+getLogger().debug("This instance of NiFi is configured for 
clustering, but the Cluster Node Identifier is not yet available. "
++ "Will wait for Node Identifier to be established.");
+return;
+}
+
+final VirtualMachineMetrics virtualMachineMetrics = 
VirtualMachineMetrics.getInstance();
+final Map config = Collections.emptyMap();
+

[GitHub] nifi pull request #2430: NIFI-4809 - Implement a SiteToSiteMetricsReportingT...

[pvillard31]

GitHub user pvillard31 opened a pull request:

https://github.com/apache/nifi/pull/2430

NIFI-4809 - Implement a SiteToSiteMetricsReportingTask

To avoid some code duplication, I moved few utilitary classes into the 
reporting-utils package. And I also added two metrics (available cores and load 
average) as we have in System Diagnostic but that we don't send with the 
current Ambari reporting task.

Thank you for submitting a contribution to Apache NiFi.

In order to streamline the review of the contribution we ask you
to ensure the following steps have been taken:

### For all changes:
- [ ] Is there a JIRA ticket associated with this PR? Is it referenced 
 in the commit message?

- [ ] Does your PR title start with NIFI- where  is the JIRA number 
you are trying to resolve? Pay particular attention to the hyphen "-" character.

- [ ] Has your PR been rebased against the latest commit within the target 
branch (typically master)?

- [ ] Is your initial contribution a single, squashed commit?

### For code changes:
- [ ] Have you ensured that the full suite of tests is executed via mvn 
-Pcontrib-check clean install at the root nifi folder?
- [ ] Have you written or updated unit tests to verify your changes?
- [ ] If adding new dependencies to the code, are these dependencies 
licensed in a way that is compatible for inclusion under [ASF 
2.0](http://www.apache.org/legal/resolved.html#category-a)? 
- [ ] If applicable, have you updated the LICENSE file, including the main 
LICENSE file under nifi-assembly?
- [ ] If applicable, have you updated the NOTICE file, including the main 
NOTICE file found under nifi-assembly?
- [ ] If adding new Properties, have you added .displayName in addition to 
.name (programmatic access) for each of the new properties?

### For documentation related changes:
- [ ] Have you ensured that format looks appropriate for the output in 
which it is rendered?

### Note:
Please ensure that once the PR is submitted, you check travis-ci for build 
issues and submit an update to your PR as soon as possible.


You can merge this pull request into a Git repository by running:

$ git pull https://github.com/pvillard31/nifi NIFI-4809

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/nifi/pull/2430.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #2430


commit c49dd0e9ef08b34f30939b2771cd43bf580dc5d4
Author: Pierre Villard 
Date:   2018-01-23T22:15:18Z

NIFI-4809 - Implement a SiteToSiteMetricsReportingTask




---