[
https://issues.apache.org/jira/browse/METRON-1483?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16395852#comment-16395852
]
ASF GitHub Bot commented on METRON-1483:
----------------------------------------
Github user justinleet commented on a diff in the pull request:
https://github.com/apache/metron/pull/958#discussion_r173929511
--- Diff:
metron-contrib/metron-performance/src/main/java/org/apache/metron/performance/load/monitor/writers/Writer.java
---
@@ -0,0 +1,91 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.metron.performance.load.monitor.writers;
+
+import org.apache.commons.math3.stat.descriptive.DescriptiveStatistics;
+import org.apache.metron.performance.load.monitor.AbstractMonitor;
+import org.apache.metron.performance.load.monitor.Results;
+
+import java.util.ArrayList;
+import java.util.Date;
+import java.util.LinkedList;
+import java.util.List;
+import java.util.Optional;
+import java.util.function.Consumer;
+
+public class Writer {
+
+ private int summaryLookback;
+ private List<LinkedList<Double>> summaries = new ArrayList<>();
+ private List<Consumer<Writable>> writers;
+ private List<AbstractMonitor> monitors;
+
+ public Writer(List<AbstractMonitor> monitors, int summaryLookback,
List<Consumer<Writable>> writers) {
+ this.summaryLookback = summaryLookback;
+ this.writers = writers;
+ this.monitors = monitors;
+ for(AbstractMonitor m : monitors) {
+ this.summaries.add(new LinkedList<>());
+ }
+ }
+
+ public void writeAll() {
+ int i = 0;
+ Date dateOf = new Date();
+ List<Results> results = new ArrayList<>();
+ for(AbstractMonitor m : monitors) {
+ Long eps = m.get();
+ if(eps != null) {
+ if (summaryLookback > 0) {
+ LinkedList<Double> summary = summaries.get(i);
+ addToLookback(eps == null ? Double.NaN : eps.doubleValue(),
summary);
+ results.add(new Results(m.format(), m.name(), eps,
Optional.of(getStats(summary))));
+ }
+ else {
+ results.add(new Results(m.format(),m.name(), eps,
Optional.empty()));
+ }
+ }
+ else {
--- End diff --
Since the two else's are the same, I think this should simplify down to
(with the above change):
```
if (eps != null && summaryLookback > 0) {
LinkedList<Double> summary = summaries.get(i);
addToLookback(eps.doubleValue(), summary);
results.add(new Results(m.format(), m.name(), eps,
Optional.of(getStats(summary))));
} else {
results.add(new Results(m.format(), m.name(), eps,
Optional.empty()));
}
```
> Create a tool to monitor performance of the topologies
> ------------------------------------------------------
>
> Key: METRON-1483
> URL: https://issues.apache.org/jira/browse/METRON-1483
> Project: Metron
> Issue Type: New Feature
> Reporter: Casey Stella
> Priority: Major
>
> In performance evaluation, generating synthetic load and monitoring the write
> throughput of our kafka-to-kafka topologies has required a lot of custom
> scripting. We should have a tool that could do the following:
> * Take a file representing a message template and generate synthetic load at
> a given events per second
> * Monitor the kafka offsets of a topic and report throughput numbers in
> events per second
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
