[jira] [Commented] (NIFI-4164) Realistic Time Series Processor Simulator
[ https://issues.apache.org/jira/browse/NIFI-4164?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16384680#comment-16384680 ] ASF GitHub Bot commented on NIFI-4164: -- Github user MikeThomsen commented on the issue: https://github.com/apache/nifi/pull/1997 @cherrera2001 Do you want me to take a stab at completing this ticket? > Realistic Time Series Processor Simulator > - > > Key: NIFI-4164 > URL: https://issues.apache.org/jira/browse/NIFI-4164 > Project: Apache NiFi > Issue Type: New Feature > Components: Extensions >Reporter: Chris Herrera >Assignee: Chris Herrera >Priority: Minor > Labels: features > Original Estimate: 168h > Remaining Estimate: 168h > > In order to validate several flows that deal with sensor data, it would be > good to have a built in time series simulator processor that generates data > and can send it out via a flow file. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (NIFI-4164) Realistic Time Series Processor Simulator
[ https://issues.apache.org/jira/browse/NIFI-4164?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16362298#comment-16362298 ] ASF GitHub Bot commented on NIFI-4164: -- Github user cherrera2001 commented on the issue: https://github.com/apache/nifi/pull/1997 Thanks Mike I did see it, and as such it is getting re written :) > Realistic Time Series Processor Simulator > - > > Key: NIFI-4164 > URL: https://issues.apache.org/jira/browse/NIFI-4164 > Project: Apache NiFi > Issue Type: New Feature > Components: Extensions >Reporter: Chris Herrera >Assignee: Chris Herrera >Priority: Minor > Labels: features > Original Estimate: 168h > Remaining Estimate: 168h > > In order to validate several flows that deal with sensor data, it would be > good to have a built in time series simulator processor that generates data > and can send it out via a flow file. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (NIFI-4164) Realistic Time Series Processor Simulator
[ https://issues.apache.org/jira/browse/NIFI-4164?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16362294#comment-16362294 ] ASF GitHub Bot commented on NIFI-4164: -- Github user MikeThomsen commented on the issue: https://github.com/apache/nifi/pull/1997 @cherrera2001 I haven't seen any activity here, but before we go for round two I would recommend reading the thread "Will you accept contributions in Scala" on nifi-dev. If you're not subscribed, go [here](https://mail-archives.apache.org/mod_mbox/nifi-dev/201802.mbox/browser). Most of the committers seem to be against contributions in Scala so that small Scala file should be rewritten in Java. (As a Groovy fan I can commiserate on not getting to use one's preferred language) > Realistic Time Series Processor Simulator > - > > Key: NIFI-4164 > URL: https://issues.apache.org/jira/browse/NIFI-4164 > Project: Apache NiFi > Issue Type: New Feature > Components: Extensions >Reporter: Chris Herrera >Assignee: Chris Herrera >Priority: Minor > Labels: features > Original Estimate: 168h > Remaining Estimate: 168h > > In order to validate several flows that deal with sensor data, it would be > good to have a built in time series simulator processor that generates data > and can send it out via a flow file. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (NIFI-4164) Realistic Time Series Processor Simulator
[ https://issues.apache.org/jira/browse/NIFI-4164?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16355605#comment-16355605 ] ASF GitHub Bot commented on NIFI-4164: -- Github user cherrera2001 commented on a diff in the pull request: https://github.com/apache/nifi/pull/1997#discussion_r166655562 --- Diff: nifi-nar-bundles/nifi-simulator-bundle/nifi-simulator-processors/src/main/java/com/apache/nifi/processors/simulator/GenerateTimeSeriesFlowFile.java --- @@ -0,0 +1,180 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one or more + * contributor license agreements. See the NOTICE file distributed with + * this work for additional information regarding copyright ownership. + * The ASF licenses this file to You under the Apache License, Version 2.0 + * (the "License"); you may not use this file except in compliance with + * the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package com.apache.nifi.processors.simulator; + +import be.cetic.tsimulus.config.Configuration; +import org.apache.nifi.annotation.behavior.InputRequirement; +import org.apache.nifi.annotation.documentation.CapabilityDescription; +import org.apache.nifi.annotation.documentation.Tags; +import org.apache.nifi.annotation.lifecycle.OnScheduled; +import org.apache.nifi.components.PropertyDescriptor; +import org.apache.nifi.flowfile.FlowFile; +import org.apache.nifi.logging.ComponentLog; +import org.apache.nifi.processor.AbstractProcessor; +import org.apache.nifi.processor.Relationship; +import org.apache.nifi.processor.ProcessorInitializationContext; +import org.apache.nifi.processor.ProcessContext; +import org.apache.nifi.processor.exception.ProcessException; +import org.apache.nifi.processor.ProcessSession; +import org.apache.nifi.processor.util.StandardValidators; +import org.joda.time.LocalDateTime; +import scala.Some; +import scala.Tuple3; +import scala.collection.JavaConverters; + +import java.util.List; +import java.util.Set; +import java.util.Collections; +import java.util.HashSet; +import java.util.ArrayList; + +@Tags({"Simulator, Timeseries, IOT, Testing"}) +@InputRequirement(InputRequirement.Requirement.INPUT_FORBIDDEN) +@CapabilityDescription("Generates realistic time series data using the TSimulus time series generator, and places the values into the flowfile in a CSV format.") +public class GenerateTimeSeriesFlowFile extends AbstractProcessor { + +private Configuration simConfig = null; +private boolean isTest = false; + +public static final PropertyDescriptor SIMULATOR_CONFIG = new PropertyDescriptor +.Builder().name("SIMULATOR_CONFIG") +.displayName("Simulator Configuration File") +.description("The JSON configuration file to use to configure TSimulus") +.required(true) +.addValidator(StandardValidators.FILE_EXISTS_VALIDATOR) +.build(); + +public static final PropertyDescriptor PRINT_HEADER = new PropertyDescriptor +.Builder().name("PRINT_HEADER") +.displayName("Print Header") +.description("Directs the processor whether to print a header line or not.") +.required(true) +.allowableValues("true", "false") +.defaultValue("false") +.addValidator(StandardValidators.BOOLEAN_VALIDATOR) +.build(); + +public static final Relationship SUCCESS = new Relationship.Builder() +.name("Success") +.description("When the flowfile is successfully generated") +.build(); + +private List descriptors; + +private Set relationships; + +@Override +protected void init(final ProcessorInitializationContext context) { + +final List descriptors = new ArrayList<>(); + +descriptors.add(SIMULATOR_CONFIG); +descriptors.add(PRINT_HEADER); + +this.descriptors = Collections.unmodifiableList(descriptors); + +final Set relationships = new HashSet<>(); +relationships.add(SUCCESS); +this.relationships = Collections.unmodifiableSet(relationships); +} + +@Override +public Set getRelationships() {
[jira] [Commented] (NIFI-4164) Realistic Time Series Processor Simulator
[ https://issues.apache.org/jira/browse/NIFI-4164?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16355575#comment-16355575 ] ASF GitHub Bot commented on NIFI-4164: -- Github user YolandaMDavis commented on a diff in the pull request: https://github.com/apache/nifi/pull/1997#discussion_r166648706 --- Diff: nifi-nar-bundles/nifi-simulator-bundle/nifi-simulator-processors/src/main/java/com/apache/nifi/processors/simulator/GenerateTimeSeriesFlowFile.java --- @@ -0,0 +1,180 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one or more + * contributor license agreements. See the NOTICE file distributed with + * this work for additional information regarding copyright ownership. + * The ASF licenses this file to You under the Apache License, Version 2.0 + * (the "License"); you may not use this file except in compliance with + * the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package com.apache.nifi.processors.simulator; + +import be.cetic.tsimulus.config.Configuration; +import org.apache.nifi.annotation.behavior.InputRequirement; +import org.apache.nifi.annotation.documentation.CapabilityDescription; +import org.apache.nifi.annotation.documentation.Tags; +import org.apache.nifi.annotation.lifecycle.OnScheduled; +import org.apache.nifi.components.PropertyDescriptor; +import org.apache.nifi.flowfile.FlowFile; +import org.apache.nifi.logging.ComponentLog; +import org.apache.nifi.processor.AbstractProcessor; +import org.apache.nifi.processor.Relationship; +import org.apache.nifi.processor.ProcessorInitializationContext; +import org.apache.nifi.processor.ProcessContext; +import org.apache.nifi.processor.exception.ProcessException; +import org.apache.nifi.processor.ProcessSession; +import org.apache.nifi.processor.util.StandardValidators; +import org.joda.time.LocalDateTime; +import scala.Some; +import scala.Tuple3; +import scala.collection.JavaConverters; + +import java.util.List; +import java.util.Set; +import java.util.Collections; +import java.util.HashSet; +import java.util.ArrayList; + +@Tags({"Simulator, Timeseries, IOT, Testing"}) +@InputRequirement(InputRequirement.Requirement.INPUT_FORBIDDEN) +@CapabilityDescription("Generates realistic time series data using the TSimulus time series generator, and places the values into the flowfile in a CSV format.") +public class GenerateTimeSeriesFlowFile extends AbstractProcessor { + +private Configuration simConfig = null; +private boolean isTest = false; + +public static final PropertyDescriptor SIMULATOR_CONFIG = new PropertyDescriptor +.Builder().name("SIMULATOR_CONFIG") +.displayName("Simulator Configuration File") +.description("The JSON configuration file to use to configure TSimulus") +.required(true) +.addValidator(StandardValidators.FILE_EXISTS_VALIDATOR) +.build(); + +public static final PropertyDescriptor PRINT_HEADER = new PropertyDescriptor +.Builder().name("PRINT_HEADER") +.displayName("Print Header") +.description("Directs the processor whether to print a header line or not.") +.required(true) +.allowableValues("true", "false") +.defaultValue("false") +.addValidator(StandardValidators.BOOLEAN_VALIDATOR) +.build(); + +public static final Relationship SUCCESS = new Relationship.Builder() +.name("Success") +.description("When the flowfile is successfully generated") +.build(); + +private List descriptors; + +private Set relationships; + +@Override +protected void init(final ProcessorInitializationContext context) { + +final List descriptors = new ArrayList<>(); + +descriptors.add(SIMULATOR_CONFIG); +descriptors.add(PRINT_HEADER); + +this.descriptors = Collections.unmodifiableList(descriptors); + +final Set relationships = new HashSet<>(); +relationships.add(SUCCESS); +this.relationships = Collections.unmodifiableSet(relationships); +} + +@Override +public Set getRelationships() {
[jira] [Commented] (NIFI-4164) Realistic Time Series Processor Simulator
[ https://issues.apache.org/jira/browse/NIFI-4164?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16351961#comment-16351961 ] ASF GitHub Bot commented on NIFI-4164: -- Github user cherrera2001 commented on the issue: https://github.com/apache/nifi/pull/1997 Should be done by Tuesday night CST > Realistic Time Series Processor Simulator > - > > Key: NIFI-4164 > URL: https://issues.apache.org/jira/browse/NIFI-4164 > Project: Apache NiFi > Issue Type: New Feature > Components: Extensions >Reporter: Chris Herrera >Assignee: Chris Herrera >Priority: Minor > Labels: features > Original Estimate: 168h > Remaining Estimate: 168h > > In order to validate several flows that deal with sensor data, it would be > good to have a built in time series simulator processor that generates data > and can send it out via a flow file. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (NIFI-4164) Realistic Time Series Processor Simulator
[ https://issues.apache.org/jira/browse/NIFI-4164?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16351907#comment-16351907 ] ASF GitHub Bot commented on NIFI-4164: -- Github user MikeThomsen commented on the issue: https://github.com/apache/nifi/pull/1997 @cherrera2001 Do you have any updates? > Realistic Time Series Processor Simulator > - > > Key: NIFI-4164 > URL: https://issues.apache.org/jira/browse/NIFI-4164 > Project: Apache NiFi > Issue Type: New Feature > Components: Extensions >Reporter: Chris Herrera >Assignee: Chris Herrera >Priority: Minor > Labels: features > Original Estimate: 168h > Remaining Estimate: 168h > > In order to validate several flows that deal with sensor data, it would be > good to have a built in time series simulator processor that generates data > and can send it out via a flow file. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (NIFI-4164) Realistic Time Series Processor Simulator
[ https://issues.apache.org/jira/browse/NIFI-4164?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16348564#comment-16348564 ] ASF GitHub Bot commented on NIFI-4164: -- Github user MikeThomsen commented on a diff in the pull request: https://github.com/apache/nifi/pull/1997#discussion_r165352409 --- Diff: nifi-nar-bundles/nifi-simulator-bundle/nifi-simulator-processors/src/main/java/com/apache/nifi/processors/simulator/GenerateTimeSeriesFlowFile.java --- @@ -0,0 +1,180 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one or more + * contributor license agreements. See the NOTICE file distributed with + * this work for additional information regarding copyright ownership. + * The ASF licenses this file to You under the Apache License, Version 2.0 + * (the "License"); you may not use this file except in compliance with + * the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package com.apache.nifi.processors.simulator; + +import be.cetic.tsimulus.config.Configuration; +import org.apache.nifi.annotation.behavior.InputRequirement; +import org.apache.nifi.annotation.documentation.CapabilityDescription; +import org.apache.nifi.annotation.documentation.Tags; +import org.apache.nifi.annotation.lifecycle.OnScheduled; +import org.apache.nifi.components.PropertyDescriptor; +import org.apache.nifi.flowfile.FlowFile; +import org.apache.nifi.logging.ComponentLog; +import org.apache.nifi.processor.AbstractProcessor; +import org.apache.nifi.processor.Relationship; +import org.apache.nifi.processor.ProcessorInitializationContext; +import org.apache.nifi.processor.ProcessContext; +import org.apache.nifi.processor.exception.ProcessException; +import org.apache.nifi.processor.ProcessSession; +import org.apache.nifi.processor.util.StandardValidators; +import org.joda.time.LocalDateTime; +import scala.Some; +import scala.Tuple3; +import scala.collection.JavaConverters; + +import java.util.List; +import java.util.Set; +import java.util.Collections; +import java.util.HashSet; +import java.util.ArrayList; + +@Tags({"Simulator, Timeseries, IOT, Testing"}) +@InputRequirement(InputRequirement.Requirement.INPUT_FORBIDDEN) +@CapabilityDescription("Generates realistic time series data using the TSimulus time series generator, and places the values into the flowfile in a CSV format.") +public class GenerateTimeSeriesFlowFile extends AbstractProcessor { + +private Configuration simConfig = null; +private boolean isTest = false; + +public static final PropertyDescriptor SIMULATOR_CONFIG = new PropertyDescriptor +.Builder().name("SIMULATOR_CONFIG") +.displayName("Simulator Configuration File") +.description("The JSON configuration file to use to configure TSimulus") +.required(true) +.addValidator(StandardValidators.FILE_EXISTS_VALIDATOR) --- End diff -- I think you should consider creating your own validator that does a combination of file existence check and testing whether or not you can create a config that the API will accept from it. > Realistic Time Series Processor Simulator > - > > Key: NIFI-4164 > URL: https://issues.apache.org/jira/browse/NIFI-4164 > Project: Apache NiFi > Issue Type: New Feature > Components: Extensions >Reporter: Chris Herrera >Assignee: Chris Herrera >Priority: Minor > Labels: features > Original Estimate: 168h > Remaining Estimate: 168h > > In order to validate several flows that deal with sensor data, it would be > good to have a built in time series simulator processor that generates data > and can send it out via a flow file. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (NIFI-4164) Realistic Time Series Processor Simulator
[ https://issues.apache.org/jira/browse/NIFI-4164?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16346734#comment-16346734 ] ASF GitHub Bot commented on NIFI-4164: -- Github user cherrera2001 commented on a diff in the pull request: https://github.com/apache/nifi/pull/1997#discussion_r165036667 --- Diff: nifi-nar-bundles/nifi-simulator-bundle/nifi-simulator-processors/src/main/java/com/apache/nifi/processors/simulator/GenerateTimeSeriesFlowFile.java --- @@ -0,0 +1,180 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one or more + * contributor license agreements. See the NOTICE file distributed with + * this work for additional information regarding copyright ownership. + * The ASF licenses this file to You under the Apache License, Version 2.0 + * (the "License"); you may not use this file except in compliance with + * the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package com.apache.nifi.processors.simulator; + +import be.cetic.tsimulus.config.Configuration; +import org.apache.nifi.annotation.behavior.InputRequirement; +import org.apache.nifi.annotation.documentation.CapabilityDescription; +import org.apache.nifi.annotation.documentation.Tags; +import org.apache.nifi.annotation.lifecycle.OnScheduled; +import org.apache.nifi.components.PropertyDescriptor; +import org.apache.nifi.flowfile.FlowFile; +import org.apache.nifi.logging.ComponentLog; +import org.apache.nifi.processor.AbstractProcessor; +import org.apache.nifi.processor.Relationship; +import org.apache.nifi.processor.ProcessorInitializationContext; +import org.apache.nifi.processor.ProcessContext; +import org.apache.nifi.processor.exception.ProcessException; +import org.apache.nifi.processor.ProcessSession; +import org.apache.nifi.processor.util.StandardValidators; +import org.joda.time.LocalDateTime; +import scala.Some; +import scala.Tuple3; +import scala.collection.JavaConverters; + +import java.util.List; +import java.util.Set; +import java.util.Collections; +import java.util.HashSet; +import java.util.ArrayList; + +@Tags({"Simulator, Timeseries, IOT, Testing"}) +@InputRequirement(InputRequirement.Requirement.INPUT_FORBIDDEN) +@CapabilityDescription("Generates realistic time series data using the TSimulus time series generator, and places the values into the flowfile in a CSV format.") +public class GenerateTimeSeriesFlowFile extends AbstractProcessor { + +private Configuration simConfig = null; +private boolean isTest = false; + +public static final PropertyDescriptor SIMULATOR_CONFIG = new PropertyDescriptor +.Builder().name("SIMULATOR_CONFIG") +.displayName("Simulator Configuration File") +.description("The JSON configuration file to use to configure TSimulus") --- End diff -- Good Point, I think I will update the description as well as allow for the inclusion of the JSON directly in a property. This goes well with allowing someone to pass in a configuration as part of a flow. > Realistic Time Series Processor Simulator > - > > Key: NIFI-4164 > URL: https://issues.apache.org/jira/browse/NIFI-4164 > Project: Apache NiFi > Issue Type: New Feature > Components: Extensions >Reporter: Chris Herrera >Assignee: Chris Herrera >Priority: Minor > Labels: features > Original Estimate: 168h > Remaining Estimate: 168h > > In order to validate several flows that deal with sensor data, it would be > good to have a built in time series simulator processor that generates data > and can send it out via a flow file. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (NIFI-4164) Realistic Time Series Processor Simulator
[ https://issues.apache.org/jira/browse/NIFI-4164?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16346737#comment-16346737 ] ASF GitHub Bot commented on NIFI-4164: -- Github user cherrera2001 commented on a diff in the pull request: https://github.com/apache/nifi/pull/1997#discussion_r165036864 --- Diff: nifi-nar-bundles/nifi-simulator-bundle/nifi-simulator-processors/src/main/java/com/apache/nifi/processors/simulator/GenerateTimeSeriesFlowFile.java --- @@ -0,0 +1,180 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one or more + * contributor license agreements. See the NOTICE file distributed with + * this work for additional information regarding copyright ownership. + * The ASF licenses this file to You under the Apache License, Version 2.0 + * (the "License"); you may not use this file except in compliance with + * the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package com.apache.nifi.processors.simulator; + +import be.cetic.tsimulus.config.Configuration; +import org.apache.nifi.annotation.behavior.InputRequirement; +import org.apache.nifi.annotation.documentation.CapabilityDescription; +import org.apache.nifi.annotation.documentation.Tags; +import org.apache.nifi.annotation.lifecycle.OnScheduled; +import org.apache.nifi.components.PropertyDescriptor; +import org.apache.nifi.flowfile.FlowFile; +import org.apache.nifi.logging.ComponentLog; +import org.apache.nifi.processor.AbstractProcessor; +import org.apache.nifi.processor.Relationship; +import org.apache.nifi.processor.ProcessorInitializationContext; +import org.apache.nifi.processor.ProcessContext; +import org.apache.nifi.processor.exception.ProcessException; +import org.apache.nifi.processor.ProcessSession; +import org.apache.nifi.processor.util.StandardValidators; +import org.joda.time.LocalDateTime; +import scala.Some; +import scala.Tuple3; +import scala.collection.JavaConverters; + +import java.util.List; +import java.util.Set; +import java.util.Collections; +import java.util.HashSet; +import java.util.ArrayList; + +@Tags({"Simulator, Timeseries, IOT, Testing"}) +@InputRequirement(InputRequirement.Requirement.INPUT_FORBIDDEN) +@CapabilityDescription("Generates realistic time series data using the TSimulus time series generator, and places the values into the flowfile in a CSV format.") +public class GenerateTimeSeriesFlowFile extends AbstractProcessor { + +private Configuration simConfig = null; +private boolean isTest = false; + +public static final PropertyDescriptor SIMULATOR_CONFIG = new PropertyDescriptor +.Builder().name("SIMULATOR_CONFIG") +.displayName("Simulator Configuration File") +.description("The JSON configuration file to use to configure TSimulus") +.required(true) +.addValidator(StandardValidators.FILE_EXISTS_VALIDATOR) +.build(); + +public static final PropertyDescriptor PRINT_HEADER = new PropertyDescriptor +.Builder().name("PRINT_HEADER") +.displayName("Print Header") +.description("Directs the processor whether to print a header line or not.") +.required(true) +.allowableValues("true", "false") +.defaultValue("false") +.addValidator(StandardValidators.BOOLEAN_VALIDATOR) +.build(); + +public static final Relationship SUCCESS = new Relationship.Builder() +.name("Success") +.description("When the flowfile is successfully generated") +.build(); + +private List descriptors; + +private Set relationships; + +@Override +protected void init(final ProcessorInitializationContext context) { + +final List descriptors = new ArrayList<>(); + +descriptors.add(SIMULATOR_CONFIG); +descriptors.add(PRINT_HEADER); + +this.descriptors = Collections.unmodifiableList(descriptors); + +final Set relationships = new HashSet<>(); +relationships.add(SUCCESS); +this.relationships = Collections.unmodifiableSet(relationships); +} + +@Override +public Set getRelationships() {
[jira] [Commented] (NIFI-4164) Realistic Time Series Processor Simulator
[ https://issues.apache.org/jira/browse/NIFI-4164?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16346736#comment-16346736 ] ASF GitHub Bot commented on NIFI-4164: -- Github user cherrera2001 commented on a diff in the pull request: https://github.com/apache/nifi/pull/1997#discussion_r165036821 --- Diff: nifi-nar-bundles/nifi-simulator-bundle/nifi-simulator-processors/src/main/java/com/apache/nifi/processors/simulator/GenerateTimeSeriesFlowFile.java --- @@ -0,0 +1,180 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one or more + * contributor license agreements. See the NOTICE file distributed with + * this work for additional information regarding copyright ownership. + * The ASF licenses this file to You under the Apache License, Version 2.0 + * (the "License"); you may not use this file except in compliance with + * the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package com.apache.nifi.processors.simulator; + +import be.cetic.tsimulus.config.Configuration; +import org.apache.nifi.annotation.behavior.InputRequirement; +import org.apache.nifi.annotation.documentation.CapabilityDescription; +import org.apache.nifi.annotation.documentation.Tags; +import org.apache.nifi.annotation.lifecycle.OnScheduled; +import org.apache.nifi.components.PropertyDescriptor; +import org.apache.nifi.flowfile.FlowFile; +import org.apache.nifi.logging.ComponentLog; +import org.apache.nifi.processor.AbstractProcessor; +import org.apache.nifi.processor.Relationship; +import org.apache.nifi.processor.ProcessorInitializationContext; +import org.apache.nifi.processor.ProcessContext; +import org.apache.nifi.processor.exception.ProcessException; +import org.apache.nifi.processor.ProcessSession; +import org.apache.nifi.processor.util.StandardValidators; +import org.joda.time.LocalDateTime; +import scala.Some; +import scala.Tuple3; +import scala.collection.JavaConverters; + +import java.util.List; +import java.util.Set; +import java.util.Collections; +import java.util.HashSet; +import java.util.ArrayList; + +@Tags({"Simulator, Timeseries, IOT, Testing"}) +@InputRequirement(InputRequirement.Requirement.INPUT_FORBIDDEN) +@CapabilityDescription("Generates realistic time series data using the TSimulus time series generator, and places the values into the flowfile in a CSV format.") +public class GenerateTimeSeriesFlowFile extends AbstractProcessor { + +private Configuration simConfig = null; +private boolean isTest = false; + +public static final PropertyDescriptor SIMULATOR_CONFIG = new PropertyDescriptor +.Builder().name("SIMULATOR_CONFIG") +.displayName("Simulator Configuration File") +.description("The JSON configuration file to use to configure TSimulus") +.required(true) +.addValidator(StandardValidators.FILE_EXISTS_VALIDATOR) +.build(); + +public static final PropertyDescriptor PRINT_HEADER = new PropertyDescriptor +.Builder().name("PRINT_HEADER") +.displayName("Print Header") +.description("Directs the processor whether to print a header line or not.") +.required(true) +.allowableValues("true", "false") +.defaultValue("false") +.addValidator(StandardValidators.BOOLEAN_VALIDATOR) +.build(); + +public static final Relationship SUCCESS = new Relationship.Builder() +.name("Success") +.description("When the flowfile is successfully generated") +.build(); + +private List descriptors; + +private Set relationships; + +@Override +protected void init(final ProcessorInitializationContext context) { + +final List descriptors = new ArrayList<>(); + +descriptors.add(SIMULATOR_CONFIG); +descriptors.add(PRINT_HEADER); + +this.descriptors = Collections.unmodifiableList(descriptors); + +final Set relationships = new HashSet<>(); +relationships.add(SUCCESS); +this.relationships = Collections.unmodifiableSet(relationships); +} + +@Override +public Set getRelationships() {
[jira] [Commented] (NIFI-4164) Realistic Time Series Processor Simulator
[ https://issues.apache.org/jira/browse/NIFI-4164?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16346735#comment-16346735 ] ASF GitHub Bot commented on NIFI-4164: -- Github user cherrera2001 commented on a diff in the pull request: https://github.com/apache/nifi/pull/1997#discussion_r165036757 --- Diff: nifi-nar-bundles/nifi-simulator-bundle/nifi-simulator-processors/src/main/java/com/apache/nifi/processors/simulator/GenerateTimeSeriesFlowFile.java --- @@ -0,0 +1,180 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one or more + * contributor license agreements. See the NOTICE file distributed with + * this work for additional information regarding copyright ownership. + * The ASF licenses this file to You under the Apache License, Version 2.0 + * (the "License"); you may not use this file except in compliance with + * the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package com.apache.nifi.processors.simulator; + +import be.cetic.tsimulus.config.Configuration; +import org.apache.nifi.annotation.behavior.InputRequirement; +import org.apache.nifi.annotation.documentation.CapabilityDescription; +import org.apache.nifi.annotation.documentation.Tags; +import org.apache.nifi.annotation.lifecycle.OnScheduled; +import org.apache.nifi.components.PropertyDescriptor; +import org.apache.nifi.flowfile.FlowFile; +import org.apache.nifi.logging.ComponentLog; +import org.apache.nifi.processor.AbstractProcessor; +import org.apache.nifi.processor.Relationship; +import org.apache.nifi.processor.ProcessorInitializationContext; +import org.apache.nifi.processor.ProcessContext; +import org.apache.nifi.processor.exception.ProcessException; +import org.apache.nifi.processor.ProcessSession; +import org.apache.nifi.processor.util.StandardValidators; +import org.joda.time.LocalDateTime; +import scala.Some; +import scala.Tuple3; +import scala.collection.JavaConverters; + +import java.util.List; +import java.util.Set; +import java.util.Collections; +import java.util.HashSet; +import java.util.ArrayList; + +@Tags({"Simulator, Timeseries, IOT, Testing"}) +@InputRequirement(InputRequirement.Requirement.INPUT_FORBIDDEN) +@CapabilityDescription("Generates realistic time series data using the TSimulus time series generator, and places the values into the flowfile in a CSV format.") +public class GenerateTimeSeriesFlowFile extends AbstractProcessor { + +private Configuration simConfig = null; +private boolean isTest = false; + +public static final PropertyDescriptor SIMULATOR_CONFIG = new PropertyDescriptor +.Builder().name("SIMULATOR_CONFIG") +.displayName("Simulator Configuration File") +.description("The JSON configuration file to use to configure TSimulus") +.required(true) +.addValidator(StandardValidators.FILE_EXISTS_VALIDATOR) +.build(); + +public static final PropertyDescriptor PRINT_HEADER = new PropertyDescriptor +.Builder().name("PRINT_HEADER") +.displayName("Print Header") +.description("Directs the processor whether to print a header line or not.") +.required(true) +.allowableValues("true", "false") +.defaultValue("false") +.addValidator(StandardValidators.BOOLEAN_VALIDATOR) +.build(); + +public static final Relationship SUCCESS = new Relationship.Builder() +.name("Success") +.description("When the flowfile is successfully generated") --- End diff -- Good Point. > Realistic Time Series Processor Simulator > - > > Key: NIFI-4164 > URL: https://issues.apache.org/jira/browse/NIFI-4164 > Project: Apache NiFi > Issue Type: New Feature > Components: Extensions >Reporter: Chris Herrera >Assignee: Chris Herrera >Priority: Minor > Labels: features > Original Estimate: 168h > Remaining Estimate: 168h > > In order to validate several flows that deal with sensor data, it would be > good to have a built in time series simulator processor that generates data > and can send it out
[jira] [Commented] (NIFI-4164) Realistic Time Series Processor Simulator
[ https://issues.apache.org/jira/browse/NIFI-4164?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16346731#comment-16346731 ] ASF GitHub Bot commented on NIFI-4164: -- Github user cherrera2001 commented on a diff in the pull request: https://github.com/apache/nifi/pull/1997#discussion_r165036464 --- Diff: nifi-nar-bundles/nifi-simulator-bundle/nifi-simulator-processors/src/main/java/com/apache/nifi/processors/simulator/GenerateTimeSeriesFlowFile.java --- @@ -0,0 +1,180 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one or more + * contributor license agreements. See the NOTICE file distributed with + * this work for additional information regarding copyright ownership. + * The ASF licenses this file to You under the Apache License, Version 2.0 + * (the "License"); you may not use this file except in compliance with + * the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package com.apache.nifi.processors.simulator; + +import be.cetic.tsimulus.config.Configuration; +import org.apache.nifi.annotation.behavior.InputRequirement; +import org.apache.nifi.annotation.documentation.CapabilityDescription; +import org.apache.nifi.annotation.documentation.Tags; +import org.apache.nifi.annotation.lifecycle.OnScheduled; +import org.apache.nifi.components.PropertyDescriptor; +import org.apache.nifi.flowfile.FlowFile; +import org.apache.nifi.logging.ComponentLog; +import org.apache.nifi.processor.AbstractProcessor; +import org.apache.nifi.processor.Relationship; +import org.apache.nifi.processor.ProcessorInitializationContext; +import org.apache.nifi.processor.ProcessContext; +import org.apache.nifi.processor.exception.ProcessException; +import org.apache.nifi.processor.ProcessSession; +import org.apache.nifi.processor.util.StandardValidators; +import org.joda.time.LocalDateTime; +import scala.Some; +import scala.Tuple3; +import scala.collection.JavaConverters; + +import java.util.List; +import java.util.Set; +import java.util.Collections; +import java.util.HashSet; +import java.util.ArrayList; + +@Tags({"Simulator, Timeseries, IOT, Testing"}) +@InputRequirement(InputRequirement.Requirement.INPUT_FORBIDDEN) --- End diff -- Its a good point I think its a valuable addition. I will modify this. > Realistic Time Series Processor Simulator > - > > Key: NIFI-4164 > URL: https://issues.apache.org/jira/browse/NIFI-4164 > Project: Apache NiFi > Issue Type: New Feature > Components: Extensions >Reporter: Chris Herrera >Assignee: Chris Herrera >Priority: Minor > Labels: features > Original Estimate: 168h > Remaining Estimate: 168h > > In order to validate several flows that deal with sensor data, it would be > good to have a built in time series simulator processor that generates data > and can send it out via a flow file. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (NIFI-4164) Realistic Time Series Processor Simulator
[ https://issues.apache.org/jira/browse/NIFI-4164?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16346707#comment-16346707 ] ASF GitHub Bot commented on NIFI-4164: -- Github user MikeThomsen commented on a diff in the pull request: https://github.com/apache/nifi/pull/1997#discussion_r165031960 --- Diff: nifi-nar-bundles/nifi-simulator-bundle/nifi-simulator-processors/src/main/java/com/apache/nifi/processors/simulator/GenerateTimeSeriesFlowFile.java --- @@ -0,0 +1,180 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one or more + * contributor license agreements. See the NOTICE file distributed with + * this work for additional information regarding copyright ownership. + * The ASF licenses this file to You under the Apache License, Version 2.0 + * (the "License"); you may not use this file except in compliance with + * the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package com.apache.nifi.processors.simulator; + +import be.cetic.tsimulus.config.Configuration; +import org.apache.nifi.annotation.behavior.InputRequirement; +import org.apache.nifi.annotation.documentation.CapabilityDescription; +import org.apache.nifi.annotation.documentation.Tags; +import org.apache.nifi.annotation.lifecycle.OnScheduled; +import org.apache.nifi.components.PropertyDescriptor; +import org.apache.nifi.flowfile.FlowFile; +import org.apache.nifi.logging.ComponentLog; +import org.apache.nifi.processor.AbstractProcessor; +import org.apache.nifi.processor.Relationship; +import org.apache.nifi.processor.ProcessorInitializationContext; +import org.apache.nifi.processor.ProcessContext; +import org.apache.nifi.processor.exception.ProcessException; +import org.apache.nifi.processor.ProcessSession; +import org.apache.nifi.processor.util.StandardValidators; +import org.joda.time.LocalDateTime; +import scala.Some; +import scala.Tuple3; +import scala.collection.JavaConverters; + +import java.util.List; +import java.util.Set; +import java.util.Collections; +import java.util.HashSet; +import java.util.ArrayList; + +@Tags({"Simulator, Timeseries, IOT, Testing"}) +@InputRequirement(InputRequirement.Requirement.INPUT_FORBIDDEN) +@CapabilityDescription("Generates realistic time series data using the TSimulus time series generator, and places the values into the flowfile in a CSV format.") +public class GenerateTimeSeriesFlowFile extends AbstractProcessor { + +private Configuration simConfig = null; +private boolean isTest = false; + +public static final PropertyDescriptor SIMULATOR_CONFIG = new PropertyDescriptor +.Builder().name("SIMULATOR_CONFIG") +.displayName("Simulator Configuration File") +.description("The JSON configuration file to use to configure TSimulus") +.required(true) +.addValidator(StandardValidators.FILE_EXISTS_VALIDATOR) +.build(); + +public static final PropertyDescriptor PRINT_HEADER = new PropertyDescriptor +.Builder().name("PRINT_HEADER") +.displayName("Print Header") +.description("Directs the processor whether to print a header line or not.") +.required(true) +.allowableValues("true", "false") +.defaultValue("false") +.addValidator(StandardValidators.BOOLEAN_VALIDATOR) +.build(); + +public static final Relationship SUCCESS = new Relationship.Builder() +.name("Success") +.description("When the flowfile is successfully generated") +.build(); + +private List descriptors; + +private Set relationships; + +@Override +protected void init(final ProcessorInitializationContext context) { + +final List descriptors = new ArrayList<>(); + +descriptors.add(SIMULATOR_CONFIG); +descriptors.add(PRINT_HEADER); + +this.descriptors = Collections.unmodifiableList(descriptors); + +final Set relationships = new HashSet<>(); +relationships.add(SUCCESS); +this.relationships = Collections.unmodifiableSet(relationships); +} + +@Override +public Set getRelationships() {
[jira] [Commented] (NIFI-4164) Realistic Time Series Processor Simulator
[ https://issues.apache.org/jira/browse/NIFI-4164?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16346700#comment-16346700 ] ASF GitHub Bot commented on NIFI-4164: -- Github user MikeThomsen commented on a diff in the pull request: https://github.com/apache/nifi/pull/1997#discussion_r165031639 --- Diff: nifi-nar-bundles/nifi-simulator-bundle/nifi-simulator-processors/src/main/java/com/apache/nifi/processors/simulator/GenerateTimeSeriesFlowFile.java --- @@ -0,0 +1,180 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one or more + * contributor license agreements. See the NOTICE file distributed with + * this work for additional information regarding copyright ownership. + * The ASF licenses this file to You under the Apache License, Version 2.0 + * (the "License"); you may not use this file except in compliance with + * the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package com.apache.nifi.processors.simulator; + +import be.cetic.tsimulus.config.Configuration; +import org.apache.nifi.annotation.behavior.InputRequirement; +import org.apache.nifi.annotation.documentation.CapabilityDescription; +import org.apache.nifi.annotation.documentation.Tags; +import org.apache.nifi.annotation.lifecycle.OnScheduled; +import org.apache.nifi.components.PropertyDescriptor; +import org.apache.nifi.flowfile.FlowFile; +import org.apache.nifi.logging.ComponentLog; +import org.apache.nifi.processor.AbstractProcessor; +import org.apache.nifi.processor.Relationship; +import org.apache.nifi.processor.ProcessorInitializationContext; +import org.apache.nifi.processor.ProcessContext; +import org.apache.nifi.processor.exception.ProcessException; +import org.apache.nifi.processor.ProcessSession; +import org.apache.nifi.processor.util.StandardValidators; +import org.joda.time.LocalDateTime; +import scala.Some; +import scala.Tuple3; +import scala.collection.JavaConverters; + +import java.util.List; +import java.util.Set; +import java.util.Collections; +import java.util.HashSet; +import java.util.ArrayList; + +@Tags({"Simulator, Timeseries, IOT, Testing"}) +@InputRequirement(InputRequirement.Requirement.INPUT_FORBIDDEN) +@CapabilityDescription("Generates realistic time series data using the TSimulus time series generator, and places the values into the flowfile in a CSV format.") +public class GenerateTimeSeriesFlowFile extends AbstractProcessor { + +private Configuration simConfig = null; +private boolean isTest = false; + +public static final PropertyDescriptor SIMULATOR_CONFIG = new PropertyDescriptor +.Builder().name("SIMULATOR_CONFIG") +.displayName("Simulator Configuration File") +.description("The JSON configuration file to use to configure TSimulus") +.required(true) +.addValidator(StandardValidators.FILE_EXISTS_VALIDATOR) +.build(); + +public static final PropertyDescriptor PRINT_HEADER = new PropertyDescriptor +.Builder().name("PRINT_HEADER") +.displayName("Print Header") +.description("Directs the processor whether to print a header line or not.") +.required(true) +.allowableValues("true", "false") +.defaultValue("false") +.addValidator(StandardValidators.BOOLEAN_VALIDATOR) +.build(); + +public static final Relationship SUCCESS = new Relationship.Builder() +.name("Success") +.description("When the flowfile is successfully generated") +.build(); + +private List descriptors; + +private Set relationships; + +@Override +protected void init(final ProcessorInitializationContext context) { + +final List descriptors = new ArrayList<>(); + +descriptors.add(SIMULATOR_CONFIG); +descriptors.add(PRINT_HEADER); + +this.descriptors = Collections.unmodifiableList(descriptors); + +final Set relationships = new HashSet<>(); +relationships.add(SUCCESS); +this.relationships = Collections.unmodifiableSet(relationships); +} + +@Override +public Set getRelationships() {
[jira] [Commented] (NIFI-4164) Realistic Time Series Processor Simulator
[ https://issues.apache.org/jira/browse/NIFI-4164?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16346706#comment-16346706 ] ASF GitHub Bot commented on NIFI-4164: -- Github user MikeThomsen commented on a diff in the pull request: https://github.com/apache/nifi/pull/1997#discussion_r165030357 --- Diff: nifi-nar-bundles/nifi-simulator-bundle/nifi-simulator-processors/src/main/java/com/apache/nifi/processors/simulator/GenerateTimeSeriesFlowFile.java --- @@ -0,0 +1,180 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one or more + * contributor license agreements. See the NOTICE file distributed with + * this work for additional information regarding copyright ownership. + * The ASF licenses this file to You under the Apache License, Version 2.0 + * (the "License"); you may not use this file except in compliance with + * the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package com.apache.nifi.processors.simulator; + +import be.cetic.tsimulus.config.Configuration; +import org.apache.nifi.annotation.behavior.InputRequirement; +import org.apache.nifi.annotation.documentation.CapabilityDescription; +import org.apache.nifi.annotation.documentation.Tags; +import org.apache.nifi.annotation.lifecycle.OnScheduled; +import org.apache.nifi.components.PropertyDescriptor; +import org.apache.nifi.flowfile.FlowFile; +import org.apache.nifi.logging.ComponentLog; +import org.apache.nifi.processor.AbstractProcessor; +import org.apache.nifi.processor.Relationship; +import org.apache.nifi.processor.ProcessorInitializationContext; +import org.apache.nifi.processor.ProcessContext; +import org.apache.nifi.processor.exception.ProcessException; +import org.apache.nifi.processor.ProcessSession; +import org.apache.nifi.processor.util.StandardValidators; +import org.joda.time.LocalDateTime; +import scala.Some; +import scala.Tuple3; +import scala.collection.JavaConverters; + +import java.util.List; +import java.util.Set; +import java.util.Collections; +import java.util.HashSet; +import java.util.ArrayList; + +@Tags({"Simulator, Timeseries, IOT, Testing"}) +@InputRequirement(InputRequirement.Requirement.INPUT_FORBIDDEN) +@CapabilityDescription("Generates realistic time series data using the TSimulus time series generator, and places the values into the flowfile in a CSV format.") +public class GenerateTimeSeriesFlowFile extends AbstractProcessor { + +private Configuration simConfig = null; +private boolean isTest = false; + +public static final PropertyDescriptor SIMULATOR_CONFIG = new PropertyDescriptor +.Builder().name("SIMULATOR_CONFIG") +.displayName("Simulator Configuration File") +.description("The JSON configuration file to use to configure TSimulus") --- End diff -- I think you're going to need to get a little more detailed here. Depending on how big the config file is, you might be better off trying to make it constructable from the processor's properties. > Realistic Time Series Processor Simulator > - > > Key: NIFI-4164 > URL: https://issues.apache.org/jira/browse/NIFI-4164 > Project: Apache NiFi > Issue Type: New Feature > Components: Extensions >Reporter: Chris Herrera >Assignee: Chris Herrera >Priority: Minor > Labels: features > Original Estimate: 168h > Remaining Estimate: 168h > > In order to validate several flows that deal with sensor data, it would be > good to have a built in time series simulator processor that generates data > and can send it out via a flow file. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (NIFI-4164) Realistic Time Series Processor Simulator
[ https://issues.apache.org/jira/browse/NIFI-4164?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16346709#comment-16346709 ] ASF GitHub Bot commented on NIFI-4164: -- Github user MikeThomsen commented on a diff in the pull request: https://github.com/apache/nifi/pull/1997#discussion_r165030668 --- Diff: nifi-nar-bundles/nifi-simulator-bundle/nifi-simulator-processors/src/main/java/com/apache/nifi/processors/simulator/GenerateTimeSeriesFlowFile.java --- @@ -0,0 +1,180 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one or more + * contributor license agreements. See the NOTICE file distributed with + * this work for additional information regarding copyright ownership. + * The ASF licenses this file to You under the Apache License, Version 2.0 + * (the "License"); you may not use this file except in compliance with + * the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package com.apache.nifi.processors.simulator; + +import be.cetic.tsimulus.config.Configuration; +import org.apache.nifi.annotation.behavior.InputRequirement; +import org.apache.nifi.annotation.documentation.CapabilityDescription; +import org.apache.nifi.annotation.documentation.Tags; +import org.apache.nifi.annotation.lifecycle.OnScheduled; +import org.apache.nifi.components.PropertyDescriptor; +import org.apache.nifi.flowfile.FlowFile; +import org.apache.nifi.logging.ComponentLog; +import org.apache.nifi.processor.AbstractProcessor; +import org.apache.nifi.processor.Relationship; +import org.apache.nifi.processor.ProcessorInitializationContext; +import org.apache.nifi.processor.ProcessContext; +import org.apache.nifi.processor.exception.ProcessException; +import org.apache.nifi.processor.ProcessSession; +import org.apache.nifi.processor.util.StandardValidators; +import org.joda.time.LocalDateTime; +import scala.Some; +import scala.Tuple3; +import scala.collection.JavaConverters; + +import java.util.List; +import java.util.Set; +import java.util.Collections; +import java.util.HashSet; +import java.util.ArrayList; + +@Tags({"Simulator, Timeseries, IOT, Testing"}) +@InputRequirement(InputRequirement.Requirement.INPUT_FORBIDDEN) +@CapabilityDescription("Generates realistic time series data using the TSimulus time series generator, and places the values into the flowfile in a CSV format.") +public class GenerateTimeSeriesFlowFile extends AbstractProcessor { + +private Configuration simConfig = null; +private boolean isTest = false; + +public static final PropertyDescriptor SIMULATOR_CONFIG = new PropertyDescriptor +.Builder().name("SIMULATOR_CONFIG") +.displayName("Simulator Configuration File") +.description("The JSON configuration file to use to configure TSimulus") +.required(true) +.addValidator(StandardValidators.FILE_EXISTS_VALIDATOR) +.build(); + +public static final PropertyDescriptor PRINT_HEADER = new PropertyDescriptor +.Builder().name("PRINT_HEADER") +.displayName("Print Header") +.description("Directs the processor whether to print a header line or not.") +.required(true) +.allowableValues("true", "false") --- End diff -- AllowableValue objects are a better way to go here because they provide a value and a display label. > Realistic Time Series Processor Simulator > - > > Key: NIFI-4164 > URL: https://issues.apache.org/jira/browse/NIFI-4164 > Project: Apache NiFi > Issue Type: New Feature > Components: Extensions >Reporter: Chris Herrera >Assignee: Chris Herrera >Priority: Minor > Labels: features > Original Estimate: 168h > Remaining Estimate: 168h > > In order to validate several flows that deal with sensor data, it would be > good to have a built in time series simulator processor that generates data > and can send it out via a flow file. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (NIFI-4164) Realistic Time Series Processor Simulator
[ https://issues.apache.org/jira/browse/NIFI-4164?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16346715#comment-16346715 ] ASF GitHub Bot commented on NIFI-4164: -- Github user MikeThomsen commented on a diff in the pull request: https://github.com/apache/nifi/pull/1997#discussion_r165032832 --- Diff: pom.xml --- @@ -1300,6 +1300,12 @@ 1.4.0-SNAPSHOT nar + +org.apache.nifi +nifi-simulator-nar +1.4.0-SNAPSHOT --- End diff -- 1.6.0-SNAPSHOT > Realistic Time Series Processor Simulator > - > > Key: NIFI-4164 > URL: https://issues.apache.org/jira/browse/NIFI-4164 > Project: Apache NiFi > Issue Type: New Feature > Components: Extensions >Reporter: Chris Herrera >Assignee: Chris Herrera >Priority: Minor > Labels: features > Original Estimate: 168h > Remaining Estimate: 168h > > In order to validate several flows that deal with sensor data, it would be > good to have a built in time series simulator processor that generates data > and can send it out via a flow file. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (NIFI-4164) Realistic Time Series Processor Simulator
[ https://issues.apache.org/jira/browse/NIFI-4164?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16346697#comment-16346697 ] ASF GitHub Bot commented on NIFI-4164: -- Github user MikeThomsen commented on a diff in the pull request: https://github.com/apache/nifi/pull/1997#discussion_r165029072 --- Diff: nifi-nar-bundles/nifi-simulator-bundle/nifi-simulator-nar/pom.xml --- @@ -0,0 +1,41 @@ + + +http://maven.apache.org/POM/4.0.0; xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance; xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd;> +4.0.0 + + +org.apache.nifi +nifi-simulator-bundle +1.4.0-SNAPSHOT --- End diff -- 1.6.0-SNAPSHOT > Realistic Time Series Processor Simulator > - > > Key: NIFI-4164 > URL: https://issues.apache.org/jira/browse/NIFI-4164 > Project: Apache NiFi > Issue Type: New Feature > Components: Extensions >Reporter: Chris Herrera >Assignee: Chris Herrera >Priority: Minor > Labels: features > Original Estimate: 168h > Remaining Estimate: 168h > > In order to validate several flows that deal with sensor data, it would be > good to have a built in time series simulator processor that generates data > and can send it out via a flow file. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (NIFI-4164) Realistic Time Series Processor Simulator
[ https://issues.apache.org/jira/browse/NIFI-4164?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16346716#comment-16346716 ] ASF GitHub Bot commented on NIFI-4164: -- Github user MikeThomsen commented on a diff in the pull request: https://github.com/apache/nifi/pull/1997#discussion_r165033444 --- Diff: nifi-nar-bundles/nifi-simulator-bundle/nifi-simulator-processors/src/main/scala/com/apache/nifi/processors/simulator/SimController.scala --- @@ -0,0 +1,59 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one or more + * contributor license agreements. See the NOTICE file distributed with + * this work for additional information regarding copyright ownership. + * The ASF licenses this file to You under the Apache License, Version 2.0 + * (the "License"); you may not use this file except in compliance with + * the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package com.apache.nifi.processors.simulator + +import java.io.File +import be.cetic.tsimulus.config.Configuration +import be.cetic.tsimulus.timeseries._ +import com.github.nscala_time.time.Imports._ +import spray.json._ +import scala.io.Source + +object SimController --- End diff -- Since the Scala APIs can be called from Java, is there any reason to have this written in Scala? > Realistic Time Series Processor Simulator > - > > Key: NIFI-4164 > URL: https://issues.apache.org/jira/browse/NIFI-4164 > Project: Apache NiFi > Issue Type: New Feature > Components: Extensions >Reporter: Chris Herrera >Assignee: Chris Herrera >Priority: Minor > Labels: features > Original Estimate: 168h > Remaining Estimate: 168h > > In order to validate several flows that deal with sensor data, it would be > good to have a built in time series simulator processor that generates data > and can send it out via a flow file. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (NIFI-4164) Realistic Time Series Processor Simulator
[ https://issues.apache.org/jira/browse/NIFI-4164?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16346704#comment-16346704 ] ASF GitHub Bot commented on NIFI-4164: -- Github user MikeThomsen commented on a diff in the pull request: https://github.com/apache/nifi/pull/1997#discussion_r165030487 --- Diff: nifi-nar-bundles/nifi-simulator-bundle/nifi-simulator-processors/src/main/java/com/apache/nifi/processors/simulator/GenerateTimeSeriesFlowFile.java --- @@ -0,0 +1,180 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one or more + * contributor license agreements. See the NOTICE file distributed with + * this work for additional information regarding copyright ownership. + * The ASF licenses this file to You under the Apache License, Version 2.0 + * (the "License"); you may not use this file except in compliance with + * the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package com.apache.nifi.processors.simulator; + +import be.cetic.tsimulus.config.Configuration; +import org.apache.nifi.annotation.behavior.InputRequirement; +import org.apache.nifi.annotation.documentation.CapabilityDescription; +import org.apache.nifi.annotation.documentation.Tags; +import org.apache.nifi.annotation.lifecycle.OnScheduled; +import org.apache.nifi.components.PropertyDescriptor; +import org.apache.nifi.flowfile.FlowFile; +import org.apache.nifi.logging.ComponentLog; +import org.apache.nifi.processor.AbstractProcessor; +import org.apache.nifi.processor.Relationship; +import org.apache.nifi.processor.ProcessorInitializationContext; +import org.apache.nifi.processor.ProcessContext; +import org.apache.nifi.processor.exception.ProcessException; +import org.apache.nifi.processor.ProcessSession; +import org.apache.nifi.processor.util.StandardValidators; +import org.joda.time.LocalDateTime; +import scala.Some; +import scala.Tuple3; +import scala.collection.JavaConverters; + +import java.util.List; +import java.util.Set; +import java.util.Collections; +import java.util.HashSet; +import java.util.ArrayList; + +@Tags({"Simulator, Timeseries, IOT, Testing"}) +@InputRequirement(InputRequirement.Requirement.INPUT_FORBIDDEN) +@CapabilityDescription("Generates realistic time series data using the TSimulus time series generator, and places the values into the flowfile in a CSV format.") +public class GenerateTimeSeriesFlowFile extends AbstractProcessor { + +private Configuration simConfig = null; +private boolean isTest = false; + +public static final PropertyDescriptor SIMULATOR_CONFIG = new PropertyDescriptor +.Builder().name("SIMULATOR_CONFIG") +.displayName("Simulator Configuration File") +.description("The JSON configuration file to use to configure TSimulus") +.required(true) +.addValidator(StandardValidators.FILE_EXISTS_VALIDATOR) +.build(); + +public static final PropertyDescriptor PRINT_HEADER = new PropertyDescriptor +.Builder().name("PRINT_HEADER") +.displayName("Print Header") +.description("Directs the processor whether to print a header line or not.") --- End diff -- Some additional details describing how this would impact the output, such as a minimalist sample might be really helpful to users. > Realistic Time Series Processor Simulator > - > > Key: NIFI-4164 > URL: https://issues.apache.org/jira/browse/NIFI-4164 > Project: Apache NiFi > Issue Type: New Feature > Components: Extensions >Reporter: Chris Herrera >Assignee: Chris Herrera >Priority: Minor > Labels: features > Original Estimate: 168h > Remaining Estimate: 168h > > In order to validate several flows that deal with sensor data, it would be > good to have a built in time series simulator processor that generates data > and can send it out via a flow file. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (NIFI-4164) Realistic Time Series Processor Simulator
[ https://issues.apache.org/jira/browse/NIFI-4164?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16346714#comment-16346714 ] ASF GitHub Bot commented on NIFI-4164: -- Github user MikeThomsen commented on a diff in the pull request: https://github.com/apache/nifi/pull/1997#discussion_r165033904 --- Diff: nifi-nar-bundles/nifi-simulator-bundle/nifi-simulator-processors/src/test/java/com/apache/nifi/processors/simulator/GenerateTimeSeriesFlowfileTest.java --- @@ -0,0 +1,72 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one or more + * contributor license agreements. See the NOTICE file distributed with + * this work for additional information regarding copyright ownership. + * The ASF licenses this file to You under the Apache License, Version 2.0 + * (the "License"); you may not use this file except in compliance with + * the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package com.apache.nifi.processors.simulator; + +import org.apache.nifi.util.TestRunner; +import org.apache.nifi.util.TestRunners; +import org.junit.Before; +import org.junit.Test; + + +public class GenerateTimeSeriesFlowfileTest { + +private TestRunner testRunner; + +@Before +public void init() { +testRunner = TestRunners.newTestRunner(GenerateTimeSeriesFlowFile.class); +} + +@Test +public void testGenerateTimeSeries() { +TestRunner runner = TestRunners.newTestRunner(new GenerateTimeSeriesFlowFile()); +runner.setProperty(GenerateTimeSeriesFlowFile.SIMULATOR_CONFIG, getClass().getResource("/configs/basicConfig.json").getPath()); +runner.assertValid(); + +runner.run(); + +runner.assertTransferCount(GenerateTimeSeriesFlowFile.SUCCESS, 1); +} + +@Test +public void testInvalidConfig() { +TestRunner runner = TestRunners.newTestRunner(new GenerateTimeSeriesFlowFile()); +runner.setProperty(GenerateTimeSeriesFlowFile.SIMULATOR_CONFIG, "/my/invalid/path"); +runner.assertNotValid(); +} + +@Test +public void testFalseHeaderCreation() { +TestRunner runner = TestRunners.newTestRunner(new GenerateTimeSeriesFlowFile()); +runner.setProperty(GenerateTimeSeriesFlowFile.PRINT_HEADER, "false"); +runner.setProperty(GenerateTimeSeriesFlowFile.SIMULATOR_CONFIG, getClass().getResource("/configs/unitTestConfig.json").getPath()); +runner.run(); +runner.assertTransferCount(GenerateTimeSeriesFlowFile.SUCCESS, 1); + runner.getFlowFilesForRelationship(GenerateTimeSeriesFlowFile.SUCCESS).get(0).assertContentEquals("test,2016-01-01T00:00:00.000,17.5"); --- End diff -- You should subclass the processor if you need to inject that timestamp in there somehow. > Realistic Time Series Processor Simulator > - > > Key: NIFI-4164 > URL: https://issues.apache.org/jira/browse/NIFI-4164 > Project: Apache NiFi > Issue Type: New Feature > Components: Extensions >Reporter: Chris Herrera >Assignee: Chris Herrera >Priority: Minor > Labels: features > Original Estimate: 168h > Remaining Estimate: 168h > > In order to validate several flows that deal with sensor data, it would be > good to have a built in time series simulator processor that generates data > and can send it out via a flow file. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (NIFI-4164) Realistic Time Series Processor Simulator
[ https://issues.apache.org/jira/browse/NIFI-4164?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16346701#comment-16346701 ] ASF GitHub Bot commented on NIFI-4164: -- Github user MikeThomsen commented on a diff in the pull request: https://github.com/apache/nifi/pull/1997#discussion_r165031839 --- Diff: nifi-nar-bundles/nifi-simulator-bundle/nifi-simulator-processors/src/main/java/com/apache/nifi/processors/simulator/GenerateTimeSeriesFlowFile.java --- @@ -0,0 +1,180 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one or more + * contributor license agreements. See the NOTICE file distributed with + * this work for additional information regarding copyright ownership. + * The ASF licenses this file to You under the Apache License, Version 2.0 + * (the "License"); you may not use this file except in compliance with + * the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package com.apache.nifi.processors.simulator; + +import be.cetic.tsimulus.config.Configuration; +import org.apache.nifi.annotation.behavior.InputRequirement; +import org.apache.nifi.annotation.documentation.CapabilityDescription; +import org.apache.nifi.annotation.documentation.Tags; +import org.apache.nifi.annotation.lifecycle.OnScheduled; +import org.apache.nifi.components.PropertyDescriptor; +import org.apache.nifi.flowfile.FlowFile; +import org.apache.nifi.logging.ComponentLog; +import org.apache.nifi.processor.AbstractProcessor; +import org.apache.nifi.processor.Relationship; +import org.apache.nifi.processor.ProcessorInitializationContext; +import org.apache.nifi.processor.ProcessContext; +import org.apache.nifi.processor.exception.ProcessException; +import org.apache.nifi.processor.ProcessSession; +import org.apache.nifi.processor.util.StandardValidators; +import org.joda.time.LocalDateTime; +import scala.Some; +import scala.Tuple3; +import scala.collection.JavaConverters; + +import java.util.List; +import java.util.Set; +import java.util.Collections; +import java.util.HashSet; +import java.util.ArrayList; + +@Tags({"Simulator, Timeseries, IOT, Testing"}) +@InputRequirement(InputRequirement.Requirement.INPUT_FORBIDDEN) +@CapabilityDescription("Generates realistic time series data using the TSimulus time series generator, and places the values into the flowfile in a CSV format.") +public class GenerateTimeSeriesFlowFile extends AbstractProcessor { + +private Configuration simConfig = null; +private boolean isTest = false; + +public static final PropertyDescriptor SIMULATOR_CONFIG = new PropertyDescriptor +.Builder().name("SIMULATOR_CONFIG") +.displayName("Simulator Configuration File") +.description("The JSON configuration file to use to configure TSimulus") +.required(true) +.addValidator(StandardValidators.FILE_EXISTS_VALIDATOR) +.build(); + +public static final PropertyDescriptor PRINT_HEADER = new PropertyDescriptor +.Builder().name("PRINT_HEADER") +.displayName("Print Header") +.description("Directs the processor whether to print a header line or not.") +.required(true) +.allowableValues("true", "false") +.defaultValue("false") +.addValidator(StandardValidators.BOOLEAN_VALIDATOR) +.build(); + +public static final Relationship SUCCESS = new Relationship.Builder() +.name("Success") +.description("When the flowfile is successfully generated") +.build(); + +private List descriptors; + +private Set relationships; + +@Override +protected void init(final ProcessorInitializationContext context) { + +final List descriptors = new ArrayList<>(); + +descriptors.add(SIMULATOR_CONFIG); +descriptors.add(PRINT_HEADER); + +this.descriptors = Collections.unmodifiableList(descriptors); + +final Set relationships = new HashSet<>(); +relationships.add(SUCCESS); +this.relationships = Collections.unmodifiableSet(relationships); +} + +@Override +public Set getRelationships() {
[jira] [Commented] (NIFI-4164) Realistic Time Series Processor Simulator
[ https://issues.apache.org/jira/browse/NIFI-4164?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16346702#comment-16346702 ] ASF GitHub Bot commented on NIFI-4164: -- Github user MikeThomsen commented on a diff in the pull request: https://github.com/apache/nifi/pull/1997#discussion_r165029951 --- Diff: nifi-nar-bundles/nifi-simulator-bundle/nifi-simulator-processors/src/main/java/com/apache/nifi/processors/simulator/GenerateTimeSeriesFlowFile.java --- @@ -0,0 +1,180 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one or more + * contributor license agreements. See the NOTICE file distributed with + * this work for additional information regarding copyright ownership. + * The ASF licenses this file to You under the Apache License, Version 2.0 + * (the "License"); you may not use this file except in compliance with + * the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package com.apache.nifi.processors.simulator; + +import be.cetic.tsimulus.config.Configuration; +import org.apache.nifi.annotation.behavior.InputRequirement; +import org.apache.nifi.annotation.documentation.CapabilityDescription; +import org.apache.nifi.annotation.documentation.Tags; +import org.apache.nifi.annotation.lifecycle.OnScheduled; +import org.apache.nifi.components.PropertyDescriptor; +import org.apache.nifi.flowfile.FlowFile; +import org.apache.nifi.logging.ComponentLog; +import org.apache.nifi.processor.AbstractProcessor; +import org.apache.nifi.processor.Relationship; +import org.apache.nifi.processor.ProcessorInitializationContext; +import org.apache.nifi.processor.ProcessContext; +import org.apache.nifi.processor.exception.ProcessException; +import org.apache.nifi.processor.ProcessSession; +import org.apache.nifi.processor.util.StandardValidators; +import org.joda.time.LocalDateTime; +import scala.Some; +import scala.Tuple3; +import scala.collection.JavaConverters; + +import java.util.List; +import java.util.Set; +import java.util.Collections; +import java.util.HashSet; +import java.util.ArrayList; + +@Tags({"Simulator, Timeseries, IOT, Testing"}) +@InputRequirement(InputRequirement.Requirement.INPUT_FORBIDDEN) --- End diff -- Are you sure that you don't want INPUT_ALLOWED so that users can provide input to dynamically change the behavior through EL or content in the flowfile? If not, that's fine. > Realistic Time Series Processor Simulator > - > > Key: NIFI-4164 > URL: https://issues.apache.org/jira/browse/NIFI-4164 > Project: Apache NiFi > Issue Type: New Feature > Components: Extensions >Reporter: Chris Herrera >Assignee: Chris Herrera >Priority: Minor > Labels: features > Original Estimate: 168h > Remaining Estimate: 168h > > In order to validate several flows that deal with sensor data, it would be > good to have a built in time series simulator processor that generates data > and can send it out via a flow file. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (NIFI-4164) Realistic Time Series Processor Simulator
[ https://issues.apache.org/jira/browse/NIFI-4164?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16346705#comment-16346705 ] ASF GitHub Bot commented on NIFI-4164: -- Github user MikeThomsen commented on a diff in the pull request: https://github.com/apache/nifi/pull/1997#discussion_r165030818 --- Diff: nifi-nar-bundles/nifi-simulator-bundle/nifi-simulator-processors/src/main/java/com/apache/nifi/processors/simulator/GenerateTimeSeriesFlowFile.java --- @@ -0,0 +1,180 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one or more + * contributor license agreements. See the NOTICE file distributed with + * this work for additional information regarding copyright ownership. + * The ASF licenses this file to You under the Apache License, Version 2.0 + * (the "License"); you may not use this file except in compliance with + * the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package com.apache.nifi.processors.simulator; + +import be.cetic.tsimulus.config.Configuration; +import org.apache.nifi.annotation.behavior.InputRequirement; +import org.apache.nifi.annotation.documentation.CapabilityDescription; +import org.apache.nifi.annotation.documentation.Tags; +import org.apache.nifi.annotation.lifecycle.OnScheduled; +import org.apache.nifi.components.PropertyDescriptor; +import org.apache.nifi.flowfile.FlowFile; +import org.apache.nifi.logging.ComponentLog; +import org.apache.nifi.processor.AbstractProcessor; +import org.apache.nifi.processor.Relationship; +import org.apache.nifi.processor.ProcessorInitializationContext; +import org.apache.nifi.processor.ProcessContext; +import org.apache.nifi.processor.exception.ProcessException; +import org.apache.nifi.processor.ProcessSession; +import org.apache.nifi.processor.util.StandardValidators; +import org.joda.time.LocalDateTime; +import scala.Some; +import scala.Tuple3; +import scala.collection.JavaConverters; + +import java.util.List; +import java.util.Set; +import java.util.Collections; +import java.util.HashSet; +import java.util.ArrayList; + +@Tags({"Simulator, Timeseries, IOT, Testing"}) +@InputRequirement(InputRequirement.Requirement.INPUT_FORBIDDEN) +@CapabilityDescription("Generates realistic time series data using the TSimulus time series generator, and places the values into the flowfile in a CSV format.") +public class GenerateTimeSeriesFlowFile extends AbstractProcessor { + +private Configuration simConfig = null; +private boolean isTest = false; + +public static final PropertyDescriptor SIMULATOR_CONFIG = new PropertyDescriptor +.Builder().name("SIMULATOR_CONFIG") +.displayName("Simulator Configuration File") +.description("The JSON configuration file to use to configure TSimulus") +.required(true) +.addValidator(StandardValidators.FILE_EXISTS_VALIDATOR) +.build(); + +public static final PropertyDescriptor PRINT_HEADER = new PropertyDescriptor +.Builder().name("PRINT_HEADER") +.displayName("Print Header") +.description("Directs the processor whether to print a header line or not.") +.required(true) +.allowableValues("true", "false") +.defaultValue("false") +.addValidator(StandardValidators.BOOLEAN_VALIDATOR) +.build(); + +public static final Relationship SUCCESS = new Relationship.Builder() +.name("Success") +.description("When the flowfile is successfully generated") --- End diff -- Needs a little cleanup. Consider something like this: > "When the flowfile is successfully generated, it is transferred to this relationship." > Realistic Time Series Processor Simulator > - > > Key: NIFI-4164 > URL: https://issues.apache.org/jira/browse/NIFI-4164 > Project: Apache NiFi > Issue Type: New Feature > Components: Extensions >Reporter: Chris Herrera >Assignee: Chris Herrera >Priority: Minor > Labels: features > Original Estimate: 168h > Remaining Estimate: 168h > > In order to validate several flows
[jira] [Commented] (NIFI-4164) Realistic Time Series Processor Simulator
[ https://issues.apache.org/jira/browse/NIFI-4164?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16346711#comment-16346711 ] ASF GitHub Bot commented on NIFI-4164: -- Github user MikeThomsen commented on a diff in the pull request: https://github.com/apache/nifi/pull/1997#discussion_r165032066 --- Diff: nifi-nar-bundles/nifi-simulator-bundle/nifi-simulator-processors/src/main/java/com/apache/nifi/processors/simulator/GenerateTimeSeriesFlowFile.java --- @@ -0,0 +1,180 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one or more + * contributor license agreements. See the NOTICE file distributed with + * this work for additional information regarding copyright ownership. + * The ASF licenses this file to You under the Apache License, Version 2.0 + * (the "License"); you may not use this file except in compliance with + * the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package com.apache.nifi.processors.simulator; + +import be.cetic.tsimulus.config.Configuration; +import org.apache.nifi.annotation.behavior.InputRequirement; +import org.apache.nifi.annotation.documentation.CapabilityDescription; +import org.apache.nifi.annotation.documentation.Tags; +import org.apache.nifi.annotation.lifecycle.OnScheduled; +import org.apache.nifi.components.PropertyDescriptor; +import org.apache.nifi.flowfile.FlowFile; +import org.apache.nifi.logging.ComponentLog; +import org.apache.nifi.processor.AbstractProcessor; +import org.apache.nifi.processor.Relationship; +import org.apache.nifi.processor.ProcessorInitializationContext; +import org.apache.nifi.processor.ProcessContext; +import org.apache.nifi.processor.exception.ProcessException; +import org.apache.nifi.processor.ProcessSession; +import org.apache.nifi.processor.util.StandardValidators; +import org.joda.time.LocalDateTime; +import scala.Some; +import scala.Tuple3; +import scala.collection.JavaConverters; + +import java.util.List; +import java.util.Set; +import java.util.Collections; +import java.util.HashSet; +import java.util.ArrayList; + +@Tags({"Simulator, Timeseries, IOT, Testing"}) +@InputRequirement(InputRequirement.Requirement.INPUT_FORBIDDEN) +@CapabilityDescription("Generates realistic time series data using the TSimulus time series generator, and places the values into the flowfile in a CSV format.") +public class GenerateTimeSeriesFlowFile extends AbstractProcessor { + +private Configuration simConfig = null; +private boolean isTest = false; + +public static final PropertyDescriptor SIMULATOR_CONFIG = new PropertyDescriptor +.Builder().name("SIMULATOR_CONFIG") +.displayName("Simulator Configuration File") +.description("The JSON configuration file to use to configure TSimulus") +.required(true) +.addValidator(StandardValidators.FILE_EXISTS_VALIDATOR) +.build(); + +public static final PropertyDescriptor PRINT_HEADER = new PropertyDescriptor +.Builder().name("PRINT_HEADER") +.displayName("Print Header") +.description("Directs the processor whether to print a header line or not.") +.required(true) +.allowableValues("true", "false") +.defaultValue("false") +.addValidator(StandardValidators.BOOLEAN_VALIDATOR) +.build(); + +public static final Relationship SUCCESS = new Relationship.Builder() +.name("Success") +.description("When the flowfile is successfully generated") +.build(); + +private List descriptors; + +private Set relationships; + +@Override +protected void init(final ProcessorInitializationContext context) { + +final List descriptors = new ArrayList<>(); + +descriptors.add(SIMULATOR_CONFIG); +descriptors.add(PRINT_HEADER); + +this.descriptors = Collections.unmodifiableList(descriptors); + +final Set relationships = new HashSet<>(); +relationships.add(SUCCESS); +this.relationships = Collections.unmodifiableSet(relationships); +} + +@Override +public Set getRelationships() {
[jira] [Commented] (NIFI-4164) Realistic Time Series Processor Simulator
[ https://issues.apache.org/jira/browse/NIFI-4164?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16346713#comment-16346713 ] ASF GitHub Bot commented on NIFI-4164: -- Github user MikeThomsen commented on a diff in the pull request: https://github.com/apache/nifi/pull/1997#discussion_r165033644 --- Diff: nifi-nar-bundles/nifi-simulator-bundle/nifi-simulator-processors/src/test/java/com/apache/nifi/processors/simulator/GenerateTimeSeriesFlowfileTest.java --- @@ -0,0 +1,72 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one or more + * contributor license agreements. See the NOTICE file distributed with + * this work for additional information regarding copyright ownership. + * The ASF licenses this file to You under the Apache License, Version 2.0 + * (the "License"); you may not use this file except in compliance with + * the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package com.apache.nifi.processors.simulator; + +import org.apache.nifi.util.TestRunner; +import org.apache.nifi.util.TestRunners; +import org.junit.Before; +import org.junit.Test; + + +public class GenerateTimeSeriesFlowfileTest { + +private TestRunner testRunner; + +@Before +public void init() { --- End diff -- Not used. It can be deleted. > Realistic Time Series Processor Simulator > - > > Key: NIFI-4164 > URL: https://issues.apache.org/jira/browse/NIFI-4164 > Project: Apache NiFi > Issue Type: New Feature > Components: Extensions >Reporter: Chris Herrera >Assignee: Chris Herrera >Priority: Minor > Labels: features > Original Estimate: 168h > Remaining Estimate: 168h > > In order to validate several flows that deal with sensor data, it would be > good to have a built in time series simulator processor that generates data > and can send it out via a flow file. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (NIFI-4164) Realistic Time Series Processor Simulator
[ https://issues.apache.org/jira/browse/NIFI-4164?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16346698#comment-16346698 ] ASF GitHub Bot commented on NIFI-4164: -- Github user MikeThomsen commented on a diff in the pull request: https://github.com/apache/nifi/pull/1997#discussion_r165029448 --- Diff: nifi-nar-bundles/nifi-simulator-bundle/nifi-simulator-processors/pom.xml --- @@ -0,0 +1,109 @@ + + +http://maven.apache.org/POM/4.0.0; xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance; xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd;> +4.0.0 + + +org.apache.nifi +nifi-simulator-bundle +1.4.0-SNAPSHOT --- End diff -- 1.6.0-SNAPSHOT > Realistic Time Series Processor Simulator > - > > Key: NIFI-4164 > URL: https://issues.apache.org/jira/browse/NIFI-4164 > Project: Apache NiFi > Issue Type: New Feature > Components: Extensions >Reporter: Chris Herrera >Assignee: Chris Herrera >Priority: Minor > Labels: features > Original Estimate: 168h > Remaining Estimate: 168h > > In order to validate several flows that deal with sensor data, it would be > good to have a built in time series simulator processor that generates data > and can send it out via a flow file. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (NIFI-4164) Realistic Time Series Processor Simulator
[ https://issues.apache.org/jira/browse/NIFI-4164?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16346710#comment-16346710 ] ASF GitHub Bot commented on NIFI-4164: -- Github user MikeThomsen commented on a diff in the pull request: https://github.com/apache/nifi/pull/1997#discussion_r165033604 --- Diff: nifi-nar-bundles/nifi-simulator-bundle/nifi-simulator-processors/src/test/java/com/apache/nifi/processors/simulator/GenerateTimeSeriesFlowfileTest.java --- @@ -0,0 +1,72 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one or more + * contributor license agreements. See the NOTICE file distributed with + * this work for additional information regarding copyright ownership. + * The ASF licenses this file to You under the Apache License, Version 2.0 + * (the "License"); you may not use this file except in compliance with + * the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package com.apache.nifi.processors.simulator; + +import org.apache.nifi.util.TestRunner; +import org.apache.nifi.util.TestRunners; +import org.junit.Before; +import org.junit.Test; + + +public class GenerateTimeSeriesFlowfileTest { + +private TestRunner testRunner; --- End diff -- This can be deleted. > Realistic Time Series Processor Simulator > - > > Key: NIFI-4164 > URL: https://issues.apache.org/jira/browse/NIFI-4164 > Project: Apache NiFi > Issue Type: New Feature > Components: Extensions >Reporter: Chris Herrera >Assignee: Chris Herrera >Priority: Minor > Labels: features > Original Estimate: 168h > Remaining Estimate: 168h > > In order to validate several flows that deal with sensor data, it would be > good to have a built in time series simulator processor that generates data > and can send it out via a flow file. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (NIFI-4164) Realistic Time Series Processor Simulator
[ https://issues.apache.org/jira/browse/NIFI-4164?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16346712#comment-16346712 ] ASF GitHub Bot commented on NIFI-4164: -- Github user MikeThomsen commented on a diff in the pull request: https://github.com/apache/nifi/pull/1997#discussion_r165029171 --- Diff: nifi-nar-bundles/nifi-simulator-bundle/nifi-simulator-nar/pom.xml --- @@ -0,0 +1,41 @@ + + +http://maven.apache.org/POM/4.0.0; xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance; xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd;> +4.0.0 + + +org.apache.nifi +nifi-simulator-bundle +1.4.0-SNAPSHOT + + +nifi-simulator-nar +1.4.0-SNAPSHOT +nar + +true +true + + + + +org.apache.nifi +nifi-simulator-processors +1.4.0-SNAPSHOT --- End diff -- 1.6.0-SNAPSHOT > Realistic Time Series Processor Simulator > - > > Key: NIFI-4164 > URL: https://issues.apache.org/jira/browse/NIFI-4164 > Project: Apache NiFi > Issue Type: New Feature > Components: Extensions >Reporter: Chris Herrera >Assignee: Chris Herrera >Priority: Minor > Labels: features > Original Estimate: 168h > Remaining Estimate: 168h > > In order to validate several flows that deal with sensor data, it would be > good to have a built in time series simulator processor that generates data > and can send it out via a flow file. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (NIFI-4164) Realistic Time Series Processor Simulator
[ https://issues.apache.org/jira/browse/NIFI-4164?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16346703#comment-16346703 ] ASF GitHub Bot commented on NIFI-4164: -- Github user MikeThomsen commented on a diff in the pull request: https://github.com/apache/nifi/pull/1997#discussion_r165032740 --- Diff: nifi-nar-bundles/nifi-simulator-bundle/pom.xml --- @@ -0,0 +1,35 @@ + + +http://maven.apache.org/POM/4.0.0; xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance; xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd;> +4.0.0 + + +org.apache.nifi +nifi-nar-bundles +1.4.0-SNAPSHOT + + +org.apache.nifi +nifi-simulator-bundle +1.4.0-SNAPSHOT --- End diff -- 1.6.0-SNAPSHOT > Realistic Time Series Processor Simulator > - > > Key: NIFI-4164 > URL: https://issues.apache.org/jira/browse/NIFI-4164 > Project: Apache NiFi > Issue Type: New Feature > Components: Extensions >Reporter: Chris Herrera >Assignee: Chris Herrera >Priority: Minor > Labels: features > Original Estimate: 168h > Remaining Estimate: 168h > > In order to validate several flows that deal with sensor data, it would be > good to have a built in time series simulator processor that generates data > and can send it out via a flow file. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (NIFI-4164) Realistic Time Series Processor Simulator
[ https://issues.apache.org/jira/browse/NIFI-4164?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16346708#comment-16346708 ] ASF GitHub Bot commented on NIFI-4164: -- Github user MikeThomsen commented on a diff in the pull request: https://github.com/apache/nifi/pull/1997#discussion_r165032723 --- Diff: nifi-nar-bundles/nifi-simulator-bundle/pom.xml --- @@ -0,0 +1,35 @@ + + +http://maven.apache.org/POM/4.0.0; xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance; xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd;> +4.0.0 + + +org.apache.nifi +nifi-nar-bundles +1.4.0-SNAPSHOT --- End diff -- 1.6.0-SNAPSHOT > Realistic Time Series Processor Simulator > - > > Key: NIFI-4164 > URL: https://issues.apache.org/jira/browse/NIFI-4164 > Project: Apache NiFi > Issue Type: New Feature > Components: Extensions >Reporter: Chris Herrera >Assignee: Chris Herrera >Priority: Minor > Labels: features > Original Estimate: 168h > Remaining Estimate: 168h > > In order to validate several flows that deal with sensor data, it would be > good to have a built in time series simulator processor that generates data > and can send it out via a flow file. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (NIFI-4164) Realistic Time Series Processor Simulator
[ https://issues.apache.org/jira/browse/NIFI-4164?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16346699#comment-16346699 ] ASF GitHub Bot commented on NIFI-4164: -- Github user MikeThomsen commented on a diff in the pull request: https://github.com/apache/nifi/pull/1997#discussion_r165029401 --- Diff: nifi-nar-bundles/nifi-simulator-bundle/nifi-simulator-nar/resources/LICENSE --- @@ -0,0 +1,210 @@ + Apache License --- End diff -- I don't think this file is necessary since all of the dependencies you listed are Apache 2.0 licensed. > Realistic Time Series Processor Simulator > - > > Key: NIFI-4164 > URL: https://issues.apache.org/jira/browse/NIFI-4164 > Project: Apache NiFi > Issue Type: New Feature > Components: Extensions >Reporter: Chris Herrera >Assignee: Chris Herrera >Priority: Minor > Labels: features > Original Estimate: 168h > Remaining Estimate: 168h > > In order to validate several flows that deal with sensor data, it would be > good to have a built in time series simulator processor that generates data > and can send it out via a flow file. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (NIFI-4164) Realistic Time Series Processor Simulator
[ https://issues.apache.org/jira/browse/NIFI-4164?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16078948#comment-16078948 ] ASF GitHub Bot commented on NIFI-4164: -- GitHub user cherrera2001 opened a pull request: https://github.com/apache/nifi/pull/1997 NIFI-4164 Adding a realistic time simulator processor to NiFi This is the initial commit of the processor. It can be used by using the bundled basicConfig.json or unitTestConfig.json files found within /test/Resources of the processor. Thank you for submitting a contribution to Apache NiFi. In order to streamline the review of the contribution we ask you to ensure the following steps have been taken: ### For all changes: - [X] Is there a JIRA ticket associated with this PR? Is it referenced in the commit message? - [X] Does your PR title start with NIFI- where is the JIRA number you are trying to resolve? Pay particular attention to the hyphen "-" character. - [X] Has your PR been rebased against the latest commit within the target branch (typically master)? - [X] Is your initial contribution a single, squashed commit? ### For code changes: - [X] Have you ensured that the full suite of tests is executed via mvn -Pcontrib-check clean install at the root nifi folder? - [X] Have you written or updated unit tests to verify your changes? - [X] If adding new dependencies to the code, are these dependencies licensed in a way that is compatible for inclusion under [ASF 2.0](http://www.apache.org/legal/resolved.html#category-a)? - [X] If applicable, have you updated the LICENSE file, including the main LICENSE file under nifi-assembly? - [X] If applicable, have you updated the NOTICE file, including the main NOTICE file found under nifi-assembly? - [ ] If adding new Properties, have you added .displayName in addition to .name (programmatic access) for each of the new properties? ### For documentation related changes: - [X] Have you ensured that format looks appropriate for the output in which it is rendered? ### Note: Please ensure that once the PR is submitted, you check travis-ci for build issues and submit an update to your PR as soon as possible. You can merge this pull request into a Git repository by running: $ git pull https://github.com/hashmapinc/nifi NIFI-4164 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/nifi/pull/1997.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #1997 commit e980796ed93b3053567f029f3d1ddb1b9c0ae46c Author: Chris HerreraDate: 2017-07-08T04:18:00Z Inital commit of the processor Inital commit of the processor > Realistic Time Series Processor Simulator > - > > Key: NIFI-4164 > URL: https://issues.apache.org/jira/browse/NIFI-4164 > Project: Apache NiFi > Issue Type: New Feature > Components: Extensions >Reporter: Chris Herrera >Assignee: Chris Herrera >Priority: Minor > Labels: features > Original Estimate: 168h > Remaining Estimate: 168h > > In order to validate several flows that deal with sensor data, it would be > good to have a built in time series simulator processor that generates data > and can send it out via a flow file. -- This message was sent by Atlassian JIRA (v6.4.14#64029)