Re: Processors on the fly for many sensor devices

Davy, The processor I have been working on may meet your needs. You are correct, at this time I have not pushed the source for it, still working through some hurdles. The one thing to work out is how you would dynamically add the processors -- suppose you may be able to use the REST API for NiFi.

Re: Processors on the fly for many sensor devices

Just wanted to clarify something about ListenTCP... it does support multiple incoming connections, however if you using the batch output capability, one flow file will contain data across all the connections. I do agree with Andrew that based on the description it sounds like NiFi is expected to

Re: Processors on the fly for many sensor devices

Correct... But I don't think that one is in the official nifi distribution. I did stumble upon your repo / jira issue. (The repo didn't contain any sources, only binaries I think). But I guess there we would also need some way of dynamically adding these processors (as it would require 1

Re: Processors on the fly for many sensor devices

Hi, Thanks for the response ... it's an existing network of sensors. The sensors spit out data over a serial interface that is exposed over a tcp connection. (rs232 -> ethernet converter in the sensor). The current sensor architecture involves clients making direct connections to the individual

Re: Processors on the fly for many sensor devices

Hello Can you talk a bit about why you'd want ListenTCP processors tied to a given sensor? You should be able to have many sensors to a single ListenTCP. Each stream will be between a source/sensor and nifi so data won't be getting intermingled there. If we're not providing enough

Processors on the fly for many sensor devices

We have a large number of sensors that send out data via TCP. The idea is to use a ListenTCP processor in Nifi to capture the data, do some filtering / basic transformation before sending it upstream into our stack. We can configure individual ListenTCP processors for each sensor, and that works

Re: UI: feedback on the processor 'color' in NiFi 1.0

Rob - I really like your idea of layers! We could have a 'traffic' layer that could highlight areas of back pressure while the data is flowing. We could even allow the user to customize the threshold for warnings or alert values as well as the colors used for the different states of data flowing

Re: State management not captured in cluster mode

At the moment the sate management description (if any) of a processor is available by right clicking on it and going into "View state". Pierre 2016-09-28 16:15 GMT+02:00 Bryan Bende : > What I was referring to is in the code of each processor, it is annotated > with something

Re: State management not captured in cluster mode

Hi Bryan, Thanks for the information. Can you please share a picture where i can able to see the state (local or cluster). I could not see anywhere. On Wed, Sep 28, 2016 at 1:22 PM, Bryan Bende wrote: > Hi Selvam, > > It depends what processor you are using. For example,

Re: UI: feedback on the processor 'color' in NiFi 1.0

After thinking on it a bit, I agree that Manish' suggestion could be a good idea as an option (the way /additionalDetails.html/ is an option). It would be easier if they were /.png/ files rather than formal icon files only with a "width x length" limit. My two cents, Russ On 09/28/2016

Re: Create NiFi Templates

Ashish, I don't have the 0.7 docs in front of me so I'm not sure if/how it is possible there, but it is definitely possible via the 1.0 REST API [1]. Procedure is as follows: If the process group already exists in the flow (and you know its ID), here are some REST API calls that should create a

Re: Create NiFi Templates

Ashish As Joe pointed out, while “possible”, at the moment we don’t expose a direct public API to accomplish that. When I say “possible” I am of course referring to the internal API that is used by NIFi (NIFI UI that is) to wire up flows, but at the moment it is neither public nor it is the

RE: Create NiFi Templates

Hello, I am using Nifi 0.7.0. I want to create the template of the processor groups. Instead of creating it from UI. I want to design a flow that creates a processor group template based on its id, download it and store it somewhere locally. Is it possible ? If yes, Is there any method/api

RE: Remove top N lines from a text file

Hi Peter , the simplest way I see, need to transverse the all file, using a processor property. Ex: N_lines (number of Lines to Strip on file) import java.nio.charset.StandardCharsets //Read the flowFile def flowFile = session.get() if (!flowFile) return //Read Processor property def

State management not captured in cluster mode

Hi, This is my state-management.xml attribute local-provider org.apache.nifi.controller.state.providers.local.WriteAheadLocalStateProvider ./state/local zk-provider org.apache.nifi.controller.state.providers.zookeeper.ZooKeeperStateProvider

Re: nifi Rest API to get full details of the flow.

This isn't an ideal approach, IMO. There is a standard API to get a summary of the flow and,status of every processor, check what the Summary tab is invoking for a URL. You can then drill into any specific component by ID. Andrew On Wed, Sep 28, 2016, 6:42 AM Sandeep Khurana

RE: logging all transformed flowfiles

Hello Manish Thx for the very helpful answer , but I was thinking that this functional perimeter ( ie logging, storing transformations of data, data lineage ) was built in Nifi and available through REST API ... Or internal calls ... The point is that I am not ready to hook devoted logging