Hi Davy, Sorry for the slow response, I am traveling today. However, I should be able to get a build of it for 1.0.0 out later today or tomorrow.
Thanks, Andrew On Fri, Sep 30, 2016 at 6:54 PM, Davy De Waele <[email protected]> wrote: > Hi Andrew, > > We tested your 0.0.6 binary and it seemed to do what we want. > Unfortunately it is not compatible with the current Nifi version. > Do you have a binary available that would work with nifi 1.0.0 ? > > Thx. > > On Wed, Sep 28, 2016 at 11:39 PM, Andrew Psaltis <[email protected] > > wrote: > >> Davy, >> The processor I have been working on may meet your needs. You are >> correct, at this time I have not pushed the source for it, still working >> through some hurdles. The one thing to work out is how you would >> dynamically add the processors -- suppose you may be able to use the REST >> API for NiFi. I would imagine there are quite a number of these devices >> that you would need to have processors for. In the use case I have been >> working on, there may be 600 or so endpoints that I need to connect to and >> I'm trying to figure out does it make sense to do it this way. >> >> I'll hopefully be in a place soon that I can push the code I have for the >> GetTCP processor. >> >> >> >> >> On Thu, Sep 29, 2016 at 5:22 AM, Bryan Bende <[email protected]> wrote: >> >>> Just wanted to clarify something about ListenTCP... it does support >>> multiple incoming connections, however if you using the batch output >>> capability, one flow file will contain data across all the connections. >>> >>> I do agree with Andrew that based on the description it sounds like NiFi >>> is expected to be the client that initiates a connection and keeps reading >>> for some amount of time/threshold, like a GetTCP processor. >>> >>> Currently we have ListenTCP which is waiting for incoming connections, >>> and PutTCP which makes a connection, but only writes data. >>> >>> On Wed, Sep 28, 2016 at 5:02 PM, Andrew Psaltis < >>> [email protected]> wrote: >>> >>>> Davy, >>>> It sounds like you need a GetTCP type of processor that connects from >>>> NiFi to the TCP endpoint on the sensor, is that correct? >>>> >>>> Thanks, >>>> Andrew >>>> >>>> On Thu, Sep 29, 2016 at 4:46 AM, Davy De Waele <[email protected]> >>>> wrote: >>>> >>>>> Hi, >>>>> >>>>> Thanks for the response ... it's an existing network of sensors. The >>>>> sensors spit out data over a serial interface that is exposed over a tcp >>>>> connection. (rs232 -> ethernet converter in the sensor). >>>>> The current sensor architecture involves clients making direct >>>>> connections to the individual sensors. (establishing a tcp connection to >>>>> the specific ip of the sensor). >>>>> >>>>> If I understand correctly, ListenTCP would not work in this case for >>>>> multiple sensors >>>>> >>>>> Are you talking about a setup where the sensors would be in a "client" >>>>> mode where each sensor would each establish a tcp connections to a single >>>>> ListTCP processor ? >>>>> >>>>> Thx >>>>> >>>>> >>>>> >>>>> On Wed, Sep 28, 2016 at 10:03 PM, Joe Witt <[email protected]> wrote: >>>>> >>>>>> Hello >>>>>> >>>>>> Can you talk a bit about why you'd want ListenTCP processors tied to a >>>>>> given sensor? You should be able to have many sensors to a single >>>>>> ListenTCP. Each stream will be between a source/sensor and nifi so >>>>>> data won't be getting intermingled there. If we're not providing >>>>>> enough session/stream metadata on the flow files to make demux of the >>>>>> streams easy using something like RouteOnAttribute or whatnot we >>>>>> definitely should. >>>>>> >>>>>> Now, that said, you could certainly programmatically deploy (via the >>>>>> REST API) instances of these processors along the lines of what your >>>>>> endpoint registry tells you. It just seems on the surface like doing >>>>>> so would be avoidable at least for the listening of data. Typically >>>>>> such a registry would be useful to do additional tagging/enrichment of >>>>>> the data and would occur once it is in the flow. >>>>>> >>>>>> Thanks >>>>>> Joe >>>>>> >>>>>> On Wed, Sep 28, 2016 at 3:39 PM, Davy De Waele <[email protected]> >>>>>> wrote: >>>>>> > We have a large number of sensors that send out data via TCP. The >>>>>> idea is to >>>>>> > use a ListenTCP processor in Nifi to capture the data, do some >>>>>> filtering / >>>>>> > basic transformation before sending it upstream into our stack. >>>>>> > >>>>>> > We can configure individual ListenTCP processors for each sensor, >>>>>> and that >>>>>> > works fine when the number of sensors is small, but once you hit a >>>>>> larger >>>>>> > number if becomes cumbersome and difficult to manage. >>>>>> > >>>>>> > We have an inventory of those sensors (exposed via a REST service >>>>>> endpoint), >>>>>> > containing the sensor tcp information like ip and port) >>>>>> > >>>>>> > Is there an easy way to create these ListenTCP processors on the >>>>>> fly based >>>>>> > on a REST endpoint or some other external configuration ? How would >>>>>> that >>>>>> > work ? >>>>>> > >>>>>> > Thx. >>>>>> >>>>> >>>>> >>>> >>>> >>>> -- >>>> Thanks, >>>> Andrew >>>> >>>> Subscribe to my book: Streaming Data <http://manning.com/psaltis> >>>> <https://www.linkedin.com/pub/andrew-psaltis/1/17b/306> >>>> twiiter: @itmdata <http://twitter.com/intent/user?screen_name=itmdata> >>>> >>> >>> >> >> >> -- >> Thanks, >> Andrew >> >> Subscribe to my book: Streaming Data <http://manning.com/psaltis> >> <https://www.linkedin.com/pub/andrew-psaltis/1/17b/306> >> twiiter: @itmdata <http://twitter.com/intent/user?screen_name=itmdata> >> > > -- Thanks, Andrew Subscribe to my book: Streaming Data <http://manning.com/psaltis> <https://www.linkedin.com/pub/andrew-psaltis/1/17b/306> twiiter: @itmdata <http://twitter.com/intent/user?screen_name=itmdata>
