Hello I think it is fair to say that building listeners, where NiFi is acting as a recipient of data being pushed to it, is among the harder extension points to build. I also think that the outline of the general pattern is fair at a high level but that a good API to make that more repeatable generically is pretty hard for the reasons Tony mentions.
That said there are things we can do and should do to just keep making such things a little bit easier over time. We're happy to help you walk through NIFI-856 and if we find things to chip away at generically then let's identify them and work them off. Thanks Joe On Sun, Nov 29, 2015 at 9:37 PM, Tony Kurc <[email protected]> wrote: > Many of these use blockingqueues to pass between the "listener" and the > "workers". I do think some library support for this would be a good idea, > to help provide thread safe publication and make exception handling a > little easier. I think providing any of the networking support may be > tricky because of the diversity of client apis and libraries. But a nice > interprocessor pub/sub api seems like a good move. I noticed Joe Witt > mentioned adding a little delay on poll in his listensyslog, I mentioned > how coverity and findbugs complain about ignoring the return code of > offer... Can you put a ticket in for this? I wouldn't mind working it. > On Nov 29, 2015 9:01 PM, "Andre" <[email protected]> wrote: > >> Hi, >> >> I am trying to give it a go on NIFI-856 (I'm not a coder either but >> decided to take the challenge). >> >> As I try to get my head around it I've noticed that coding "Listeners" >> for NiFi is sort of a nightmare. (please take no offense, it is just a >> sincere unskilled code statement). >> >> I've looked at ListenHTTP, ListenUDP and ListenSyslog for inspiration, >> I got the impression they all seem to have a need to tackle the >> transmission of messages between the Listener "daemon" and the >> onTrigger method. (e.g. syslogEvents within ListenSyslog). >> >> Is this understanding correct? >> >> If so, Shouldn't the framework contain something that makes it easier >> to write this sort of processor instead of having to recreate them? >> >> I say that because nearly every Listen processors will: >> >> 1. Bind to a socket; >> 2. Receive messages; >> 3. Read messages; >> 4. Add them flowfile; >> 5. Acknowledge receipt; >> >> Where 4 and 5 are usually applicable to some protocols (like Logstash, >> RELP, RLTP, etc). >> >> Keen to hear your thoughts >> >> Andre >>
