Hi Joe, Damn! I should have suspected it... :-(
Anyhow, the code is starting to take shape here : https://github.com/trixpan/nifi-lumberjack-bundle (I will send a PR when I get closer to complete) I believe I've cleaned most of the netty dependencies, except for the issues I faced trying to get TLS to work (netty is so much easier for this sort of things! ), so meanwhile I am simply using SOCAT to play the TLS termination point until I can get my head around. The code heavily follows the ListenSyslog pattern (don't pay attention to the mess, I will clean it later) but still doesn't inject flowfiles. So far I have been able to receive the initial window size packets, however, no traffic is processed yet (I suspect it is a problem with the framedecoder switch statement but will have to confirm tomorrow morning. Cheers On Mon, Nov 30, 2015 at 1:45 PM, Joe Witt <[email protected]> wrote: > Hello > > I think it is fair to say that building listeners, where NiFi is > acting as a recipient of data being pushed to it, is among the harder > extension points to build. I also think that the outline of the > general pattern is fair at a high level but that a good API to make > that more repeatable generically is pretty hard for the reasons Tony > mentions. > > That said there are things we can do and should do to just keep making > such things a little bit easier over time. We're happy to help you > walk through NIFI-856 and if we find things to chip away at > generically then let's identify them and work them off. > > Thanks > Joe > > On Sun, Nov 29, 2015 at 9:37 PM, Tony Kurc <[email protected]> wrote: >> Many of these use blockingqueues to pass between the "listener" and the >> "workers". I do think some library support for this would be a good idea, >> to help provide thread safe publication and make exception handling a >> little easier. I think providing any of the networking support may be >> tricky because of the diversity of client apis and libraries. But a nice >> interprocessor pub/sub api seems like a good move. I noticed Joe Witt >> mentioned adding a little delay on poll in his listensyslog, I mentioned >> how coverity and findbugs complain about ignoring the return code of >> offer... Can you put a ticket in for this? I wouldn't mind working it. >> On Nov 29, 2015 9:01 PM, "Andre" <[email protected]> wrote: >> >>> Hi, >>> >>> I am trying to give it a go on NIFI-856 (I'm not a coder either but >>> decided to take the challenge). >>> >>> As I try to get my head around it I've noticed that coding "Listeners" >>> for NiFi is sort of a nightmare. (please take no offense, it is just a >>> sincere unskilled code statement). >>> >>> I've looked at ListenHTTP, ListenUDP and ListenSyslog for inspiration, >>> I got the impression they all seem to have a need to tackle the >>> transmission of messages between the Listener "daemon" and the >>> onTrigger method. (e.g. syslogEvents within ListenSyslog). >>> >>> Is this understanding correct? >>> >>> If so, Shouldn't the framework contain something that makes it easier >>> to write this sort of processor instead of having to recreate them? >>> >>> I say that because nearly every Listen processors will: >>> >>> 1. Bind to a socket; >>> 2. Receive messages; >>> 3. Read messages; >>> 4. Add them flowfile; >>> 5. Acknowledge receipt; >>> >>> Where 4 and 5 are usually applicable to some protocols (like Logstash, >>> RELP, RLTP, etc). >>> >>> Keen to hear your thoughts >>> >>> Andre >>>
