I wonder if the cause of zero length messages is the health check from the f5 balancer. Worth verifying with your team.
Andrew On Tue, Apr 25, 2017, 3:15 PM Andy LoPresto <[email protected]> wrote: > PR 1694 [1] is available for this issue. > > [1] https://github.com/apache/nifi/pull/1694 > > Andy LoPresto > [email protected] > *[email protected] <[email protected]>* > PGP Fingerprint: 70EC B3E5 98A6 5A3F D3C4 BACE 3C6E F65B 2F7D EF69 > > On Apr 25, 2017, at 10:07 AM, Conrad Crampton <[email protected] > <[email protected]>> wrote: > > Hi, > Thanks for the swift reply (as usual). > NIFI-3738 created [1]. > > I have passed over to infrastructure to try and establish cause of the > zero length datagrams, but at least I now know there isn’t anything > fundamentally wrong here and can (safely) ignore the errors. > > Thanks > Conrad > > > [1] https://issues.apache.org/jira/browse/NIFI-3738 > > On 25/04/2017, 17:46, "Bryan Bende" <[email protected]> wrote: > > Hi Conrad, > > Line 431 of ListenSyslog has the following code: > > if (!valid || !event.isValid()) > > So to get an NPE there means event must be null, and event comes from > this code: > > boolean valid = true; > try { > event = parser.parseEvent(rawSyslogEvent.getData(), sender); > } catch (final ProcessException pe) { > getLogger().warn("Failed to parse Syslog event; routing to > invalid"); > valid = false; > } > > The parser returns null if the bytes sent in are null or length 0. > > We should be checking if (!valid || event == null || !event.isValid()) > to avoid this case, and I think a similar situation exists in the > ParseSyslog processor. It appears this would only happen if parsing > messages is enabled in ListenSyslog. > > Do you want to create a JIRA for this? > > The other question is why you are ending up with these 0 length > messages, but that one I am not sure about. In the case of UDP, its > just reading from a datagram channel into a byte buffer and passing > those bytes a long, so I think it means its receiving a 0 byte > datagram from the sender. > > Thanks, > > Bryan > > > On Tue, Apr 25, 2017 at 12:31 PM, Conrad Crampton > <[email protected]> wrote: > > Hi, > > Been away for a bit from this community due to other work pressures, but > picking up Nifi again and successfully upgraded to 1.1.2 (apart from > screwing up one of the nodes temporarily). > > So, with the renewed interest in log processing our infrastructure team has > put in an F5 load balancer to distribute the syslog traffic I am collecting > to my 6 node cluster. This is to stop one node being the only workhorse for > receiving syslog traffic. I had previously used the ‘standard’ pattern of > having the ListenSyslog processor connect to a RPG and then the rest of my > data processing flow receive via a local port – to effectively distribute > the processing load. I was finding though that the single node was getting > too many warnings about buffer, sockets being full etc. – hence the > external > load balancing. > > > > I am no load balancing expert, but what I believe happens is the F5 load > balancer receives syslog traffic (over UDP) then distributes this load to > all Nifi nodes (gives a bit of syslog traffic to each I believe). All > appears fine, but then I start getting NPE in my node logs thus: > > > > 2017-04-25 17:16:34,832 ERROR [Timer-Driven Process Thread-7] > o.a.n.processors.standard.ListenSyslog > ListenSyslog[id=0a932c37-0158-1000-0000-0000656754bf] > ListenSyslog[id=0a932c37-0158-1000-0000-0000656754bf] failed to process due > to java.lang.NullPointerException; rolling back session: > java.lang.NullPointerException > > 2017-04-25 17:16:34,833 ERROR [Timer-Driven Process Thread-7] > o.a.n.processors.standard.ListenSyslog > > java.lang.NullPointerException: null > > at > > org.apache.nifi.processors.standard.ListenSyslog.onTrigger(ListenSyslog.java:431) > ~[nifi-standard-processors-1.1.2.jar:1.1.2] > > at > > org.apache.nifi.processor.AbstractProcessor.onTrigger(AbstractProcessor.java:27) > ~[nifi-api-1.1.2.jar:1.1.2] > > at > > org.apache.nifi.controller.StandardProcessorNode.onTrigger(StandardProcessorNode.java:1099) > [nifi-framework-core-1.1.2.jar:1.1.2] > > at > > org.apache.nifi.controller.tasks.ContinuallyRunProcessorTask.call(ContinuallyRunProcessorTask.java:136) > [nifi-framework-core-1.1.2.jar:1.1.2] > > at > > org.apache.nifi.controller.tasks.ContinuallyRunProcessorTask.call(ContinuallyRunProcessorTask.java:47) > [nifi-framework-core-1.1.2.jar:1.1.2] > > at > > org.apache.nifi.controller.scheduling.TimerDrivenSchedulingAgent$1.run(TimerDrivenSchedulingAgent.java:132) > [nifi-framework-core-1.1.2.jar:1.1.2] > > at > java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) > [na:1.8.0_51] > > at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308) > [na:1.8.0_51] > > at > > java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180) > [na:1.8.0_51] > > at > > java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294) > [na:1.8.0_51] > > at > > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) > [na:1.8.0_51] > > at > > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) > [na:1.8.0_51] > > at java.lang.Thread.run(Thread.java:745) [na:1.8.0_51] > > 2017-04-25 17:16:34,833 ERROR [Timer-Driven Process Thread-7] > o.a.n.processors.standard.ListenSyslog > ListenSyslog[id=0a932c37-0158-1000-0000-0000656754bf] > ListenSyslog[id=0a932c37-0158-1000-0000-0000656754bf] failed to process > session due to java.lang.NullPointerException: > java.lang.NullPointerException > > 2017-04-25 17:16:34,833 ERROR [Timer-Driven Process Thread-7] > o.a.n.processors.standard.ListenSyslog > > java.lang.NullPointerException: null > > at > > org.apache.nifi.processors.standard.ListenSyslog.onTrigger(ListenSyslog.java:431) > ~[na:na] > > at > > org.apache.nifi.processor.AbstractProcessor.onTrigger(AbstractProcessor.java:27) > ~[nifi-api-1.1.2.jar:1.1.2] > > at > > org.apache.nifi.controller.StandardProcessorNode.onTrigger(StandardProcessorNode.java:1099) > ~[nifi-framework-core-1.1.2.jar:1.1.2] > > at > > org.apache.nifi.controller.tasks.ContinuallyRunProcessorTask.call(ContinuallyRunProcessorTask.java:136) > [nifi-framework-core-1.1.2.jar:1.1.2] > > at > > org.apache.nifi.controller.tasks.ContinuallyRunProcessorTask.call(ContinuallyRunProcessorTask.java:47) > [nifi-framework-core-1.1.2.jar:1.1.2] > > at > > org.apache.nifi.controller.scheduling.TimerDrivenSchedulingAgent$1.run(TimerDrivenSchedulingAgent.java:132) > [nifi-framework-core-1.1.2.jar:1.1.2] > > at > java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) > [na:1.8.0_51] > > at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308) > [na:1.8.0_51] > > at > > java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180) > [na:1.8.0_51] > > at > > java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294) > [na:1.8.0_51] > > at > > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) > [na:1.8.0_51] > > at > > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) > [na:1.8.0_51] > > at java.lang.Thread.run(Thread.java:745) [na:1.8.0_51] > > 2017-04-25 17:16:34,833 WARN [Timer-Driven Process Thread-7] > o.a.n.processors.standard.ListenSyslog > ListenSyslog[id=0a932c37-0158-1000-0000-0000656754bf] Processor > Administratively Yielded for 1 sec due to processing failure > > 2017-04-25 17:16:34,833 WARN [Timer-Driven Process Thread-7] > o.a.n.c.t.ContinuallyRunProcessorTask Administratively Yielding > ListenSyslog[id=0a932c37-0158-1000-0000-0000656754bf] due to uncaught > Exception: java.lang.NullPointerException > > 2017-04-25 17:16:34,834 WARN [Timer-Driven Process Thread-7] > o.a.n.c.t.ContinuallyRunProcessorTask > > > > is what I’m trying to do screwing up the processor? If so, is this a) > something that can be done with a load balancer – I’ve seen blog posts > about > putting an HA proxy in front of Nifi for other ListXX processors but not > ListenSyslog, and b) if can’t be done is there suggestions on how I can > better distribute load other than the standard pattern using RPG (which > only > distributes the processing of the data once it is in). > > > > Many thanks > > Conrad > > > > SecureData, combating cyber threats > > ________________________________ > > The information contained in this message or any of its attachments may be > privileged and confidential and intended for the exclusive use of the > intended recipient. If you are not the intended recipient any disclosure, > reproduction, distribution or other dissemination or use of this > communications is strictly prohibited. The views expressed in this email > are > those of the individual and not necessarily of SecureData Europe Ltd. Any > prices quoted are only valid if followed up by a formal written quote. > > SecureData Europe Limited. Registered in England & Wales 04365896. > Registered Address: SecureData House, Hermitage Court, Hermitage Lane, > Maidstone, Kent, ME16 9NT > > > > ***This email originated outside SecureData*** > > Click > https://www.mailcontrol.com/sr/Ua65DeG3YiLGX2PQPOmvUqEZpHzWxmiIfINdU0qyFhoA5x+dEtpBxXgn6ouHwVqxHTbTRgxltt8YNBM7q0TQEQ== > to report this email as spam. > > > >
