Hi Shamsudeen, Thanks for the reply and clarifying the use of the protocol. UDP does not offer any kind of delivery guarantees at the protocol level, so it is even less reliable than TCP for syslog. In other words, data loss can happen without any type of error being logged.
If the data loss is frequent, a network packet monitor such as tcpdump [1] is one tool for evaluating network communication. Regards, David Handermann [1] https://www.tcpdump.org/ On Fri, Mar 29, 2024 at 8:44 AM Shamsudeen Jameer <shamsudeen.jam...@prth.com> wrote: > > Hi David, > Thanks for the information. I'm actually using the UDP port to transfer logs > to the syslog system. Planning to upgrade to TCP in the future but just need > to figure out why it's hit \ miss. > > Regards, > Shamsudeen Jameer > > > > On Thu, Mar 28, 2024 at 10:37 PM David Handermann > <exceptionfact...@apache.org> wrote: >> >> Hi Shamsudeen, >> >> Although TCP incorporates packet acknowledgement as a core feature of >> the protocol, syslog does not provide any type of delivery guarantees >> on top of TCP. This means that it is possible for a system such as >> NiFi to send a syslog message, without logging a socket communication >> error, but for the message to be lost. This may not be common, but it >> is a known possibility. >> >> The following post is from 2008, but the explanation on the >> unreliability of syslog still applies: >> >> https://rainer.gerhards.net/2008/04/on-unreliability-of-plain-tcp-syslog.html >> >> There are several modern alternatives to syslog, with OpenTelemetry >> Logging [1] as one example of a transport protocol that includes >> delivery acknowledgement, providing much better reliability. >> >> Recent versions of NiFi have incorporated both improvements to syslog >> processing and support for OpenTelemetry [2]. Upgrading to a more >> recent version of NiFi would be a useful step, regardless of the >> protocol used. >> >> Regards, >> David Handermann >> >> [1] https://opentelemetry.io/docs/specs/otel/logs/ >> [2] >> https://exceptionfactory.com/posts/2024/02/26/building-opentelemetry-collection-in-apache-nifi-with-netty/ >> >> On Thu, Mar 28, 2024 at 9:03 PM Shamsudeen Jameer >> <shamsudeen.jam...@prth.com> wrote: >> > >> > Hello everyone, >> > I'm currently running Nifi version 1.8.0, starting from last week I >> > noticed that I'm missing log from my syslog system. Some logs would >> > appear and some wouldn't write to the syslog system. I checked my syslog >> > system and was unable to find any issue. No other systems are >> > experiencing issue writing to the syslog system. I enabled DEBUG on >> > PutSysLog and it shows it was successful in sending the message to the >> > syslog system but no entry exists. At the same time other PutSysLog does >> > record the entries and it flows through the process group. Any >> > suggestions on how to troubleshoot this issue? >> > Thanks! >> > >> > PROPRIETARY / CONFIDENTIALITY NOTICE >> > >> > This message (including any attachments) contains information that is >> > proprietary and confidential to PRIORITY TECHNOLOGY HOLDINGS, INC. and its >> > affiliates and subsidiaries, including the sender hereof, and is for the >> > sole use of the intended recipients. If you are not an intended recipient, >> > you may not read, print, retain, use, copy, distribute, forward or >> > disclose to anyone this message or any information contained in this >> > message (including any attachments). If you have received this message in >> > error, please advise the sender of this error by reply e-mail, and please >> > destroy all copies of this message (including any attachments). > > > PROPRIETARY / CONFIDENTIALITY NOTICE > > This message (including any attachments) contains information that is > proprietary and confidential to PRIORITY TECHNOLOGY HOLDINGS, INC. and its > affiliates and subsidiaries, including the sender hereof, and is for the sole > use of the intended recipients. If you are not an intended recipient, you may > not read, print, retain, use, copy, distribute, forward or disclose to anyone > this message or any information contained in this message (including any > attachments). If you have received this message in error, please advise the > sender of this error by reply e-mail, and please destroy all copies of this > message (including any attachments).
