Looking at https://datatracker.ietf.org/doc/html/draft-ietf-idpr-specv1#section-2.4
I think youll want to write an IDPRRecordReader Thanks On Wed, Feb 9, 2022 at 8:05 PM Nathan Gough <thena...@apache.org> wrote: > You should be able to set the message delimiter in ListenTCP with the > "Message Delimiter" property. The default is \n. I don't believe there's a > great way to see how data is being interpreted other than using interactive > debug mode with NiFi set through the bootstrap.conf. > > On Wed, Feb 9, 2022 at 9:29 PM Breno Cesar <brenocrs230...@gmail.com> > wrote: > >> Hi guys, >> >> First of all i apologize for my english, and thank you very much for your >> answers. >> The data type i need is a data stream, which starts when the ipdr >> protocol connect process ends. >> Is there any way to change the line terminator? or somehow see how nifi >> is interpreting the data when It its received ? >> >> Em qua., 9 de fev. de 2022 22:31, Joe Witt <joe.w...@gmail.com> escreveu: >> >>> Correct. With both ListenTCP and ListenTCPRecord it listens for data >>> and needs some way of framing them to generate records/bundles of data as >>> flowfiles. >>> >>> What kind of data do you want to listen for? >>> >>> Thanks >>> >>> On Wed, Feb 9, 2022 at 6:26 PM Michael Loftis <mlof...@wgops.com> wrote: >>> >>>> ListenTCP isn’t fit for ingesting arbitrary data. It’s record oriented, >>>> default separator is Unix newline “\n”. I don’t think it will generate a >>>> flow if it never receives a newline before the connection closes. >>>> >>>> On Wed, Feb 9, 2022 at 18:17 Breno Cesar <brenocrs230...@gmail.com> >>>> wrote: >>>> >>>>> i'm tring to receive a tcp stream using the listentcp module, but when >>>>> the connection start, i see the traffic reaching my server (usgin >>>>> tcpdump), >>>>> but the nifi does note generate a flow file. >>>>> Ther is no error ow warning in the module or in the nifi cluster. >>>>> Changing in the processor the bulleting level to "debug" , i can see >>>>> the following message for every package that i receive: >>>>> >>>>> SPOLPAVMW131:8080ListenTCP[id=e083186d-017e-1000-0000-000040efa66e] >>>>> Accepted incoming connection from /192.0.2.12:55209 >>>>> SPOLPAVMW131:8080ListenTCP[id=e083186d-017e-1000-0000-000040efa66e] >>>>> bytes read 46 >>>>> SPOLPAVMW131:8080ListenTCP[id=e083186d-017e-1000-0000-000040efa66e] No >>>>> more data available, returning for selection >>>>> SPOLPAVMW131:8080ListenTCP[id=e083186d-017e-1000-0000-000040efa66e] >>>>> Reached EOF, closing connection >>>>> >>>>> bellow is the same message, but in the tcpdump format. >>>>> >>>>> Internet Protocol Version 4, Src: 192.0.2.12, Dst: 192.0.2.128 >>>>> Transmission Control Protocol, Src Port: 55209 , Dst Port: 4737, >>>>> Seq: 1, Ack: 1, Len: 46 >>>>> Source Port: 55209 >>>>> Destination Port: 4737 >>>>> <Source or Destination Port: 55209 > >>>>> <Source or Destination Port: 4737> >>>>> [Stream index: 0] >>>>> [TCP Segment Len: 46] >>>>> Sequence number: 1 (relative sequence number) >>>>> Sequence number (raw): 3765145522 >>>>> [Next sequence number: 47 (relative sequence number)] >>>>> Acknowledgment number: 1 (relative ack number) >>>>> Acknowledgment number (raw): 274707205 >>>>> 0101 .... = Header Length: 20 bytes (5) >>>>> Flags: 0x018 (PSH, ACK) >>>>> Window size value: 10000 >>>>> [Calculated window size: 10000] >>>>> [Window size scaling factor: 1] >>>>> Checksum: 0x5fd7 [unverified] >>>>> [Checksum Status: Unverified] >>>>> Urgent pointer: 0 >>>>> [SEQ/ACK analysis] >>>>> [Timestamps] >>>>> TCP payload (46 bytes) >>>>> [PDU Size: 46] >>>>> IPDR >>>>> Version: 2 >>>>> Message id: CONNECT (5) >>>>> Session id: 0 >>>>> Message flags: 0x00 >>>>> Message length: 46 >>>>> Initiator id: 0.0.0.0 >>>>> Initiator port: 0 >>>>> Capabilities: 0x00000006 >>>>> Keep-alive interval: 90 >>>>> Vendor id: ARRIS E6000 Exporter >>>>> >>>>> Here somethings that i tried: >>>>> >>>>> - Increase the number of simultaneos connections and batch size, but >>>>> it does not change at all. >>>>> - Use listentcprecord but the behavior is the same, it only change >>>>> when i select the avroreader, then i got an error informing that is a >>>>> "wrong data format". >>>>> >>>>> So i belive that nifi is receving the messages that come from tcp , >>>>> but for some reason is not generating flowfiles. >>>>> >>>>> Does anyone got same behavior ? >>>>> There is anything else that i can do ? >>>>> >>>>> i'm using nifi 1.15.3 with a 4 nodes cluster. >>>>> >>>>> >>>>> Breno >>>>> >>>>> >>>>> >>>>> -- >>>> >>>> "Genius might be described as a supreme capacity for getting its >>>> possessors >>>> into trouble of all kinds." >>>> -- Samuel Butler >>>> >>>
