Re: PutTCP connector not cleaning up dangling connections

This is a shot in the dark, but I'd be curious if you saw any different behavior running your test with this change: https://github.com/apache/nifi/pull/2168 On Thu, Sep 21, 2017 at 1:31 AM, ddewaele wrote: > After looking at the tcp dumps (data packets) the only thing I

Re: PutTCP connector not cleaning up dangling connections

The issue is on a CentOS linux server, running Nifi 1.4.0 snapshot. There it communicates with a dozen of Moxa nports over tcp on various remote sites. All moxa's are connected over a (slow) cellular connection. Will do a tcpdump with the new / old PutTCP processor. On Wed, 20 Sep 2017 at 22:10,

Re: PutTCP connector not cleaning up dangling connections

The sender/output stream can only ever be in use by one thread since it acquires it from the pool at the beginning of onTrigger, which makes it unavailable to any other threads until it is placed back in the pool. The flush is there just because now it is writing to an output stream and we may be

Re: PutTCP connector not cleaning up dangling connections

Don't get it either... new code looks more or less the same. been trying various things all day. Glad I got it working with a "patched" PutTCP. Better from what I got but still not ideal. Can it be a threading issue (we interact with multiple modules) in the input / output stream handling ? Also

Re: PutTCP connector not cleaning up dangling connections

For what it is worth, I can't reproduce this on my Mac laptop either... Setup GenerateFlowFile with same sample text your provided, followed by PutTCP with the \r\n delimiter, this sends to ListenTCP in the same NiFi instance. If I look at the content of the flow files coming out of ListenTCP,

Re: PutTCP connector not cleaning up dangling connections

Correct about the delimiter. Think character set is (was) automatically filled in with utf-8. (Never noticed a default option) On Wed, 20 Sep 2017 at 19:11, Bryan Bende wrote: > Thanks for the info and for digging into this.. > > Just to confirm, in your PutTCP configuration,

Re: PutTCP connector not cleaning up dangling connections

Thanks for the info and for digging into this.. Just to confirm, in your PutTCP configuration, you are saying that you have "Outgoing Message Delimiter" set to \r\n ? And do you have the "Character Set" property set to the default of "UTF-8" ? On Wed, Sep 20, 2017 at 12:59 PM, ddewaele

Re: PutTCP connector not cleaning up dangling connections

Sounds good, interested to hear about it when you get a chance. On Wed, Sep 20, 2017 at 12:32 PM, Davy De Waele wrote: > Having dinner now without laptop :) Have more updates. Isolated the problem > in the puttcp class in the processor. Rest of the 1.4.0-snapshot seems fine.

Re: PutTCP connector not cleaning up dangling connections

Having dinner now without laptop :) Have more updates. Isolated the problem in the puttcp class in the processor. Rest of the 1.4.0-snapshot seems fine. Will post more info after dinner. On Wed, 20 Sep 2017 at 18:27, Bryan Bende wrote: > Is the data good at other points in

Re: PutTCP connector not cleaning up dangling connections

Small update : No garbage / noise on 1.3.0 also so it must be in 1.4.0-SNAPSHOT. Noticed that the PutTCP processor has changed the way it processes incoming flowfiles. Might be related to that. Looked into the data provenance and noticed 3 bytes EF BF BD (in hex) when things go bad. (This

Re: PutTCP connector not cleaning up dangling connections

I've let it run overnight on 1.4.0-SNAPSHOT. Didn't see any hanging connections and after timeouts they were cleaned up. However, I noticed something else (perhaps unrelated). About 40% of the messages that we "get" from the tcp connection contained "noise / garbage" and didn't pass their

Re: PutTCP connector not cleaning up dangling connections

thanks davy - i'll get it reviewed and merged if you can give it a go and verify it makes your case better. Thanks On Tue, Sep 19, 2017 at 1:57 PM, ddewaele wrote: > Hi, > > Trying it out now. forgot how long it takes to build :) > > Will give feedback here. > > Thx for the

Re: PutTCP connector not cleaning up dangling connections

Hi, Trying it out now. forgot how long it takes to build :) Will give feedback here. Thx for the client port logging also ... that is always useful for debugging perhaps we can check later in what way we can retrieve it in the timeout scenarios / standard close scenario Really hope this

Re: PutTCP connector not cleaning up dangling connections

Davy If you could give the PR a try and see if it helps I'd be happy to help get it reviewed and in for 1.4 if timing works out. Thanks On Mon, Sep 18, 2017 at 4:39 PM, Bryan Bende wrote: > Davy, > > I just pushed a second commit to the PR that will log the port from > the

Re: PutTCP connector not cleaning up dangling connections

Davy, I just pushed a second commit to the PR that will log the port from the local address of the socket being used by the sender, which I think is what you mean by the client port. If you turn on debug for PutTCP you will see something like... o.apache.nifi.processors.standard.PutTCP

Re: PutTCP connector not cleaning up dangling connections

Thx a lot for the quick response. Looking forward to the PR and the release :) Would this for example still make the 1.4.0 release ? It would also be very interesting to log client ports in debug mode don't know how easy that is with nio. There is Keep Alive Timeout = 2min specified on

Re: PutTCP connector not cleaning up dangling connections

Thx a lot for the quick response. Looking forward to the PR and the release :) Would this for example still make the 1.4.0 release ? It would also be very interesting to log client ports in debug mode don't know how easy that is with nio. There is Keep Alive Timeout = 2min specified on

Re: PutTCP connector not cleaning up dangling connections

I just created a JIRA and will put up a PR shortly: https://issues.apache.org/jira/browse/NIFI-4391 The processor is catching the exception while attempting to obtain a connection, and then logs an error and transfers to failure which is where we see this message: 2017-09-17 14:20:20,860 ERROR

Re: PutTCP connector not cleaning up dangling connections

Good catch. Can you please be sure to cover in a JIRA? That said, wouldn't we see that in the stack trace during the problematic condition? On Mon, Sep 18, 2017 at 9:16 AM, Bryan Bende wrote: > The code in SocketChannelSender that Davy pointed out could definitely > be the

Re: PutTCP connector not cleaning up dangling connections

The code in SocketChannelSender that Davy pointed out could definitely be the problem... It makes a non-blocking channel and calls connect, then goes into a loop waiting for finishConnect() to return true, but if that doesn't happen before the configured timeout, then it throws an exception, but

Re: PutTCP connector not cleaning up dangling connections

Davy Interesting. So in looking through the stack trace I don't see anything related to sockets nifi has initiated to another service and nothing for PutTCP. I'm not saying that means there is nothing but the stack traces only show the custom GetTCP processors. You can use netstat to show open

Re: PutTCP connector not cleaning up dangling connections

Stopping the processor doesn't cleanup the tcp connection. It remains ESTABLISHED. There are 2 ways of getting out of it (none of them are ideal). - Restarting Nifi - Restarting the Moxa serial ports I've dumped the output in the following gist :

Re: PutTCP connector not cleaning up dangling connections

If you stop the processor manually does it clean them up? When the connections appear stuck can you please get a thread dump? bin/nifi.sh dump The results end up in bootstrap.log. Thanks Joe On Sep 17, 2017 2:22 PM, "ddewaele" wrote: > We are using NiFi PutTCP processors