This is a shot in the dark, but I'd be curious if you saw any
different behavior running your test with this change:
https://github.com/apache/nifi/pull/2168
On Thu, Sep 21, 2017 at 1:31 AM, ddewaele wrote:
> After looking at the tcp dumps (data packets) the only thing I
The issue is on a CentOS linux server, running Nifi 1.4.0 snapshot. There
it communicates with a dozen of Moxa nports over tcp on various remote
sites. All moxa's are connected over a (slow) cellular connection.
Will do a tcpdump with the new / old PutTCP processor.
On Wed, 20 Sep 2017 at 22:10,
The sender/output stream can only ever be in use by one thread since
it acquires it from the pool at the beginning of onTrigger, which
makes it unavailable to any other threads until it is placed back in
the pool.
The flush is there just because now it is writing to an output stream
and we may be
Don't get it either... new code looks more or less the same. been trying
various things all day. Glad I got it working with a "patched" PutTCP.
Better from what I got but still not ideal.
Can it be a threading issue (we interact with multiple modules) in the
input / output stream handling ? Also
For what it is worth, I can't reproduce this on my Mac laptop either...
Setup GenerateFlowFile with same sample text your provided, followed
by PutTCP with the \r\n delimiter, this sends to ListenTCP in the same
NiFi instance.
If I look at the content of the flow files coming out of ListenTCP,
Correct about the delimiter.
Think character set is (was) automatically filled in with utf-8. (Never
noticed a default option)
On Wed, 20 Sep 2017 at 19:11, Bryan Bende wrote:
> Thanks for the info and for digging into this..
>
> Just to confirm, in your PutTCP configuration,
Thanks for the info and for digging into this..
Just to confirm, in your PutTCP configuration, you are saying that you
have "Outgoing Message Delimiter" set to \r\n ?
And do you have the "Character Set" property set to the default of "UTF-8" ?
On Wed, Sep 20, 2017 at 12:59 PM, ddewaele
Sounds good, interested to hear about it when you get a chance.
On Wed, Sep 20, 2017 at 12:32 PM, Davy De Waele wrote:
> Having dinner now without laptop :) Have more updates. Isolated the problem
> in the puttcp class in the processor. Rest of the 1.4.0-snapshot seems fine.
Having dinner now without laptop :) Have more updates. Isolated the problem
in the puttcp class in the processor. Rest of the 1.4.0-snapshot seems fine.
Will post more info after dinner.
On Wed, 20 Sep 2017 at 18:27, Bryan Bende wrote:
> Is the data good at other points in
Small update : No garbage / noise on 1.3.0 also so it must be in
1.4.0-SNAPSHOT.
Noticed that the PutTCP processor has changed the way it processes incoming
flowfiles. Might be related to that.
Looked into the data provenance and noticed 3 bytes EF BF BD (in hex) when
things go bad. (This
I've let it run overnight on 1.4.0-SNAPSHOT. Didn't see any hanging
connections and after timeouts they were cleaned up.
However, I noticed something else (perhaps unrelated). About 40% of the
messages that we "get" from the tcp connection contained "noise / garbage"
and didn't pass their
thanks davy - i'll get it reviewed and merged if you can give it a go
and verify it makes your case better.
Thanks
On Tue, Sep 19, 2017 at 1:57 PM, ddewaele wrote:
> Hi,
>
> Trying it out now. forgot how long it takes to build :)
>
> Will give feedback here.
>
> Thx for the
Hi,
Trying it out now. forgot how long it takes to build :)
Will give feedback here.
Thx for the client port logging also ... that is always useful for
debugging perhaps we can check later in what way we can retrieve it in
the timeout scenarios / standard close scenario
Really hope this
Davy
If you could give the PR a try and see if it helps I'd be happy to
help get it reviewed and in for 1.4 if timing works out.
Thanks
On Mon, Sep 18, 2017 at 4:39 PM, Bryan Bende wrote:
> Davy,
>
> I just pushed a second commit to the PR that will log the port from
> the
Davy,
I just pushed a second commit to the PR that will log the port from
the local address of the socket being used by the sender, which I
think is what you mean by the client port.
If you turn on debug for PutTCP you will see something like...
o.apache.nifi.processors.standard.PutTCP
Thx a lot for the quick response. Looking forward to the PR and the
release :)
Would this for example still make the 1.4.0 release ?
It would also be very interesting to log client ports in debug mode
don't know how easy that is with nio.
There is Keep Alive Timeout = 2min specified on
Thx a lot for the quick response. Looking forward to the PR and the
release :)
Would this for example still make the 1.4.0 release ?
It would also be very interesting to log client ports in debug mode
don't know how easy that is with nio.
There is Keep Alive Timeout = 2min specified on
I just created a JIRA and will put up a PR shortly:
https://issues.apache.org/jira/browse/NIFI-4391
The processor is catching the exception while attempting to obtain a
connection, and then logs an error and transfers to failure which is
where we see this message:
2017-09-17 14:20:20,860 ERROR
Good catch. Can you please be sure to cover in a JIRA?
That said, wouldn't we see that in the stack trace during the
problematic condition?
On Mon, Sep 18, 2017 at 9:16 AM, Bryan Bende wrote:
> The code in SocketChannelSender that Davy pointed out could definitely
> be the
The code in SocketChannelSender that Davy pointed out could definitely
be the problem...
It makes a non-blocking channel and calls connect, then goes into a
loop waiting for finishConnect() to return true, but if that doesn't
happen before the configured timeout, then it throws an exception, but
Davy
Interesting. So in looking through the stack trace I don't see
anything related to sockets nifi has initiated to another service and
nothing for PutTCP. I'm not saying that means there is nothing but
the stack traces only show the custom GetTCP processors.
You can use netstat to show open
Stopping the processor doesn't cleanup the tcp connection. It remains
ESTABLISHED.
There are 2 ways of getting out of it (none of them are ideal).
- Restarting Nifi
- Restarting the Moxa serial ports
I've dumped the output in the following gist :
If you stop the processor manually does it clean them up?
When the connections appear stuck can you please get a thread dump?
bin/nifi.sh dump
The results end up in bootstrap.log.
Thanks
Joe
On Sep 17, 2017 2:22 PM, "ddewaele" wrote:
> We are using NiFi PutTCP processors
23 matches
Mail list logo