Robert, Thanks for the update, that's very interesting. Version 3.8.1 is several years old and is missing a number of updates related to HTTP/2 and TLS. OkHttp 3.12.0 introduced support for TLS 1.3 when running on a supported JVM, so if the connection is occurring over HTTPS, that may be part of the equation. Java 11 supports TLS 1.3, whereas Java 8 did not support TLS 1.3 until more recent updates. It would be interesting to know if your configuration still works with a more recent version of OkHttp in the 3.x series. Thanks again for providing the feedback.
Regards, David Handermann On Tue, Jun 1, 2021 at 1:31 PM Robert R. Bruno <[email protected]> wrote: > David, > > Quick update for you. We decided after a bit of troubleshooting with zero > luck to just downgrade the OKHttp to 3.8.1 and the okhttp-digest to > 1.18, and no more errors. Not sure what to say. > > Robert > > On Mon, May 31, 2021 at 8:41 AM David Handermann < > [email protected]> wrote: > >> Hi Robert, >> >> Thanks for providing the additional details. It should be possible to >> replace the current version of OkHttp 4.9.1 with an older version to see if >> that makes a difference. It would also be helpful to know whether the >> remote server supports HTTP/2. Newer versions of OkHttp have improved >> support for HTTP/2, but it also has different connection characteristics. >> Setting the Disable HTTP/2 property to True in InvokeHTTP would force the >> use of HTTP/1.1. I would not necessarily expect to see errors on the >> server side, but knowing whether the remote server has a connection or >> write timeout property would be useful. >> >> Regards, >> David Handermann >> >> On Sun, May 30, 2021 at 4:54 AM Robert R. Bruno <[email protected]> >> wrote: >> >>> When seeing the error we put our timeouts values in the processor both >>> to 5 mins as a test and still saw the errors and well before 5 minutes. We >>> also slowed the processor down a lot and still were seeing the error. >>> Failed attempts will often succeed just fine but not always. >>> >>> As an easy test could we just rebuild with the older http client library >>> or did a lot more change with the processor? >>> >>> We do have access to both endpoints and plan to dig deeper there as >>> well, but initial looking did not show errors on server side. >>> >>> On Sat, May 29, 2021, 23:26 David Handermann < >>> [email protected]> wrote: >>> >>>> Hi Robert, >>>> >>>> It would be helpful to know the settings for the Read Timeout and Idle >>>> Timeout properties on the InvokeHTTP processors. If you have access to the >>>> remote service being called, it would also be interesting to know if there >>>> are timeouts on that side of the connection. NiFi 1.13.2 includes a much >>>> newer version of the OkHttp client library, which InvokeHTTP uses, so the >>>> internal connection handling has gone through some changes. In general, >>>> broken pipe errors suggest that the connection is timing out at some point, >>>> which may be related to a number of a variety of factors such as the number >>>> of connections, payload sizes, network latency, or local resource >>>> consumption. >>>> >>>> Regards, >>>> David Handermann >>>> >>>> On Sat, May 29, 2021 at 2:08 PM Joe Witt <[email protected]> wrote: >>>> >>>>> K. We have seen specific jvm versions causing issues with socket >>>>> handling. But had not seen it on Java 11 though may be possible. Is >>>>> there a full stack trace? >>>>> >>>>> On Sat, May 29, 2021 at 12:00 PM Robert R. Bruno <[email protected]> >>>>> wrote: >>>>> >>>>>> We upgraded to java 11 when we upgrade to 1.13.2 we were on java 8 >>>>>> with 1.9.2. >>>>>> >>>>>> On Sat, May 29, 2021, 14:21 Joe Witt <[email protected]> wrote: >>>>>> >>>>>>> What JVM are you using? >>>>>>> >>>>>>> Thanks >>>>>>> >>>>>>> On Sat, May 29, 2021 at 11:16 AM Juan Pablo Gardella < >>>>>>> [email protected]> wrote: >>>>>>> >>>>>>>> Not related to Nifi, but I faced the same type of issue for >>>>>>>> endpoints behind a proxy which takes more than 30 seconds to answer. >>>>>>>> Fixed >>>>>>>> by replacing Apache Http client by OkHttp. I did not investigate >>>>>>>> further, >>>>>>>> just simply replaced one library by another and the error was fixed. >>>>>>>> >>>>>>>> >>>>>>>> Juan >>>>>>>> >>>>>>>> On Sat, 29 May 2021 at 15:08, Robert R. Bruno <[email protected]> >>>>>>>> wrote: >>>>>>>> >>>>>>>>> I wanted to see if anyone has any ideas on this one. Since >>>>>>>>> upgrading to 1.13.2 from 1.9.2 we are starting to see broken pipe >>>>>>>>> (write >>>>>>>>> failed) errors from a few invokeHttp processers. >>>>>>>>> >>>>>>>>> It is happening to processors talking to different endpoints, so I >>>>>>>>> am suspecting it is on the nifi side. We are now using load balanced >>>>>>>>> queues throughout our flow. Is it possible we are hitting a http >>>>>>>>> connection resource issue or something like that? A total guess I'll >>>>>>>>> admit. >>>>>>>>> >>>>>>>>> If this could be it, does anyone know which parameter(s) to play >>>>>>>>> with in the properties file? I know there is one setting for jetty >>>>>>>>> threads >>>>>>>>> and another for max concurrent requests, but it isn't quite clear to >>>>>>>>> me of >>>>>>>>> they are at all involved with invokeHttp calls. >>>>>>>>> >>>>>>>>> Thanks in advance! >>>>>>>>> >>>>>>>>> Robert >>>>>>>>> >>>>>>>>
