David, Quick update for you. We decided after a bit of troubleshooting with zero luck to just downgrade the OKHttp to 3.8.1 and the okhttp-digest to 1.18, and no more errors. Not sure what to say.
Robert On Mon, May 31, 2021 at 8:41 AM David Handermann <exceptionfact...@gmail.com> wrote: > Hi Robert, > > Thanks for providing the additional details. It should be possible to > replace the current version of OkHttp 4.9.1 with an older version to see if > that makes a difference. It would also be helpful to know whether the > remote server supports HTTP/2. Newer versions of OkHttp have improved > support for HTTP/2, but it also has different connection characteristics. > Setting the Disable HTTP/2 property to True in InvokeHTTP would force the > use of HTTP/1.1. I would not necessarily expect to see errors on the > server side, but knowing whether the remote server has a connection or > write timeout property would be useful. > > Regards, > David Handermann > > On Sun, May 30, 2021 at 4:54 AM Robert R. Bruno <rbru...@gmail.com> wrote: > >> When seeing the error we put our timeouts values in the processor both to >> 5 mins as a test and still saw the errors and well before 5 minutes. We >> also slowed the processor down a lot and still were seeing the error. >> Failed attempts will often succeed just fine but not always. >> >> As an easy test could we just rebuild with the older http client library >> or did a lot more change with the processor? >> >> We do have access to both endpoints and plan to dig deeper there as well, >> but initial looking did not show errors on server side. >> >> On Sat, May 29, 2021, 23:26 David Handermann <exceptionfact...@apache.org> >> wrote: >> >>> Hi Robert, >>> >>> It would be helpful to know the settings for the Read Timeout and Idle >>> Timeout properties on the InvokeHTTP processors. If you have access to the >>> remote service being called, it would also be interesting to know if there >>> are timeouts on that side of the connection. NiFi 1.13.2 includes a much >>> newer version of the OkHttp client library, which InvokeHTTP uses, so the >>> internal connection handling has gone through some changes. In general, >>> broken pipe errors suggest that the connection is timing out at some point, >>> which may be related to a number of a variety of factors such as the number >>> of connections, payload sizes, network latency, or local resource >>> consumption. >>> >>> Regards, >>> David Handermann >>> >>> On Sat, May 29, 2021 at 2:08 PM Joe Witt <joe.w...@gmail.com> wrote: >>> >>>> K. We have seen specific jvm versions causing issues with socket >>>> handling. But had not seen it on Java 11 though may be possible. Is >>>> there a full stack trace? >>>> >>>> On Sat, May 29, 2021 at 12:00 PM Robert R. Bruno <rbru...@gmail.com> >>>> wrote: >>>> >>>>> We upgraded to java 11 when we upgrade to 1.13.2 we were on java 8 >>>>> with 1.9.2. >>>>> >>>>> On Sat, May 29, 2021, 14:21 Joe Witt <joe.w...@gmail.com> wrote: >>>>> >>>>>> What JVM are you using? >>>>>> >>>>>> Thanks >>>>>> >>>>>> On Sat, May 29, 2021 at 11:16 AM Juan Pablo Gardella < >>>>>> gardellajuanpa...@gmail.com> wrote: >>>>>> >>>>>>> Not related to Nifi, but I faced the same type of issue for >>>>>>> endpoints behind a proxy which takes more than 30 seconds to answer. >>>>>>> Fixed >>>>>>> by replacing Apache Http client by OkHttp. I did not investigate >>>>>>> further, >>>>>>> just simply replaced one library by another and the error was fixed. >>>>>>> >>>>>>> >>>>>>> Juan >>>>>>> >>>>>>> On Sat, 29 May 2021 at 15:08, Robert R. Bruno <rbru...@gmail.com> >>>>>>> wrote: >>>>>>> >>>>>>>> I wanted to see if anyone has any ideas on this one. Since >>>>>>>> upgrading to 1.13.2 from 1.9.2 we are starting to see broken pipe >>>>>>>> (write >>>>>>>> failed) errors from a few invokeHttp processers. >>>>>>>> >>>>>>>> It is happening to processors talking to different endpoints, so I >>>>>>>> am suspecting it is on the nifi side. We are now using load balanced >>>>>>>> queues throughout our flow. Is it possible we are hitting a http >>>>>>>> connection resource issue or something like that? A total guess I'll >>>>>>>> admit. >>>>>>>> >>>>>>>> If this could be it, does anyone know which parameter(s) to play >>>>>>>> with in the properties file? I know there is one setting for jetty >>>>>>>> threads >>>>>>>> and another for max concurrent requests, but it isn't quite clear to >>>>>>>> me of >>>>>>>> they are at all involved with invokeHttp calls. >>>>>>>> >>>>>>>> Thanks in advance! >>>>>>>> >>>>>>>> Robert >>>>>>>> >>>>>>>
