Luke, I have not yet heard of anyone else running into this, personally. Typically, when I've used these processors, I am expecting sub-second response times, not 30+ second response times. Of course, your use case, though, is perfectly valid - just not something that I've ever run into myself.
I have submitted a new JIRA [1] to address this issue. Thanks -Mark [1] https://issues.apache.org/jira/browse/NIFI-1732 <https://issues.apache.org/jira/browse/NIFI-1732> > On Apr 1, 2016, at 12:32 PM, Stephen Coder - US <[email protected]> wrote: > > Hi Nifi Team, > > > I've been experimenting with the HandleHttpRequest/Response processors in > Nifi 0.5.1, and have noticed an issue that I've not been able to resolve. I'm > hoping that I'm simply missing a configuration item, but I've been unable to > find the solution. > > > The scenario is this: HandleHttpRequest --> Long Processing (> 30 seconds) > --> HandleHttpResponse. It appears that the jetty server backing the > HandleHttpRequest has a built in idle time timeout of 30000 ms (see > jetty-server/src/main/java/org/eclipse/jetty/server/AbstractConnector.java > _idleTimeout value). In my test flow, 30 seconds after a HTTP requests comes > in, a second request comes into the flow. It has the same information, except > the http.context.identifier and the FlowFile UUID has changed, and the > http.dispatcher.type has changed from REQUEST to ERROR. From my online > research > (http://stackoverflow.com/questions/30786939/jetty-replay-request-on-timeout?), > this re-request with a type of error comes in after jetty determines that a > request has timed out. > > > This would not normally be a big deal. I was able to RouteOnAttribute and > capture all ERROR requests without responding. However, those requests are > never cleared from the StandardHttpContextMap. I've tested this by setting > the number of requests allowed by the StandardHttpContextMap to 4, and done 4 > of my long Request/Response tests. Each request is correctly responded to > eventually in my test, but because they take over 30 seconds each also > generates an ERROR request that is stored in the StandardHttpContextMap. If I > then leave the system alone for much longer than the Request Timeout > parameter in the StandardHttpContextMap and then attempt a request, I get a > 503 response saying that the queue is full and no requests are allowed. No > requests are allowed at all until I delete and recreate the Map. > > > It seems unlikely to me that no one has attempted to use these processors in > this fashion. However, looking through the unit test for this processor it > seems like no where was a timeout tested over 30 seconds, so I thought it > worth a conversation. > > > So finally, is there a configuration item to extend the jetty server's idle > timeout? Or is there a better way to ensure that the bogus requests don't get > stuck permanently in the StandardHttpContextMap? I appreciate any pointers > you can give. > > > Thanks, > Luke Coder > BIT Systems > CACI - NCS > 941-907-8803 x705 > 6851 Professional Pkwy W > Sarasota, FL 34240
