exceptionfactory commented on PR #7830: URL: https://github.com/apache/nifi/pull/7830#issuecomment-1746013182
Thanks for the testing and feedback @joewitt! After reviewing the request content buffering and the available Netty Handlers, I consolidated the HTTP processing to work with the Netty [FullHttpRequest](https://netty.io/4.1/api/io/netty/handler/codec/http/FullHttpRequest.html) for both HTTP/1.1 and HTTP/2. This streamlined the implementation in `HttpRequestHandler`, avoiding the previous buffer handling issue that resulted in the OutOfMemoryError. Following these changes, running a continuous stream of JSON files through `InvokeHTTP` did not produce any errors. Heap and non-heap memory usage also followed a consistent pattern. As part of streamlining the handler pipeline, I added a 10 MB maximum content length limitation. Various service providers set request limits for OpenTelemetry payloads to 4 MB or 5 MB. This could be changed to a configurable property, but larger sizes also introduce potential memory usage concerns. Along similar lines, the default Queue Capacity is now 1000 and the default Batch Size is now 100, also limiting potential memory issues in the default configuration. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
