Nice test case. Is the JVM able to garbage collect fast enough? Are you releasing the resources properly in your components as early as possible? Java VM's achieve a steady-state after a certain period of time, but I've found in some cases garbage collection doesn't keep-up with my load tests before steady state is reached. Just an idea. Again - nice test. :)
On 3/4/19, Santhosh Kumar <santhosh...@gmail.com> wrote: >> Do you actually need all those values and where do they come from in the > first place ? Do you understand what they do ? > > I was just experimenting with all available attributes related to socket > and multiplexing. I have followed the documentations, > https://tomcat.apache.org/tomcat-9.0-doc/config/http.html#SSL_Support_-_SSLHostConfig > >> The buffer sizes seem way >> too large, and maxConcurrentStreamExecution is as well (multiplexing 200 >> threads output over a single socket is not a very good idea usually, >> which >> is why the setting was added). >> The attributes are from java tcp socket attributes, you can check here > > I have given the 10x of default values to handle high load, and > maxConcurrentStreamExecution is also increased assuming it will allow large > data to be exchanged. I ll set back to default if it doesn't help > > From h2load, the payload of 1000b approx sent with 100 requests with > multiplexing as any, it fails after 65 requests approximately. > > I don't know whether tomcat have any limit set on buffer to restrict data > pushed in a same connection. > > On Mon, Mar 4, 2019 at 7:27 PM Rémy Maucherat <r...@apache.org> wrote: > >> On Mon, Mar 4, 2019 at 10:40 AM Santhosh Kumar <santhosh...@gmail.com> >> wrote: >> >> > Hi, >> > >> > We have a tomcat instance which is http2 enabled and it needs to serve >> > large number of requests using multiplexing, so we have configured our >> > instance as follows, >> > >> > <Connector port="9191" URIEncoding="UTF-8" >> > sslImplementationName="org.apache.tomcat.util.net >> > .openssl.OpenSSLImplementation" >> > protocol="org.apache.coyote.http11.Http11Nio2Protocol" >> > maxThreads="50000" SSLEnabled="true" >> > >> > >> compressibleMimeType="text/html,text/xml,text/plain,text/css,text/javascript,application/javascript,application/json,application/xml" >> > compression="on" minSpareThreads="25" >> > noCompressionUserAgents="gozilla, traviata" scheme="https" >> > secure="true" >> > keystoreFile="conf/myfile.keystore" keystorePass="password" >> > socket.appReadBufSize="81920" >> > socket.appWriteBufSize="81920" socket.rxBufSize="251880" >> > socket.txBufSize="438000"> >> > <UpgradeProtocol compression="on" >> > maxConcurrentStreamExecution="200" >> > maxConcurrentStreams="200" >> > className="org.apache.coyote.http2.Http2Protocol"/> >> > </Connector> >> > >> >> Do you actually need all those values and where do they come from in the >> first place ? Do you understand what they do ? The buffer sizes seem way >> too large, and maxConcurrentStreamExecution is as well (multiplexing 200 >> threads output over a single socket is not a very good idea usually, >> which >> is why the setting was added). >> >> Rémy >> >> >> > >> > This instance mainly serves concurrent POST request which will have >> payload >> > of size, approx 1000-1500, which can be verified by tomcat logs >> > >> > org.apache.coyote.http2.Http2Parser.validateFrame Connection [0], >> > Stream >> > [19], Frame type [DATA], Flags [1], Payload size [*1195*] >> > >> > We tested our server with the help of h2load as follows, >> > >> > h2load -n100 -c1 -m100 https://localhost:9191/ -d '/agentRequest.txt' >> > >> > We are getting this error as follows, >> > >> > >> > org.apache.coyote.http2.Http2UpgradeHandler.upgradeDispatch Connection >> [0] >> > java.io.IOException: Unable to unwrap data, invalid status >> > [BUFFER_OVERFLOW] >> > at >> > org.apache.tomcat.util.net >> > .SecureNio2Channel$2.completed(SecureNio2Channel.java:1041) >> > at >> > org.apache.tomcat.util.net >> > .SecureNio2Channel$2.completed(SecureNio2Channel.java:1000) >> > at >> > java.base/sun.nio.ch.Invoker.invokeUnchecked(Invoker.java:127) >> > at java.base/sun.nio.ch.Invoker.invokeDirect(Invoker.java:158) >> > at >> > java.base/sun.nio.ch >> > >> .UnixAsynchronousSocketChannelImpl.implRead(UnixAsynchronousSocketChannelImpl.java:552) >> > at >> > java.base/sun.nio.ch >> > >> .AsynchronousSocketChannelImpl.read(AsynchronousSocketChannelImpl.java:276) >> > at >> > java.base/sun.nio.ch >> > >> .AsynchronousSocketChannelImpl.read(AsynchronousSocketChannelImpl.java:297) >> > at >> > org.apache.tomcat.util.net >> > .SecureNio2Channel$2.completed(SecureNio2Channel.java:1027) >> > at >> > org.apache.tomcat.util.net >> > .SecureNio2Channel$2.completed(SecureNio2Channel.java:1000) >> > at >> > org.apache.tomcat.util.net >> > .SecureNio2Channel.read(SecureNio2Channel.java:1067) >> > at >> > org.apache.tomcat.util.net >> > >> .Nio2Endpoint$Nio2SocketWrapper$VectoredIOCompletionHandler.completed(Nio2Endpoint.java:1153) >> > at >> > org.apache.tomcat.util.net >> > .Nio2Endpoint$Nio2SocketWrapper.read(Nio2Endpoint.java:1026) >> > at >> > org.apache.tomcat.util.net >> > .SocketWrapperBase.read(SocketWrapperBase.java:1012) >> > at >> > >> > >> org.apache.coyote.http2.Http2AsyncParser.readFrame(Http2AsyncParser.java:61) >> > at >> > org.apache.coyote.http2.Http2Parser.readFrame(Http2Parser.java:69) >> > at >> > >> > >> org.apache.coyote.http2.Http2UpgradeHandler.upgradeDispatch(Http2UpgradeHandler.java:322) >> > at >> > >> > >> org.apache.coyote.http2.Http2AsyncUpgradeHandler.upgradeDispatch(Http2AsyncUpgradeHandler.java:37) >> > at >> > >> > >> org.apache.coyote.http11.upgrade.UpgradeProcessorInternal.dispatch(UpgradeProcessorInternal.java:54) >> > at >> > >> > >> org.apache.coyote.AbstractProcessorLight.process(AbstractProcessorLight.java:53) >> > at >> > >> > >> org.apache.coyote.AbstractProtocol$ConnectionHandler.process(AbstractProtocol.java:834) >> > at >> > org.apache.tomcat.util.net >> > .Nio2Endpoint$SocketProcessor.doRun(Nio2Endpoint.java:1769) >> > at >> > org.apache.tomcat.util.net >> > .SocketProcessorBase.run(SocketProcessorBase.java:49) >> > at >> > org.apache.tomcat.util.net >> > .AbstractEndpoint.processSocket(AbstractEndpoint.java:1048) >> > at >> > org.apache.tomcat.util.net >> > >> .SecureNio2Channel$HandshakeWriteCompletionHandler.completed(SecureNio2Channel.java:116) >> > at >> > org.apache.tomcat.util.net >> > >> .SecureNio2Channel$HandshakeWriteCompletionHandler.completed(SecureNio2Channel.java:109) >> > at >> > java.base/sun.nio.ch.Invoker.invokeUnchecked(Invoker.java:127) >> > at java.base/sun.nio.ch.Invoker.invokeDirect(Invoker.java:158) >> > >> > Why is this error is thrown? How can I configure tomcat to handle >> > concurrent POST requests which have a decent payload? >> > >> > >> > We have tried with various java clients like http-client-5-beta, jetty >> > or >> > okhttp3 and spam requests to our tomcat using http2 multiplexing and we >> > found the time taken to process a requests increases(sometimes even >> > 10x) >> > when load is increased. >> > We have tweaked all common configuration related to http2 on both >> > client >> > and server side with no luck. >> > >> > But same tomcat configuration can handle 10s of 1000s of get request >> > concurrently without a problem, its only creating problem with POST >> > requests. >> > >> > What is wrong in our configuration? >> > >> > Kindly someone shed some light. >> > >> > Tomcat - 9.0.16 >> > APR-1.2.18 >> > OpenSSL-1.1.1a >> > JDK-10.0.2 >> > OS - Ubuntu/Centos >> > HeapSize - 4GB >> > RAM -16GB >> > >> > >> > Kindly help >> > >> > -- >> > *With Regards,* >> > *Santhosh Kumar J* >> > >> > > > -- > *With Regards,* > *Santhosh Kumar J* > --------------------------------------------------------------------- To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org For additional commands, e-mail: users-h...@tomcat.apache.org
