I reported this issue to Oracle several month ago, and it took a long time that I received the mail from Oracle support team that they need a producer. Before I got time to come back to have a look at this again and provided them code to reproduce this, they already closed as a non-reproducible bug. I also wrote several emails after that to report this bug still exists in latest JDK7/8 with some of my analysis, and haven't get their response.
This is another findings I've talked with Oracle: This performance regression is actually introduced by https://bugs.openjdk.java.net/browse/JDK-7199862. From changeset http://hg.openjdk.java.net/jdk7u/jdk7u40/jdk/rev/e6dc1d9bc70b, it tries to read serverSocket's inputStream to know if the connection is still alive. But it dramatically slows down the things. Remove setFixLengthStreamMode() is a temporary workaround, and this will lead client side consumes more memory for the message doesn't reach default 4k size, especially in some concurrent environment. I agreed with Sergey to enable/disable this mode with system property and wait JDK team finally get this fixed. I created a simple test case to verify if this issue still exists in the latest jdk if you are interested : https://github.com/jimma/jdk7u40-httpurlconnection-issue Cheers, Jim On Wed, Jan 28, 2015 at 5:09 AM, Sergey Beryozkin <[email protected]> wrote: > Hi Dan > On 27/01/15 18:00, Daniel Kulp wrote: > >> Chatted with Alessio about this on IRC. It looks like a bug in the JDK >> introduced in Java7 update 40. A bug was logged: >> >> https://bugs.openjdk.java.net/browse/JDK-8044726 >> >> but then closed a non-reproducible. Alessio will look into seeing if >> that can be re-opened. >> >> Meanwhile we’ll likely remove the call to setFixedLengthStreamingMode. >> That will result in an extra byte[] copy of the message, but this is only >> for smaller messages (hasn’t hit the threshold yet) and thus is fairly >> quick and is certainly MUCH MUCH faster than the problems we’re seeing with >> the call to setFixedLengthStreamingMode(). >> >> Should we make it optional (system + contextual property) so that people > can tell the runtime to actually call this method if confirmed it actually > improves the performance (may be in newer JDK, etc) ? > > Thanks, Sergey > > Dan >> >> >> >> On Jan 27, 2015, at 12:14 PM, Alessio Soldano <[email protected]> >>> wrote: >>> >>> Hi, >>> my attention has been recently brought to a scenario in which an Apache >>> CXF client invokes an endpoint operation in a loop and the number of >>> invocations performed in a given amount of time (say, 2 minutes) is used as >>> benchmark for measuring WS stack performances. It's actually a very >>> simplistic scenario, with a plain JAX-WS single thread client sending and >>> receiving small RPC/Lit SOAP messages [1]. The reason why I've been asked >>> to have a look is that with default settings the Apache CXF JAX-WS impl >>> seems to perform *shamefully* bad compared to the Metro (JAX-WS RI) >>> implementation. I've been blaming the user log configuration, etc but when >>> I eventually tried on my own I could actually reproduce the bad results. >>> I've been profiling a bit and found few hot spot area where CXF could >>> possibly be optimized, but the big issue really seems to be at the >>> HTTPCounduit / HTTPURLConnection level. >>> I found that almost all the invocations end up into >>> sun.net.www.http.HttpClient.New(..) calling available() method [2] as >>> part of the process for re-using cached connections [3]; that goes to the >>> wire to try reading and takes a lot of time. >>> When the RI does the equivalent operation, the available() method is not >>> called [4], resulting in much better performances. >>> By looking at the JDK code, it looks to me that the problem boils down >>> to sun.net.www.protocol.http.HttpURLConnection#streaming() [5] >>> returning different values, as a consequence of the fixedContentLenght >>> attribute being set to a value different from -1 when running on CXF only. >>> As a matter of fact, that is set when >>> HTTPConduit.WrappedOutputStream#thresholdNotReached() >>> is called, whenever a message is completely written to the outpustream >>> buffer before the chunking threshold is reached (at least AFAIU). I've >>> searched through the JAX-WS RI and could not find any place where >>> setFixedLengthStreamingMode is called on the connection instead. >>> So, I've performed two quick and dirty tries: the first time I forced >>> allowChunking = false on the client policy, the second time I commented out >>> the code in HTTPConduit.WrappedOutputStream#thresholdNotReached(). In >>> both cases I managed to get performances comparable to what I can get with >>> the JAX-WS RI. >>> Now, few questions: >>> - are we really required to call setFixedLengthStreamingMode as we >>> currently do? what's the drawback of not calling it? >>> - should we actually do something for getting decent performances by >>> default in this scenario? (not sure expecting the user to disable chunking >>> is that an option...) >>> As a side note, the relevant part of the JDK HttpClient code changed >>> between JDK6 and JDK7, so things have not always been as explained above... >>> >>> Cheers >>> Alessio >>> >>> >>> [1] http://www.fpaste.org/176166/14223765/ >>> [2] http://pasteboard.co/FR5QVrP.png >>> [3] http://grepcode.com/file/repository.grepcode.com/java/ >>> root/jdk/openjdk/7u40-b43/sun/net/www/http/HttpClient.java#276 >>> [4] http://pasteboard.co/FR8okYM.png >>> [5] http://grepcode.com/file/repository.grepcode.com/java/ >>> root/jdk/openjdk/7u40-b43/sun/net/www/protocol/http/ >>> HttpURLConnection.java#HttpURLConnection.streaming%28%29 >>> >>> -- >>> Alessio Soldano >>> Web Service Lead, JBoss >>> >>> >> >
