Hi Christopher, Thank you for your contribution to this thread. I think we we have made good progress on the subject, here are some elements i'd like to share :
- The fact that the response time was increasing with the the number of JSP loaded was linked to our monitoring tool... This tool hadn't the same impact with websphere. Without monitoring the response time remains stable no matter how many jsp are already loaded in the permgen. - There is no permgen defined in the IBM JVM running Websphere and i was wondering how much space the JVM was allocating to host this huge number of JSP. The memory footprint of the process on the system was quite big : Xmx : 1,5 Go, memory footprint of the JVM : 3,5 Go. It lets me think that Websphere allocates a large space to host these JSP, i increase accordingly the permgen size of my JVM to 1Go. - I finally noticed that when the permgen is undersized (ie it cannot host all the JSP of my application and has to unload class), the CPU impact is much more important with the CMS garbage policy than with the parallel GC. Our main concern so far was the CPU comsumption, we finally solved this by tuning our monitoring tool correctly and by increasing the size of the permgen. Sylvain On Mon, Apr 21, 2014 at 3:54 PM, Christopher Schultz < ch...@christopherschultz.net> wrote: > -----BEGIN PGP SIGNED MESSAGE----- > Hash: SHA256 > > Sylvain, > > This thread is bit old, but there were no conclusions si I figured I'd > pick it back up. > > On 4/11/14, 11:52 AM, Sylvain Goulmy wrote: > > Thank you all for your returns. > > > > Maybe also, just to give us an idea, you could tell us how much > > memory that > >> system has, and how much is given to use by Tomcat ? > > > > > > Xmx is 1500 Mo. The PermSize is set so that it can easily load the > > max number of JSP set with the maxLoadedJsp (set to 5000) > > parameter. > > Assuming this is 1500MiB, then 16000 JSPs might have trouble fitting > in here. Can you see how much space java.lang.Class objects are > taking? How about how many JSPs are actually loaded? Presumably, you > have 16000 .jsp files on the disk, but maybe only 1/2 of them are > actually loaded before you have any problems. > > >> 1. Personally I am pessimistic about 'maxLoadedJsps' option, > >> because even if you unload a JSP, the strings used in its code > >> will still remain in JVM constant strings pool. Anyway, I expect > >> the best performance would be if you do not try unloading the > >> JSPs. 2. Did you set development=false on JspServlet? > > If Class objects are unloaded, they will not pin any data in the JVM > string constant pool. You don't have anything to fear, here. > > What you *do* have to worry about is the buffers that JSP ends up > using for custom tags. I can't remember the exact mechanics, but > whenever a tag is executed, the content generated within it must be > buffered in case of an error -- so an error message can be generated > instead of failing "silently" to the remote user due to a > response-already-committed problem. > > Anyway, when those buffers are expanded, they never shrink again. So, > over time, all your buffers will grow enormous if you have a few > places where this kind of thing can happen. The number of JSPs does > not matter: you can do this with a single unfortunately-written JSP or > with 16k of them. But given the extensive use of JSPs on your site, > this is a likely candidate for any problems you are encountering. > > Are you getting OOMEs? Have you enabled verbose GC to see if the > slowdowns are associated with GC pauses? It seems like you are coming > to us with the problem "things are slow" and asking us to figure it > out for you. Sorry, but you are going to need to provide more information. > > >> > http://tomcat.apache.org/tomcat-7.0-doc/jasper-howto.html#Production_Configuration > >> > >> > Note: I would not recommend using 'genStringAsCharArray', > >> 3. There are a number of ways to monitor your memory usage. 4. > >> Does the number of requests remain the same and only the number > >> of JSPs increases? How much is "longer" in "the response time > >> becomes longer" ? > > > > > > 1. This is indeed a scenario i want to test, ie to size the PermGen > > so it can load all the JSP without having to unload them. This > > could require a huge PermGen size that i have to assess. > > java.lang.Class objects don't take a huge amount of memory. You will > likely have to increase PermGen from the default, but you won't need a > gigabyte or anything like that. > > > 2. development=false and i don't use the "genStringAsCharArray" > > parameter > > > > 3. I'm using many tools which allows me to monitor the GC > > behaviour > > What do those tools tell you? > > > 4. Here is the behaviour that i observed : i request the URL of a > > jsp in a loop. The content of this JSP is always the same but it's > > name is different in each URL so that it is considered as a new > > one. This JSP is quite big (100 ko). Here is the evolution of the > > response time : > > > > - First call : 70 ms - 1000th call : 100ms - 2000th call : 140 ms - > > 3000th call : 200 ms > > The size of the .jsp file is not relevant. What is relevant is the > "complexity". IF you just have 100kib of text, then there is no > work/buffering to be done, for instance. > > > The more the permgen hosts many JSP, the more the response time > > increases. On this test, the PermGen was big enough to host all the > > JSP and the maxLoadedJsp parameter is set to 5000. > > What happens if you remove the maxLoadedJsp setting? Do you get an OOME? > > >> Is this performance problem something you didn't have before > >> today? > > > > No indeed, we were on a different technology (Websphere). > > Same technology (JSP), different implementation. So, you can't give > anyone some samples, eh? But you said you were able to replicate this > with some simpler example? Did you use an example from your own web > application or did you just create a sample .jsp and copy it to > different names over and over again? If you help us reproduce the > problem, we may be able to find a solution. > > - -chris > -----BEGIN PGP SIGNATURE----- > Version: GnuPG v1 > Comment: GPGTools - http://gpgtools.org > Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/ > > iQIcBAEBCAAGBQJTVSMsAAoJEBzwKT+lPKRYMS8P/2qOP8A74qPtRQL/1wXcw39p > uJ1i9f3ml+vP2PB1Ujcxv2zeZIL8ztL/ClpJwk5nWh2mZW+S9ZUE15eozddzMn4d > qbKT9brtTZWstX0HROY11yaMG52lQwsktw5TPwsLeRtSyvUSG2qTEohXYe8sRf/s > ZMfYNdpgTED7RW+6iIooIN2YZMFSLUF6dyx0Up8sPRLFacDlGQw+v0BiLxyTSAev > Fn1nsccGJ5wbSqVOZR8m8AtgCMzouAgxE7Ct0B/TI7d4ek7M1k9ba9mjk5xEvN/Y > 4W/gJxr6f3NA0Mqw1zjhsIWjSUUf4o4q5ZhtoeihWWcBFaltO7dLymCFSxiXe/l3 > rM73PfFZTKSo8SIGi0sXPfCFqH+2hUMaABLLGhRI0UzFU65nO3mKYbLFaNNc+sej > djNROjcmKt+MZs2JpUGttePjefeqhIlHWzgsvowb4QOob/ACm/r0xKN+byDR0NzP > DQwLfxglnbxO95TwiuhEBVAKuMI4fVwNX0xNnoCQlh6K92R+nP9NZ4/PLWjkDKk/ > bB6HUJWJMw36i08xq1u+O/6bOYAeJZtt7fZFBlSz9ASGtpXDZOXhiao0JUOmrj9y > Pd3zUFz1uY0+pymp3aamXTwSEI49OY3LfM1TIT09h2bqsZtIA+UYiMSLMTgY591/ > pwg8Ts0K4enWoVhq0P+5 > =1DhS > -----END PGP SIGNATURE----- > > --------------------------------------------------------------------- > To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org > For additional commands, e-mail: users-h...@tomcat.apache.org > >
