Thanks Rob, Jeff. Some excellent ideas to go on there.

Rob Lockstone wrote:
> I just looked in resin 3.1.x's bin/wrapper.pl file and it sets the  
> stack size to 1Meg:
>
> #
> # Default stack size.  The 1m is a good tradeoff between stack size and
> # allowing more threads.  The default stack size doesn't allow enough
> # threads for several operating systems.
> #
> $DEFAULT_STACK_SIZE="-Xss1m";
>
> I don't know what Fedora 7's default stack size is or what stack size  
> is being set by whatever program you're using when you're not using  
> resin, but on a 1G machine, depending on how big your java heap is, it  
> might not take very many threads to eat up whatever "native" memory  
> you have left (keeping in mind that you're using that same memory pool  
> for everything else that's running).
>
> Rob
>
> On Mar 22, 2009, at 11:58, Jeff Schnitzer wrote:
>
>   
>> There are profiling/memory allocation tracking tools for C and
>> Linux... if you're seeing what looks like a memory leak, it might be
>> worth looking into.  They're quite a bit more complicated than the
>> equivalent java tools but years ago in another life I got a lot of
>> mileage out of Rational Purify, which has a free trial version.
>>
>> Jeff
>>
>> On Sun, Mar 22, 2009 at 10:03 AM, Rob Lockstone <lockst...@mac.com>  
>> wrote:
>>     
>>> On Mar 22, 2009, at 04:29, Stargazer wrote:
>>>
>>>       
>>>> Rob Lockstone wrote:
>>>>         
>>>>> On Mar 21, 2009, at 14:59, Stargazer wrote:
>>>>>
>>>>>
>>>>>           
>>>>>> Adam Allgaier wrote:
>>>>>>
>>>>>>             
>>>>>>> I would plug jconsole into your resin instance and watch what's
>>>>>>> happening to the JVM memory.  Could be loose open threads (and
>>>>>>> large thread size) that grows over time and eats up free memory.
>>>>>>> Restarting would kill all the threads and free the memory.
>>>>>>>
>>>>>>> http://java.sun.com/j2se/1.5.0/docs/guide/management/ 
>>>>>>> jconsole.html
>>>>>>>
>>>>>>> Adam
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>               
>>>>>> Thanks, but as I'd hoped I'd made clear Resin itself shows flat
>>>>>> memory
>>>>>> use, whether using jconsole, jprofiler, /resin-admin or any of the
>>>>>> other
>>>>>> jvmti tools I've used to try to fix this. The memory consumption
>>>>>> occurs
>>>>>> outside java.
>>>>>>
>>>>>>             
>>>>> I don't know about Ronan's problem, but yours sounds like non-java-
>>>>> heap memory or "native" memory is being exhausted. In my  
>>>>> experience,
>>>>> the biggest consumers of that are threads and graphic operations.
>>>>> Depending on the system you're running on, your default stack size
>>>>> could be quite large. Have you tried setting your stack size to a
>>>>> much
>>>>> smaller value via -Xss? Unless you're doing a lot of heavy  
>>>>> recursion,
>>>>> the chance that you'll need a large stack is fairly remote. On our
>>>>> (very busy) Windows boxes, we use a stack size of 128K (-Xss128K).
>>>>> Keep in mind that on a 32-bit system (not sure what you're running
>>>>> on), you have only 2G of addressable memory per process. So if you
>>>>> define a larg(ish) java heap, you have only 2G - YourHeapSize to  
>>>>> use
>>>>> for allocating memory for threads and other non-java operations,  
>>>>> such
>>>>> as some graphics operations.
>>>>>
>>>>> Rob
>>>>>
>>>>>
>>>>>           
>>>> How come this doesn't happen when I move the non-resin stuff (i.e.  
>>>> the
>>>> PHP apps) to another server? Its Linux Fed 7, dual core, 1G Ram/4G
>>>> swap.
>>>> What I'm really asking is how can you be sure its not mod_caucho?  
>>>> Btw
>>>> there are no graphics operations involved, and the PHP apps are all
>>>> popular packages like Joomla and Pligg, so if these were at fault  
>>>> I'm
>>>> sure the users would be screaming in their forums about it.
>>>>         
>>> I don't know that it's not mod_caucho. I'm only suggesting a possible
>>> cause and something to try. Iirc, at least some versions of resin
>>> (perhaps all) set their own stack size if the user does not specify
>>> one. If this is the case, it may be that the version you're using is
>>> setting a stack size that is larger than the default stack size of
>>> your OS (or larger than the stack size set by whatever other software
>>> you're using to run you PHP apps). Anyway, it's easy to set and easy
>>> to test.
>>>
>>> Rob
>>>
>>>       
>>>>>>> ----- Original Message ----
>>>>>>> From: Stargazer <starga...@blueyonder.co.uk>
>>>>>>> To: General Discussion for the Resin application server 
>>>>>>> <resin-interest@caucho.com
>>>>>>>
>>>>>>> Sent: Saturday, March 21, 2009 4:39:56 AM
>>>>>>> Subject: Re: [Resin-interest] Perf Issues
>>>>>>>
>>>>>>> Ronan Lucio wrote:
>>>>>>>
>>>>>>>
>>>>>>>               
>>>>>>>> Hi,
>>>>>>>>
>>>>>>>> We have had a perf issue.
>>>>>>>>
>>>>>>>> Our servers have 4Gb RAM. It has just Resin and Apache  
>>>>>>>> installed.
>>>>>>>>
>>>>>>>> The problem is, when I start Resin, the whole server uses about
>>>>>>>> 2.5 Gb RAM
>>>>>>>> After that memory usage keep growing til it reachs the 4Gb RAM,
>>>>>>>> use swap
>>>>>>>> and so on.
>>>>>>>>
>>>>>>>> After few hours the application start getting slow.
>>>>>>>> Analyzing the server sources, it's using so low CPU, load about
>>>>>>>> 1... I
>>>>>>>> see none overload evidence, except for RAM memory.
>>>>>>>>
>>>>>>>> So I just restart Resin and/or Apache and application gets fast
>>>>>>>> again,
>>>>>>>> but few ours later it will raise the same issue.
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>                 
>>>>>>> I have been suffering from exactly those symptoms for years.
>>>>>>> Do your httpd processes consume all the swap, with top showing  
>>>>>>> some
>>>>>>> at
>>>>>>> 450Mb? (default httpd.conf values)  I.e does restarting apache
>>>>>>> alone,
>>>>>>> and not resin, cause the swap to drop back down to normal until  
>>>>>>> the
>>>>>>> next
>>>>>>> slow growth starts it all again?
>>>>>>>
>>>>>>> That is my pattern. So when I profile resin theres no unusual
>>>>>>> growth
>>>>>>> even through the whole server is effectiviley dead until restart.
>>>>>>> Given
>>>>>>> that, whos to blame? Could mod_caucho somehow be at fault even
>>>>>>> though
>>>>>>> resin itself is ok?
>>>>>>>
>>>>>>> I have another identical server running a couple of PHP CMS apps,
>>>>>>> no
>>>>>>> resin or java in sight - because of this problem actually. The  
>>>>>>> plan
>>>>>>> is
>>>>>>> to move everything over when stable but these have become too
>>>>>>> critical
>>>>>>> to play with. Their typical httpd swap use is 25Mb, and its the
>>>>>>> default
>>>>>>> httpd.conf. I would dearly love to know what the httpd on the
>>>>>>> failing
>>>>>>> server thinks it needs to hold onto 450Mb for, without tweaking
>>>>>>> there
>>>>>>> could be 20 of these. There are other non-quercus PHP apps
>>>>>>> running on
>>>>>>> that server btw.
>>>>>>>
>>>>>>> My "solution" is to kill child httpd processes at a far quicker
>>>>>>> rate
>>>>>>> than you'd normally want, and it works of sorts:
>>>>>>>
>>>>>>> <IfModule prefork.c>
>>>>>>> StartServers       8
>>>>>>> MinSpareServers    5
>>>>>>> MaxSpareServers   20
>>>>>>> ServerLimit      256
>>>>>>> MaxClients       256
>>>>>>> MaxRequestsPerChild  96
>>>>>>> </IfModule>
>>>>>>>
>>>>>>> <IfModule worker.c>
>>>>>>> StartServers         2
>>>>>>> MaxClients         150
>>>>>>> MinSpareThreads     25
>>>>>>> MaxSpareThreads     75
>>>>>>> ThreadsPerChild     25
>>>>>>> MaxRequestsPerChild  96
>>>>>>> </IfModule>
>>>>>>>
>>>>>>>
>>>>>>>               
>>>>>>>> This problem usually happens on peaks hours.
>>>>>>>>
>>>>>>>> So we upgraded RAM memory to 8Gb with a PAE kernel.
>>>>>>>> Although it doesn't reach the 8G RAM, slow moments gots for
>>>>>>>> frequent. It
>>>>>>>> seems to work worse that way (8Gb + PAE kernel).
>>>>>>>>
>>>>>>>> The question is:
>>>>>>>> Is there everyone having the same issue with Resin (3.1.6)?
>>>>>>>> My doubt if such problem resides either on Resin or on my
>>>>>>>> application.
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>                 
>>>>>>> All versions prior to 3.1.6, and currently with 3.2.1 Pro
>>>>>>>
>>>>>>>
>>>>>>>               
>>>>>>>> Thanks,
>>>>>>>> Ronan
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>> _______________________________________________
>>>>>>>> resin-interest mailing list
>>>>>>>> resin-interest@caucho.com
>>>>>>>> http://maillist.caucho.com/mailman/listinfo/resin-interest
>>>>>>>>
>>>>>>>> ------------------------------------------------------------------------
>>>>>>>>
>>>>>>>>
>>>>>>>> No virus found in this incoming message.
>>>>>>>> Checked by AVG - www.avg.com
>>>>>>>> Version: 8.0.238 / Virus Database: 270.11.21/2014 - Release  
>>>>>>>> Date:
>>>>>>>> 03/20/09 06:59:00
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>                 
>>>>>>> _______________________________________________
>>>>>>> resin-interest mailing list
>>>>>>> resin-interest@caucho.com
>>>>>>> http://maillist.caucho.com/mailman/listinfo/resin-interest
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> _______________________________________________
>>>>>>> resin-interest mailing list
>>>>>>> resin-interest@caucho.com
>>>>>>> http://maillist.caucho.com/mailman/listinfo/resin-interest
>>>>>>>
>>>>>>> ------------------------------------------------------------------------
>>>>>>>
>>>>>>>
>>>>>>> No virus found in this incoming message.
>>>>>>> Checked by AVG - www.avg.com
>>>>>>> Version: 8.0.238 / Virus Database: 270.11.22/2015 - Release Date:
>>>>>>> 03/20/09 19:01:00
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>               
>>>>>> _______________________________________________
>>>>>> resin-interest mailing list
>>>>>> resin-interest@caucho.com
>>>>>> http://maillist.caucho.com/mailman/listinfo/resin-interest
>>>>>>
>>>>>>             
>>>>>
>>>>> _______________________________________________
>>>>> resin-interest mailing list
>>>>> resin-interest@caucho.com
>>>>> http://maillist.caucho.com/mailman/listinfo/resin-interest
>>>>>
>>>>> ------------------------------------------------------------------------
>>>>>
>>>>>
>>>>> No virus found in this incoming message.
>>>>> Checked by AVG - www.avg.com
>>>>> Version: 8.0.238 / Virus Database: 270.11.23/2016 - Release Date:
>>>>> 03/21/09 17:58:00
>>>>>
>>>>>
>>>>>           
>>>>
>>>>
>>>> _______________________________________________
>>>> resin-interest mailing list
>>>> resin-interest@caucho.com
>>>> http://maillist.caucho.com/mailman/listinfo/resin-interest
>>>>         
>>>
>>> _______________________________________________
>>> resin-interest mailing list
>>> resin-interest@caucho.com
>>> http://maillist.caucho.com/mailman/listinfo/resin-interest
>>>
>>>       
>> _______________________________________________
>> resin-interest mailing list
>> resin-interest@caucho.com
>> http://maillist.caucho.com/mailman/listinfo/resin-interest
>>     
>
>
>
> _______________________________________________
> resin-interest mailing list
> resin-interest@caucho.com
> http://maillist.caucho.com/mailman/listinfo/resin-interest
>   
> ------------------------------------------------------------------------
>
>
> No virus found in this incoming message.
> Checked by AVG - www.avg.com 
> Version: 8.0.238 / Virus Database: 270.11.23/2016 - Release Date: 03/21/09 
> 17:58:00
>
>   




_______________________________________________
resin-interest mailing list
resin-interest@caucho.com
http://maillist.caucho.com/mailman/listinfo/resin-interest

Reply via email to