Re: [Resin-interest] implicit stack memory size?

[Jay Ballinger]

Thanks much, guys, for the replies.

We've arrived at the same conclusion that Scott suggests - a GB for the 
heap and the other GB for the product (roughly) of the threads and the 
stack size in our windows 32-bit environment.

Now it's an exercise in testing the actual -Xss size that we want to run 
with - which is proving to be a bit elusive. Understanding that local 
primitives and heap-object addresses are stored on the stack, I can't 
envision a need for a stack size anywhere bigger than 1MB (for our app, 
YMMV). Even 1MB is probably way over doing it.



There's a few of us here at work who have been lurkers on this list for 
a long time - or at least a couple of years anyway. We've always found 
this list valuable and appreciate everyone's involvement. Thanks again.


+ jay



Scott Ferguson wrote:
> On Jan 8, 2009, at 5:23 PM, Rob Lockstone wrote:
> 
>> On Jan 8, 2009, at 13:52, Jay Ballinger wrote:
>>
>>> Hi everyone,
>>>
>>> We're running Resin 3.0.18 Pro on a Windows 32-bit system and we're
>>> looking to understand/tune our memory settings a bit.
>>>
>>> According to http://www.caucho.com/resin-3.0/performance/jvm-tuning.xtp
>>> the stack size is set to 2048k unless set otherwise. Is this true?
>>> In using JConsole we can't see where any explicit -Xss setting is
>>> passed and we can't readily determine what the current stack size is
>>> set to.
> 
> Try -verbose when starting.  Since the -Xss is a command line option,  
> it should be reported.  Also, JConsole should be able to see the  
> command line options on one of the system beans.
> 
>> I think Caucho tends to err on the side of caution in this regard, and
>> is also (understandably) more *nix-central than Windows. So take that
>> 2048K as being "appropriate for most situations" with a grain of salt
>> because situations can vary wildly.
> 
> Basically.  There's no way for us to know if your application needs  
> more stack or more threads.  So I assumed 1G heap, leaving 1G for  
> other stuff, and then figured the default should support 1000 threads,  
> giving 1m as a default, which should be enough stack unless you have  
> massively recursive applications, and enough threads for most apps.   
> You know your own requirements best, of course.
> 
> I believe Linux is the most finicky about the stack allocation, so it  
> was the OS we needed to solve (and it was the OS which exposed the  
> issue first.)
> 
> I'm really looking forward for the 64-bit transition to complete so we  
> can use as much VM as possible.
> 
> -- Scott
> 
> 
>>
>>> So, here's a short list of questions. And thanks for your help.
>>>
>>> - If JConsole doesn't show -Xss, is it really there for resin?
>> Whether it's explicitly set or not, there IS a stack size, which would
>> end up being the default stack size of the operating system if the JVM
>> itself doesn't set it (I do not think it does, but could be wrong and
>> don't feel like looking it up now).
>>
>>> - How was "2048k is an appropriate value for most situations."
>>> determined?
>> See above.
>>
>>> - Can resin/jvm/jconsole report the current stack memory setting?
>> I don't know if there's a way to query a running JVM for this
>> information. It's best if you explicitly set the stack size using the
>> JVM option -Xss (but see below).
>>
>>> (general sun jvm on windows questions)
>>> - What is the default stack memory size?
>> That's too general of a question. It depends on the OS, and (perhaps)
>> the version of the OS.
>>
>>> - When a thread is created, does it grab all of the stack memory, or
>>> can the stack memory start small and grow to the default or -Xss  
>>> size?
>> Again, might depend on the operating system. Contrary to what Knut
>> said, and to the best of my knowledge and in the testing that I did
>> (it's been a while), all of the memory for the stack is pre-allocated
>> (at least from the perspective of the JVM itself). But it's easy
>> enough to write a program to check this.
>>
>> Thread stacks, at least on Windows, are more complicated than just a
>> single memory allocation (down at the Windows API level) because
>> you've got both reserved and committed memory allocations, and one
>> always has to be >= the other (I forget which), and there's a 64K
>> granularity in sizing, and you're dealing with physical and virtual
>> memory, and this is all set in the linker and therefore configurable,
>> but can also be set programmatically on a per-thread basis. In other
>> words, it's quite involved. See this page as an example of how complex
>> this kind of memory management is when you go into how the underlying
>> OS is actually managing all of this 
>> >> .
>> Remember, thread allocation/destruction as well as context switching
>> and just about everything else is managed by the operating system, not
>> Java (unless you're running "green" threads, which I don't think is
>> even around anymore?). This is why the memory us

Re: [Resin-interest] implicit stack memory size?

[Scott Ferguson]

On Jan 8, 2009, at 5:23 PM, Rob Lockstone wrote:

> On Jan 8, 2009, at 13:52, Jay Ballinger wrote:
>
>> Hi everyone,
>>
>> We're running Resin 3.0.18 Pro on a Windows 32-bit system and we're
>> looking to understand/tune our memory settings a bit.
>>
>> According to http://www.caucho.com/resin-3.0/performance/jvm-tuning.xtp
>> the stack size is set to 2048k unless set otherwise. Is this true?
>> In using JConsole we can't see where any explicit -Xss setting is
>> passed and we can't readily determine what the current stack size is
>> set to.

Try -verbose when starting.  Since the -Xss is a command line option,  
it should be reported.  Also, JConsole should be able to see the  
command line options on one of the system beans.

> I think Caucho tends to err on the side of caution in this regard, and
> is also (understandably) more *nix-central than Windows. So take that
> 2048K as being "appropriate for most situations" with a grain of salt
> because situations can vary wildly.

Basically.  There's no way for us to know if your application needs  
more stack or more threads.  So I assumed 1G heap, leaving 1G for  
other stuff, and then figured the default should support 1000 threads,  
giving 1m as a default, which should be enough stack unless you have  
massively recursive applications, and enough threads for most apps.   
You know your own requirements best, of course.

I believe Linux is the most finicky about the stack allocation, so it  
was the OS we needed to solve (and it was the OS which exposed the  
issue first.)

I'm really looking forward for the 64-bit transition to complete so we  
can use as much VM as possible.

-- Scott


>
>
>> So, here's a short list of questions. And thanks for your help.
>>
>> - If JConsole doesn't show -Xss, is it really there for resin?
>
> Whether it's explicitly set or not, there IS a stack size, which would
> end up being the default stack size of the operating system if the JVM
> itself doesn't set it (I do not think it does, but could be wrong and
> don't feel like looking it up now).
>
>> - How was "2048k is an appropriate value for most situations."
>> determined?
>
> See above.
>
>> - Can resin/jvm/jconsole report the current stack memory setting?
>
> I don't know if there's a way to query a running JVM for this
> information. It's best if you explicitly set the stack size using the
> JVM option -Xss (but see below).
>
>> (general sun jvm on windows questions)
>> - What is the default stack memory size?
>
> That's too general of a question. It depends on the OS, and (perhaps)
> the version of the OS.
>
>> - When a thread is created, does it grab all of the stack memory, or
>> can the stack memory start small and grow to the default or -Xss  
>> size?
>
> Again, might depend on the operating system. Contrary to what Knut
> said, and to the best of my knowledge and in the testing that I did
> (it's been a while), all of the memory for the stack is pre-allocated
> (at least from the perspective of the JVM itself). But it's easy
> enough to write a program to check this.
>
> Thread stacks, at least on Windows, are more complicated than just a
> single memory allocation (down at the Windows API level) because
> you've got both reserved and committed memory allocations, and one
> always has to be >= the other (I forget which), and there's a 64K
> granularity in sizing, and you're dealing with physical and virtual
> memory, and this is all set in the linker and therefore configurable,
> but can also be set programmatically on a per-thread basis. In other
> words, it's quite involved. See this page as an example of how complex
> this kind of memory management is when you go into how the underlying
> OS is actually managing all of this 
> > .
>
> Remember, thread allocation/destruction as well as context switching
> and just about everything else is managed by the operating system, not
> Java (unless you're running "green" threads, which I don't think is
> even around anymore?). This is why the memory used by the thread's
> stack is not part of the Java heap. On a 32-bit machine, the maximum
> addressable space for an individual java program is 2G. So, for
> example, if you run java with these parameters on a 32-bit machine:
>
> java -server -Xms1500M -Xmx1500M -Xss1M
>
> then you'll leave enough room to run ~548 threads (2048M - 1500M =
> 548M / 1M stack = 548 threads). Of course, that assumes that none of
> the 548M of non-java-heap memory is being used for anything else,
> which is a false assumption, perhaps hugely false  depending on what
> your java program is doing, e.g. graphics operations or the like.
>
> Something I discovered while optimizing our Resin on Windows
> environments is that the version of Windows and Java we use (64-bit
> Windows 2003 Server with Java 1.5) will allow a minimum stack size of
> 128K, and if you try to specify a stack size of 256K or 512K, Windows
> (or maybe the JVM

Re: [Resin-interest] implicit stack memory size?

[Rob Lockstone]

As you can see, it's all over the map, which is probably why: (1)  
Caucho sets it explicitly (and to a very conservative value); (2) it's  
always best to set the stack size yourself so you know what you're  
getting; and (3) test it yourself with the value(s) you'd like to use  
in the environments you want to use it to make sure it works (see my  
previous comment about only 128K and 1M working when I did my testing).

Rob

On Jan 8, 2009, at 17:28, Serge Knystautas wrote:

> Rob Lockstone wrote:
>>> (general sun jvm on windows questions)
>>> - What is the default stack memory size?
>>
>> That's too general of a question. It depends on the OS, and (perhaps)
>> the version of the OS.
>
> I did this research a while ago, but seems relevant:
>
>   Default in Sun JDK 1.2 on unix was 128kb
>   Default in Sun JDK 1.3 on unix was 256kb
>   Default in Sun JDK 1.4 on windows is 512kb
>   Default in IBM JDK 1.5 is 256kb on all platforms
> (http://tinyurl.com/gbfd4)
>   Default in JRockit JDK 1.5 is 64k (win) or 128kb (linux)
> (http://tinyurl.com/2hevdw)
>   The old volano chat scalability reports used 32kb
>
> -- 
> Serge Knystautas
> Lokitech >> software . strategy . design >> http://www.lokitech.com
> p. 301.656.5501
> e. ser...@lokitech.com
>



___
resin-interest mailing list
resin-interest@caucho.com
http://maillist.caucho.com/mailman/listinfo/resin-interest

Re: [Resin-interest] implicit stack memory size?

[Serge Knystautas]

Rob Lockstone wrote:
>> (general sun jvm on windows questions)
>> - What is the default stack memory size?
> 
> That's too general of a question. It depends on the OS, and (perhaps)  
> the version of the OS.

I did this research a while ago, but seems relevant:

   Default in Sun JDK 1.2 on unix was 128kb
   Default in Sun JDK 1.3 on unix was 256kb
   Default in Sun JDK 1.4 on windows is 512kb
   Default in IBM JDK 1.5 is 256kb on all platforms 
(http://tinyurl.com/gbfd4)
   Default in JRockit JDK 1.5 is 64k (win) or 128kb (linux) 
(http://tinyurl.com/2hevdw)
   The old volano chat scalability reports used 32kb

-- 
Serge Knystautas
Lokitech >> software . strategy . design >> http://www.lokitech.com
p. 301.656.5501
e. ser...@lokitech.com


___
resin-interest mailing list
resin-interest@caucho.com
http://maillist.caucho.com/mailman/listinfo/resin-interest

Re: [Resin-interest] implicit stack memory size?

[Rob Lockstone]

On Jan 8, 2009, at 13:52, Jay Ballinger wrote:

> Hi everyone,
>
> We're running Resin 3.0.18 Pro on a Windows 32-bit system and we're  
> looking to understand/tune our memory settings a bit.
>
> According to http://www.caucho.com/resin-3.0/performance/jvm-tuning.xtp 
>  the stack size is set to 2048k unless set otherwise. Is this true?  
> In using JConsole we can't see where any explicit -Xss setting is  
> passed and we can't readily determine what the current stack size is  
> set to.

I think Caucho tends to err on the side of caution in this regard, and  
is also (understandably) more *nix-central than Windows. So take that  
2048K as being "appropriate for most situations" with a grain of salt  
because situations can vary wildly.

> So, here's a short list of questions. And thanks for your help.
>
> - If JConsole doesn't show -Xss, is it really there for resin?

Whether it's explicitly set or not, there IS a stack size, which would  
end up being the default stack size of the operating system if the JVM  
itself doesn't set it (I do not think it does, but could be wrong and  
don't feel like looking it up now).

> - How was "2048k is an appropriate value for most situations."  
> determined?

See above.

> - Can resin/jvm/jconsole report the current stack memory setting?

I don't know if there's a way to query a running JVM for this  
information. It's best if you explicitly set the stack size using the  
JVM option -Xss (but see below).

> (general sun jvm on windows questions)
> - What is the default stack memory size?

That's too general of a question. It depends on the OS, and (perhaps)  
the version of the OS.

> - When a thread is created, does it grab all of the stack memory, or  
> can the stack memory start small and grow to the default or -Xss size?

Again, might depend on the operating system. Contrary to what Knut  
said, and to the best of my knowledge and in the testing that I did  
(it's been a while), all of the memory for the stack is pre-allocated  
(at least from the perspective of the JVM itself). But it's easy  
enough to write a program to check this.

Thread stacks, at least on Windows, are more complicated than just a  
single memory allocation (down at the Windows API level) because  
you've got both reserved and committed memory allocations, and one  
always has to be >= the other (I forget which), and there's a 64K  
granularity in sizing, and you're dealing with physical and virtual  
memory, and this is all set in the linker and therefore configurable,  
but can also be set programmatically on a per-thread basis. In other  
words, it's quite involved. See this page as an example of how complex  
this kind of memory management is when you go into how the underlying  
OS is actually managing all of this 
.

Remember, thread allocation/destruction as well as context switching  
and just about everything else is managed by the operating system, not  
Java (unless you're running "green" threads, which I don't think is  
even around anymore?). This is why the memory used by the thread's  
stack is not part of the Java heap. On a 32-bit machine, the maximum  
addressable space for an individual java program is 2G. So, for  
example, if you run java with these parameters on a 32-bit machine:

java -server -Xms1500M -Xmx1500M -Xss1M

then you'll leave enough room to run ~548 threads (2048M - 1500M =  
548M / 1M stack = 548 threads). Of course, that assumes that none of  
the 548M of non-java-heap memory is being used for anything else,  
which is a false assumption, perhaps hugely false  depending on what  
your java program is doing, e.g. graphics operations or the like.

Something I discovered while optimizing our Resin on Windows  
environments is that the version of Windows and Java we use (64-bit  
Windows 2003 Server with Java 1.5) will allow a minimum stack size of  
128K, and if you try to specify a stack size of 256K or 512K, Windows  
(or maybe the JVM, I don't know which one for sure) automatically  
jumps you up to 1M. So you have a choice between 128K, 1M, 2M, ??? (I  
didn't test above 2M).

Fwiw, we've got dozens of Windows servers running Resin Pro 3.0.x  
(upgrading to 3.1.x in the not-too-distant future), and we run all of  
them with a 128K stack size. In general, unless you're running code  
that's highly recursive, you don't need very large stack sizes in Java  
programming.

Rob



___
resin-interest mailing list
resin-interest@caucho.com
http://maillist.caucho.com/mailman/listinfo/resin-interest

Re: [Resin-interest] implicit stack memory size?

[Jay Ballinger]

Knut,

Thanks for the info.

Just a check on the math - if Win32 has a 2GB max addressable space, 
that includes heap and stack and misc. overhead, right?

If we set resin heap to 1GB and allow each thread to use the 'default' 
of 2048k, and we see the number of threads running at about 700, then we 
have the potential of 2.4GB total memory usage [ 1024MB + (700*2MB) ]. 
This exceeds the 2GB limit. However, if, as you suggest, each thread 
only uses a fraction of the 2MB available to each thread, we could see 
the total memory somewhere between 1GB and 2GB.

So, my question is if we use -Xss1024k, each thread is really using 
between 0 and 1024k depending on what that thread really needs?


+ jay


Knut Forkalsrud wrote:
> On Thu, Jan 8, 2009 at 1:52 PM, Jay Ballinger  wrote:
>> - When a thread is created, does it grab all of the stack memory, or can
>> the stack memory start small and grow to the default or -Xss size?
> 
> If I'm not mistaken the memory is allocated, but it is virtual memory,
> so unless it is actually used there is no penalty.
> 
> The only problem with allocating too much stack space is that you can
> run out of address space on a 32 bit JVM.
> 
> With typically 2GB max addressable for the app and 10 MBytes per stack
> you are limited to 200 threads.
> 
> -Knut
> 
> 
> ___
> resin-interest mailing list
> resin-interest@caucho.com
> http://maillist.caucho.com/mailman/listinfo/resin-interest


___
resin-interest mailing list
resin-interest@caucho.com
http://maillist.caucho.com/mailman/listinfo/resin-interest

Re: [Resin-interest] implicit stack memory size?

[Knut Forkalsrud]

On Thu, Jan 8, 2009 at 1:52 PM, Jay Ballinger  wrote:
> - When a thread is created, does it grab all of the stack memory, or can
> the stack memory start small and grow to the default or -Xss size?

If I'm not mistaken the memory is allocated, but it is virtual memory,
so unless it is actually used there is no penalty.

The only problem with allocating too much stack space is that you can
run out of address space on a 32 bit JVM.

With typically 2GB max addressable for the app and 10 MBytes per stack
you are limited to 200 threads.

-Knut


___
resin-interest mailing list
resin-interest@caucho.com
http://maillist.caucho.com/mailman/listinfo/resin-interest

[Resin-interest] implicit stack memory size?

[Jay Ballinger]

Hi everyone,

We're running Resin 3.0.18 Pro on a Windows 32-bit system and we're 
looking to understand/tune our memory settings a bit.

According to http://www.caucho.com/resin-3.0/performance/jvm-tuning.xtp 
the stack size is set to 2048k unless set otherwise. Is this true? In 
using JConsole we can't see where any explicit -Xss setting is passed 
and we can't readily determine what the current stack size is set to.

So, here's a short list of questions. And thanks for your help.

- If JConsole doesn't show -Xss, is it really there for resin?
- How was "2048k is an appropriate value for most situations." determined?
- Can resin/jvm/jconsole report the current stack memory setting?

(general sun jvm on windows questions)
- What is the default stack memory size?
- When a thread is created, does it grab all of the stack memory, or can 
the stack memory start small and grow to the default or -Xss size?


Thanks, everyone.

+ jay



___
resin-interest mailing list
resin-interest@caucho.com
http://maillist.caucho.com/mailman/listinfo/resin-interest