Re: Tomcat died on "java.lang.OutOfMemoryError: requested 2147483664 bytes for Chunk::new. Out of swap space?" message

[Rainer Jung]

On 28.05.2010 05:12, Leon Kolchinsky wrote:

Hello Jung,

Thanks for your insights. Learning some new stuff about Java every day ;)
I've read this nice blog -
http://www.talkingtree.com/blog/index.cfm/2006/4/28/Understanding-HotSpot-in-Plain-English
  (
UNDERSTANDING HOTSPOT IN PLAIN
ENGLISH
)

Now I can see how to avoid those HotSpot compilation errors and eventual JVM
crashes.
I could use -Xint option to force "interpretedmode execution only"

like this:
# cat /srv/tomcat/bin/setenv.sh
export JAVA_HOME=/srv/jdk
export PATH="${PATH}:${JAVA_HOME}/bin/"
CATALINA_OPTS="-Xint -Xms512m -Xmx1024m -XX:MaxPermSize=512m $JAVA_OPTS
-Djava.awt.headless=true "

What are you saying?
Am I right?


As Chuck and others have stated: "-Xint" is not really a good solution. 
You definitely want HotSpot and only work around your special problem. 
Disabling HotSpot is very negative for performance and not something you 
can get forward with.


Yes: file a bug!

-client: As Chuck wrote it might not be available or 64Bit VMs. So take 
the commandline parameters and add -version and check what the output 
is, i.e. if there is any diference between -server and -client. 
Otherwise don't be surprised that it doesn't help.


I really suggest you go the way of disabling special compilations. Only 
that way you learn about your real problem. Is it only a few methods 
that you need to disable for compilation, or is it many methods, so that 
there is a bigger problem underneath.


And don't forget to try other JVM patch versions (older or newer). 
HotSpot compiler bug fixes (and possibly regressions) are frequent in 
the release notes for the JVM.


Regards,

Rainer

-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org

Re: Tomcat died on "java.lang.OutOfMemoryError: requested 2147483664 bytes for Chunk::new. Out of swap space?" message

[Kees Jan Koster]

Dear Leon,

>
> It'd be nice to see some speed comparisons between "-client" "-server" and
> "-Xint" options and not only "-server" vs "-Xint".
>

I did some benchmarks with these settings. I was actually exploring
hyperthreading in P4 hardware when I found -server to be twice as fast
for even simple programs.

http://java-monitor.com/forum/showthread.php?t=552

Ever since I have been calling -server the "go faster flag". :)
-- 
Kees Jan

http://java-monitor.com/forum/
kjkos...@gmail.com
06-51838192

-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org

Re: Tomcat died on "java.lang.OutOfMemoryError: requested 2147483664 bytes for Chunk::new. Out of swap space?" message

[Leon Kolchinsky]

Thanks Charles,

It's clear now.
I'll try to run Tomcat with "-client" instead of "-server" option.

It'd be nice to see some speed comparisons between "-client" "-server" and
"-Xint" options and not only "-server" vs "-Xint".


Regards,
Leon

On Fri, May 28, 2010 at 15:28, Caldarale, Charles R <
chuck.caldar...@unisys.com> wrote:

> > From: Leon Kolchinsky [mailto:lkolc...@gmail.com]
> > Subject: Re: Tomcat died on "java.lang.OutOfMemoryError: requested
> > 2147483664 bytes for Chunk::new. Out of swap space?" message
> >
> > My understanding that the difference between "-client"
> > and "-server" is in the number of code execution before
> > compilation occurred.
>
> No, that's one of the most trivial differences.  The client and server JITs
> are two completely separate and mostly independent compiler implementations.
>
> > Why is it "less fragile"?
>
> Because the server JIT (commonly known as C2) is much more complex and
> includes many more optimization mechanisms; consequently, there is some
> increased chance of error.
>
> Just for grins, I compared Tomcat startup times between server mode and
> -Xint with a 64-bit JDK 6u20 running on a 2.0 Ghz dual-core AMD with 4 GB of
> RAM and the following options:
>
> -Dcom.sun.management.jmxremote -XX:MaxPermSize=128m
> -XX:+HeapDumpOnOutOfMemoryError
>
> Using Tomcat 7 RC2 with full logging enabled, server mode startup took just
> under 4 seconds, whereas -Xint required over 6.5 seconds - a rather
> significant difference, especially since it was purely startup.
>
>  - Chuck
>
>
> THIS COMMUNICATION MAY CONTAIN CONFIDENTIAL AND/OR OTHERWISE PROPRIETARY
> MATERIAL and is thus for use only by the intended recipient. If you received
> this in error, please contact the sender and delete the e-mail and its
> attachments from all computers.
>
>
> -
> To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
> For additional commands, e-mail: users-h...@tomcat.apache.org
>
>

RE: Tomcat died on "java.lang.OutOfMemoryError: requested 2147483664 bytes for Chunk::new. Out of swap space?" message

[Caldarale, Charles R]

> From: Leon Kolchinsky [mailto:lkolc...@gmail.com]
> Subject: Re: Tomcat died on "java.lang.OutOfMemoryError: requested
> 2147483664 bytes for Chunk::new. Out of swap space?" message
> 
> My understanding that the difference between "-client"
> and "-server" is in the number of code execution before
> compilation occurred.

No, that's one of the most trivial differences.  The client and server JITs are 
two completely separate and mostly independent compiler implementations.

> Why is it "less fragile"?

Because the server JIT (commonly known as C2) is much more complex and includes 
many more optimization mechanisms; consequently, there is some increased chance 
of error.

Just for grins, I compared Tomcat startup times between server mode and -Xint 
with a 64-bit JDK 6u20 running on a 2.0 Ghz dual-core AMD with 4 GB of RAM and 
the following options:

-Dcom.sun.management.jmxremote -XX:MaxPermSize=128m 
-XX:+HeapDumpOnOutOfMemoryError

Using Tomcat 7 RC2 with full logging enabled, server mode startup took just 
under 4 seconds, whereas -Xint required over 6.5 seconds - a rather significant 
difference, especially since it was purely startup.

 - Chuck


THIS COMMUNICATION MAY CONTAIN CONFIDENTIAL AND/OR OTHERWISE PROPRIETARY 
MATERIAL and is thus for use only by the intended recipient. If you received 
this in error, please contact the sender and delete the e-mail and its 
attachments from all computers.


-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org

Re: Tomcat died on "java.lang.OutOfMemoryError: requested 2147483664 bytes for Chunk::new. Out of swap space?" message

[Leon Kolchinsky]

What do you mean by "less aggressive compilation"?
My understanding that the difference between "-client" and "-server" is in
the number of code execution before compilation occurred.

Why is it "less fragile"?

On Fri, May 28, 2010 at 13:34, Caldarale, Charles R <
chuck.caldar...@unisys.com> wrote:

> > From: Leon Kolchinsky [mailto:lkolc...@gmail.com]
> > Subject: Re: Tomcat died on "java.lang.OutOfMemoryError: requested
> > 2147483664 bytes for Chunk::new. Out of swap space?" message
> >
> > I've read this nice blog -
> > http://www.talkingtree.com/blog/index.cfm/2006/4/28/Understanding-
> > HotSpot-in-Plain-English
>
> A somewhat naive bit of writing; rather outdated, and tad on the dangerous
> side.  Rainer's suggestions and references are much more appropriate.
>
> > I could use -Xint option to force "interpretedmode execution only"
>
> You could, but your code execution would slow to a crawl.  Not an
> appropriate thing to do in a production environment.  Don't forget that not
> only is your webapp converted to native code by the JIT, but the thousands
> of methods in the JRE and Tomcat itself are also processed.
>
> The client mode (as suggested by Rainer, and almost completely ignored by
> the above article) is a less aggressive compilation option, and is thus less
> fragile than the server JIT.  That should suffice until the Oracle/Sun
> people fix the server bug.  (You did file a bug report, didn't you?)
>
>  - Chuck
>
>
> THIS COMMUNICATION MAY CONTAIN CONFIDENTIAL AND/OR OTHERWISE PROPRIETARY
> MATERIAL and is thus for use only by the intended recipient. If you received
> this in error, please contact the sender and delete the e-mail and its
> attachments from all computers.
>
>
> -
> To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
> For additional commands, e-mail: users-h...@tomcat.apache.org
>
>

RE: Tomcat died on "java.lang.OutOfMemoryError: requested 2147483664 bytes for Chunk::new. Out of swap space?" message

[Caldarale, Charles R]

> From: Leon Kolchinsky [mailto:lkolc...@gmail.com]
> Subject: Re: Tomcat died on "java.lang.OutOfMemoryError: requested
> 2147483664 bytes for Chunk::new. Out of swap space?" message
> 
> I've read this nice blog -
> http://www.talkingtree.com/blog/index.cfm/2006/4/28/Understanding-
> HotSpot-in-Plain-English

A somewhat naive bit of writing; rather outdated, and tad on the dangerous 
side.  Rainer's suggestions and references are much more appropriate.

> I could use -Xint option to force "interpretedmode execution only"

You could, but your code execution would slow to a crawl.  Not an appropriate 
thing to do in a production environment.  Don't forget that not only is your 
webapp converted to native code by the JIT, but the thousands of methods in the 
JRE and Tomcat itself are also processed.

The client mode (as suggested by Rainer, and almost completely ignored by the 
above article) is a less aggressive compilation option, and is thus less 
fragile than the server JIT.  That should suffice until the Oracle/Sun people 
fix the server bug.  (You did file a bug report, didn't you?)

 - Chuck


THIS COMMUNICATION MAY CONTAIN CONFIDENTIAL AND/OR OTHERWISE PROPRIETARY 
MATERIAL and is thus for use only by the intended recipient. If you received 
this in error, please contact the sender and delete the e-mail and its 
attachments from all computers.


-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org

Re: Tomcat died on "java.lang.OutOfMemoryError: requested 2147483664 bytes for Chunk::new. Out of swap space?" message

[Leon Kolchinsky]

Hello Jung,

Thanks for your insights. Learning some new stuff about Java every day ;)
I've read this nice blog -
http://www.talkingtree.com/blog/index.cfm/2006/4/28/Understanding-HotSpot-in-Plain-English
 (
UNDERSTANDING HOTSPOT IN PLAIN
ENGLISH
)

Now I can see how to avoid those HotSpot compilation errors and eventual JVM
crashes.
I could use -Xint option to force "interpretedmode execution only"

like this:
# cat /srv/tomcat/bin/setenv.sh
export JAVA_HOME=/srv/jdk
export PATH="${PATH}:${JAVA_HOME}/bin/"
CATALINA_OPTS="-Xint -Xms512m -Xmx1024m -XX:MaxPermSize=512m $JAVA_OPTS
-Djava.awt.headless=true "

What are you saying?
Am I right?


Best Regards,
Leon Kolchinsky

On Thu, May 27, 2010 at 00:38, Rainer Jung  wrote:

> On 26.05.2010 14:49, Caldarale, Charles R wrote:
>
>> From: Leon Kolchinsky [mailto:lkolc...@gmail.com]
>>> Subject: Tomcat died on "java.lang.OutOfMemoryError: requested
>>> 2147483664 bytes for Chunk::new. Out of swap space?" message
>>>
>>> Yesterday I've got this error in catalina.out and Tomcat died (I'm
>>> attaching hs_err_pid5460.log file):
>>>
>>
>> This is not a Tomcat error - it's a bug in the JVM.  The C2 (server)
>> compiler is requesting an extremely large expansion of the C (not Java)
>> heap.  You can try increasing the swap space to see if that's a viable
>> workaround, but you really should do this:
>>
>>  # If you would like to submit a bug report, please visit:
>>> #   http://java.sun.com/webapps/bugreport/crash.jsp
>>>
>>
>> The current JVM version is 6u20, so you might want to try running with
>> that before filing a bug report or expanding the swap file.
>>
>
> If we stick to the theory, that it is actually the Hot Spot compiler, which
> demands the memory, one could also try to work around the issue.
>
> Option 1: Use the client Hot Sopt compiler (Add "-client" when starting the
> JVM). It could help,
>
> Option 2: Disable the Hot Spot compilation for the class and method that
> resulted in the crash:
>
>  Current CompileTask:
>> C2:4743  !
>> org.apache.velocity.runtime.directive.Foreach.render(Lorg/apache/velocity/context/InternalContextAdapter;Ljava/io/Writer;Lorg/apache/velocity/runtime/parser/node/Node;)Z
>> (514 bytes)
>>
>
> To do this, you need to create a file named ".hotspot_compiler" in the
> working directory of your Tomcat. You would put a line like that into the
> file:
>
> exclude org/apache/velocity/runtime/directive/Foreach render
>
> Some more info about that is contained in
>
>
> http://java.sun.com/javase/6/webnotes/trouble/TSG-VM/html/crashes.html#gbyzd
>
> An alternative is adding
> -XX:CompileCommand=exclude,org/apache/velocity/runtime/directive/Foreach,render
>
> to the command line.
>
> See also
>
> http://forums.sun.com/thread.jspa?threadID=417123
>
> http://blogs.sun.com/javawithjiva/entry/hotspotrc_and_hotspot_compiler
>
> If you want to follow the Hot Spot compilation, you can also add
> "-XX:-PrintCompilation".
>
> Regards,
>
> Rainer
>
>
> -
> To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
> For additional commands, e-mail: users-h...@tomcat.apache.org
>
>

Re: Tomcat died on "java.lang.OutOfMemoryError: requested 2147483664 bytes for Chunk::new. Out of swap space?" message

[Leon Kolchinsky]

Thanks guys for you replys,

Carl - Please update this thread with results of your check for 6u20.

I've decided to install x86 version of JDK - jdk-6u7-linux-i586.bin
(stability is my main concern now)

So in my installation I just installed jdk-6u7-linux-i586.bin version.
1. Changed symlink:
# rm jdk
# ln -s jdk1.6.0_07 jdk
2. Updated paths with:
# alternatives --remove jar /srv/jdk1.6.0_18/bin/jar
# alternatives --remove java /srv/jdk1.6.0_18/bin/java
# alternatives --remove javac /srv/jdk1.6.0_18/bin/javac 100

# alternatives --install /usr/bin/java java /srv/jdk1.6.0_07/bin/java 100
# alternatives --install /usr/bin/jar jar /srv/jdk1.6.0_07/bin/jar 100
# alternatives --install /usr/bin/javac javac /srv/jdk1.6.0_07/bin/javac 100

# /usr/sbin/alternatives --config java
There are 2 programs which provide 'java'.
  SelectionCommand
---
*+ 1   /srv/jdk1.6.0_18/bin/java
   2   /srv/jdk1.6.0_07/bin/java
Enter to keep the current selection[+], or type selection number: 2
# java -version
java version "1.6.0_07"
Java(TM) SE Runtime Environment (build 1.6.0_07-b06)
Java HotSpot(TM) Client VM (build 10.0-b23, mixed mode, sharing)

3. Updated setenv.sh to some basic defaults:

# cat /srv/tomcat/bin/setenv.sh
export JAVA_HOME=/srv/jdk
export PATH="${PATH}:${JAVA_HOME}/bin/"

# CATALINA_OPTS="-server -XX:+UseParallelGC -Xms2048m -Xmx2048m
-XX:MaxPermSize=1024m -XX:+CMSClassUnloadingEnabled
-XX:+CMSPermGenSweepingEnabled -Djava.awt.headless=true"
CATALINA_OPTS="-Xms512m -Xmx1024m -XX:MaxPermSize=512m $JAVA_OPTS
-Djava.awt.headless=true "
export CATALINA_OPTS

export CATALINA_PID="$CATALINA_HOME/tomcat.pid"

I'm going to monitor this server and see how it goes.

Best Regards,
Leon Kolchinsky

On Thu, May 27, 2010 at 00:20, Carl  wrote:

> I am the person who reported occasional but persistent abrupt terminations
> of the 1.6 JVM on levels 6u10 and above.  I did go back to 6u7 and the
> application has run without a burp for three months.  I had tested
> 6u18/6u19, both of which produced the same result.  I am getting ready to
> start testing 6u20 as I would like to keep current so I don't have any
> security issues hanging out.
>
> Thanks,
>
> Carl
> - Original Message - From: "Caldarale, Charles R" <
> chuck.caldar...@unisys.com>
> To: "Tomcat Users List" 
> Sent: Wednesday, May 26, 2010 8:00 AM
>
> Subject: RE: Tomcat died on "java.lang.OutOfMemoryError: requested
> 2147483664 bytes for Chunk::new. Out of swap space?" message
>
>
>  From: Caldarale, Charles R
>> Subject: RE: Tomcat died on "java.lang.OutOfMemoryError: requested
>> 2147483664 bytes for Chunk::new. Out of swap space?" message
>>
>> The current JVM version is 6u20, so you might want to try running with
>> that before filing a bug report or expanding the swap file.
>>
>
> Should also mention that another Tomcat admin reported occasional but
> persistent abrupt terminations of the 1.6 JVM on levels 6u10 and above, but
> went back to 6u7 and that appeared to rectify the problem.  I think the
> highest level he had tested was 6u17, but I'm not positive.
>
> - Chuck
>
>
> THIS COMMUNICATION MAY CONTAIN CONFIDENTIAL AND/OR OTHERWISE PROPRIETARY
> MATERIAL and is thus for use only by the intended recipient. If you received
> this in error, please contact the sender and delete the e-mail and its
> attachments from all computers.
>
>
> -
> To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
> For additional commands, e-mail: users-h...@tomcat.apache.org
>
>
>
> -
> To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
> For additional commands, e-mail: users-h...@tomcat.apache.org
>
>

Re: Tomcat died on "java.lang.OutOfMemoryError: requested 2147483664 bytes for Chunk::new. Out of swap space?" message

[Rainer Jung]

On 26.05.2010 16:38, Rainer Jung wrote:

On 26.05.2010 14:49, Caldarale, Charles R wrote:

From: Leon Kolchinsky [mailto:lkolc...@gmail.com]
Subject: Tomcat died on "java.lang.OutOfMemoryError: requested
2147483664 bytes for Chunk::new. Out of swap space?" message

Yesterday I've got this error in catalina.out and Tomcat died (I'm
attaching hs_err_pid5460.log file):


This is not a Tomcat error - it's a bug in the JVM. The C2 (server)
compiler is requesting an extremely large expansion of the C (not
Java) heap. You can try increasing the swap space to see if that's a
viable workaround, but you really should do this:


# If you would like to submit a bug report, please visit:
# http://java.sun.com/webapps/bugreport/crash.jsp


The current JVM version is 6u20, so you might want to try running with
that before filing a bug report or expanding the swap file.


If we stick to the theory, that it is actually the Hot Spot compiler,
which demands the memory, one could also try to work around the issue.

Option 1: Use the client Hot Sopt compiler (Add "-client" when starting
the JVM). It could help,

Option 2: Disable the Hot Spot compilation for the class and method that
resulted in the crash:


Current CompileTask:
C2:4743 !
org.apache.velocity.runtime.directive.Foreach.render(Lorg/apache/velocity/context/InternalContextAdapter;Ljava/io/Writer;Lorg/apache/velocity/runtime/parser/node/Node;)Z
(514 bytes)


To do this, you need to create a file named ".hotspot_compiler" in the
working directory of your Tomcat. You would put a line like that into
the file:

exclude org/apache/velocity/runtime/directive/Foreach render

Some more info about that is contained in

http://java.sun.com/javase/6/webnotes/trouble/TSG-VM/html/crashes.html#gbyzd


An alternative is adding
-XX:CompileCommand=exclude,org/apache/velocity/runtime/directive/Foreach,render


to the command line.

See also

http://forums.sun.com/thread.jspa?threadID=417123

http://blogs.sun.com/javawithjiva/entry/hotspotrc_and_hotspot_compiler

If you want to follow the Hot Spot compilation, you can also add
"-XX:-PrintCompilation".


I meant "-XX:+PrintCompilation"

-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org

RE: Tomcat died on "java.lang.OutOfMemoryError: requested 2147483664 bytes for Chunk::new. Out of swap space?" message

[Caldarale, Charles R]

> From: Rainer Jung [mailto:rainer.j...@kippdata.de]
> Subject: Re: Tomcat died on "java.lang.OutOfMemoryError: requested
> 2147483664 bytes for Chunk::new. Out of swap space?" message
> 
> Option 1: Use the client Hot Sopt compiler (Add "-client" when 
> starting the JVM).

Usually not available on 64-bit JVMs, unless you build the JVM yourself (not 
recommended :-).

 - Chuck


THIS COMMUNICATION MAY CONTAIN CONFIDENTIAL AND/OR OTHERWISE PROPRIETARY 
MATERIAL and is thus for use only by the intended recipient. If you received 
this in error, please contact the sender and delete the e-mail and its 
attachments from all computers.


-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org

Re: Tomcat died on "java.lang.OutOfMemoryError: requested 2147483664 bytes for Chunk::new. Out of swap space?" message

[Rainer Jung]

On 26.05.2010 14:49, Caldarale, Charles R wrote:

From: Leon Kolchinsky [mailto:lkolc...@gmail.com]
Subject: Tomcat died on "java.lang.OutOfMemoryError: requested
2147483664 bytes for Chunk::new. Out of swap space?" message

Yesterday I've got this error in catalina.out and Tomcat died (I'm
attaching hs_err_pid5460.log file):


This is not a Tomcat error - it's a bug in the JVM.  The C2 (server) compiler 
is requesting an extremely large expansion of the C (not Java) heap.  You can 
try increasing the swap space to see if that's a viable workaround, but you 
really should do this:


# If you would like to submit a bug report, please visit:
#   http://java.sun.com/webapps/bugreport/crash.jsp


The current JVM version is 6u20, so you might want to try running with that 
before filing a bug report or expanding the swap file.


If we stick to the theory, that it is actually the Hot Spot compiler, 
which demands the memory, one could also try to work around the issue.


Option 1: Use the client Hot Sopt compiler (Add "-client" when starting 
the JVM). It could help,


Option 2: Disable the Hot Spot compilation for the class and method that 
resulted in the crash:



Current CompileTask:
C2:4743  !   
org.apache.velocity.runtime.directive.Foreach.render(Lorg/apache/velocity/context/InternalContextAdapter;Ljava/io/Writer;Lorg/apache/velocity/runtime/parser/node/Node;)Z
 (514 bytes)


To do this, you need to create a file named ".hotspot_compiler" in the 
working directory of your Tomcat. You would put a line like that into 
the file:


exclude org/apache/velocity/runtime/directive/Foreach render

Some more info about that is contained in

http://java.sun.com/javase/6/webnotes/trouble/TSG-VM/html/crashes.html#gbyzd

An alternative is adding 
-XX:CompileCommand=exclude,org/apache/velocity/runtime/directive/Foreach,render


to the command line.

See also

http://forums.sun.com/thread.jspa?threadID=417123

http://blogs.sun.com/javawithjiva/entry/hotspotrc_and_hotspot_compiler

If you want to follow the Hot Spot compilation, you can also add 
"-XX:-PrintCompilation".


Regards,

Rainer

-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org

Re: Tomcat died on "java.lang.OutOfMemoryError: requested 2147483664 bytes for Chunk::new. Out of swap space?" message

[Carl]

I am the person who reported occasional but persistent abrupt terminations 
of the 1.6 JVM on levels 6u10 and above.  I did go back to 6u7 and the 
application has run without a burp for three months.  I had tested 
6u18/6u19, both of which produced the same result.  I am getting ready to 
start testing 6u20 as I would like to keep current so I don't have any 
security issues hanging out.


Thanks,

Carl
- Original Message - 
From: "Caldarale, Charles R" 

To: "Tomcat Users List" 
Sent: Wednesday, May 26, 2010 8:00 AM
Subject: RE: Tomcat died on "java.lang.OutOfMemoryError: requested 
2147483664 bytes for Chunk::new. Out of swap space?" message




From: Caldarale, Charles R
Subject: RE: Tomcat died on "java.lang.OutOfMemoryError: requested
2147483664 bytes for Chunk::new. Out of swap space?" message

The current JVM version is 6u20, so you might want to try running with
that before filing a bug report or expanding the swap file.


Should also mention that another Tomcat admin reported occasional but 
persistent abrupt terminations of the 1.6 JVM on levels 6u10 and above, but 
went back to 6u7 and that appeared to rectify the problem.  I think the 
highest level he had tested was 6u17, but I'm not positive.


- Chuck


THIS COMMUNICATION MAY CONTAIN CONFIDENTIAL AND/OR OTHERWISE PROPRIETARY 
MATERIAL and is thus for use only by the intended recipient. If you received 
this in error, please contact the sender and delete the e-mail and its 
attachments from all computers.



-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org



-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org

RE: Tomcat died on "java.lang.OutOfMemoryError: requested 2147483664 bytes for Chunk::new. Out of swap space?" message

[Caldarale, Charles R]

> From: Caldarale, Charles R
> Subject: RE: Tomcat died on "java.lang.OutOfMemoryError: requested
> 2147483664 bytes for Chunk::new. Out of swap space?" message
> 
> The current JVM version is 6u20, so you might want to try running with
> that before filing a bug report or expanding the swap file.

Should also mention that another Tomcat admin reported occasional but 
persistent abrupt terminations of the 1.6 JVM on levels 6u10 and above, but 
went back to 6u7 and that appeared to rectify the problem.  I think the highest 
level he had tested was 6u17, but I'm not positive.

 - Chuck


THIS COMMUNICATION MAY CONTAIN CONFIDENTIAL AND/OR OTHERWISE PROPRIETARY 
MATERIAL and is thus for use only by the intended recipient. If you received 
this in error, please contact the sender and delete the e-mail and its 
attachments from all computers.


-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org

Re: Tomcat died on "java.lang.OutOfMemoryError: requested 2147483664 bytes for Chunk::new. Out of swap space?" message

[Pid]

On 26/05/2010 11:15, Rainer Jung wrote:
> On 26.05.2010 11:48, Pid wrote:
>> On 26/05/2010 04:08, Leon Kolchinsky wrote:
>>> Hello
>>>
>>> I've installed Tomcat 6.0.26
>>> and running Confluence software on it.
>>> I think that maximum 2 concurrent users using this application in at
>>> this stage (testing now).
>>>
>>>
>>> Yesterday I've got this error in catalina.out and Tomcat died (I'm
>>> attaching hs_err_pid5460.log file):
>>> #
>>> # A fatal error has been detected by the Java Runtime Environment:
>>> #
>>> # java.lang.OutOfMemoryError: requested 2147483664 bytes for Chunk::new.
>>> Out of swap space?
>>
>> What was the last task that it was doing before requesting 2Gb more
>> memory?
>>
>> What's in catalina.out, the matching host access log and any application
>> logs immediately before the OOM?
> 
> Did you notice the full hotspot error file was attached to the OP's
> mail? Active threads seemed to be the hotspot compiler, but that
> wouldn't match with the 2GB allocation.

Nope.  Didn't get the file.


p

> Regards,
> 
> Rainer
> 
> -
> To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
> For additional commands, e-mail: users-h...@tomcat.apache.org
> 




signature.asc
Description: OpenPGP digital signature

RE: Tomcat died on "java.lang.OutOfMemoryError: requested 2147483664 bytes for Chunk::new. Out of swap space?" message

[Caldarale, Charles R]

> From: Leon Kolchinsky [mailto:lkolc...@gmail.com]
> Subject: Tomcat died on "java.lang.OutOfMemoryError: requested
> 2147483664 bytes for Chunk::new. Out of swap space?" message
> 
> Yesterday I've got this error in catalina.out and Tomcat died (I'm
> attaching hs_err_pid5460.log file):

This is not a Tomcat error - it's a bug in the JVM.  The C2 (server) compiler 
is requesting an extremely large expansion of the C (not Java) heap.  You can 
try increasing the swap space to see if that's a viable workaround, but you 
really should do this:

> # If you would like to submit a bug report, please visit:
> #   http://java.sun.com/webapps/bugreport/crash.jsp

The current JVM version is 6u20, so you might want to try running with that 
before filing a bug report or expanding the swap file.

 - Chuck


THIS COMMUNICATION MAY CONTAIN CONFIDENTIAL AND/OR OTHERWISE PROPRIETARY 
MATERIAL and is thus for use only by the intended recipient. If you received 
this in error, please contact the sender and delete the e-mail and its 
attachments from all computers.


-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org

Re: Tomcat died on "java.lang.OutOfMemoryError: requested 2147483664 bytes for Chunk::new. Out of swap space?" message

[Rainer Jung]

On 26.05.2010 11:48, Pid wrote:

On 26/05/2010 04:08, Leon Kolchinsky wrote:

Hello

I've installed Tomcat 6.0.26
and running Confluence software on it.
I think that maximum 2 concurrent users using this application in at
this stage (testing now).


Yesterday I've got this error in catalina.out and Tomcat died (I'm
attaching hs_err_pid5460.log file):
#
# A fatal error has been detected by the Java Runtime Environment:
#
# java.lang.OutOfMemoryError: requested 2147483664 bytes for Chunk::new.
Out of swap space?


What was the last task that it was doing before requesting 2Gb more memory?

What's in catalina.out, the matching host access log and any application
logs immediately before the OOM?


Did you notice the full hotspot error file was attached to the OP's 
mail? Active threads seemed to be the hotspot compiler, but that 
wouldn't match with the 2GB allocation.


Regards,

Rainer

-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org

Re: Tomcat died on "java.lang.OutOfMemoryError: requested 2147483664 bytes for Chunk::new. Out of swap space?" message

[Tobias Crefeld]

Am Wed, 26 May 2010 13:08:23 +1000
schrieb Leon Kolchinsky :

> # A fatal error has been detected by the Java Runtime Environment:
> #
> # java.lang.OutOfMemoryError: requested 2147483664 bytes for
> Chunk::new. Out of swap space?

Without knowing the application I would strongly estimate that it is
an application bug if it requests a block of 2 GB memory.

It might be a good idea to monitor the JVM and its memory management
with JMX-tools like jconsole (X11) or LambdaProbe (Tomcat).


> CATALINA_OPTS="-server -XX:+UseParallelGC -Xms2048m -Xmx2048m
> -XX:MaxPermSize=1024m -XX:+CMSClassUnloadingEnabled
> -XX:+CMSPermGenSweepingEnabled -Djava.awt.headless=true"

We are using in our standard setups some more parameters although I'm
not sure if they are necessary always or just still valid, e.g.:

# Maximum Size of new generation; default: function of NewRatio
JAVA_OPTS="$JAVA_OPTS "-XX:MaxNewSize=400m
# Default size of new generation; default: 30% larger
JAVA_OPTS="$JAVA_OPTS "-XX:NewSize=400m
# Ratio of new/old generation sizes; default: 8
JAVA_OPTS="$JAVA_OPTS "-XX:NewRatio=3
JAVA_OPTS="$JAVA_OPTS "-XX:+UseParNewGC

Here you find some parameter docs:
 http://java.sun.com/javase/6/docs/technotes/tools/solaris/java.html
 http://java.sun.com/javase/technologies/hotspot/vmoptions.jsp


Generally I would ask the application vendor for suggested parameters.


Gruß,
 Tobias.

-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org

Re: Tomcat died on "java.lang.OutOfMemoryError: requested 2147483664 bytes for Chunk::new. Out of swap space?" message

[Pid]

On 26/05/2010 04:08, Leon Kolchinsky wrote:
> Hello
> 
> I've installed Tomcat 6.0.26
> and running Confluence software on it.
> I think that maximum 2 concurrent users using this application in at
> this stage (testing now).
> 
> 
> Yesterday I've got this error in catalina.out and Tomcat died (I'm
> attaching hs_err_pid5460.log file):
> #
> # A fatal error has been detected by the Java Runtime Environment:
> #
> # java.lang.OutOfMemoryError: requested 2147483664 bytes for Chunk::new.
> Out of swap space?

What was the last task that it was doing before requesting 2Gb more memory?

What's in catalina.out, the matching host access log and any application
logs immediately before the OOM?


p

> #  Internal Error (allocation.cpp:215), pid=5460, tid=1085901120
> #  Error: Chunk::new
> #
> # JRE version: 6.0_18-b07
> # Java VM: Java HotSpot(TM) 64-Bit Server VM (16.0-b13 mixed mode
> linux-amd64 )
> # An error report file with more information is saved as:
> # /tmp/hs_err_pid5460.log
> #
> # If you would like to submit a bug report, please visit:
> #   http://java.sun.com/webapps/bugreport/crash.jsp
> #
> 
> 
> Any suggestion on how to avoid those errors and Tomcat failure in the
> future?
> May be some CATALINA_OPTS tuning options?
> 
> 
> Some system details:
> ---
> # cat /srv/tomcat/bin/setenv.sh
> export JAVA_HOME=/srv/jdk
> export PATH="${PATH}:${JAVA_HOME}/bin/"
> 
> CATALINA_OPTS="-server -XX:+UseParallelGC -Xms2048m -Xmx2048m
> -XX:MaxPermSize=1024m -XX:+CMSClassUnloadingEnabled
> -XX:+CMSPermGenSweepingEnabled -Djava.awt.headless=true"
> export CATALINA_OPTS
> export CATALINA_PID="$CATALINA_HOME/tomcat.pid"
> 
> # java -version
> java version "1.6.0_18"
> Java(TM) SE Runtime Environment (build 1.6.0_18-b07)
> Java HotSpot(TM) 64-Bit Server VM (build 16.0-b13, mixed mode)
> 
> # free -m
>  total   used   free sharedbuffers cached
> Mem:  3775730   3045  0280304
> -/+ buffers/cache:144   3630
> Swap: 1023  0   1023
> 
> ---  S Y S T E M  ---
> 
> OS:Scientific Linux SL release 5.5 (Boron)
> 
> uname:Linux 2.6.18-164.15.1.el5 #1 SMP Tue Mar 16 18:44:51 EDT 2010 x86_64
> libc:glibc 2.5 NPTL 2.5
> rlimit: STACK 10240k, CORE infinity, NPROC infinity, NOFILE 1024, AS
> infinity
> load average:0.88 0.29 0.10
> 
> CPU:total 1 (1 cores per cpu, 1 threads per core) family 16 model 4
> stepping 2, cmov, cx8, fxsr, mmx, sse, sse2, sse3, popcnt, mmxext,
> 3dnow, 3dnowext, lzcnt, sse4a
> 
> Memory: 4k page, physical 3866464k(13372k free), swap 1048568k(1048500k
> free)
> 
> vm_info: Java HotSpot(TM) 64-Bit Server VM (16.0-b13) for linux-amd64
> JRE (1.6.0_18-b07), built on Dec 17 2009 13:42:22 by "java_re" with gcc
> 3.2.2 (SuSE Linux)
> 
> 
> Thanks for any help,
> Leon
> 
> 
> 
> 
> -
> To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
> For additional commands, e-mail: users-h...@tomcat.apache.org




signature.asc
Description: OpenPGP digital signature