Re: Tomcat 8.0.33 Crashed

[Christopher Schultz]

-BEGIN PGP SIGNED MESSAGE-
Hash: SHA256

Atanu,

On 10/26/17 5:23 AM, Mark Thomas wrote:
> On 25/10/2017 13:08, Atanu Pal wrote:
>> Still can you help me explain little more about the root cause?
>> 
>> That will help me to explain it to customer and setup a test 
>> environment for future. Thanks and Regards Atanu Pal 
>> E2Infosystems - Chennai
> 
> Difficult to say but a possible sequence of events is something
> like:
> 
> - thread A adds socket S to poller - thread B closes socket S -
> poller tries of access socket S -> crash

If you switch from the APR connector to NIO does the problem still occur
?

Are you sure you need the APR connector at all? Usually, it's only
used to get access to the OpenSSL crypto engine. Perhaps you can do
TLS offloading onto another device such as a load-balancer?

If the problem goes away when switching connectors, can you try to use
Tomcat 8.5.x with the OpenSSL JSSE provider? That's an NIO connector
with OpenSSL providing the crypto under the hood (so it's actually
fast, unlike the JVM-provided crypto engine).

- -chris
-BEGIN PGP SIGNATURE-
Comment: GPGTools - http://gpgtools.org
Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/

iQIzBAEBCAAdFiEEMmKgYcQvxMe7tcJcHPApP6U8pFgFAlnx5agACgkQHPApP6U8
pFhufA/+LspuZv9BHOJzcbdF/T7VBTFK+Vp+gidCxDwhSb/hK9MXZ64vDjNixenb
Za8vkm4irDpurYKOWYaz31tiANyIF4kLWw0FUdiCDS2DgFNEyZhmyBjj3wi0C4eY
3e/Ck8947Rc3Ewz3/DLfe9bqQRsRsi4wb3dkWSCgKemWVM9L9sr+ROknOCHr8NMW
tgTgu2/StAKEd8ZI1E6fVA4/7h8bgoKuchlViHJ6+HFHhZkKFMOPNH3AMpYtr0sq
oXSXGGXbQ5EwUFnOCJ/nVWJb8uSaPYnyYfgne4RDR1zD0LJaTESYyrEgDmBa185B
BxNcpUP1FH+9IGjpcf0idZcazxn/r1ZFzL0/tKkEttby8BUyXrdFHa3c3qYs4MwA
p7Y9cdqZ8PtGHXEHZoe5vZxkO6JRCFAWau2+ipPc/ea2kwf7riqeABr7CLVZfRKx
65x3UYuh18zIpI6xSg0V1gVQswccaKq/o7Jim9duf6rkl2p7QcCEPnpUhXHMfRkg
nQopfhwjACyylZiD9dgFOo4JI/7uUoP+cjOSAXNoDy5rVq9CkE0kUoZxkL4z4OdF
j7vhKIHosTvFfMURS2Xlcm1N50TPvkFcTqRTR1mhh+rd+G8F25CMCB8m5nxMD4Rj
PZ4yLiahlFyc4t8KXONFrh9Yg7qHQFIjbdYduv8KOQkv89rCrgg=
=P/D+
-END PGP SIGNATURE-

-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org

Re: Tomcat 8.0.33 Crashed

[Mark Thomas]

On 25/10/2017 13:08, Atanu Pal wrote:
> Still can you help me explain little more about the root cause?
> 
> That will help me to explain it to customer and setup a test
> environment for future.
> Thanks and Regards
> Atanu Pal
> E2Infosystems - Chennai

Difficult to say but a possible sequence of events is something like:

- thread A adds socket S to poller
- thread B closes socket S
- poller tries of access socket S -> crash

Mark

-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org

Re: Tomcat 8.0.33 Crashed

[Atanu Pal]

Still can you help me explain little more about the root cause?

That will help me to explain it to customer and setup a test
environment for future.
Thanks and Regards
Atanu Pal
E2Infosystems - Chennai


On Wed, Oct 25, 2017 at 5:36 PM, Atanu Pal  wrote:
> Thanks Mark, I was inclining to the same, I will try 8.0.47 first
> before jumping to 8.5.x, will let you know what happens.
> Thanks and Regards
> Atanu Pal
> E2Infosystems - Chennai
>
>
> On Wed, Oct 25, 2017 at 5:30 PM, Mark Thomas  wrote:
>> On 25/10/17 11:43, Atanu Pal wrote:
>>> Hi,
>>>
>>> We have application deployed on tomcat & its working good for most of
>>> the customer, but recently we installed it for two customers on a
>>> virtual machine & it looks like tomcat is crashing after few days of
>>> successful run.
>>>
>>> Machine Details:
>>> - It is a Virtual Machine
>>> - Windows Server 2012 R2
>>> - 64-bit, 8GB RAM
>>> - Intel Xeon E5-2665 dual processor @2.4GHz
>>>
>>> Both the server crashed with almost exact same crash log.
>>>
>>> As per my google research & if I am reading this crash log right, then
>>> this issue happened due to APR Module while doing polling.
>>>
>>> We are trying to find root cause of this problem, these are my top suspect:
>>> 1. Its a VM machine thing.
>>> 2. Bug in tomcat APR module.
>>
>> 2 is the more likely option.
>>
>> There have been a number of APR/native bugs that are very timing
>> dependent. It is quite possible that the move the a VM altering the
>> timing slightly and did so in such a way that you hit a APR/native bug.
>>
>> I'd recommend updating to the latest 8.0.x release. Better still, the
>> latest 8.5.x. (8.5.x should be a drop in replacement for 8.0.x for
>> nearly all users). Also make sure you are using the latest Tomcat native
>> 1.2.x library.
>>
>> Mark
>>
>>
>>>
>>> Let me know how should we fix this issue & if its already fixed please
>>> guide to fixed version.
>>>
>>> I have posted same question on stackoverflow :
>>> https://stackoverflow.com/questions/46925905/tomcat-8-0-33-crashed-due-to-apr-in-a-virtual-machine
>>>
>>> Copying 1 Crash log below, I have one more very similar to this & two
>>> memory dump. let me know if you need any of those to help me out here.
>>> Memory dump is quite big so I can upload it on Google-drive & share it
>>> with you.
>>>
>>> Crash Log 1:
>>> 
>>> #
>>> # A fatal error has been detected by the Java Runtime Environment:
>>> #
>>> #  EXCEPTION_ACCESS_VIOLATION (0xc005) at pc=0x23a56eb6, pid=3084, 
>>> tid=5852
>>> #
>>> # JRE version: Java(TM) SE Runtime Environment (8.0_77-b03) (build 
>>> 1.8.0_77-b03)
>>> # Java VM: Java HotSpot(TM) Server VM (25.77-b03 mixed mode windows-x86 )
>>> # Problematic frame:
>>> # C  [tcnative-1.dll+0x6eb6]
>>> #
>>> # Core dump written. Default location: <>
>>> #
>>> # If you would like to submit a bug report, please visit:
>>> #   http://bugreport.java.com/bugreport/crash.jsp
>>> # The crash happened outside the Java Virtual Machine in native code.
>>> # See problematic frame for where to report the bug.
>>> #
>>>
>>> ---  T H R E A D  ---
>>>
>>> Current thread (0x24c7ac00):  JavaThread "http-apr-8082-Poller" daemon
>>> [_thread_in_native, id=5852, stack(0x2b06,0x2b0d)]
>>>
>>> siginfo: ExceptionCode=0xc005, reading address 0x3941f0b4
>>>
>>> Registers:
>>> EAX=0x3941f098, EBX=0x23be18e0, ECX=0x23be27b0, EDX=0x
>>> ESP=0x2b0cfac4, EBP=0x2b0cfae8, ESI=0x3941f098, EDI=0x
>>> EIP=0x23a56eb6, EFLAGS=0x00010202
>>>
>>> Top of Stack: (sp=0x2b0cfac4)
>>> 0x2b0cfac4:   24c7ac00 2b0cfb18  00fa
>>> 0x2b0cfad4:    838e735e 00055c23 23bdc7a8
>>> 0x2b0cfae4:   00db 2b0cfb28 014aad14 24c7ad40
>>> 0x2b0cfaf4:   2b0cfb18 23be18e0  00fa
>>> 0x2b0cfb04:    2b0cfb10 0001 11d94a90
>>> 0x2b0cfb14:   2710 11e05348 0400 f8ce8c91
>>> 0x2b0cfb24:   01d34b41 00fa 021c140c 23be18e0
>>> 0x2b0cfb34:    00fa  11d8e430
>>>
>>> Instructions: (pc=0x23a56eb6)
>>> 0x23a56e96:   41 0a 8b 71 10 8b 4b 10 99 89 04 0f 89 54 0f 04
>>> 0x23a56ea6:   8b 4b 10 8b c6 99 89 44 0f 08 89 54 0f 0c 74 59
>>> 0x23a56eb6:   83 7e 1c 00 74 5f 8b 45 f8 8b 4b 0c 50 51 e8 37
>>> 0x23a56ec6:   53 01 00 8b 56 1c 8b 02 8b 4e 1c 8b 51 04 89 02
>>>
>>>
>>> Register to memory mapping:
>>>
>>> EAX=0x3941f098 is an unknown value
>>> EBX=0x23be18e0 is an unknown value
>>> ECX=0x23be27b0 is an unknown value
>>> EDX=0x is an unknown value
>>> ESP=0x2b0cfac4 is pointing into the stack for thread: 0x24c7ac00
>>> EBP=0x2b0cfae8 is pointing into the stack for thread: 0x24c7ac00
>>> ESI=0x3941f098 is an unknown value
>>> EDI=0x is an unknown value
>>>
>>>
>>> Stack: [0x2b06,0x2b0d],  sp=0x2b0cfac4,  free space=446k
>>> Native frames: (J=compiled Java code, j=interpreted, Vv=VM code, C=native 
>>> code)
>>> C  

Re: Tomcat 8.0.33 Crashed

[Atanu Pal]

Thanks Mark, I was inclining to the same, I will try 8.0.47 first
before jumping to 8.5.x, will let you know what happens.
Thanks and Regards
Atanu Pal
E2Infosystems - Chennai


On Wed, Oct 25, 2017 at 5:30 PM, Mark Thomas  wrote:
> On 25/10/17 11:43, Atanu Pal wrote:
>> Hi,
>>
>> We have application deployed on tomcat & its working good for most of
>> the customer, but recently we installed it for two customers on a
>> virtual machine & it looks like tomcat is crashing after few days of
>> successful run.
>>
>> Machine Details:
>> - It is a Virtual Machine
>> - Windows Server 2012 R2
>> - 64-bit, 8GB RAM
>> - Intel Xeon E5-2665 dual processor @2.4GHz
>>
>> Both the server crashed with almost exact same crash log.
>>
>> As per my google research & if I am reading this crash log right, then
>> this issue happened due to APR Module while doing polling.
>>
>> We are trying to find root cause of this problem, these are my top suspect:
>> 1. Its a VM machine thing.
>> 2. Bug in tomcat APR module.
>
> 2 is the more likely option.
>
> There have been a number of APR/native bugs that are very timing
> dependent. It is quite possible that the move the a VM altering the
> timing slightly and did so in such a way that you hit a APR/native bug.
>
> I'd recommend updating to the latest 8.0.x release. Better still, the
> latest 8.5.x. (8.5.x should be a drop in replacement for 8.0.x for
> nearly all users). Also make sure you are using the latest Tomcat native
> 1.2.x library.
>
> Mark
>
>
>>
>> Let me know how should we fix this issue & if its already fixed please
>> guide to fixed version.
>>
>> I have posted same question on stackoverflow :
>> https://stackoverflow.com/questions/46925905/tomcat-8-0-33-crashed-due-to-apr-in-a-virtual-machine
>>
>> Copying 1 Crash log below, I have one more very similar to this & two
>> memory dump. let me know if you need any of those to help me out here.
>> Memory dump is quite big so I can upload it on Google-drive & share it
>> with you.
>>
>> Crash Log 1:
>> 
>> #
>> # A fatal error has been detected by the Java Runtime Environment:
>> #
>> #  EXCEPTION_ACCESS_VIOLATION (0xc005) at pc=0x23a56eb6, pid=3084, 
>> tid=5852
>> #
>> # JRE version: Java(TM) SE Runtime Environment (8.0_77-b03) (build 
>> 1.8.0_77-b03)
>> # Java VM: Java HotSpot(TM) Server VM (25.77-b03 mixed mode windows-x86 )
>> # Problematic frame:
>> # C  [tcnative-1.dll+0x6eb6]
>> #
>> # Core dump written. Default location: <>
>> #
>> # If you would like to submit a bug report, please visit:
>> #   http://bugreport.java.com/bugreport/crash.jsp
>> # The crash happened outside the Java Virtual Machine in native code.
>> # See problematic frame for where to report the bug.
>> #
>>
>> ---  T H R E A D  ---
>>
>> Current thread (0x24c7ac00):  JavaThread "http-apr-8082-Poller" daemon
>> [_thread_in_native, id=5852, stack(0x2b06,0x2b0d)]
>>
>> siginfo: ExceptionCode=0xc005, reading address 0x3941f0b4
>>
>> Registers:
>> EAX=0x3941f098, EBX=0x23be18e0, ECX=0x23be27b0, EDX=0x
>> ESP=0x2b0cfac4, EBP=0x2b0cfae8, ESI=0x3941f098, EDI=0x
>> EIP=0x23a56eb6, EFLAGS=0x00010202
>>
>> Top of Stack: (sp=0x2b0cfac4)
>> 0x2b0cfac4:   24c7ac00 2b0cfb18  00fa
>> 0x2b0cfad4:    838e735e 00055c23 23bdc7a8
>> 0x2b0cfae4:   00db 2b0cfb28 014aad14 24c7ad40
>> 0x2b0cfaf4:   2b0cfb18 23be18e0  00fa
>> 0x2b0cfb04:    2b0cfb10 0001 11d94a90
>> 0x2b0cfb14:   2710 11e05348 0400 f8ce8c91
>> 0x2b0cfb24:   01d34b41 00fa 021c140c 23be18e0
>> 0x2b0cfb34:    00fa  11d8e430
>>
>> Instructions: (pc=0x23a56eb6)
>> 0x23a56e96:   41 0a 8b 71 10 8b 4b 10 99 89 04 0f 89 54 0f 04
>> 0x23a56ea6:   8b 4b 10 8b c6 99 89 44 0f 08 89 54 0f 0c 74 59
>> 0x23a56eb6:   83 7e 1c 00 74 5f 8b 45 f8 8b 4b 0c 50 51 e8 37
>> 0x23a56ec6:   53 01 00 8b 56 1c 8b 02 8b 4e 1c 8b 51 04 89 02
>>
>>
>> Register to memory mapping:
>>
>> EAX=0x3941f098 is an unknown value
>> EBX=0x23be18e0 is an unknown value
>> ECX=0x23be27b0 is an unknown value
>> EDX=0x is an unknown value
>> ESP=0x2b0cfac4 is pointing into the stack for thread: 0x24c7ac00
>> EBP=0x2b0cfae8 is pointing into the stack for thread: 0x24c7ac00
>> ESI=0x3941f098 is an unknown value
>> EDI=0x is an unknown value
>>
>>
>> Stack: [0x2b06,0x2b0d],  sp=0x2b0cfac4,  free space=446k
>> Native frames: (J=compiled Java code, j=interpreted, Vv=VM code, C=native 
>> code)
>> C  [tcnative-1.dll+0x6eb6]
>> J 6544  org.apache.tomcat.jni.Poll.poll(JJ[JZ)I (0 bytes) @ 0x014aad14
>> [0x014aac60+0xb4]
>> J 8272% C2 org.apache.tomcat.util.net.AprEndpoint$Poller.run()V (2267
>> bytes) @ 0x021c140c [0x021c11c0+0x24c]
>> j  java.lang.Thread.run()V+11
>> v  ~StubRoutines::call_stub
>> V  [jvm.dll+0x16db95]
>> V  [jvm.dll+0x24047e]
>> V  [jvm.dll+0x16dc2e]
>> V  [jvm.dll+0x16ddb6]
>> V  [jvm.dll+0x16de27]
>> V  

Re: Tomcat 8.0.33 Crashed

[Mark Thomas]

On 25/10/17 11:43, Atanu Pal wrote:
> Hi,
> 
> We have application deployed on tomcat & its working good for most of
> the customer, but recently we installed it for two customers on a
> virtual machine & it looks like tomcat is crashing after few days of
> successful run.
> 
> Machine Details:
> - It is a Virtual Machine
> - Windows Server 2012 R2
> - 64-bit, 8GB RAM
> - Intel Xeon E5-2665 dual processor @2.4GHz
> 
> Both the server crashed with almost exact same crash log.
> 
> As per my google research & if I am reading this crash log right, then
> this issue happened due to APR Module while doing polling.
> 
> We are trying to find root cause of this problem, these are my top suspect:
> 1. Its a VM machine thing.
> 2. Bug in tomcat APR module.

2 is the more likely option.

There have been a number of APR/native bugs that are very timing
dependent. It is quite possible that the move the a VM altering the
timing slightly and did so in such a way that you hit a APR/native bug.

I'd recommend updating to the latest 8.0.x release. Better still, the
latest 8.5.x. (8.5.x should be a drop in replacement for 8.0.x for
nearly all users). Also make sure you are using the latest Tomcat native
1.2.x library.

Mark


> 
> Let me know how should we fix this issue & if its already fixed please
> guide to fixed version.
> 
> I have posted same question on stackoverflow :
> https://stackoverflow.com/questions/46925905/tomcat-8-0-33-crashed-due-to-apr-in-a-virtual-machine
> 
> Copying 1 Crash log below, I have one more very similar to this & two
> memory dump. let me know if you need any of those to help me out here.
> Memory dump is quite big so I can upload it on Google-drive & share it
> with you.
> 
> Crash Log 1:
> 
> #
> # A fatal error has been detected by the Java Runtime Environment:
> #
> #  EXCEPTION_ACCESS_VIOLATION (0xc005) at pc=0x23a56eb6, pid=3084, 
> tid=5852
> #
> # JRE version: Java(TM) SE Runtime Environment (8.0_77-b03) (build 
> 1.8.0_77-b03)
> # Java VM: Java HotSpot(TM) Server VM (25.77-b03 mixed mode windows-x86 )
> # Problematic frame:
> # C  [tcnative-1.dll+0x6eb6]
> #
> # Core dump written. Default location: <>
> #
> # If you would like to submit a bug report, please visit:
> #   http://bugreport.java.com/bugreport/crash.jsp
> # The crash happened outside the Java Virtual Machine in native code.
> # See problematic frame for where to report the bug.
> #
> 
> ---  T H R E A D  ---
> 
> Current thread (0x24c7ac00):  JavaThread "http-apr-8082-Poller" daemon
> [_thread_in_native, id=5852, stack(0x2b06,0x2b0d)]
> 
> siginfo: ExceptionCode=0xc005, reading address 0x3941f0b4
> 
> Registers:
> EAX=0x3941f098, EBX=0x23be18e0, ECX=0x23be27b0, EDX=0x
> ESP=0x2b0cfac4, EBP=0x2b0cfae8, ESI=0x3941f098, EDI=0x
> EIP=0x23a56eb6, EFLAGS=0x00010202
> 
> Top of Stack: (sp=0x2b0cfac4)
> 0x2b0cfac4:   24c7ac00 2b0cfb18  00fa
> 0x2b0cfad4:    838e735e 00055c23 23bdc7a8
> 0x2b0cfae4:   00db 2b0cfb28 014aad14 24c7ad40
> 0x2b0cfaf4:   2b0cfb18 23be18e0  00fa
> 0x2b0cfb04:    2b0cfb10 0001 11d94a90
> 0x2b0cfb14:   2710 11e05348 0400 f8ce8c91
> 0x2b0cfb24:   01d34b41 00fa 021c140c 23be18e0
> 0x2b0cfb34:    00fa  11d8e430
> 
> Instructions: (pc=0x23a56eb6)
> 0x23a56e96:   41 0a 8b 71 10 8b 4b 10 99 89 04 0f 89 54 0f 04
> 0x23a56ea6:   8b 4b 10 8b c6 99 89 44 0f 08 89 54 0f 0c 74 59
> 0x23a56eb6:   83 7e 1c 00 74 5f 8b 45 f8 8b 4b 0c 50 51 e8 37
> 0x23a56ec6:   53 01 00 8b 56 1c 8b 02 8b 4e 1c 8b 51 04 89 02
> 
> 
> Register to memory mapping:
> 
> EAX=0x3941f098 is an unknown value
> EBX=0x23be18e0 is an unknown value
> ECX=0x23be27b0 is an unknown value
> EDX=0x is an unknown value
> ESP=0x2b0cfac4 is pointing into the stack for thread: 0x24c7ac00
> EBP=0x2b0cfae8 is pointing into the stack for thread: 0x24c7ac00
> ESI=0x3941f098 is an unknown value
> EDI=0x is an unknown value
> 
> 
> Stack: [0x2b06,0x2b0d],  sp=0x2b0cfac4,  free space=446k
> Native frames: (J=compiled Java code, j=interpreted, Vv=VM code, C=native 
> code)
> C  [tcnative-1.dll+0x6eb6]
> J 6544  org.apache.tomcat.jni.Poll.poll(JJ[JZ)I (0 bytes) @ 0x014aad14
> [0x014aac60+0xb4]
> J 8272% C2 org.apache.tomcat.util.net.AprEndpoint$Poller.run()V (2267
> bytes) @ 0x021c140c [0x021c11c0+0x24c]
> j  java.lang.Thread.run()V+11
> v  ~StubRoutines::call_stub
> V  [jvm.dll+0x16db95]
> V  [jvm.dll+0x24047e]
> V  [jvm.dll+0x16dc2e]
> V  [jvm.dll+0x16ddb6]
> V  [jvm.dll+0x16de27]
> V  [jvm.dll+0x10a73f]
> V  [jvm.dll+0x192a7c]
> V  [jvm.dll+0x192b6a]
> V  [jvm.dll+0x1d7356]
> C  [MSVCR100.dll+0x5c556]
> C  [MSVCR100.dll+0x5c600]
> C  [KERNEL32.DLL+0x17c04]
> C  [ntdll.dll+0x5ad2f]
> C  [ntdll.dll+0x5acfa]
> C  0x
> 
> Java frames: (J=compiled Java code, j=interpreted, Vv=VM code)
> J 6544  org.apache.tomcat.jni.Poll.poll(JJ[JZ)I (0 bytes) @ 

Tomcat 8.0.33 Crashed

[Atanu Pal]

Hi,

We have application deployed on tomcat & its working good for most of
the customer, but recently we installed it for two customers on a
virtual machine & it looks like tomcat is crashing after few days of
successful run.

Machine Details:
- It is a Virtual Machine
- Windows Server 2012 R2
- 64-bit, 8GB RAM
- Intel Xeon E5-2665 dual processor @2.4GHz

Both the server crashed with almost exact same crash log.

As per my google research & if I am reading this crash log right, then
this issue happened due to APR Module while doing polling.

We are trying to find root cause of this problem, these are my top suspect:
1. Its a VM machine thing.
2. Bug in tomcat APR module.

Let me know how should we fix this issue & if its already fixed please
guide to fixed version.

I have posted same question on stackoverflow :
https://stackoverflow.com/questions/46925905/tomcat-8-0-33-crashed-due-to-apr-in-a-virtual-machine

Copying 1 Crash log below, I have one more very similar to this & two
memory dump. let me know if you need any of those to help me out here.
Memory dump is quite big so I can upload it on Google-drive & share it
with you.

Crash Log 1:

#
# A fatal error has been detected by the Java Runtime Environment:
#
#  EXCEPTION_ACCESS_VIOLATION (0xc005) at pc=0x23a56eb6, pid=3084, tid=5852
#
# JRE version: Java(TM) SE Runtime Environment (8.0_77-b03) (build 1.8.0_77-b03)
# Java VM: Java HotSpot(TM) Server VM (25.77-b03 mixed mode windows-x86 )
# Problematic frame:
# C  [tcnative-1.dll+0x6eb6]
#
# Core dump written. Default location: <>
#
# If you would like to submit a bug report, please visit:
#   http://bugreport.java.com/bugreport/crash.jsp
# The crash happened outside the Java Virtual Machine in native code.
# See problematic frame for where to report the bug.
#

---  T H R E A D  ---

Current thread (0x24c7ac00):  JavaThread "http-apr-8082-Poller" daemon
[_thread_in_native, id=5852, stack(0x2b06,0x2b0d)]

siginfo: ExceptionCode=0xc005, reading address 0x3941f0b4

Registers:
EAX=0x3941f098, EBX=0x23be18e0, ECX=0x23be27b0, EDX=0x
ESP=0x2b0cfac4, EBP=0x2b0cfae8, ESI=0x3941f098, EDI=0x
EIP=0x23a56eb6, EFLAGS=0x00010202

Top of Stack: (sp=0x2b0cfac4)
0x2b0cfac4:   24c7ac00 2b0cfb18  00fa
0x2b0cfad4:    838e735e 00055c23 23bdc7a8
0x2b0cfae4:   00db 2b0cfb28 014aad14 24c7ad40
0x2b0cfaf4:   2b0cfb18 23be18e0  00fa
0x2b0cfb04:    2b0cfb10 0001 11d94a90
0x2b0cfb14:   2710 11e05348 0400 f8ce8c91
0x2b0cfb24:   01d34b41 00fa 021c140c 23be18e0
0x2b0cfb34:    00fa  11d8e430

Instructions: (pc=0x23a56eb6)
0x23a56e96:   41 0a 8b 71 10 8b 4b 10 99 89 04 0f 89 54 0f 04
0x23a56ea6:   8b 4b 10 8b c6 99 89 44 0f 08 89 54 0f 0c 74 59
0x23a56eb6:   83 7e 1c 00 74 5f 8b 45 f8 8b 4b 0c 50 51 e8 37
0x23a56ec6:   53 01 00 8b 56 1c 8b 02 8b 4e 1c 8b 51 04 89 02


Register to memory mapping:

EAX=0x3941f098 is an unknown value
EBX=0x23be18e0 is an unknown value
ECX=0x23be27b0 is an unknown value
EDX=0x is an unknown value
ESP=0x2b0cfac4 is pointing into the stack for thread: 0x24c7ac00
EBP=0x2b0cfae8 is pointing into the stack for thread: 0x24c7ac00
ESI=0x3941f098 is an unknown value
EDI=0x is an unknown value


Stack: [0x2b06,0x2b0d],  sp=0x2b0cfac4,  free space=446k
Native frames: (J=compiled Java code, j=interpreted, Vv=VM code, C=native code)
C  [tcnative-1.dll+0x6eb6]
J 6544  org.apache.tomcat.jni.Poll.poll(JJ[JZ)I (0 bytes) @ 0x014aad14
[0x014aac60+0xb4]
J 8272% C2 org.apache.tomcat.util.net.AprEndpoint$Poller.run()V (2267
bytes) @ 0x021c140c [0x021c11c0+0x24c]
j  java.lang.Thread.run()V+11
v  ~StubRoutines::call_stub
V  [jvm.dll+0x16db95]
V  [jvm.dll+0x24047e]
V  [jvm.dll+0x16dc2e]
V  [jvm.dll+0x16ddb6]
V  [jvm.dll+0x16de27]
V  [jvm.dll+0x10a73f]
V  [jvm.dll+0x192a7c]
V  [jvm.dll+0x192b6a]
V  [jvm.dll+0x1d7356]
C  [MSVCR100.dll+0x5c556]
C  [MSVCR100.dll+0x5c600]
C  [KERNEL32.DLL+0x17c04]
C  [ntdll.dll+0x5ad2f]
C  [ntdll.dll+0x5acfa]
C  0x

Java frames: (J=compiled Java code, j=interpreted, Vv=VM code)
J 6544  org.apache.tomcat.jni.Poll.poll(JJ[JZ)I (0 bytes) @ 0x014aacd0
[0x014aac60+0x70]
J 8272% C2 org.apache.tomcat.util.net.AprEndpoint$Poller.run()V (2267
bytes) @ 0x021c140c [0x021c11c0+0x24c]
j  java.lang.Thread.run()V+11
v  ~StubRoutines::call_stub

---  P R O C E S S  ---

Java Threads: ( => current thread )
  0x2315c800 JavaThread "Thread-176703" daemon [_thread_blocked,
id=5448, stack(0x3d50,0x3d57)]
  0x2316 JavaThread "Keep-Alive-Timer" daemon [_thread_blocked,
id=5276, stack(0x3d45,0x3d4c)]
  0x2315fc00 JavaThread "http-apr-8082-exec-2223" daemon
[_thread_blocked, id=6724, stack(0x3d3a,0x3d41)]
  0x2315b400 JavaThread "http-apr-8082-exec-" daemon
[_thread_blocked, id=5016, stack(0x3d2f,0x3d36)]
  0x2315c000 JavaThread