Re: Apache Tomcat/7.0.39 crashed with fatal error

2017-01-27 Thread Coty Sutherland
> # Problematic frame:
> # J  
> org.apache.http.impl.cookie.BestMatchSpec.formatCookies(Ljava/util/List;)Ljava/util/List;

Generally a crash in a java frame is a JVM bug. A quick google search
of the problematic frame yields the following first result,
https://github.com/rholder/jvm-loop-unswitching-bug. Looking at the
java7 bug linked on the page
(http://bugs.java.com/view_bug.do?bug_id=8025398) and further details
from the bug that the fix was backported from
(http://bugs.java.com/view_bug.do?bug_id=8021898) shows evidence that
this is likely your issue. You'll need to update your JDK or use the
suggested workaround to prevent future occurrences.

On Thu, Jan 26, 2017 at 4:40 PM, Christopher Schultz
<ch...@christopherschultz.net> wrote:
> -BEGIN PGP SIGNED MESSAGE-
> Hash: SHA256
>
> Aurélien,
>
> On 1/26/17 4:31 PM, Aurélien Terrestris wrote:
>> maybe you're just sending cookies with non-compliant characters.
>> Please check what you're sending if you can reproduce this problem
>> yourself
>>
>> RFC 6265 says  :
>>
>> cookie-value  = *cookie-octet / ( DQUOTE *cookie-octet DQUOTE )
>> cookie-octet  = %x21 / %x23-2B / %x2D-3A / %x3C-5B / %x5D-7E ;
>> US-ASCII characters excluding CTLs, ; whitespace DQUOTE, comma,
>> semicolon, ; and backslash
>
> Even if the client is sending a malformed HTTP header (or cookie,
> specifically), it shouldn't crash the JVM.
>
> - -chris
>
>> 2017-01-26 22:22 GMT+01:00 Satish Chhatpar 02
>> <chhatp...@cpwplc.com>:
>>
>>> Yes all of them failed in the same way.
>>>
>>>
>>> # Problematic frame: # J
>>> org.apache.http.impl.cookie.BestMatchSpec.formatCookies(
>>> Ljava/util/List;)Ljava/util/List;
>>>
>>>
>>>
>>> Regards
>>>
>>> Satish Chhatpar
>>>
>>>
>>>  From: Christopher Schultz
>>> <ch...@christopherschultz.net> Sent: Friday, January 27, 2017
>>> 2:44:54 AM To: Tomcat Users List Subject: Re: Apache
>>> Tomcat/7.0.39 crashed with fatal error
>>>
>> Satish,
>>
>> On 1/26/17 3:42 PM, Satish Chhatpar 02 wrote:
>>>>> Thanks Chris. I appreciate your help.
>>>>>
>>>>> All 4 tomcats are on diff machines. One on each, with same
>>>>> tomcat version, same java version and same OS for all.
>>
>> Did they all fail in the same way (JVM crash @
>> org.apache.http.impl.cookie.BestMatchSpec.formatCookies)?
>>
>>>>> Tomcats are not in cluster.
>>
>> I would highly recommend upgrading the JVM on one of those servers
>> to 1.7.latest to see if everything still works. If things go well,
>>  upgrade all of them.
>>
>> Then deploy the 1.8.latest to one of them. Tomcat shouldn't have
>> any compatibility issues with Java 8, but you will definitely want
>> to test everything in your application of course.
>>
>> -chris
>>
>>>>>  From: Christopher Schultz
>>>>> <ch...@christopherschultz.net> Sent: Friday, January 27, 2017
>>>>> 1:52:47 AM To: Tomcat Users List Subject: Re: Apache
>>>>> Tomcat/7.0.39 crashed with fatal error
>>>>>
>>>>> Satish,
>>>>>
>>>>> On 1/26/17 2:28 PM, Satish Chhatpar 02 wrote:
>>>>>> we are using Apache Tomcat/7.0.39 for our java
>>>>>> application.
>>>>>
>>>>> I highly recommend an upgrade for both Tomcat and Java.
>>>>> There are published vulnerabilities for both product versions
>>>>> you are using.
>>>>>
>>>>>> There are 4 tomcat instances using same tomcat version and
>>>>>> java version. yesterday all 4 tomcats crashed with below
>>>>>> error in hs_err_pid log file.
>>>>>
>>>>> All on the same hardware? Or separate machines?
>>>>>
>>>>>> This log file was created for all 4 tomcats.
>>>>>
>>>>>> Its very peculiar behaviour that all 4 crashed around same
>>>>>> time.
>>>>>
>>>>> If they are in a cluster, one going down could cause the
>>>>> load on the others to go up, increasing the chances of a
>>>>> problem.
>>>>>
>>>>>> Any information can help us to mitigate this incident.
>>>>>
>>>>>> Apache Tomcat/7.0.39
>>>>>
>>>>> Unless this is a package-mana

Re: Apache Tomcat/7.0.39 crashed with fatal error

2017-01-26 Thread Christopher Schultz
-BEGIN PGP SIGNED MESSAGE-
Hash: SHA256

Aurélien,

On 1/26/17 4:31 PM, Aurélien Terrestris wrote:
> maybe you're just sending cookies with non-compliant characters. 
> Please check what you're sending if you can reproduce this problem 
> yourself
> 
> RFC 6265 says  :
> 
> cookie-value  = *cookie-octet / ( DQUOTE *cookie-octet DQUOTE )
> cookie-octet  = %x21 / %x23-2B / %x2D-3A / %x3C-5B / %x5D-7E ;
> US-ASCII characters excluding CTLs, ; whitespace DQUOTE, comma, 
> semicolon, ; and backslash

Even if the client is sending a malformed HTTP header (or cookie,
specifically), it shouldn't crash the JVM.

- -chris

> 2017-01-26 22:22 GMT+01:00 Satish Chhatpar 02 
> <chhatp...@cpwplc.com>:
> 
>> Yes all of them failed in the same way.
>> 
>> 
>> # Problematic frame: # J 
>> org.apache.http.impl.cookie.BestMatchSpec.formatCookies( 
>> Ljava/util/List;)Ljava/util/List;
>> 
>> 
>> 
>> Regards
>> 
>> Satish Chhatpar
>> 
>> 
>>  From: Christopher Schultz 
>> <ch...@christopherschultz.net> Sent: Friday, January 27, 2017 
>> 2:44:54 AM To: Tomcat Users List Subject: Re: Apache 
>> Tomcat/7.0.39 crashed with fatal error
>> 
> Satish,
> 
> On 1/26/17 3:42 PM, Satish Chhatpar 02 wrote:
>>>> Thanks Chris. I appreciate your help.
>>>> 
>>>> All 4 tomcats are on diff machines. One on each, with same 
>>>> tomcat version, same java version and same OS for all.
> 
> Did they all fail in the same way (JVM crash @ 
> org.apache.http.impl.cookie.BestMatchSpec.formatCookies)?
> 
>>>> Tomcats are not in cluster.
> 
> I would highly recommend upgrading the JVM on one of those servers 
> to 1.7.latest to see if everything still works. If things go well,
>  upgrade all of them.
> 
> Then deploy the 1.8.latest to one of them. Tomcat shouldn't have 
> any compatibility issues with Java 8, but you will definitely want 
> to test everything in your application of course.
> 
> -chris
> 
>>>>  From: Christopher Schultz 
>>>> <ch...@christopherschultz.net> Sent: Friday, January 27, 2017
>>>> 1:52:47 AM To: Tomcat Users List Subject: Re: Apache 
>>>> Tomcat/7.0.39 crashed with fatal error
>>>> 
>>>> Satish,
>>>> 
>>>> On 1/26/17 2:28 PM, Satish Chhatpar 02 wrote:
>>>>> we are using Apache Tomcat/7.0.39 for our java 
>>>>> application.
>>>> 
>>>> I highly recommend an upgrade for both Tomcat and Java.
>>>> There are published vulnerabilities for both product versions
>>>> you are using.
>>>> 
>>>>> There are 4 tomcat instances using same tomcat version and 
>>>>> java version. yesterday all 4 tomcats crashed with below 
>>>>> error in hs_err_pid log file.
>>>> 
>>>> All on the same hardware? Or separate machines?
>>>> 
>>>>> This log file was created for all 4 tomcats.
>>>> 
>>>>> Its very peculiar behaviour that all 4 crashed around same 
>>>>> time.
>>>> 
>>>> If they are in a cluster, one going down could cause the
>>>> load on the others to go up, increasing the chances of a
>>>> problem.
>>>> 
>>>>> Any information can help us to mitigate this incident.
>>>> 
>>>>> Apache Tomcat/7.0.39
>>>> 
>>>> Unless this is a package-managed version of Tomcat with an 
>>>> unfortunately inaccurate version number, that version of 
>>>> Tomcat is nearly 3 years old. The current version in the 
>>>> 7.0.x line is 7.0.75 (released yesterday).
>>>> 
>>>>> java version "1.7.0_21" Java(TM) SE Runtime Environment 
>>>>> (build 1.7.0_21-b11) Java HotSpot(TM) 64-Bit Server VM 
>>>>> (build 23.21-b01, mixed mode)
>>>> 
>>>> That version of Java is also nearly 3 years old. Latest 1.7 
>>>> build is 1.7.0_80 release nearly 3 years ago. Note that Java 
>>>> 7 is no longer supported unless you have a long-term support 
>>>> contract with Oracle, in which case the latest version is 
>>>> 1.7.0_131, released earlier this month.
>>>> 
>>>>> OS used
>>>> 
>>>> 
>>>>> Red Hat Enterprise Linux Server release 6.3 (Santiago)
>>>> 
>>>> Ouch! 5 years old!
>>>> 
>>>>> # # A fatal er

Re: Apache Tomcat/7.0.39 crashed with fatal error

2017-01-26 Thread Aurélien Terrestris
Hello

maybe you're just sending cookies with non-compliant characters. Please
check what you're sending if you can reproduce this problem yourself

RFC 6265 says  :

 cookie-value  = *cookie-octet / ( DQUOTE *cookie-octet DQUOTE )
 cookie-octet  = %x21 / %x23-2B / %x2D-3A / %x3C-5B / %x5D-7E
   ; US-ASCII characters excluding CTLs,
   ; whitespace DQUOTE, comma, semicolon,
   ; and backslash




2017-01-26 22:22 GMT+01:00 Satish Chhatpar 02 <chhatp...@cpwplc.com>:

> Yes all of them failed in the same way.
>
>
> # Problematic frame:
> # J  org.apache.http.impl.cookie.BestMatchSpec.formatCookies(
> Ljava/util/List;)Ljava/util/List;
>
>
>
> Regards
>
> Satish Chhatpar
>
>
> 
> From: Christopher Schultz <ch...@christopherschultz.net>
> Sent: Friday, January 27, 2017 2:44:54 AM
> To: Tomcat Users List
> Subject: Re: Apache Tomcat/7.0.39 crashed with fatal error
>
> -BEGIN PGP SIGNED MESSAGE-
> Hash: SHA256
>
> Satish,
>
> On 1/26/17 3:42 PM, Satish Chhatpar 02 wrote:
> > Thanks Chris. I appreciate your help.
> >
> > All 4 tomcats are on diff machines. One on each, with same tomcat
> > version, same java version and same OS for all.
>
> Did they all fail in the same way (JVM crash @
> org.apache.http.impl.cookie.BestMatchSpec.formatCookies)?
>
> > Tomcats are not in cluster.
>
> I would highly recommend upgrading the JVM on one of those servers to
> 1.7.latest to see if everything still works. If things go well,
> upgrade all of them.
>
> Then deploy the 1.8.latest to one of them. Tomcat shouldn't have any
> compatibility issues with Java 8, but you will definitely want to test
> everything in your application of course.
>
> - -chris
>
> > ____________ From: Christopher Schultz
> > <ch...@christopherschultz.net> Sent: Friday, January 27, 2017
> > 1:52:47 AM To: Tomcat Users List Subject: Re: Apache Tomcat/7.0.39
> > crashed with fatal error
> >
> > Satish,
> >
> > On 1/26/17 2:28 PM, Satish Chhatpar 02 wrote:
> >> we are using Apache Tomcat/7.0.39 for our java application.
> >
> > I highly recommend an upgrade for both Tomcat and Java. There are
> > published vulnerabilities for both product versions you are using.
> >
> >> There are 4 tomcat instances using same tomcat version and java
> >> version. yesterday all 4 tomcats crashed with below error in
> >> hs_err_pid log file.
> >
> > All on the same hardware? Or separate machines?
> >
> >> This log file was created for all 4 tomcats.
> >
> >> Its very peculiar behaviour that all 4 crashed around same time.
> >
> > If they are in a cluster, one going down could cause the load on
> > the others to go up, increasing the chances of a problem.
> >
> >> Any information can help us to mitigate this incident.
> >
> >> Apache Tomcat/7.0.39
> >
> > Unless this is a package-managed version of Tomcat with an
> > unfortunately inaccurate version number, that version of Tomcat is
> > nearly 3 years old. The current version in the 7.0.x line is
> > 7.0.75 (released yesterday).
> >
> >> java version "1.7.0_21" Java(TM) SE Runtime Environment (build
> >> 1.7.0_21-b11) Java HotSpot(TM) 64-Bit Server VM (build
> >> 23.21-b01, mixed mode)
> >
> > That version of Java is also nearly 3 years old. Latest 1.7 build
> > is 1.7.0_80 release nearly 3 years ago. Note that Java 7 is no
> > longer supported unless you have a long-term support contract with
> > Oracle, in which case the latest version is 1.7.0_131, released
> > earlier this month.
> >
> >> OS used
> >
> >
> >> Red Hat Enterprise Linux Server release 6.3 (Santiago)
> >
> > Ouch! 5 years old!
> >
> >> # # A fatal error has been detected by the Java Runtime
> >> Environment: # #  SIGSEGV (0xb) at pc=0x7fed24ecfe9e,
> >> pid=21352, tid=140656275650304 # # JRE version: 7.0_21-b11 #
> >> Java VM: Java HotSpot(TM) 64-Bit Server VM (23.21-b01 mixed mode
> >> linux-amd64 compressed oops) # Problematic frame: # J
> >> org.apache.http.impl.cookie.BestMatchSpec.formatCookies(Ljava/util/Li
> s
> >
> >>
> t;)Ljava/util/List;
> >
> >
> > #
> >> # Failed to write core dump. Core dumps have been disabled. To
> >> enable core dumping, try "ulimit -c unlimited" before starting
> >> Java again # # If you would like to submit a bug report, please
> >> vi

Re: Apache Tomcat/7.0.39 crashed with fatal error

2017-01-26 Thread Satish Chhatpar 02
Yes all of them failed in the same way.


# Problematic frame:
# J  
org.apache.http.impl.cookie.BestMatchSpec.formatCookies(Ljava/util/List;)Ljava/util/List;



Regards

Satish Chhatpar



From: Christopher Schultz <ch...@christopherschultz.net>
Sent: Friday, January 27, 2017 2:44:54 AM
To: Tomcat Users List
Subject: Re: Apache Tomcat/7.0.39 crashed with fatal error

-BEGIN PGP SIGNED MESSAGE-
Hash: SHA256

Satish,

On 1/26/17 3:42 PM, Satish Chhatpar 02 wrote:
> Thanks Chris. I appreciate your help.
>
> All 4 tomcats are on diff machines. One on each, with same tomcat
> version, same java version and same OS for all.

Did they all fail in the same way (JVM crash @
org.apache.http.impl.cookie.BestMatchSpec.formatCookies)?

> Tomcats are not in cluster.

I would highly recommend upgrading the JVM on one of those servers to
1.7.latest to see if everything still works. If things go well,
upgrade all of them.

Then deploy the 1.8.latest to one of them. Tomcat shouldn't have any
compatibility issues with Java 8, but you will definitely want to test
everything in your application of course.

- -chris

>  From: Christopher Schultz
> <ch...@christopherschultz.net> Sent: Friday, January 27, 2017
> 1:52:47 AM To: Tomcat Users List Subject: Re: Apache Tomcat/7.0.39
> crashed with fatal error
>
> Satish,
>
> On 1/26/17 2:28 PM, Satish Chhatpar 02 wrote:
>> we are using Apache Tomcat/7.0.39 for our java application.
>
> I highly recommend an upgrade for both Tomcat and Java. There are
> published vulnerabilities for both product versions you are using.
>
>> There are 4 tomcat instances using same tomcat version and java
>> version. yesterday all 4 tomcats crashed with below error in
>> hs_err_pid log file.
>
> All on the same hardware? Or separate machines?
>
>> This log file was created for all 4 tomcats.
>
>> Its very peculiar behaviour that all 4 crashed around same time.
>
> If they are in a cluster, one going down could cause the load on
> the others to go up, increasing the chances of a problem.
>
>> Any information can help us to mitigate this incident.
>
>> Apache Tomcat/7.0.39
>
> Unless this is a package-managed version of Tomcat with an
> unfortunately inaccurate version number, that version of Tomcat is
> nearly 3 years old. The current version in the 7.0.x line is
> 7.0.75 (released yesterday).
>
>> java version "1.7.0_21" Java(TM) SE Runtime Environment (build
>> 1.7.0_21-b11) Java HotSpot(TM) 64-Bit Server VM (build
>> 23.21-b01, mixed mode)
>
> That version of Java is also nearly 3 years old. Latest 1.7 build
> is 1.7.0_80 release nearly 3 years ago. Note that Java 7 is no
> longer supported unless you have a long-term support contract with
> Oracle, in which case the latest version is 1.7.0_131, released
> earlier this month.
>
>> OS used
>
>
>> Red Hat Enterprise Linux Server release 6.3 (Santiago)
>
> Ouch! 5 years old!
>
>> # # A fatal error has been detected by the Java Runtime
>> Environment: # #  SIGSEGV (0xb) at pc=0x7fed24ecfe9e,
>> pid=21352, tid=140656275650304 # # JRE version: 7.0_21-b11 #
>> Java VM: Java HotSpot(TM) 64-Bit Server VM (23.21-b01 mixed mode
>> linux-amd64 compressed oops) # Problematic frame: # J
>> org.apache.http.impl.cookie.BestMatchSpec.formatCookies(Ljava/util/Li
s
>
>>
t;)Ljava/util/List;
>
>
> #
>> # Failed to write core dump. Core dumps have been disabled. To
>> enable core dumping, try "ulimit -c unlimited" before starting
>> Java again # # If you would like to submit a bug report, please
>> visit: # http://bugreport.sun.com/bugreport/crash.jsp #
>
> This is either a JVM bug is a hardware error. Given that the OS if
> 5 years old, I'm guessing the hardware is at least that old. I'd
> expect 5-year old hardware to be fairly trustworthy, but it may not
> have been properly-tested before going into production.
>
> If it's all on a single piece of hardware (all 4 Tomcats), I'd
> blame the hardware and look for a speedy replacement
> (properly-tested this time). If it was on different machines, I'd
> suspect a JVM bug.
>
> -chris
>
> -
>
>
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
> For additional commands, e-mail: users-h...@tomcat.apache.org
>
> ::DISCLAIMER::
> __
__
>
>
Confidentiality Notice from Dixons Carphone plc (registered in England &
Wales No.07105905) of 1 Portal Way, London, W3 6RS ("Dixons Carp

Re: Apache Tomcat/7.0.39 crashed with fatal error

2017-01-26 Thread Christopher Schultz
-BEGIN PGP SIGNED MESSAGE-
Hash: SHA256

Satish,

On 1/26/17 3:42 PM, Satish Chhatpar 02 wrote:
> Thanks Chris. I appreciate your help.
> 
> All 4 tomcats are on diff machines. One on each, with same tomcat 
> version, same java version and same OS for all.

Did they all fail in the same way (JVM crash @
org.apache.http.impl.cookie.BestMatchSpec.formatCookies)?

> Tomcats are not in cluster.

I would highly recommend upgrading the JVM on one of those servers to
1.7.latest to see if everything still works. If things go well,
upgrade all of them.

Then deploy the 1.8.latest to one of them. Tomcat shouldn't have any
compatibility issues with Java 8, but you will definitely want to test
everything in your application of course.

- -chris

>  From: Christopher Schultz
> <ch...@christopherschultz.net> Sent: Friday, January 27, 2017
> 1:52:47 AM To: Tomcat Users List Subject: Re: Apache Tomcat/7.0.39
> crashed with fatal error
> 
> Satish,
> 
> On 1/26/17 2:28 PM, Satish Chhatpar 02 wrote:
>> we are using Apache Tomcat/7.0.39 for our java application.
> 
> I highly recommend an upgrade for both Tomcat and Java. There are 
> published vulnerabilities for both product versions you are using.
> 
>> There are 4 tomcat instances using same tomcat version and java 
>> version. yesterday all 4 tomcats crashed with below error in 
>> hs_err_pid log file.
> 
> All on the same hardware? Or separate machines?
> 
>> This log file was created for all 4 tomcats.
> 
>> Its very peculiar behaviour that all 4 crashed around same time.
> 
> If they are in a cluster, one going down could cause the load on
> the others to go up, increasing the chances of a problem.
> 
>> Any information can help us to mitigate this incident.
> 
>> Apache Tomcat/7.0.39
> 
> Unless this is a package-managed version of Tomcat with an 
> unfortunately inaccurate version number, that version of Tomcat is 
> nearly 3 years old. The current version in the 7.0.x line is
> 7.0.75 (released yesterday).
> 
>> java version "1.7.0_21" Java(TM) SE Runtime Environment (build 
>> 1.7.0_21-b11) Java HotSpot(TM) 64-Bit Server VM (build
>> 23.21-b01, mixed mode)
> 
> That version of Java is also nearly 3 years old. Latest 1.7 build
> is 1.7.0_80 release nearly 3 years ago. Note that Java 7 is no
> longer supported unless you have a long-term support contract with
> Oracle, in which case the latest version is 1.7.0_131, released
> earlier this month.
> 
>> OS used
> 
> 
>> Red Hat Enterprise Linux Server release 6.3 (Santiago)
> 
> Ouch! 5 years old!
> 
>> # # A fatal error has been detected by the Java Runtime 
>> Environment: # #  SIGSEGV (0xb) at pc=0x7fed24ecfe9e, 
>> pid=21352, tid=140656275650304 # # JRE version: 7.0_21-b11 #
>> Java VM: Java HotSpot(TM) 64-Bit Server VM (23.21-b01 mixed mode 
>> linux-amd64 compressed oops) # Problematic frame: # J 
>> org.apache.http.impl.cookie.BestMatchSpec.formatCookies(Ljava/util/Li
s
>
>> 
t;)Ljava/util/List;
> 
> 
> #
>> # Failed to write core dump. Core dumps have been disabled. To 
>> enable core dumping, try "ulimit -c unlimited" before starting
>> Java again # # If you would like to submit a bug report, please
>> visit: # http://bugreport.sun.com/bugreport/crash.jsp #
> 
> This is either a JVM bug is a hardware error. Given that the OS if
> 5 years old, I'm guessing the hardware is at least that old. I'd
> expect 5-year old hardware to be fairly trustworthy, but it may not
> have been properly-tested before going into production.
> 
> If it's all on a single piece of hardware (all 4 Tomcats), I'd
> blame the hardware and look for a speedy replacement
> (properly-tested this time). If it was on different machines, I'd
> suspect a JVM bug.
> 
> -chris
> 
> -
>
> 
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
> For additional commands, e-mail: users-h...@tomcat.apache.org
> 
> ::DISCLAIMER:: 
> __
__
>
> 
Confidentiality Notice from Dixons Carphone plc (registered in England &
Wales No.07105905) of 1 Portal Way, London, W3 6RS ("Dixons Carphone").
The information contained in this e-mail and any attachments may be
legally privileged, proprietary and/or confidential. If you received
this e-mail in error, please notify the sender by return, permanently
delete the e-mail and destroy all hard copies immediately. No warranty
is made as to the completeness or accuracy of the information contained
in 

Re: Apache Tomcat/7.0.39 crashed with fatal error

2017-01-26 Thread Satish Chhatpar 02
Thanks Chris. I appreciate your help.


All 4 tomcats are on diff machines. One on each, with same tomcat version, same 
java version and same OS for all.


Tomcats are not in cluster.


Regards

Satish Chhatpar



From: Christopher Schultz <ch...@christopherschultz.net>
Sent: Friday, January 27, 2017 1:52:47 AM
To: Tomcat Users List
Subject: Re: Apache Tomcat/7.0.39 crashed with fatal error

-BEGIN PGP SIGNED MESSAGE-
Hash: SHA256

Satish,

On 1/26/17 2:28 PM, Satish Chhatpar 02 wrote:
> we are using Apache Tomcat/7.0.39 for our java application.

I highly recommend an upgrade for both Tomcat and Java. There are
published vulnerabilities for both product versions you are using.

> There are 4 tomcat instances using same tomcat version and java
> version. yesterday all 4 tomcats crashed with below error in
> hs_err_pid log file.

All on the same hardware? Or separate machines?

> This log file was created for all 4 tomcats.
>
> Its very peculiar behaviour that all 4 crashed around same time.

If they are in a cluster, one going down could cause the load on the
others to go up, increasing the chances of a problem.

> Any information can help us to mitigate this incident.
>
> Apache Tomcat/7.0.39

Unless this is a package-managed version of Tomcat with an
unfortunately inaccurate version number, that version of Tomcat is
nearly 3 years old. The current version in the 7.0.x line is 7.0.75
(released yesterday).

> java version "1.7.0_21" Java(TM) SE Runtime Environment (build
> 1.7.0_21-b11) Java HotSpot(TM) 64-Bit Server VM (build 23.21-b01,
> mixed mode)

That version of Java is also nearly 3 years old. Latest 1.7 build is
1.7.0_80 release nearly 3 years ago. Note that Java 7 is no longer
supported unless you have a long-term support contract with Oracle, in
which case the latest version is 1.7.0_131, released earlier this month.

> OS used
>
>
> Red Hat Enterprise Linux Server release 6.3 (Santiago)

Ouch! 5 years old!

> # # A fatal error has been detected by the Java Runtime
> Environment: # #  SIGSEGV (0xb) at pc=0x7fed24ecfe9e,
> pid=21352, tid=140656275650304 # # JRE version: 7.0_21-b11 # Java
> VM: Java HotSpot(TM) 64-Bit Server VM (23.21-b01 mixed mode
> linux-amd64 compressed oops) # Problematic frame: # J
> org.apache.http.impl.cookie.BestMatchSpec.formatCookies(Ljava/util/Lis
t;)Ljava/util/List;
>
>
#
> # Failed to write core dump. Core dumps have been disabled. To
> enable core dumping, try "ulimit -c unlimited" before starting Java
> again # # If you would like to submit a bug report, please visit: #
> http://bugreport.sun.com/bugreport/crash.jsp #

This is either a JVM bug is a hardware error. Given that the OS if 5
years old, I'm guessing the hardware is at least that old. I'd expect
5-year old hardware to be fairly trustworthy, but it may not have been
properly-tested before going into production.

If it's all on a single piece of hardware (all 4 Tomcats), I'd blame
the hardware and look for a speedy replacement (properly-tested this
time). If it was on different machines, I'd suspect a JVM bug.

- -chris
-BEGIN PGP SIGNATURE-
Comment: GPGTools - http://gpgtools.org
Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/

iQIcBAEBCAAGBQJYilqXAAoJEBzwKT+lPKRYk8gQAJxyGn+KWUb6V0fzWdiF0gzi
xh5p9tjmszS7ixEkMNZmkJH5cwWL8SjrXy0bO3s/50bfzEAOIKnqii+AEBWvKSsw
W/zmh/OFJzONOPhSWFvzkI1q9pND+gwRqyIVIkHYMs/l0Vi8GeqoWwJHjaIfSRtG
ZW9mDg1cH6p8/QrEynTU/B9XzNxKwIxg7YdUu97/uYPXVICDoRyF6Z8y+ISPMSpF
6OT36NDpdQXTL3xWnYM9149HE37dXjAiz1T+ms0FvP4UKLWN8hy58OdUnjW9EQpj
5IpsruGyRKsAKZzUfwWS68t4d2pq8/QUSvi+t23ZVnOkvdUy99thJ+rIUigb2942
NN6mMzpexOxkq36S7DyJrfXUVWdseeoYgEBFz+9mso2TaTf+9RD4JgLIPXjJ2LZp
CvUuHZ1G45uP62qgEsek4LQdPQ8urOzPuDvW578JleRXxE0w80eNPlnFfjb5mFmz
TrjGMWZE9ErnfFu6WB0ef5aNpGC5vh3TMXvT5NCVZvqRz/cBP4cXape0CZZdxxkf
/lfXqOWb0fBGrzS6BoeI44nyL+Nyqei9UZDl8RZaWjah1aGE9bGYSuvvhWXhgorb
Fs/4yvllgIpjRx9gDVcreE0oUFtPYh+FWLSEBYETjEFZCIZVsAGUNNDKa4kDDtXK
pxsK5m9udi1kAN+sqBzC
=lACD
-END PGP SIGNATURE-

-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org

::DISCLAIMER::

Confidentiality Notice from Dixons Carphone plc (registered in England & Wales 
No.07105905) of 1 Portal Way, London, W3 6RS ("Dixons Carphone"). The 
information contained in this e-mail and any attachments may be legally 
privileged, proprietary and/or confidential. If you received this e-mail in 
error, please notify the sender by return, permanently delete the e-mail and 
destroy all hard copies immediately. No warranty is made as to the completeness 
or accuracy of the information contained in this e-mail. Opin

Re: Apache Tomcat/7.0.39 crashed with fatal error

2017-01-26 Thread Christopher Schultz
-BEGIN PGP SIGNED MESSAGE-
Hash: SHA256

Satish,

On 1/26/17 2:28 PM, Satish Chhatpar 02 wrote:
> we are using Apache Tomcat/7.0.39 for our java application.

I highly recommend an upgrade for both Tomcat and Java. There are
published vulnerabilities for both product versions you are using.

> There are 4 tomcat instances using same tomcat version and java 
> version. yesterday all 4 tomcats crashed with below error in 
> hs_err_pid log file.

All on the same hardware? Or separate machines?

> This log file was created for all 4 tomcats.
> 
> Its very peculiar behaviour that all 4 crashed around same time.

If they are in a cluster, one going down could cause the load on the
others to go up, increasing the chances of a problem.

> Any information can help us to mitigate this incident.
> 
> Apache Tomcat/7.0.39

Unless this is a package-managed version of Tomcat with an
unfortunately inaccurate version number, that version of Tomcat is
nearly 3 years old. The current version in the 7.0.x line is 7.0.75
(released yesterday).

> java version "1.7.0_21" Java(TM) SE Runtime Environment (build
> 1.7.0_21-b11) Java HotSpot(TM) 64-Bit Server VM (build 23.21-b01,
> mixed mode)

That version of Java is also nearly 3 years old. Latest 1.7 build is
1.7.0_80 release nearly 3 years ago. Note that Java 7 is no longer
supported unless you have a long-term support contract with Oracle, in
which case the latest version is 1.7.0_131, released earlier this month.

> OS used
> 
> 
> Red Hat Enterprise Linux Server release 6.3 (Santiago)

Ouch! 5 years old!

> # # A fatal error has been detected by the Java Runtime
> Environment: # #  SIGSEGV (0xb) at pc=0x7fed24ecfe9e,
> pid=21352, tid=140656275650304 # # JRE version: 7.0_21-b11 # Java
> VM: Java HotSpot(TM) 64-Bit Server VM (23.21-b01 mixed mode
> linux-amd64 compressed oops) # Problematic frame: # J
> org.apache.http.impl.cookie.BestMatchSpec.formatCookies(Ljava/util/Lis
t;)Ljava/util/List;
>
> 
#
> # Failed to write core dump. Core dumps have been disabled. To
> enable core dumping, try "ulimit -c unlimited" before starting Java
> again # # If you would like to submit a bug report, please visit: #
> http://bugreport.sun.com/bugreport/crash.jsp #

This is either a JVM bug is a hardware error. Given that the OS if 5
years old, I'm guessing the hardware is at least that old. I'd expect
5-year old hardware to be fairly trustworthy, but it may not have been
properly-tested before going into production.

If it's all on a single piece of hardware (all 4 Tomcats), I'd blame
the hardware and look for a speedy replacement (properly-tested this
time). If it was on different machines, I'd suspect a JVM bug.

- -chris
-BEGIN PGP SIGNATURE-
Comment: GPGTools - http://gpgtools.org
Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/

iQIcBAEBCAAGBQJYilqXAAoJEBzwKT+lPKRYk8gQAJxyGn+KWUb6V0fzWdiF0gzi
xh5p9tjmszS7ixEkMNZmkJH5cwWL8SjrXy0bO3s/50bfzEAOIKnqii+AEBWvKSsw
W/zmh/OFJzONOPhSWFvzkI1q9pND+gwRqyIVIkHYMs/l0Vi8GeqoWwJHjaIfSRtG
ZW9mDg1cH6p8/QrEynTU/B9XzNxKwIxg7YdUu97/uYPXVICDoRyF6Z8y+ISPMSpF
6OT36NDpdQXTL3xWnYM9149HE37dXjAiz1T+ms0FvP4UKLWN8hy58OdUnjW9EQpj
5IpsruGyRKsAKZzUfwWS68t4d2pq8/QUSvi+t23ZVnOkvdUy99thJ+rIUigb2942
NN6mMzpexOxkq36S7DyJrfXUVWdseeoYgEBFz+9mso2TaTf+9RD4JgLIPXjJ2LZp
CvUuHZ1G45uP62qgEsek4LQdPQ8urOzPuDvW578JleRXxE0w80eNPlnFfjb5mFmz
TrjGMWZE9ErnfFu6WB0ef5aNpGC5vh3TMXvT5NCVZvqRz/cBP4cXape0CZZdxxkf
/lfXqOWb0fBGrzS6BoeI44nyL+Nyqei9UZDl8RZaWjah1aGE9bGYSuvvhWXhgorb
Fs/4yvllgIpjRx9gDVcreE0oUFtPYh+FWLSEBYETjEFZCIZVsAGUNNDKa4kDDtXK
pxsK5m9udi1kAN+sqBzC
=lACD
-END PGP SIGNATURE-

-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org



Apache Tomcat/7.0.39 crashed with fatal error

2017-01-26 Thread Satish Chhatpar 02
Hi,


we are using Apache Tomcat/7.0.39 for our java application. There are 4 tomcat 
instances using same tomcat version and java version. yesterday all 4 tomcats 
crashed with below error in hs_err_pid log file.

This log file was created for all 4 tomcats.

Its very peculiar behaviour that all 4 crashed around same time.


Any information can help us to mitigate this incident.




Apache Tomcat/7.0.39



java version "1.7.0_21"
Java(TM) SE Runtime Environment (build 1.7.0_21-b11)
Java HotSpot(TM) 64-Bit Server VM (build 23.21-b01, mixed mode)


OS used


Red Hat Enterprise Linux Server release 6.3 (Santiago)



#
# A fatal error has been detected by the Java Runtime Environment:
#
#  SIGSEGV (0xb) at pc=0x7fed24ecfe9e, pid=21352, tid=140656275650304
#
# JRE version: 7.0_21-b11
# Java VM: Java HotSpot(TM) 64-Bit Server VM (23.21-b01 mixed mode linux-amd64 
compressed oops)
# Problematic frame:
# J  
org.apache.http.impl.cookie.BestMatchSpec.formatCookies(Ljava/util/List;)Ljava/util/List;
#
# Failed to write core dump. Core dumps have been disabled. To enable core 
dumping, try "ulimit -c unlimited" before starting Java again
#
# If you would like to submit a bug report, please visit:
#   http://bugreport.sun.com/bugreport/crash.jsp
#

---  T H R E A D  ---

Current thread (0x7fecf832f000):  JavaThread "AD Thread-Metric Reporter0" 
daemon [_thread_in_Java, id=21367, stack(0x7fed17487000,0x7fed17588000)]

siginfo:si_signo=SIGSEGV: si_errno=0, si_code=1 (SEGV_MAPERR), 
si_addr=0x003f09daccec

Registers:
RAX=0x0007e14683e0, RBX=0x, RCX=0x003f09dacce0, 
RDX=0x0005
RSP=0x7fed17585ee0, RBP=0xeda7c0dd, RSI=0x0007e13b59a0, 
RDI=0x0007e13b599c
R8 =0x0004, R9 =0x0003, R10=0x, 
R11=0x0001
R12=0x, R13=0x000660a90c70, R14=0x, 
R15=0x7fecf832f000
RIP=0x7fed24ecfe9e, EFLAGS=0x00010206, CSGSFS=0x0033, 
ERR=0x0004
  TRAPNO=0x000e

Top of Stack: (sp=0x7fed17585ee0)
0x7fed17585ee0:   cc15218e0001 7fed000a
0x7fed17585ef0:   0007e11fec98 000660a90c58
0x7fed17585f00:   000660a90b90 0005
0x7fed17585f10:   0007e11fecd0 
0x7fed17585f20:   7fed17585f50 00072057de50
0x7fed17585f30:   7fed24abd218 000660a90b70
0x7fed17585f40:   00072057de50 7fed24abe508
0x7fed17585f50:   000660a90b90 000660a904a8
0x7fed17585f60:   0007220723e0 000660a90c58
0x7fed17585f70:   000660a90b90 000660a90bb0
0x7fed17585f80:   0007220723e0 000660a90ca8
0x7fed17585f90:   000660a90c58 000660a90b70
0x7fed17585fa0:   00040005 0006
0x7fed17585fb0:   000660a90bb0 0007220cc188
0x7fed17585fc0:   24188870 00072206f478
0x7fed17585fd0:   000660a90b90 0006602f73e8
0x7fed17585fe0:   0009 0007220723e0
0x7fed17585ff0:   0006 7fed24188870
0x7fed17586000:   000660a904a8 0007220723e0
0x7fed17586010:   0007220bea48 7fed0009
0x7fed17586020:   0007e0279348 0007eb43afd0
0x7fed17586030:   7fed175860a0 7fed23173806
0x7fed17586040:   7fed23173806 0007220723e0
0x7fed17586050:   000660a904a8 0007220d20f0
0x7fed17586060:   7fed17586060 0007e0a8092e
0x7fed17586070:   7fed17586108 0007e0a80db8
0x7fed17586080:   0007eb367008 0007e0a80ab0
0x7fed17586090:   7fed17586048 7fed175860e8
0x7fed175860a0:   7fed17586150 7fed231739e1
0x7fed175860b0:    
0x7fed175860c0:   00072206dfe8 
0x7fed175860d0:   000660a90878 0006602f7268

Instructions: (pc=0x7fed24ecfe9e)
0x7fed24ecfe7e:   85 04 00 00 00 48 89 46 28 41 0f 45 d9 45 8b cb
0x7fed24ecfe8e:   41 83 c1 02 49 63 cb 41 8b 6c 8d 14 49 8d 0c fc
0x7fed24ecfe9e:   8b 79 0c 41 3b fa 44 0f 4c d7 85 ed 0f 84 59 08
0x7fed24ecfeae:   00 00 41 8b 7c ec 08 81 ff 34 6b 27 fc 0f 85 b9

Register to memory mapping:

RAX=0x0007e14683e0 is an oop
{instance class}
 - klass: {other class}
RBX=0x is an unknown value
RCX=0x003f09dacce0 is an unknown value
RDX=0x0005 is an unknown value
RSP=0x7fed17585ee0 is pointing into the stack for thread: 0x7fecf832f000
RBP=0xeda7c0dd is an unknown value
RSI=0x0007e13b59a0 is an oop
{instance class}
 - klass: {other class}
RDI=0x0007e13b599c is an oop
[[I
 - klass: 'java/lang/Object'[]
 - length: 5
R8 =0x0004 is an unknown value
R9 =0x0003 is an unknown value
R10=0x is an unknown value
R11=0x0001 is an unknown value
R12=0x is an unknown value
R13=
[error occurred during error