Re: a lot of 502 error with using version 7.0.68-1

2016-03-10 Thread Kenichi MASUDA
Dear Christopher,

Thank you for your reply.

Our circumstances are so old, sorry. I want to renew them, but it seems to
be difficult for these errors for now.

>  This might not be the best list to ask about mod_proxy_http, but there
> are some people who probably have good experience with it here.

Thank for guiding to the mailing list and I'll post about my situation.


2016-03-11 10:49 GMT+09:00 Christopher Schultz :

> -BEGIN PGP SIGNED MESSAGE-
> Hash: SHA1
>
> Kenchi,
>
> On 3/10/16 8:25 PM, Kenichi MASUDA wrote:
> > Thank you for your reply and I'm sorry for less information.
> >
> >> how are you connecting httpd -> Tomcat?
> >>
> > A client sends a request to httpd which listens as 80, after that
> > httpd throw it like this to the tomcat which exists in backend and
> > serves as 8080 port with the rewrite rule.
> >
> > RewriteCond %{REQUEST_URI}
> > !\.(png|jpe?g|gif|css|js|xml|json|jsonp|ico|html?|swf|txt)$
> > RewriteRule ^/(.*)$ http://localhost:8080/core/$1 [P,L]
>
> So you are using mod_proxy_http.
>
> > The error is: httpd log say that "proxy: Error reading from remote
> > server returned by"
> >
> > and
> >
> > a client say that "502 proxy error. The proxy server received an
> > invalid response from an upstream server. The proxy server could
> > not handle the request POST /foo/bar"
> >
> > After having checked that error occurred, I've downgraded the
> > tomcat version to 7.0.39 (the version which I used to use), so that
> > the error disappeared.
>
> That is a very old Tomcat version indeed (more than 3 years old). You
> would be better upgrading to latest Tomcat 7.0.68 and figuring out
> what's wrong with the proxy.
>
> This might not be the best list to ask about mod_proxy_http, but there
> are some people who probably have good experience with it here.
>
> The best place to ask about mod_proxy_http would be on the httpd
> user's list:
> http://httpd.apache.org/lists.html#http-users
>
> - -chris
> -BEGIN PGP SIGNATURE-
> Comment: GPGTools - http://gpgtools.org
> Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/
>
> iEYEARECAAYFAlbiJEYACgkQ9CaO5/Lv0PA2KACfbdp4wR3rXQcJ82jEK/GWLls7
> /EUAoLpT6SzzPYoyZYF1Zh/481dnCvLt
> =hiN1
> -END PGP SIGNATURE-
>
> -
> To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
> For additional commands, e-mail: users-h...@tomcat.apache.org
>
>


-- 
mailto:masu...@gmail.com


RE: Understanding how to controlling what data is written to log4j appenders

2016-03-10 Thread Joleen Barker
This is great information to know. Our installations are on AIX boxes
however.

Joleen
On Mar 10, 2016 10:31 PM, "George Stanchev"  wrote:

> If you run tomcat via the windows server wrapper, you can
>
> "%TOMCAT_EXE%" //US//%TOMCAT_SERVICE_NAME% --StdOutput
> "%TOMCAT_CONSOLE_LOG%" --StdError "%TOMCAT_CONSOLE_LOG%"
>
> Which will redirect the stderr and stdoout to the corresponding log files
>
> George
>
> -Original Message-
> From: Joleen Barker [mailto:oldenuf2no...@gmail.com]
> Sent: Thursday, March 10, 2016 5:48 PM
> To: Tomcat Users List 
> Subject: Re: Understanding how to controlling what data is written to
> log4j appenders
>
> Thanks for the tips. I have to use the perl program for now to accomplish
> the task for the company but l'll continue to work this for the sake of
> learning and getting this changed through to application.
>
> Joleen
> On Mar 10, 2016 7:42 PM, "Konstantin Kolinko" 
> wrote:
>
> > 2016-03-11 2:49 GMT+03:00 Joleen Barker :
> > > I wanted to let you know that I really tried at this and feel the
> > changes I
> > > made should be working and it is a matter of the developer hard
> > > coding
> > the
> > > log messages to go to the stdout/stderr and became lazy as one of
> > > the
> > other
> > > that commented had stated. I opened a ticket with the vendor but I
> > > have
> > no
> > > idea how long it will take. So I went the route of writing a small
> > > perl script to copy the catalina.out file to a file with the same
> > > name and the date and time appended on the file name and then next I
> > > open the existing catalina.out file to clear the contents and then
> > > close it again to start the next day with a clean log file. It isn't
> > > the way I wanted to go but I ran out of time. If the vendor decides
> > > to work with me, it would be great but I have a feeling they will not
> go back and change things.
> > >
> >
> > On the logging page of the FAQ:
> > https://wiki.apache.org/tomcat/FAQ/Logging#Q10
> >
> >
> > By the way,
> > 1) It is possible to run with a debugger and put a breakpoint into
> > java.io.PrintStream.println(String).  I doubt that the PrintStream
> > class is used much anywhere except to implement System.out/System.err.
> > https://wiki.apache.org/tomcat/FAQ/Developing#Debugging
> >
> > 2) It is possible to search the class files for the message text. The
> > classes store it in UTF-8, and jar files are just zip archives.
> >
> >
> > > So are you suggesting to remove the ConsoleAppender from the
> > > log4j.properties that the vendor has in the WEB-INF/classes directory?
> >
> > Yes.
> >
> > The same for Tomcat own log configuration,
> >
> > http://tomcat.apache.org/tomcat-8.0-doc/logging.html#Considerations_fo
> > r_production_usage
> >
> > (Just mentioning for completeness. IIRC you have already reconfigured
> > Tomcat own logging.)
> >
> >
> > Best regards,
> > Konstantin Kolinko
> >
> > -
> > To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
> > For additional commands, e-mail: users-h...@tomcat.apache.org
> >
> >
>


RE: Understanding how to controlling what data is written to log4j appenders

2016-03-10 Thread George Stanchev
If you run tomcat via the windows server wrapper, you can

"%TOMCAT_EXE%" //US//%TOMCAT_SERVICE_NAME% --StdOutput "%TOMCAT_CONSOLE_LOG%" 
--StdError "%TOMCAT_CONSOLE_LOG%"

Which will redirect the stderr and stdoout to the corresponding log files

George

-Original Message-
From: Joleen Barker [mailto:oldenuf2no...@gmail.com] 
Sent: Thursday, March 10, 2016 5:48 PM
To: Tomcat Users List 
Subject: Re: Understanding how to controlling what data is written to log4j 
appenders

Thanks for the tips. I have to use the perl program for now to accomplish the 
task for the company but l'll continue to work this for the sake of learning 
and getting this changed through to application.

Joleen
On Mar 10, 2016 7:42 PM, "Konstantin Kolinko" 
wrote:

> 2016-03-11 2:49 GMT+03:00 Joleen Barker :
> > I wanted to let you know that I really tried at this and feel the
> changes I
> > made should be working and it is a matter of the developer hard 
> > coding
> the
> > log messages to go to the stdout/stderr and became lazy as one of 
> > the
> other
> > that commented had stated. I opened a ticket with the vendor but I 
> > have
> no
> > idea how long it will take. So I went the route of writing a small 
> > perl script to copy the catalina.out file to a file with the same 
> > name and the date and time appended on the file name and then next I 
> > open the existing catalina.out file to clear the contents and then 
> > close it again to start the next day with a clean log file. It isn't 
> > the way I wanted to go but I ran out of time. If the vendor decides 
> > to work with me, it would be great but I have a feeling they will not go 
> > back and change things.
> >
>
> On the logging page of the FAQ:
> https://wiki.apache.org/tomcat/FAQ/Logging#Q10
>
>
> By the way,
> 1) It is possible to run with a debugger and put a breakpoint into 
> java.io.PrintStream.println(String).  I doubt that the PrintStream 
> class is used much anywhere except to implement System.out/System.err.
> https://wiki.apache.org/tomcat/FAQ/Developing#Debugging
>
> 2) It is possible to search the class files for the message text. The 
> classes store it in UTF-8, and jar files are just zip archives.
>
>
> > So are you suggesting to remove the ConsoleAppender from the 
> > log4j.properties that the vendor has in the WEB-INF/classes directory?
>
> Yes.
>
> The same for Tomcat own log configuration,
>
> http://tomcat.apache.org/tomcat-8.0-doc/logging.html#Considerations_fo
> r_production_usage
>
> (Just mentioning for completeness. IIRC you have already reconfigured 
> Tomcat own logging.)
>
>
> Best regards,
> Konstantin Kolinko
>
> -
> To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
> For additional commands, e-mail: users-h...@tomcat.apache.org
>
>


Re: Idle Thread high CPU

2016-03-10 Thread Christopher Schultz
-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1

Rallavagu,

On 3/10/16 8:38 PM, Rallavagu wrote:
> 
> 
> On 3/10/16 5:23 PM, Christopher Schultz wrote: Rallavagu,
> 
> On 3/10/16 8:10 PM, Rallavagu wrote:
 
 
 On 3/10/16 2:33 PM, Christopher Schultz wrote: Rallavagu,
 
 On 3/10/16 5:16 PM, Rallavagu wrote:
>>> On 3/10/16 2:09 PM, Christopher Schultz wrote:
>>> Rallavagu,
>>> 
>>> On 3/10/16 4:02 PM, Rallavagu wrote:
>> On 3/10/16 11:54 AM, Christopher Schultz wrote:
>>> Are you sure you have matched-up the correct
>>> thread within the JVM that is using all that
>>> CPU?
>> 
>>> How are you measuring the CPU usage?
>> 
>> It would the ID output from "top -H" mapping to
>> "Native ID" in thread dump.
>>> 
>>> My version of 'top' (Debian Linux) doesn't show thread
>>> ids. :(
>>> 
>>> I seem to recall having to do some backflips to
>>> convert native thread id to Java thread id. Can you
>>> explain what you've done to do that?
>>> 
 A typical top -H shows the following
>>> 
 top - 11:40:11 up 190 days,  1:24,  1 user,  load
 average: 5.74, 6.09, 5.78 Tasks: 759 total,   4
 running, 755 sleeping,   0 stopped,   0 zombie
 Cpu(s): 18.4%us,  1.6%sy, 0.0%ni, 79.5%id, 0.1%wa,
 0.0%hi,  0.5%si, 0.0%st Mem: 8057664k total,
 7895252k used,   162412k free,63312k buffers
 Swap:  2064380k total, 199452k used,  1864928k free,
 2125868k cached
>>> 
 PID USER  PR  NI  VIRT  RES  SHR S %CPU %MEM
 TIME+ COMMAND 15648 tomcat20   0 9649m 4.8g 4520
 R 87.3 62.6 7:24.24 java 21710 tomcat20   0 9649m
 4.8g 4520 R 79.8 62.6 5:44.99 java 21694 tomcat20
 0 9649m 4.8g 4520 S 74.3 62.6 5:39.40 java 7889
 tomcat20   0 9649m 4.8g 4520 S 29.7 62.6 4:24.44
 java 7878 tomcat20   0 9649m 4.8g 4520 S 27.8
 62.6 4:36.82 java 21701 tomcat20   0 9649m 4.8g
 4520 S 26.0 62.6 5:49.83 java
>>> 
 After taking thread dump, I used Threadlogic which
 will show Native-ID as column which corresponds to
 PID shown above.
>>> 
 https://java.net/projects/threadlogic
>>> 
 This way it helps to determine the thread that might 
 potentially causing high cpu.
 
 Okay. Are you serving a high rate of requests? It's possible
 that the thread is just doing a lot of (legitimate) work.
 
 The BIO connector is very basic: it uses blocking reads, and
 the thread dump you showed before showed it waiting on IO, so
 it should be completely idle, using no CPU time.
 
 It's *possible* that it's in a busy-wait state where it is 
 performing a very short IO-wait in a loop that it never
 exits. But since you haven't specified any weird timeouts,
 etc. on your connector, I'm skeptical as to that being the
 cause.
 
 This thread stays at high CPU usage for quite a while? And
 every thread dump you do has the same (or very similar) stack
 trace?
 
> The symptom is that app is always high on CPU hovering
> between 75 - 85 and so looked at the thread dumps. Each
> thread dump shows few high cpu threads and some of those
> are supposedly idle. After looking at a particular thread
> by Id what it was doing in the previous thread some of
> these were reading on socket. So, that might be somewhat
> related to what you said about I/O wait.
 
> Also, if BIO is basic, what are other options?
> 
> https://tomcat.apache.org/tomcat-8.0-doc/config/http.html#Standard_Imp
le
>
> 
mentation
> 
> The NIO connector is more scalable, but the BIO should use the
> *least* resources when handling modest loads. I wasn't suggesting
> that BIO should be avoided due to its simplicity... quite the
> contrary, I was suggesting that the BIO connector *should* be
> well-behaved.
> 
>> Sounds good. So, the thread might be simply "winding down" from
>> socket read? If many of those threads behaving the same way
>> pushing CPU to high. Is there anything that I should be looking
>> into or configure perhaps to address this or update to 7.0.68?

I would first upgrade to 7.0.68 and see if the problem goes away. If
not, report back. It would be good to know what kind of load the
server is under -- mostly number of requests per second you are
actually processing, and how many request-processing threads exist for
the Connector (to get an average throughput per thread).

- -chris
-BEGIN PGP SIGNATURE-
Comment: GPGTools - http://gpgtools.org
Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/

iEYEARECAAYFAlbiJKoACgkQ9CaO5/Lv0PA8XQCdELZ8f+rCs+UV7W4sPA819FgV
uuMAnjQamWQ3MnBz2DdJuoL9+GyJC4F0
=x9ql
-END PGP SIGNATURE-

-
To unsubscribe, e-mail: 

Re: a lot of 502 error with using version 7.0.68-1

2016-03-10 Thread Christopher Schultz
-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1

Kenchi,

On 3/10/16 8:25 PM, Kenichi MASUDA wrote:
> Thank you for your reply and I'm sorry for less information.
> 
>> how are you connecting httpd -> Tomcat?
>> 
> A client sends a request to httpd which listens as 80, after that
> httpd throw it like this to the tomcat which exists in backend and 
> serves as 8080 port with the rewrite rule.
> 
> RewriteCond %{REQUEST_URI} 
> !\.(png|jpe?g|gif|css|js|xml|json|jsonp|ico|html?|swf|txt)$ 
> RewriteRule ^/(.*)$ http://localhost:8080/core/$1 [P,L]

So you are using mod_proxy_http.

> The error is: httpd log say that "proxy: Error reading from remote
> server returned by"
> 
> and
> 
> a client say that "502 proxy error. The proxy server received an
> invalid response from an upstream server. The proxy server could
> not handle the request POST /foo/bar"
> 
> After having checked that error occurred, I've downgraded the
> tomcat version to 7.0.39 (the version which I used to use), so that
> the error disappeared.

That is a very old Tomcat version indeed (more than 3 years old). You
would be better upgrading to latest Tomcat 7.0.68 and figuring out
what's wrong with the proxy.

This might not be the best list to ask about mod_proxy_http, but there
are some people who probably have good experience with it here.

The best place to ask about mod_proxy_http would be on the httpd
user's list:
http://httpd.apache.org/lists.html#http-users

- -chris
-BEGIN PGP SIGNATURE-
Comment: GPGTools - http://gpgtools.org
Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/

iEYEARECAAYFAlbiJEYACgkQ9CaO5/Lv0PA2KACfbdp4wR3rXQcJ82jEK/GWLls7
/EUAoLpT6SzzPYoyZYF1Zh/481dnCvLt
=hiN1
-END PGP SIGNATURE-

-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org



Re: How can I fix deserialization vulnerability?

2016-03-10 Thread Christopher Schultz
-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1

林慶龍,

On 3/10/16 8:07 PM, 林慶龍 Barry Lin wrote:
> These days, Everyone talks about the vulnerability in Tomcat, and
> we found that we had the same problem with “deserialization 
> vulnerability”.
> 
> How can I fix deserialization vulnerability in tomcat?

If you don't have any applications that have known problematic classes
in them (such as the famous commons-collections bug), then you aren't
really in any danger.

You can disable session serialization, but if you don't trust the
files on your own disk then you probably have bigger problems.

You can disable clustering, but if you don't trust the other members
of your cluster, then you probably have bigger problems.

You can disable session persistence, but if you don't trust your
database, then you probably have bigger problems.

The reality is that this "deserialization vulnerability" is wildly
overblown. Yes, it should be mitigated, but the attack vectors are
very, very narrow.

As of Tomcat 8.0.32, the session resumption "vulnerability" has been
mitigated in Tomcat itself if you configure it properly. It's covered
under "CVE-2016-0714" on this page:
https://tomcat.apache.org/security-8.html

You need to either run Tomcat under a SecurityManager (in which case,
you'll get a non-null default value for this configuration setting),
or you need to set sessionAttributeValueClassNameFilter on your
 element in server.xml.
https://tomcat.apache.org/tomcat-8.0-doc/config/manager.html

(I admit I find the use of that CVE a little confusing, here, but the
patches for that CVE are the ones that also fix the de-serialization
vulnerability.)

- -chris
-BEGIN PGP SIGNATURE-
Comment: GPGTools - http://gpgtools.org
Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/

iEYEARECAAYFAlbiIsMACgkQ9CaO5/Lv0PDbsACdEATTK7tFmAgw3Q8f43ZSTZYQ
GIsAoMNJOOSkpGmF+GPNKbgkbN93Okaw
=v0tc
-END PGP SIGNATURE-

-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org



Re: Idle Thread high CPU

2016-03-10 Thread Rallavagu



On 3/10/16 5:23 PM, Christopher Schultz wrote:

-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1

Rallavagu,

On 3/10/16 8:10 PM, Rallavagu wrote:



On 3/10/16 2:33 PM, Christopher Schultz wrote: Rallavagu,

On 3/10/16 5:16 PM, Rallavagu wrote:

On 3/10/16 2:09 PM, Christopher Schultz wrote: Rallavagu,

On 3/10/16 4:02 PM, Rallavagu wrote:

On 3/10/16 11:54 AM, Christopher Schultz wrote:

Are you sure you have matched-up the correct thread
within the JVM that is using all that CPU?



How are you measuring the CPU usage?


It would the ID output from "top -H" mapping to "Native
ID" in thread dump.


My version of 'top' (Debian Linux) doesn't show thread ids.
:(

I seem to recall having to do some backflips to convert
native thread id to Java thread id. Can you explain what
you've done to do that?


A typical top -H shows the following



top - 11:40:11 up 190 days,  1:24,  1 user,  load average:
5.74, 6.09, 5.78 Tasks: 759 total,   4 running, 755
sleeping,   0 stopped,   0 zombie Cpu(s): 18.4%us,  1.6%sy,
0.0%ni, 79.5%id, 0.1%wa,  0.0%hi,  0.5%si, 0.0%st Mem:
8057664k total,  7895252k used,   162412k free,63312k
buffers Swap:  2064380k total, 199452k used,  1864928k
free,  2125868k cached



PID USER  PR  NI  VIRT  RES  SHR S %CPU %MEMTIME+
COMMAND 15648 tomcat20   0 9649m 4.8g 4520 R 87.3 62.6
7:24.24 java 21710 tomcat20   0 9649m 4.8g 4520 R 79.8
62.6 5:44.99 java 21694 tomcat20   0 9649m 4.8g 4520 S
74.3 62.6 5:39.40 java 7889 tomcat20   0 9649m 4.8g
4520 S 29.7 62.6 4:24.44 java 7878 tomcat20   0 9649m
4.8g 4520 S 27.8 62.6 4:36.82 java 21701 tomcat20   0
9649m 4.8g 4520 S 26.0 62.6 5:49.83 java



After taking thread dump, I used Threadlogic which will
show Native-ID as column which corresponds to PID shown
above.



https://java.net/projects/threadlogic



This way it helps to determine the thread that might
potentially causing high cpu.


Okay. Are you serving a high rate of requests? It's possible that
the thread is just doing a lot of (legitimate) work.

The BIO connector is very basic: it uses blocking reads, and the
thread dump you showed before showed it waiting on IO, so it should
be completely idle, using no CPU time.

It's *possible* that it's in a busy-wait state where it is
performing a very short IO-wait in a loop that it never exits. But
since you haven't specified any weird timeouts, etc. on your
connector, I'm skeptical as to that being the cause.

This thread stays at high CPU usage for quite a while? And every
thread dump you do has the same (or very similar) stack trace?


The symptom is that app is always high on CPU hovering between 75
- 85 and so looked at the thread dumps. Each thread dump shows
few high cpu threads and some of those are supposedly idle. After
looking at a particular thread by Id what it was doing in the
previous thread some of these were reading on socket. So, that
might be somewhat related to what you said about I/O wait.



Also, if BIO is basic, what are other options?


https://tomcat.apache.org/tomcat-8.0-doc/config/http.html#Standard_Imple
mentation

The NIO connector is more scalable, but the BIO should use the *least*
resources when handling modest loads. I wasn't suggesting that BIO
should be avoided due to its simplicity... quite the contrary, I was
suggesting that the BIO connector *should* be well-behaved.

Sounds good. So, the thread might be simply "winding down" from socket 
read? If many of those threads behaving the same way pushing CPU to 
high. Is there anything that I should be looking into or configure 
perhaps to address this or update to 7.0.68?



- -chris
-BEGIN PGP SIGNATURE-
Comment: GPGTools - http://gpgtools.org
Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/

iEYEARECAAYFAlbiHf8ACgkQ9CaO5/Lv0PAHWQCbBHR4dIjpfv+K0qnJil50Buyk
CIAAoKUByrZhri66vkU/OLdJ/01qMc2u
=+9Fl
-END PGP SIGNATURE-

-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org



-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org



Re: a lot of 502 error with using version 7.0.68-1

2016-03-10 Thread Kenichi MASUDA
Thank you for your reply and I'm sorry for less information.

> how are you connecting httpd -> Tomcat?
>
A client sends a request to httpd which listens as 80, after
that httpd throw it like this to the tomcat which exists in backend and
serves as 8080 port with the rewrite rule.

RewriteCond %{REQUEST_URI}
!\.(png|jpe?g|gif|css|js|xml|json|jsonp|ico|html?|swf|txt)$
RewriteRule ^/(.*)$ http://localhost:8080/core/$1 [P,L]

The error is:
httpd log say that "proxy: Error reading from remote server returned by"

and

a client say that "502 proxy error. The proxy server received an invalid
response from an upstream server. The proxy server could not handle the
request POST /foo/bar"

After having checked that error occurred, I've downgraded the tomcat
version to 7.0.39(the version which I used to use), so that the error
disappeared.



2016-03-10 21:17 GMT+09:00 Christopher Schultz :

> Kenichi,
>
> On 3/10/16 4:52 AM, Kenichi MASUDA wrote:
> > Hi,
> >
> > I updated the tomcat on CentOS6 from 7.0.39-1 to 7.0.68-1,
> > and it seems to that the 502 proxy error was increased than before.
> >
> > The error is below:
> > - apache log:  proxy: Error reading from remote server returned by
> > - mobile phone display: 502 proxy error. The proxy server received an
> > invalid response from an upstream server.
> > The proxy server could not handle the request POST /foo/bar
> >
> > Is there anyone know or encounter the same situation?
> >
> > Apache: 2.24-1
> > tomcat: 7.0.68-1
>
> Can you provide more information. For example, what do the logs say?
> Also, how are you connecting httpd -> Tomcat?
>
> -chris
>
> -
> To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
> For additional commands, e-mail: users-h...@tomcat.apache.org
>
>


-- 
mailto:masu...@gmail.com


Re: Idle Thread high CPU

2016-03-10 Thread Christopher Schultz
-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1

Rallavagu,

On 3/10/16 8:10 PM, Rallavagu wrote:
> 
> 
> On 3/10/16 2:33 PM, Christopher Schultz wrote: Rallavagu,
> 
> On 3/10/16 5:16 PM, Rallavagu wrote:
 On 3/10/16 2:09 PM, Christopher Schultz wrote: Rallavagu,
 
 On 3/10/16 4:02 PM, Rallavagu wrote:
>>> On 3/10/16 11:54 AM, Christopher Schultz wrote:
 Are you sure you have matched-up the correct thread
 within the JVM that is using all that CPU?
>>> 
 How are you measuring the CPU usage?
>>> 
>>> It would the ID output from "top -H" mapping to "Native
>>> ID" in thread dump.
 
 My version of 'top' (Debian Linux) doesn't show thread ids.
 :(
 
 I seem to recall having to do some backflips to convert
 native thread id to Java thread id. Can you explain what
 you've done to do that?
 
> A typical top -H shows the following
 
> top - 11:40:11 up 190 days,  1:24,  1 user,  load average:
> 5.74, 6.09, 5.78 Tasks: 759 total,   4 running, 755
> sleeping,   0 stopped,   0 zombie Cpu(s): 18.4%us,  1.6%sy,
> 0.0%ni, 79.5%id, 0.1%wa,  0.0%hi,  0.5%si, 0.0%st Mem:
> 8057664k total,  7895252k used,   162412k free,63312k
> buffers Swap:  2064380k total, 199452k used,  1864928k
> free,  2125868k cached
 
> PID USER  PR  NI  VIRT  RES  SHR S %CPU %MEMTIME+ 
> COMMAND 15648 tomcat20   0 9649m 4.8g 4520 R 87.3 62.6 
> 7:24.24 java 21710 tomcat20   0 9649m 4.8g 4520 R 79.8
> 62.6 5:44.99 java 21694 tomcat20   0 9649m 4.8g 4520 S
> 74.3 62.6 5:39.40 java 7889 tomcat20   0 9649m 4.8g
> 4520 S 29.7 62.6 4:24.44 java 7878 tomcat20   0 9649m
> 4.8g 4520 S 27.8 62.6 4:36.82 java 21701 tomcat20   0
> 9649m 4.8g 4520 S 26.0 62.6 5:49.83 java
 
> After taking thread dump, I used Threadlogic which will
> show Native-ID as column which corresponds to PID shown
> above.
 
> https://java.net/projects/threadlogic
 
> This way it helps to determine the thread that might
> potentially causing high cpu.
> 
> Okay. Are you serving a high rate of requests? It's possible that
> the thread is just doing a lot of (legitimate) work.
> 
> The BIO connector is very basic: it uses blocking reads, and the 
> thread dump you showed before showed it waiting on IO, so it should
> be completely idle, using no CPU time.
> 
> It's *possible* that it's in a busy-wait state where it is
> performing a very short IO-wait in a loop that it never exits. But
> since you haven't specified any weird timeouts, etc. on your
> connector, I'm skeptical as to that being the cause.
> 
> This thread stays at high CPU usage for quite a while? And every 
> thread dump you do has the same (or very similar) stack trace?
> 
>> The symptom is that app is always high on CPU hovering between 75
>> - 85 and so looked at the thread dumps. Each thread dump shows
>> few high cpu threads and some of those are supposedly idle. After
>> looking at a particular thread by Id what it was doing in the
>> previous thread some of these were reading on socket. So, that
>> might be somewhat related to what you said about I/O wait.
> 
>> Also, if BIO is basic, what are other options?

https://tomcat.apache.org/tomcat-8.0-doc/config/http.html#Standard_Imple
mentation

The NIO connector is more scalable, but the BIO should use the *least*
resources when handling modest loads. I wasn't suggesting that BIO
should be avoided due to its simplicity... quite the contrary, I was
suggesting that the BIO connector *should* be well-behaved.

- -chris
-BEGIN PGP SIGNATURE-
Comment: GPGTools - http://gpgtools.org
Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/

iEYEARECAAYFAlbiHf8ACgkQ9CaO5/Lv0PAHWQCbBHR4dIjpfv+K0qnJil50Buyk
CIAAoKUByrZhri66vkU/OLdJ/01qMc2u
=+9Fl
-END PGP SIGNATURE-

-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org



How can I fix deserialization vulnerability?

2016-03-10 Thread 林慶龍 Barry Lin
Dears:

These days, Everyone talks about the vulnerability in Tomcat, and we found that 
we had the same problem with “deserialization vulnerability”.

How can I fix deserialization vulnerability in tomcat?

Thanks for your help!





Best regard,

Barry Lin

鼎捷 
(鼎新電腦股份有限公司、鼎誠資訊股份有限公司、鼎捷軟件股份有限公司及鼎捷軟件越南有限公司)將善保管您的個人資料,並於合法取得之前提下善意使用,據此本公司僅在營運範圍內之目的與您聯繫,包含鼎捷主辦或協辦之行銷活動、客戶服務、供應商聯繫等,非經由本公司上開目的下之合法授權,所寄發之資訊並不代表本公司
 。本電子郵件及附件所載訊息均為保密資訊,受合約保護或依法不得洩漏。其內容僅供指定收件人按限定範圍或特殊目的使用。未經授權者收到此資訊者均無權閱讀、 使用、 
複製、洩漏或散佈。若您因為誤傳而收到本郵件或者非本郵件之指定收件人,煩請即刻回覆郵件或並永久刪除此郵件及其附件和銷毀所有複印件。倘若有前述情形或信件誤遞至您的信箱或有相關問題,請透過下列方式聯繫更正;mail:dsc...@digiwin.biz。謝謝您的合作!


Re: Understanding how to controlling what data is written to log4j appenders

2016-03-10 Thread Joleen Barker
Thanks for the tips. I have to use the perl program for now to accomplish
the task for the company but l'll continue to work this for the sake of
learning and getting this changed through to application.

Joleen
On Mar 10, 2016 7:42 PM, "Konstantin Kolinko" 
wrote:

> 2016-03-11 2:49 GMT+03:00 Joleen Barker :
> > I wanted to let you know that I really tried at this and feel the
> changes I
> > made should be working and it is a matter of the developer hard coding
> the
> > log messages to go to the stdout/stderr and became lazy as one of the
> other
> > that commented had stated. I opened a ticket with the vendor but I have
> no
> > idea how long it will take. So I went the route of writing a small perl
> > script to copy the catalina.out file to a file with the same name and the
> > date and time appended on the file name and then next I open the existing
> > catalina.out file to clear the contents and then close it again to start
> > the next day with a clean log file. It isn't the way I wanted to go but I
> > ran out of time. If the vendor decides to work with me, it would be great
> > but I have a feeling they will not go back and change things.
> >
>
> On the logging page of the FAQ:
> https://wiki.apache.org/tomcat/FAQ/Logging#Q10
>
>
> By the way,
> 1) It is possible to run with a debugger and put a breakpoint into
> java.io.PrintStream.println(String).  I doubt that the PrintStream
> class is used much anywhere except to implement System.out/System.err.
> https://wiki.apache.org/tomcat/FAQ/Developing#Debugging
>
> 2) It is possible to search the class files for the message text. The
> classes store it in UTF-8, and jar files are just zip archives.
>
>
> > So are you suggesting to remove the ConsoleAppender from the
> > log4j.properties that the vendor has in the WEB-INF/classes directory?
>
> Yes.
>
> The same for Tomcat own log configuration,
>
> http://tomcat.apache.org/tomcat-8.0-doc/logging.html#Considerations_for_production_usage
>
> (Just mentioning for completeness. IIRC you have already reconfigured
> Tomcat own logging.)
>
>
> Best regards,
> Konstantin Kolinko
>
> -
> To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
> For additional commands, e-mail: users-h...@tomcat.apache.org
>
>


Re: Understanding how to controlling what data is written to log4j appenders

2016-03-10 Thread Konstantin Kolinko
2016-03-11 2:49 GMT+03:00 Joleen Barker :
> I wanted to let you know that I really tried at this and feel the changes I
> made should be working and it is a matter of the developer hard coding the
> log messages to go to the stdout/stderr and became lazy as one of the other
> that commented had stated. I opened a ticket with the vendor but I have no
> idea how long it will take. So I went the route of writing a small perl
> script to copy the catalina.out file to a file with the same name and the
> date and time appended on the file name and then next I open the existing
> catalina.out file to clear the contents and then close it again to start
> the next day with a clean log file. It isn't the way I wanted to go but I
> ran out of time. If the vendor decides to work with me, it would be great
> but I have a feeling they will not go back and change things.
>

On the logging page of the FAQ:
https://wiki.apache.org/tomcat/FAQ/Logging#Q10


By the way,
1) It is possible to run with a debugger and put a breakpoint into
java.io.PrintStream.println(String).  I doubt that the PrintStream
class is used much anywhere except to implement System.out/System.err.
https://wiki.apache.org/tomcat/FAQ/Developing#Debugging

2) It is possible to search the class files for the message text. The
classes store it in UTF-8, and jar files are just zip archives.


> So are you suggesting to remove the ConsoleAppender from the
> log4j.properties that the vendor has in the WEB-INF/classes directory?

Yes.

The same for Tomcat own log configuration,
http://tomcat.apache.org/tomcat-8.0-doc/logging.html#Considerations_for_production_usage

(Just mentioning for completeness. IIRC you have already reconfigured
Tomcat own logging.)


Best regards,
Konstantin Kolinko

-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org



Re: Understanding how to controlling what data is written to log4j appenders

2016-03-10 Thread Joleen Barker
So are you suggesting to remove the ConsoleAppender from the
log4j.properties that the vendor has in the WEB-INF/classes directory?

Joleen
On Mar 10, 2016 7:17 PM, "Konstantin Kolinko" 
wrote:

> 2016-03-08 18:43 GMT+03:00 Christopher Schultz <
> ch...@christopherschultz.net>:
> >
> > Everything that says log4j.logger.[something]=[level], stdout
> >
> > Is going to send those log messages to the "stdout" appender, which is
> > tied to System.out. You'll need to do one of two things to dig
> > yourself out:
> >
> > 1. Use swallowOutput="true" on your , which performs some
> > magic to take System.out from applications' calls and redirect it
> > elsewhere else (to the tomcat-defined loggers that can be configured
> > in Tomcat's log4j.properties file).
> >
> > 2. Change the "stdout" appender to be something other than
> > ConsoleAppender, and point it at a file on the disk.
> >
> > I'm not a fan of the first option, but it's sometimes the quickest way
> > to handle everything all at once, and usually doesn't require any
> > changes to the application's configuration.
>
> The swallowOutput option is a remedy for direct System.out.println()
> calls.  One should not expect it to work with something that gets a
> direct reference to System.out stream before Tomcat replaces it.
>
> For example, it Tomcat is reconfigured to use Log4J for its own
> logging, and one configures a ConsoleAppender for Tomcat, this
> ConsoleAppender is not affected by swallowOutput.
>
> The limitations are documented here:
>
> http://tomcat.apache.org/tomcat-8.0-doc/logging.html#Console
>
> If you do not want to log to stdout/stderr, you really must not
> configure any ConsoleAppender,.
>
> Best regards,
> Konstantin Kolinko
>
> -
> To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
> For additional commands, e-mail: users-h...@tomcat.apache.org
>
>


Re: Understanding how to controlling what data is written to log4j appenders

2016-03-10 Thread Konstantin Kolinko
2016-03-08 18:43 GMT+03:00 Christopher Schultz :
>
> Everything that says log4j.logger.[something]=[level], stdout
>
> Is going to send those log messages to the "stdout" appender, which is
> tied to System.out. You'll need to do one of two things to dig
> yourself out:
>
> 1. Use swallowOutput="true" on your , which performs some
> magic to take System.out from applications' calls and redirect it
> elsewhere else (to the tomcat-defined loggers that can be configured
> in Tomcat's log4j.properties file).
>
> 2. Change the "stdout" appender to be something other than
> ConsoleAppender, and point it at a file on the disk.
>
> I'm not a fan of the first option, but it's sometimes the quickest way
> to handle everything all at once, and usually doesn't require any
> changes to the application's configuration.

The swallowOutput option is a remedy for direct System.out.println()
calls.  One should not expect it to work with something that gets a
direct reference to System.out stream before Tomcat replaces it.

For example, it Tomcat is reconfigured to use Log4J for its own
logging, and one configures a ConsoleAppender for Tomcat, this
ConsoleAppender is not affected by swallowOutput.

The limitations are documented here:

http://tomcat.apache.org/tomcat-8.0-doc/logging.html#Console

If you do not want to log to stdout/stderr, you really must not
configure any ConsoleAppender,.

Best regards,
Konstantin Kolinko

-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org



RE: AJP protocol auto-switching default

2016-03-10 Thread George Stanchev


-Original Message-
From: Rémy Maucherat [mailto:r...@apache.org] 
Sent: Thursday, March 10, 2016 4:41 PM
To: Tomcat Users List 
Subject: Re: AJP protocol auto-switching default

2016-03-11 0:38 GMT+01:00 George Stanchev :

> > Perhaps I am overlooking something, but the documentation for AJP 
> > [1] states for "protocol"
> >
> > 
> > The standard protocol value for an AJP connector is AJP/1.3 which 
> > uses an auto-switching mechanism to select either a Java based 
> > connector or an APR/native based connector. If the PATH (Windows) or 
> > LD_LIBRARY_PATH (on most unix systems) environment variables contain 
> > the Tomcat native library, the native/APR connector will be used. If 
> > the native library cannot be found, the Java based connector will be
> used.
> > 
> >
> > The text above doesn't describe what is the auto-selected protocol 
> > when a Java based connector is selected. My guess is that 
> > "org.apache.coyote.ajp.AjpNioProtocol" will be auto-selected since 
> > BIO is on its way out but since the documentation doesn't explicitly 
> > state it, it leaves the reader to wonder.
> >
> > George
> >
> >
> >
> > [1] https://tomcat.apache.org/tomcat-7.0-doc/config/ajp.html
>
> 
> Tomcat 7 uses either java.io or APR as the default.
> 
>
>
> The documentation states later:
>
> To use an explicit protocol rather than rely on the auto-switching 
> mechanism described above, the following values may be used:
> org.apache.coyote.ajp.AjpProtocol - blocking Java connector 
> org.apache.coyote.ajp.AjpNioProtocol - non blocking Java connector.
> org.apache.coyote.ajp.AjpAprProtocol - the APR/native connector.
> Custom implementations may also be used.
>
> Which one of the non-Apr protocols above is "java.io"?
>


It is org.apache.coyote.ajp.AjpProtocol.


Thanks! That means my gut feeling was wrong and the documentation should be 
fixed to imply the default selection. Should I fire a BZ issue?



Re: AJP protocol auto-switching default

2016-03-10 Thread Rémy Maucherat
2016-03-11 0:38 GMT+01:00 George Stanchev :

> > Perhaps I am overlooking something, but the documentation for AJP [1]
> > states for "protocol"
> >
> > 
> > The standard protocol value for an AJP connector is AJP/1.3 which uses
> > an auto-switching mechanism to select either a Java based connector or
> > an APR/native based connector. If the PATH (Windows) or
> > LD_LIBRARY_PATH (on most unix systems) environment variables contain
> > the Tomcat native library, the native/APR connector will be used. If
> > the native library cannot be found, the Java based connector will be
> used.
> > 
> >
> > The text above doesn't describe what is the auto-selected protocol
> > when a Java based connector is selected. My guess is that
> > "org.apache.coyote.ajp.AjpNioProtocol" will be auto-selected since BIO
> > is on its way out but since the documentation doesn't explicitly state
> > it, it leaves the reader to wonder.
> >
> > George
> >
> >
> >
> > [1] https://tomcat.apache.org/tomcat-7.0-doc/config/ajp.html
>
> 
> Tomcat 7 uses either java.io or APR as the default.
> 
>
>
> The documentation states later:
>
> To use an explicit protocol rather than rely on the auto-switching
> mechanism described above, the following values may be used:
> org.apache.coyote.ajp.AjpProtocol - blocking Java connector
> org.apache.coyote.ajp.AjpNioProtocol - non blocking Java connector.
> org.apache.coyote.ajp.AjpAprProtocol - the APR/native connector.
> Custom implementations may also be used.
>
> Which one of the non-Apr protocols above is "java.io"?
>

It is org.apache.coyote.ajp.AjpProtocol.

Rémy


RE: AJP protocol auto-switching default

2016-03-10 Thread George Stanchev
> Perhaps I am overlooking something, but the documentation for AJP [1] 
> states for "protocol"
>
> 
> The standard protocol value for an AJP connector is AJP/1.3 which uses 
> an auto-switching mechanism to select either a Java based connector or 
> an APR/native based connector. If the PATH (Windows) or 
> LD_LIBRARY_PATH (on most unix systems) environment variables contain 
> the Tomcat native library, the native/APR connector will be used. If 
> the native library cannot be found, the Java based connector will be used.
> 
>
> The text above doesn't describe what is the auto-selected protocol 
> when a Java based connector is selected. My guess is that 
> "org.apache.coyote.ajp.AjpNioProtocol" will be auto-selected since BIO 
> is on its way out but since the documentation doesn't explicitly state 
> it, it leaves the reader to wonder.
>
> George
>
>
>
> [1] https://tomcat.apache.org/tomcat-7.0-doc/config/ajp.html


Tomcat 7 uses either java.io or APR as the default.



The documentation states later:

To use an explicit protocol rather than rely on the auto-switching mechanism 
described above, the following values may be used:
org.apache.coyote.ajp.AjpProtocol - blocking Java connector
org.apache.coyote.ajp.AjpNioProtocol - non blocking Java connector.
org.apache.coyote.ajp.AjpAprProtocol - the APR/native connector.
Custom implementations may also be used.

Which one of the non-Apr protocols above is "java.io"?

George



Re: AJP protocol auto-switching default

2016-03-10 Thread Rémy Maucherat
2016-03-10 23:55 GMT+01:00 George Stanchev :

> Perhaps I am overlooking something, but the documentation for AJP [1]
> states for "protocol"
>
> 
> The standard protocol value for an AJP connector is AJP/1.3 which uses an
> auto-switching mechanism to select either a Java based connector or an
> APR/native based connector. If the PATH (Windows) or LD_LIBRARY_PATH (on
> most unix systems) environment variables contain the Tomcat native library,
> the native/APR connector will be used. If the native library cannot be
> found, the Java based connector will be used.
> 
>
> The text above doesn't describe what is the auto-selected protocol when a
> Java based connector is selected. My guess is that
> "org.apache.coyote.ajp.AjpNioProtocol" will be auto-selected since BIO is
> on its way out but since the documentation doesn't explicitly state it, it
> leaves the reader to wonder.
>
> George
>
>
>
> [1] https://tomcat.apache.org/tomcat-7.0-doc/config/ajp.html


Tomcat 7 uses either java.io or APR as the default.

Rémy


AJP protocol auto-switching default

2016-03-10 Thread George Stanchev
Perhaps I am overlooking something, but the documentation for AJP [1] states 
for "protocol"


The standard protocol value for an AJP connector is AJP/1.3 which uses an 
auto-switching mechanism to select either a Java based connector or an 
APR/native based connector. If the PATH (Windows) or LD_LIBRARY_PATH (on most 
unix systems) environment variables contain the Tomcat native library, the 
native/APR connector will be used. If the native library cannot be found, the 
Java based connector will be used.


The text above doesn't describe what is the auto-selected protocol when a Java 
based connector is selected. My guess is that 
"org.apache.coyote.ajp.AjpNioProtocol" will be auto-selected since BIO is on 
its way out but since the documentation doesn't explicitly state it, it leaves 
the reader to wonder.

George



[1] https://tomcat.apache.org/tomcat-7.0-doc/config/ajp.html


Re: Idle Thread high CPU

2016-03-10 Thread Christopher Schultz
-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1

Rallavagu,

On 3/10/16 5:16 PM, Rallavagu wrote:
> On 3/10/16 2:09 PM, Christopher Schultz wrote: Rallavagu,
> 
> On 3/10/16 4:02 PM, Rallavagu wrote:
 On 3/10/16 11:54 AM, Christopher Schultz wrote:
> Are you sure you have matched-up the correct thread within
> the JVM that is using all that CPU?
 
> How are you measuring the CPU usage?
 
 It would the ID output from "top -H" mapping to "Native ID"
 in thread dump.
> 
> My version of 'top' (Debian Linux) doesn't show thread ids. :(
> 
> I seem to recall having to do some backflips to convert native
> thread id to Java thread id. Can you explain what you've done to do
> that?
> 
>> A typical top -H shows the following
> 
>> top - 11:40:11 up 190 days,  1:24,  1 user,  load average: 5.74,
>> 6.09, 5.78 Tasks: 759 total,   4 running, 755 sleeping,   0
>> stopped,   0 zombie Cpu(s): 18.4%us,  1.6%sy,  0.0%ni, 79.5%id,
>> 0.1%wa,  0.0%hi,  0.5%si, 0.0%st Mem:   8057664k total,  7895252k
>> used,   162412k free,63312k buffers Swap:  2064380k total,
>> 199452k used,  1864928k free,  2125868k cached
> 
>> PID USER  PR  NI  VIRT  RES  SHR S %CPU %MEMTIME+
>> COMMAND 15648 tomcat20   0 9649m 4.8g 4520 R 87.3 62.6
>> 7:24.24 java 21710 tomcat20   0 9649m 4.8g 4520 R 79.8 62.6
>> 5:44.99 java 21694 tomcat20   0 9649m 4.8g 4520 S 74.3 62.6
>> 5:39.40 java 7889 tomcat20   0 9649m 4.8g 4520 S 29.7 62.6
>> 4:24.44 java 7878 tomcat20   0 9649m 4.8g 4520 S 27.8 62.6
>> 4:36.82 java 21701 tomcat20   0 9649m 4.8g 4520 S 26.0 62.6
>> 5:49.83 java
> 
>> After taking thread dump, I used Threadlogic which will show
>> Native-ID as column which corresponds to PID shown above.
> 
>> https://java.net/projects/threadlogic
> 
>> This way it helps to determine the thread that might potentially
>> causing high cpu.

Okay. Are you serving a high rate of requests? It's possible that the
thread is just doing a lot of (legitimate) work.

The BIO connector is very basic: it uses blocking reads, and the
thread dump you showed before showed it waiting on IO, so it should be
completely idle, using no CPU time.

It's *possible* that it's in a busy-wait state where it is performing
a very short IO-wait in a loop that it never exits. But since you
haven't specified any weird timeouts, etc. on your connector, I'm
skeptical as to that being the cause.

This thread stays at high CPU usage for quite a while? And every
thread dump you do has the same (or very similar) stack trace?

- -chris
-BEGIN PGP SIGNATURE-
Comment: GPGTools - http://gpgtools.org
Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/

iEYEARECAAYFAlbh9igACgkQ9CaO5/Lv0PBFXQCeKARui/P0Z2/1dSBlwDcc8Dct
jA8AoLu7+oaTE2D6SlzNcXZdmIxtXVCq
=NzQR
-END PGP SIGNATURE-

-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org



Re: Prevent Sending of SSL Root Certificate

2016-03-10 Thread Tad Marko
On Thu, Mar 10, 2016 at 4:22 PM, Christopher Schultz <
ch...@christopherschultz.net> wrote:
> -BEGIN PGP SIGNED MESSAGE-
> Hash: SHA1
>
> Tad,
​...

>
> And what tool is telling you that the root cert is being served along
> with the server and intermediate certs?
>
> So the cert chain goes like this?
>
> server <- intermediate <- cross < CA (not present in keystore)
>
> ?
>

​Exactly. The tool is openssl s_client -showcerts -connect pointed towards
my server.


Re: Idle Thread high CPU

2016-03-10 Thread Christopher Schultz
-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1

Chuck,

On 3/10/16 5:18 PM, Caldarale, Charles R wrote:
>> From: Christopher Schultz [mailto:ch...@christopherschultz.net] 
>> Subject: Re: Idle Thread high CPU
> 
>> My version of 'top' (Debian Linux) doesn't show thread ids. :(
> 
> Can you try "top -H" (case sensitive option)?

I'm not seeing individual threads when I run with -H. I just see PID.
I can get top to show me the "nTH" field (thread count), and it's
non-zero, but it's not showing me the individual threads.

Kernel level is 2.6.32 SMP. Might have something to do with it.

On another 3.2.0 SMP kernel, I do get the thread-id as PID and pid as
TGID.

- -chris
-BEGIN PGP SIGNATURE-
Comment: GPGTools - http://gpgtools.org
Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/

iEYEARECAAYFAlbh9WYACgkQ9CaO5/Lv0PCjDACfSbpulgrLsZa1ddclt1R5AdjO
KCAAn0jGd3wzDD/NRRB169aKBmjzPVRc
=VzIE
-END PGP SIGNATURE-

-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org



Re: Prevent Sending of SSL Root Certificate

2016-03-10 Thread Christopher Schultz
-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1

Tad,

On 3/10/16 5:12 PM, Tad Marko wrote:
> On Thu, Mar 10, 2016 at 3:59 PM, Christopher Schultz 
>  wrote:
>> Tad,
>> 
>> On 3/10/16 4:03 PM, Tad Marko wrote:
>>> Is it possible to tell tomcat to NOT send the root for a 
>>> certificate chain?
>> 
>> Yep.
>> 
>> ...
>> 
>> Just remove the root cert from your keystore, and Tomcat will
>> stop sending it.
>> 
>> If you have further questions, please post the output of the
>> following command in your next post:
>> 
>> $ keytool -keystore  -list
>> 
> 
> The CA is not in my keystore:
> 
> Keystore type: JKS Keystore provider: SUN
> 
> Your keystore contains 3 entries
> 
> my.domain.tld, Mar 10, 2016, PrivateKeyEntry, Certificate
> fingerprint (SHA1): 
> AE:DB:AF:8D:19:D6:38:D8:EB:5A:C1:5D:E6:D2:C4:8B:5F:58:84:6F 
> intermed, Mar 10, 2016, trustedCertEntry, Certificate fingerprint
> (SHA1): 
> 27:AC:93:69:FA:F2:52:07:BB:26:27:CE:FA:CC:BE:4E:F9:C3:19:B8 cross,
> Mar 10, 2016, trustedCertEntry, Certificate fingerprint (SHA1): 
> 34:0B:28:80:F4:46:FC:C0:4E:59:ED:33:F5:2B:3D:08:D6:24:29:64

And what tool is telling you that the root cert is being served along
with the server and intermediate certs?

So the cert chain goes like this?

server <- intermediate <- cross < CA (not present in keystore)

?

- -chris
-BEGIN PGP SIGNATURE-
Comment: GPGTools - http://gpgtools.org
Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/

iEYEARECAAYFAlbh86MACgkQ9CaO5/Lv0PBH1QCfWroMlqsA1UEZmhW8R9/RGn/P
uJEAn0OpPeDIqaJ2qXPez8w9fdoIs4qB
=3MRE
-END PGP SIGNATURE-

-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org



Re: NullPointerException in MemoryRealm after upgrading to Tomcat 8.0.32 from 7.0.26

2016-03-10 Thread Christopher Schultz
-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1

Jason,

On 3/10/16 4:40 PM, Jason Overland wrote:
> Chris,
> 
> On Thu, Mar 10, 2016 at 6:18 AM, Christopher Schultz 
>  wrote:
>> Give this patch a try: ... I have no idea how the options get
>> parsed; we'll see if this simple implementation will get you
>> going again.
>> 
>> -chris
>> 
> 
> The parsing is working correctly.  After applying the patch I
> could login successfully.  Then I added digest=SHA to jaas.config
> and it stopped working ("wrong password").
> 
> On further inspection I found that our CallbackHandler was
> digesting the password before passing it back to the
> JAASMemoryModule, however CredentialHandler expects
> inputCredentials to be plaintext.  So I commented out the part of
> our CallbackHandler that digests the password and now it's working.
> That seems ok to me.

You could also just not specify the "digest=SHA" in your JAAS
configuration. My patch will always configure an otherwise-empty
MessageDigestCredentialHandler which, in the absence of any "digset"
being set, simply use plaintext passwords. (Or, rather, the
CredentialHandler won't actually mutate the credentials on the way
through.)

> So I think this patch is sufficient to get us going again.  Thanks
> for the quick turnaround.  If this patch looks good to everyone, do
> you think it can make it into the next Tomcat patch release?

I see Mark already replied saying he would work on it a bit. My patch
was a quick POC to see if it would work and not, IMO,
production-quality, etc. But we should be able to do something for you
pretty quickly. And you should be able to use it in YOUR production
system right away if you'd like. The final patch will simply be more
robust, support more options, and likely cover cases outside what you
were requesting.

Sorry about the oversight in the MemoryRealm.

- -chris
-BEGIN PGP SIGNATURE-
Comment: GPGTools - http://gpgtools.org
Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/

iEYEARECAAYFAlbh80IACgkQ9CaO5/Lv0PCudACguQJO6uAmfKG7pG23McHvJVm/
mA8AnAubCHy73F81KAdLfDdEihffkCQe
=xfMg
-END PGP SIGNATURE-

-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org



RE: Idle Thread high CPU

2016-03-10 Thread Caldarale, Charles R
> From: Christopher Schultz [mailto:ch...@christopherschultz.net] 
> Subject: Re: Idle Thread high CPU

> My version of 'top' (Debian Linux) doesn't show thread ids. :(

Can you try "top -H" (case sensitive option)?

 - Chuck


THIS COMMUNICATION MAY CONTAIN CONFIDENTIAL AND/OR OTHERWISE PROPRIETARY 
MATERIAL and is thus for use only by the intended recipient. If you received 
this in error, please contact the sender and delete the e-mail and its 
attachments from all computers.


-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org



Re: Intermittent ClassNotFoundException in Jasper EL evaluation

2016-03-10 Thread Christopher Schultz
-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1

Mark,

On 3/10/16 4:43 PM, Mark Thomas wrote:
> On 10/03/2016 21:16, jimi.hulleg...@svensktnaringsliv.se wrote:
>> On Thursday, March 10, 2016 11:20 AM, ma...@apache.org wrote:
>>> 
 3. Why is the problem not limited to the first request for a
 jsp page?
>>> 
>>> Because EL imports may be dynamic so the EL has to be evaluated
>>> on execution.
>> 
>> I'm not really sure I follow you now. Can you explain what you
>> mean with dynamic imports in this regard? I can't see any
>> mentioning of it in the specs 
>> (http://download.oracle.com/otn-pub/jcp/el-3_0-fr-eval-spec/EL3.0.FR.
pdf).
>
>> 
> There is nothing stopping a JSP author obtaining a reference to
> the ImportHandler and conditionally adding classes to import. The 
> configuration of the ImportHandler could change on every call to
> the page.

What about marking the ImportHandler as "dirty" and flushing a cache
of prior lookups? (Or are we talking about spec-defined classes only,
here?)

- -chris
-BEGIN PGP SIGNATURE-
Comment: GPGTools - http://gpgtools.org
Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/

iEYEARECAAYFAlbh8j4ACgkQ9CaO5/Lv0PDP1gCgtBnEJJA9er6w5mC1MUCLofA+
qqYAn2tBNbTYYL+fuV1rIEOklNlPOfsG
=vsTe
-END PGP SIGNATURE-

-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org



Re: Idle Thread high CPU

2016-03-10 Thread Rallavagu



On 3/10/16 2:09 PM, Christopher Schultz wrote:

-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1

Rallavagu,

On 3/10/16 4:02 PM, Rallavagu wrote:

On 3/10/16 11:54 AM, Christopher Schultz wrote:

Are you sure you have matched-up the correct thread within the
JVM that is using all that CPU?



How are you measuring the CPU usage?


It would the ID output from "top -H" mapping to "Native ID" in
thread dump.


My version of 'top' (Debian Linux) doesn't show thread ids. :(

I seem to recall having to do some backflips to convert native thread
id to Java thread id. Can you explain what you've done to do that?


A typical top -H shows the following

top - 11:40:11 up 190 days,  1:24,  1 user,  load average: 5.74, 6.09, 5.78
Tasks: 759 total,   4 running, 755 sleeping,   0 stopped,   0 zombie
Cpu(s): 18.4%us,  1.6%sy,  0.0%ni, 79.5%id,  0.1%wa,  0.0%hi,  0.5%si, 
0.0%st

Mem:   8057664k total,  7895252k used,   162412k free,63312k buffers
Swap:  2064380k total,   199452k used,  1864928k free,  2125868k cached

  PID USER  PR  NI  VIRT  RES  SHR S %CPU %MEMTIME+  COMMAND
15648 tomcat20   0 9649m 4.8g 4520 R 87.3 62.6   7:24.24 java
21710 tomcat20   0 9649m 4.8g 4520 R 79.8 62.6   5:44.99 java
21694 tomcat20   0 9649m 4.8g 4520 S 74.3 62.6   5:39.40 java
 7889 tomcat20   0 9649m 4.8g 4520 S 29.7 62.6   4:24.44 java
 7878 tomcat20   0 9649m 4.8g 4520 S 27.8 62.6   4:36.82 java
21701 tomcat20   0 9649m 4.8g 4520 S 26.0 62.6   5:49.83 java

After taking thread dump, I used Threadlogic which will show Native-ID 
as column which corresponds to PID shown above.


https://java.net/projects/threadlogic

This way it helps to determine the thread that might potentially causing 
high cpu.





Can you post your  and/or  configuration?





Okay, so you are using the default (BIO) connector with no special
configuration. I see you are running Tomcat 7.0.47. Could you please
re-test with latest 7.0.68? Your version is 2.5 years old and there
may be a performance bug fixed.


Sure.



- -chris
-BEGIN PGP SIGNATURE-
Comment: GPGTools - http://gpgtools.org
Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/

iEYEARECAAYFAlbh8LQACgkQ9CaO5/Lv0PA89wCdHBDkMdKa/9JkNLGY80Ygrl0/
NpkAoL5kQ8cx/Qr9WcOpdHw3DxvurcrD
=Ckbq
-END PGP SIGNATURE-

-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org



-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org



Re: Prevent Sending of SSL Root Certificate

2016-03-10 Thread Tad Marko
On Thu, Mar 10, 2016 at 3:59 PM, Christopher Schultz
 wrote:
> Tad,
>
> On 3/10/16 4:03 PM, Tad Marko wrote:
> > Is it possible to tell tomcat to NOT send the root for a
> > certificate chain?
>
> Yep.
>
> ...
>
> Just remove the root cert from your keystore, and Tomcat will stop
> sending it.
>
> If you have further questions, please post the output of the following
> command in your next post:
>
> $ keytool -keystore  -list
>

The CA is not in my keystore:

Keystore type: JKS
Keystore provider: SUN

Your keystore contains 3 entries

my.domain.tld, Mar 10, 2016, PrivateKeyEntry,
Certificate fingerprint (SHA1):
AE:DB:AF:8D:19:D6:38:D8:EB:5A:C1:5D:E6:D2:C4:8B:5F:58:84:6F
intermed, Mar 10, 2016, trustedCertEntry,
Certificate fingerprint (SHA1):
27:AC:93:69:FA:F2:52:07:BB:26:27:CE:FA:CC:BE:4E:F9:C3:19:B8
cross, Mar 10, 2016, trustedCertEntry,
Certificate fingerprint (SHA1):
34:0B:28:80:F4:46:FC:C0:4E:59:ED:33:F5:2B:3D:08:D6:24:29:64

-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org



Re: Idle Thread high CPU

2016-03-10 Thread Christopher Schultz
-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1

Rallavagu,

On 3/10/16 4:02 PM, Rallavagu wrote:
> On 3/10/16 11:54 AM, Christopher Schultz wrote:
>> Are you sure you have matched-up the correct thread within the
>> JVM that is using all that CPU?
> 
>> How are you measuring the CPU usage?
> 
> It would the ID output from "top -H" mapping to "Native ID" in
> thread dump.

My version of 'top' (Debian Linux) doesn't show thread ids. :(

I seem to recall having to do some backflips to convert native thread
id to Java thread id. Can you explain what you've done to do that?

>> Can you post your  and/or  configuration?
> 
>  maxThreads="500" connectionTimeout="2" redirectPort="28443" />

Okay, so you are using the default (BIO) connector with no special
configuration. I see you are running Tomcat 7.0.47. Could you please
re-test with latest 7.0.68? Your version is 2.5 years old and there
may be a performance bug fixed.

- -chris
-BEGIN PGP SIGNATURE-
Comment: GPGTools - http://gpgtools.org
Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/

iEYEARECAAYFAlbh8LQACgkQ9CaO5/Lv0PA89wCdHBDkMdKa/9JkNLGY80Ygrl0/
NpkAoL5kQ8cx/Qr9WcOpdHw3DxvurcrD
=Ckbq
-END PGP SIGNATURE-

-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org



Re: Prevent Sending of SSL Root Certificate

2016-03-10 Thread Christopher Schultz
-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1

Tad,

On 3/10/16 4:03 PM, Tad Marko wrote:
> Is it possible to tell tomcat to NOT send the root for a
> certificate chain?

Yep.

> I am trying to support some old VeriFone terminals that are pretty
> limited what they expect when dealing with SSL. I've gotten a new
> domain certificate issued by Go Daddy, and in my keystore I've
> installed this along with the Go Daddy intermediate cert and the
> cross that links it back to the older SHA-1 root that my devices
> understand. When negotiating an SSL connection, tomcat is sending
> the domain, intermediate and cross certs that are in my keystore,
> but it is also finding the root and sending that down. This is
> confusing my devices as they interpret this to mean this is a 
> self-signed key chain and they then refuse to talk to my server.

Just remove the root cert from your keystore, and Tomcat will stop
sending it.

If you have further questions, please post the output of the following
command in your next post:

$ keytool -keystore  -list

- -chris
-BEGIN PGP SIGNATURE-
Comment: GPGTools - http://gpgtools.org
Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/

iEYEARECAAYFAlbh7jYACgkQ9CaO5/Lv0PCSdACfbKVVaStFZ+hkmftdHnHhvZrp
UYwAoKSoHTTHZW/FeVlJVW7ysp7tpVGu
=qllo
-END PGP SIGNATURE-

-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org



Re: NullPointerException in MemoryRealm after upgrading to Tomcat 8.0.32 from 7.0.26

2016-03-10 Thread Mark Thomas
On 10/03/2016 21:40, Jason Overland wrote:
> Chris,
> 
> On Thu, Mar 10, 2016 at 6:18 AM, Christopher Schultz
>  wrote:
>> Give this patch a try:
>> ...
>> I have no idea how the options get parsed; we'll see if this simple
>> implementation will get you going again.
>>
>> -chris
>>
> 
> The parsing is working correctly.  After applying the patch I could
> login successfully.  Then I added digest=SHA to jaas.config and it
> stopped working ("wrong password").
> 
> On further inspection I found that our CallbackHandler was digesting
> the password before passing it back to the JAASMemoryModule, however
> CredentialHandler expects inputCredentials to be plaintext.  So I
> commented out the part of our CallbackHandler that digests the
> password and now it's working.  That seems ok to me.
> 
> So I think this patch is sufficient to get us going again.  Thanks for
> the quick turnaround.  If this patch looks good to everyone, do you
> think it can make it into the next Tomcat patch release?

Great. I've been testing a patch along the same lines locally and see
the same results. I'll get it committed before the next 8.0.x release.

Mark


-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org



Re: Intermittent ClassNotFoundException in Jasper EL evaluation

2016-03-10 Thread Mark Thomas
On 10/03/2016 21:16, jimi.hulleg...@svensktnaringsliv.se wrote:
> On Thursday, March 10, 2016 11:20 AM, ma...@apache.org wrote:
>> 
>>> 3. Why is the problem not limited to the first request for a jsp
>>> page?
>> 
>> Because EL imports may be dynamic so the EL has to be evaluated on
>> execution.
> 
> I'm not really sure I follow you now. Can you explain what you mean
> with dynamic imports in this regard? I can't see any mentioning of it
> in the specs
> (http://download.oracle.com/otn-pub/jcp/el-3_0-fr-eval-spec/EL3.0.FR.pdf).

There is nothing stopping a JSP author obtaining a reference to the
ImportHandler and conditionally adding classes to import. The
configuration of the ImportHandler could change on every call to the page.

>  Can you give a concrete example, where an EL expression element
> would need to be evaluated as a potential class, again and again for
> each request?

See above.

> Because the way I see it, "foo" in ${foo.bar} in some jsp page only
> needs to be evaluated once. If "foo" corresponded to a class,
> matching one of the imports in the jsp, then this lookup will not
> change unless the jsp code changes. And the same is true if the
> lookup failes with a ClassNotFoundException. What am I missing?

See above.

>>> 4. Why isn't the ClassNotFoundException logged by the
>>> ImportHandler?
>> 
>> Because it is expected and logging it provides no value to the
>> user.
> 
> Well, I happen to disagree. :) In our case, if we could see all these
> stacktraces in the log, by enabling DEBUG/TRACE log level on the
> ImportHandler class for example, then we would quickly see just how
> big this problem is (ie how often this happens), on what jsp pages it
> happens, and what EL expressions cause it.

We'll have to agree to disagree on that one. If you are concerned about
a performance issue then you need to know where to look to enable debug
logging. A profiler will tell you where to look and at that point you
don't need the debug logging.

> On my local machine, I was able to "catch" the ClassNotFoundException
> using a debug breakpoint in my IDE, and using the breakpoint hit
> count I could see that this exception is thrown and handled (ie
> ignored) by the ImportHandler 2700 times for a single request to the
> start page.
> 
> 
>> The JavaEE specs are very big on backwards compatibility. 
>> Therefore, the chances of changing the spec syntax to fix this are
>> zero.
> 
> OK. Fair enough. I agree with you that backwards compatibility is
> important. But there are ways to fix this while still keeping the
> backwards compatibility. For example by making it possible to turn
> off this feature (globally, per webapp, or per jsp), where the
> default is "on". Wouldn't you agree that such a change would be 100%
> backwards compatible?

The code in question is in the JSP API JAR and there is no configuration
API available. The only option would be a system property which would
make it a global configuration option.

> And at the same time it would more or less
> solve this problem completely. Because people who experience the
> mentioned problems could turn of this setting globally, and then only
> enable it for those specific jsp pages where it is needed (and these
> pages would then be cleaned up, so that no EL-references exist
> without specific scope unless they are known to never be null. Tada,
> problem solved! =)

Um, no. See above. This would have to be a global option. It would work
for some users/pages but not for others.

> Actually... this wouldn't even need to be in the specifications... I
> can't see any harm in a EL implementation introducing such settings
> on its own, before the specifications can "catch up". That way you
> could basically introduce this fix in trunk right now, and have it
> tested and out in a stable release in no time =)

We have added such spec breaking options in the past as you'll see if
you look at the configuration docs for system properties. Generally, I'm
not a fan of configuration via system property. It is usually too blunt
a tool and doesn't provide the per webapp control that most users need.

> On Thursday, March 10, 2016 1:24 PM, ma...@apache.org wrote:
>> 
>> The issue is in ScopedAttributeELResolver.
>> 
>> ScopedAttributeELResolver checks the
>> page/request/session/application context first and only if it
>> doesn't find the attribute there does it try loading the class.
> 
> When debugging this in my IDE, I could see this in action. I also
> noticed that the ImportHandler that performs the class lookup is
> fetched from the ELContext object. So if I could wrap that object, I
> could simply return null in the method getImportHandler(), thus
> disabling this functionallity and therefore solving the performance
> problem for us. But I couldn't find any way to wrap the ELContext,
> for some reason. Can it be done, using standard code or config?

No.

> Or can this "import logic" be disabled some other way?

You can do most things via reflection, 

Re: NullPointerException in MemoryRealm after upgrading to Tomcat 8.0.32 from 7.0.26

2016-03-10 Thread Jason Overland
Chris,

On Thu, Mar 10, 2016 at 6:18 AM, Christopher Schultz
 wrote:
> Give this patch a try:
> ...
> I have no idea how the options get parsed; we'll see if this simple
> implementation will get you going again.
>
> -chris
>

The parsing is working correctly.  After applying the patch I could
login successfully.  Then I added digest=SHA to jaas.config and it
stopped working ("wrong password").

On further inspection I found that our CallbackHandler was digesting
the password before passing it back to the JAASMemoryModule, however
CredentialHandler expects inputCredentials to be plaintext.  So I
commented out the part of our CallbackHandler that digests the
password and now it's working.  That seems ok to me.

So I think this patch is sufficient to get us going again.  Thanks for
the quick turnaround.  If this patch looks good to everyone, do you
think it can make it into the next Tomcat patch release?

Also, yesterday I made a bug on bugzilla.

https://bz.apache.org/bugzilla/show_bug.cgi?id=59154

Thanks again,
Jason

-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org



Re: Idle Thread high CPU

2016-03-10 Thread Rallavagu



On 3/10/16 1:02 PM, Konstantin Kolinko wrote:

2016-03-10 22:54 GMT+03:00 Christopher Schultz :

Rallavagu,

On 3/10/16 2:11 PM, Rallavagu wrote:

 From a thread dump and corresponding "top" output it is reported
that the following thread is consuming significant CPU (around
80%)

"http-bio-28080-exec-437" daemon prio=10 tid=0x7f4acc0de000
nid=0x54ce waiting on condition [0x7f4b038f7000]
java.lang.Thread.State: TIMED_WAITING (parking) at
sun.misc.Unsafe.park(Native Method) - parking to wait for
<0x00071846da50> (a
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)



at

java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:226)



at

java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.

awaitNanos(AbstractQueuedSynchronizer.java:2082)


  at
java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java

:467)




at org.apache.tomcat.util.threads.TaskQueue.poll(TaskQueue.java:86)

at
org.apache.tomcat.util.threads.TaskQueue.poll(TaskQueue.java:32)
at
java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.jav

a:1068)


  at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.j

ava:1130)


  at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.

java:615)


  at java.lang.Thread.run(Thread.java:744)

Considering it is a thread from tomcat's thread pool and it
suggests that it is "java.lang.Thread.State: TIMED_WAITING
(parking)" wonder why this shows up as consuming high CPU. Any
clues would be highly appreciated. Thanks in advance.


Are you sure you have matched-up the correct thread within the JVM
that is using all that CPU?

How are you measuring the CPU usage?

Can you post your  and/or  configuration?



Also


Sorry for not providing this information upfront.

1. Tomcat version =?

7.0.47

2. Java version =?

Java HotSpot(TM) 64-Bit Server VM (build 24.51-b03, mixed mode)

3. You need to take several (e.g. 3) thread dumps to see whether there
is some progress /change of state.


There is some progress. But, similar but different thread(s) show high 
cpu in some of those.




The last similar thread in search results for users@ list is in May
2015 "High cpu on Tomcat 8", a year ago. As there were no such threads
recently, I think that bug has already been fixed.


The only Tomcat code in that stacktrace is
org.apache.tomcat.util.threads.TaskQueue.

The connector type can be guessed as "bio" from the thread name, but
as a thread name can be configured on an Executor, this identification
is not reliable.

Best regards,
Konstantin Kolinko

-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org



-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org



RE: Intermittent ClassNotFoundException in Jasper EL evaluation

2016-03-10 Thread jimi.hullegard
On Thursday, March 10, 2016 11:20 AM, ma...@apache.org wrote:
> 
> > 3. Why is the problem not limited to the first request for a jsp page? 
> 
> Because EL imports may be dynamic so the EL has to be evaluated on execution.

I'm not really sure I follow you now. Can you explain what you mean with 
dynamic imports in this regard? I can't see any mentioning of it in the specs 
(http://download.oracle.com/otn-pub/jcp/el-3_0-fr-eval-spec/EL3.0.FR.pdf).

Can you give a concrete example, where an EL expression element would need to 
be evaluated as a potential class, again and again for each request?

Because the way I see it, "foo" in ${foo.bar} in some jsp page only needs to be 
evaluated once. If "foo" corresponded to a class, matching one of the imports 
in the jsp, then this lookup will not change unless the jsp code changes. And 
the same is true if the lookup failes with a ClassNotFoundException. What am I 
missing?


> > 4. Why isn't the ClassNotFoundException logged by the ImportHandler?
> 
> Because it is expected and logging it provides no value to the user.

Well, I happen to disagree. :)
In our case, if we could see all these stacktraces in the log, by enabling 
DEBUG/TRACE log level on the ImportHandler class for example, then we would 
quickly see just how big this problem is (ie how often this happens), on what 
jsp pages it happens, and what EL expressions cause it.

On my local machine, I was able to "catch" the ClassNotFoundException using a 
debug breakpoint in my IDE, and using the breakpoint hit count I could see that 
this exception is thrown and handled (ie ignored) by the ImportHandler 2700 
times for a single request to the start page. 


> The JavaEE specs are very big on backwards compatibility. 
> Therefore, the chances of changing the spec syntax to fix this are zero.

OK. Fair enough. I agree with you that backwards compatibility is important. 
But there are ways to fix this while still keeping the backwards compatibility. 
For example by making it possible to turn off this feature (globally, per 
webapp, or per jsp), where the default is "on". Wouldn't you agree that such a 
change would be 100% backwards compatible? And at the same time it would more 
or less solve this problem completely. Because people who experience the 
mentioned problems could turn of this setting globally, and then only enable it 
for those specific jsp pages where it is needed (and these pages would then be 
cleaned up, so that no EL-references exist without specific scope unless they 
are known to never be null. Tada, problem solved! =)

Actually... this wouldn't even need to be in the specifications... I can't see 
any harm in a EL implementation introducing such settings on its own, before 
the specifications can "catch up". That way you could basically introduce this 
fix in trunk right now, and have it tested and out in a stable release in no 
time =)



On Thursday, March 10, 2016 1:24 PM, ma...@apache.org wrote:
> 
> The issue is in ScopedAttributeELResolver.
> 
> ScopedAttributeELResolver checks the page/request/session/application context
> first and only if it doesn't find the attribute there does it try loading the 
> class.

When debugging this in my IDE, I could see this in action. I also noticed that 
the ImportHandler that performs the class lookup is fetched from the ELContext 
object. So if I could wrap that object, I could simply return null in the 
method getImportHandler(), thus disabling this functionallity and therefore 
solving the performance problem for us. But I couldn't find any way to wrap the 
ELContext, for some reason. Can it be done, using standard code or config? Or 
can this "import logic" be disabled some other way?

There really should be a way for users to disable this, if the functionallity 
is not used and it just is causing problems. And, like I said above, that could 
be done without breaking the specs. And an alternative way to the way above, 
could be like I mentioned before, to add a configuration option that forces the 
class name to begin with a capital letter. That way ${Boolean.TRUE}, 
${Integer.MAX_VALUE} and ${MyClass.myStaticField} would still work, while 
${foo.bar} etc would simply be ignored. As long as the configuration option 
would default to false (ie lower case first letter is allowed, as per the 
specification) it wouldn't break the specification unless the user deliberately 
told it to (which is fine, right?).

It would be really nice to get your input on these suggestions. And if you 
don't like them, could you explain why? If your opinion is "We need to stick 
100% to the specification, and never ever give even the expert user any way to 
override this, ever", then I would say that such a view causes more harm than 
good. :)

Regards
/Jimi

-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org



Re: Idle Thread high CPU

2016-03-10 Thread Konstantin Kolinko
2016-03-10 22:54 GMT+03:00 Christopher Schultz :
> Rallavagu,
>
> On 3/10/16 2:11 PM, Rallavagu wrote:
>> From a thread dump and corresponding "top" output it is reported
>> that the following thread is consuming significant CPU (around
>> 80%)
>>
>> "http-bio-28080-exec-437" daemon prio=10 tid=0x7f4acc0de000
>> nid=0x54ce waiting on condition [0x7f4b038f7000]
>> java.lang.Thread.State: TIMED_WAITING (parking) at
>> sun.misc.Unsafe.park(Native Method) - parking to wait for
>> <0x00071846da50> (a
>> java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
>>
>>
> at
>> java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:226)
>>
>>
> at
>> java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.
> awaitNanos(AbstractQueuedSynchronizer.java:2082)
>>
>>  at
>> java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java
> :467)
>>
>>
> at org.apache.tomcat.util.threads.TaskQueue.poll(TaskQueue.java:86)
>> at
>> org.apache.tomcat.util.threads.TaskQueue.poll(TaskQueue.java:32)
>> at
>> java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.jav
> a:1068)
>>
>>  at
>> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.j
> ava:1130)
>>
>>  at
>> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.
> java:615)
>>
>>  at java.lang.Thread.run(Thread.java:744)
>>
>> Considering it is a thread from tomcat's thread pool and it
>> suggests that it is "java.lang.Thread.State: TIMED_WAITING
>> (parking)" wonder why this shows up as consuming high CPU. Any
>> clues would be highly appreciated. Thanks in advance.
>
> Are you sure you have matched-up the correct thread within the JVM
> that is using all that CPU?
>
> How are you measuring the CPU usage?
>
> Can you post your  and/or  configuration?


Also
1. Tomcat version =?
2. Java version =?
3. You need to take several (e.g. 3) thread dumps to see whether there
is some progress /change of state.

The last similar thread in search results for users@ list is in May
2015 "High cpu on Tomcat 8", a year ago. As there were no such threads
recently, I think that bug has already been fixed.


The only Tomcat code in that stacktrace is
org.apache.tomcat.util.threads.TaskQueue.

The connector type can be guessed as "bio" from the thread name, but
as a thread name can be configured on an Executor, this identification
is not reliable.

Best regards,
Konstantin Kolinko

-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org



Re: Idle Thread high CPU

2016-03-10 Thread Rallavagu



On 3/10/16 11:54 AM, Christopher Schultz wrote:

-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1

Rallavagu,

On 3/10/16 2:11 PM, Rallavagu wrote:

 From a thread dump and corresponding "top" output it is reported
that the following thread is consuming significant CPU (around
80%)

"http-bio-28080-exec-437" daemon prio=10 tid=0x7f4acc0de000
nid=0x54ce waiting on condition [0x7f4b038f7000]
java.lang.Thread.State: TIMED_WAITING (parking) at
sun.misc.Unsafe.park(Native Method) - parking to wait for
<0x00071846da50> (a
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)



at

java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:226)



at

java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.

awaitNanos(AbstractQueuedSynchronizer.java:2082)


  at
java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java

:467)




at org.apache.tomcat.util.threads.TaskQueue.poll(TaskQueue.java:86)

at
org.apache.tomcat.util.threads.TaskQueue.poll(TaskQueue.java:32)
at
java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.jav

a:1068)


  at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.j

ava:1130)


  at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.

java:615)


  at java.lang.Thread.run(Thread.java:744)

Considering it is a thread from tomcat's thread pool and it
suggests that it is "java.lang.Thread.State: TIMED_WAITING
(parking)" wonder why this shows up as consuming high CPU. Any
clues would be highly appreciated. Thanks in advance.


Are you sure you have matched-up the correct thread within the JVM
that is using all that CPU?

How are you measuring the CPU usage?


It would the ID output from "top -H" mapping to "Native ID" in thread dump.



Can you post your  and/or  configuration?




Thanks



- -chris
-BEGIN PGP SIGNATURE-
Comment: GPGTools - http://gpgtools.org
Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/

iEYEARECAAYFAlbh0Q0ACgkQ9CaO5/Lv0PDDpACfbmueOC3FIVAImoIY6P0GYgla
iUUAnRimWHnwhvNR+q2KKyypxnFcoHBi
=o7Ac
-END PGP SIGNATURE-

-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org



-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org



Prevent Sending of SSL Root Certificate

2016-03-10 Thread Tad Marko
Howdy!

Is it possible to tell tomcat to NOT send the root for a certificate chain?
I am trying to support some old VeriFone terminals that are pretty limited
what they expect when dealing with SSL. I've gotten a new domain
certificate issued by Go Daddy, and in my keystore I've installed this
along with the Go Daddy intermediate cert and the cross that links it back
to the older SHA-1 root that my devices understand. When negotiating an SSL
connection, tomcat is sending the domain, intermediate and cross certs that
are in my keystore, but it is also finding the root and sending that down.
This is confusing my devices as they interpret this to mean this is a
self-signed key chain and they then refuse to talk to my server.

Thanks,
Tad


Re: Idle Thread high CPU

2016-03-10 Thread Christopher Schultz
-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1

Rallavagu,

On 3/10/16 2:11 PM, Rallavagu wrote:
> From a thread dump and corresponding "top" output it is reported
> that the following thread is consuming significant CPU (around
> 80%)
> 
> "http-bio-28080-exec-437" daemon prio=10 tid=0x7f4acc0de000 
> nid=0x54ce waiting on condition [0x7f4b038f7000] 
> java.lang.Thread.State: TIMED_WAITING (parking) at
> sun.misc.Unsafe.park(Native Method) - parking to wait for
> <0x00071846da50> (a 
> java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
>
> 
at
> java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:226)
>
> 
at
> java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.
awaitNanos(AbstractQueuedSynchronizer.java:2082)
>
>  at 
> java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java
:467)
>
> 
at org.apache.tomcat.util.threads.TaskQueue.poll(TaskQueue.java:86)
> at
> org.apache.tomcat.util.threads.TaskQueue.poll(TaskQueue.java:32) 
> at 
> java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.jav
a:1068)
>
>  at 
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.j
ava:1130)
>
>  at 
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.
java:615)
>
>  at java.lang.Thread.run(Thread.java:744)
> 
> Considering it is a thread from tomcat's thread pool and it
> suggests that it is "java.lang.Thread.State: TIMED_WAITING
> (parking)" wonder why this shows up as consuming high CPU. Any
> clues would be highly appreciated. Thanks in advance.

Are you sure you have matched-up the correct thread within the JVM
that is using all that CPU?

How are you measuring the CPU usage?

Can you post your  and/or  configuration?

- -chris
-BEGIN PGP SIGNATURE-
Comment: GPGTools - http://gpgtools.org
Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/

iEYEARECAAYFAlbh0Q0ACgkQ9CaO5/Lv0PDDpACfbmueOC3FIVAImoIY6P0GYgla
iUUAnRimWHnwhvNR+q2KKyypxnFcoHBi
=o7Ac
-END PGP SIGNATURE-

-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org



Idle Thread high CPU

2016-03-10 Thread Rallavagu

All,

From a thread dump and corresponding "top" output it is reported that 
the following thread is consuming significant CPU (around 80%)


"http-bio-28080-exec-437" daemon prio=10 tid=0x7f4acc0de000 
nid=0x54ce waiting on condition [0x7f4b038f7000]

   java.lang.Thread.State: TIMED_WAITING (parking)
at sun.misc.Unsafe.park(Native Method)
- parking to wait for  <0x00071846da50> (a 
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
at 
java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:226)
at 
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2082)
at 
java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467)

at org.apache.tomcat.util.threads.TaskQueue.poll(TaskQueue.java:86)
at org.apache.tomcat.util.threads.TaskQueue.poll(TaskQueue.java:32)
at 
java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)
at 
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)
at 
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)

at java.lang.Thread.run(Thread.java:744)

Considering it is a thread from tomcat's thread pool and it suggests 
that it is "java.lang.Thread.State: TIMED_WAITING (parking)" wonder why 
this shows up as consuming high CPU. Any clues would be highly 
appreciated. Thanks in advance.


-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org



Re: Tomcat mod_jk confirmation

2016-03-10 Thread Christopher Schultz
-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1

Michael,

On 3/10/16 11:31 AM, Michael Fox wrote:
> I am running Red Hat Linux version 7.2 Apache version 2.4.6 Tomcat
> version 9.0.0.M1 Tomcat connector version 1.2.41
> 
> I have configured Tomcat and Apache for Tomcat calls to be handled
> by Apache.  I am not getting any errors when starting up Tomcat or 
> Apache, but how can I tell if the Tomcat calls are being handled
> by the Apache server?

If httpd is handling the requests, there should be entries in the
access_log. httpd will log those requests even if they are being
proxied to Tomcat, so presence in that log file doesn't mean that
something is wrong.

If you turn-up the log level for mod_jk, then you should be able to
see what mod_jk is doing.

Most common mistake with mod_jk configuration: placing JkMount
directives at the top-level of httpd.conf and not in a VirtualHost.
These JkMounts are ignored unless you explicitly do a JkMountCopy in
the VirtualHost. I would recommend against global JkMounts... just
move them into the proper VirtualHost if that's the problem.

- -chris
-BEGIN PGP SIGNATURE-
Comment: GPGTools - http://gpgtools.org
Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/

iEYEARECAAYFAlbhot0ACgkQ9CaO5/Lv0PB2owCfWmm+osP0G9fbvBqEtXv+1Ql/
mFkAnRDxVEV+UkYEdwYpVLVAzc8Uq2/m
=t3LT
-END PGP SIGNATURE-

-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org



Tomcat mod_jk confirmation

2016-03-10 Thread Michael Fox
I am running
Red Hat Linux version 7.2
Apache version 2.4.6
Tomcat version 9.0.0.M1
Tomcat connector version 1.2.41

I have configured Tomcat and Apache for Tomcat calls to be handled by Apache.  
I am not getting any errors when starting up Tomcat or Apache, but how can I 
tell if the Tomcat calls are being handled by the Apache server?

Thanks,
Mike


Michael Fox
Database/System Administrator
Sidney Kimmel Comprehensive Cancer Center
Johns Hopkins University
fo...@jhmi.edu




Re: NullPointerException in MemoryRealm after upgrading to Tomcat 8.0.32 from 7.0.26

2016-03-10 Thread Christopher Schultz
Jason,

On 3/9/16 1:19 PM, Jason Overland wrote:
> For what it's worth, that analysis & approach to fixing seem
> reasonable to me.  Yes I'll be glad to file a bug report and test a
> patch.

Give this patch a try:

 CUT =

Index: java/org/apache/catalina/realm/JAASMemoryLoginModule.java
===
--- java/org/apache/catalina/realm/JAASMemoryLoginModule.java   (revision
1725851)
+++ java/org/apache/catalina/realm/JAASMemoryLoginModule.java   (working copy)
@@ -18,6 +18,7 @@

 import java.io.File;
 import java.io.IOException;
+import java.security.NoSuchAlgorithmException;
 import java.security.Principal;
 import java.util.Map;

@@ -221,6 +222,18 @@
 if (options.get("pathname") != null)
 this.pathname = (String) options.get("pathname");

+// TODO: This should probably have many more options available
+MessageDigestCredentialHandler ch = new
MessageDigestCredentialHandler();
+Object digestAlgorithm = options.get("digest");
+if(digestAlgorithm instanceof String) {
+try {
+ch.setAlgorithm((String)digestAlgorithm);
+} catch (NoSuchAlgorithmException nsae) {
+log.error("Cannot initialize credential handler", nsae);
+}
+}
+setCredentialHandler(ch);
+
 // Load our defined Principals
 load();



Once you're re-built, change your JAAS configuration to:

jaas.config:
/** JAAS Login Configuration for the Application **/

JAASTomcat {
   org.apache.catalina.realm.JAASMemoryLoginModule required debug=true
digest=SHA;
};

(note the digest=SHA is on the same line with everything else)

I have no idea how the options get parsed; we'll see if this simple
implementation will get you going again.

-chris

-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org



Re: Tomcat 8 Shared Classloader

2016-03-10 Thread Mark Thomas
On 10/03/2016 10:34, Theo Sweeny wrote:
> Hi Mark,
> 
> From: Mark Thomas 
> Sent: 10 March 2016 10:22
> To: Tomcat Users List
> Subject: Re: Tomcat 8 Shared cCassloader
> 
> On 10/03/2016 10:19, Theo Sweeny wrote:
>> Hello - I've recently noticed that there is no reference to shared.loader in 
>> this Tomcat How To guide seen here -
>>
>> https://tomcat.apache.org/tomcat-8.0-doc/class-loader-howto.html
>>
>> Has the shared.loader property from within catalina.properties been 
>> deprecated?
> 
> No. It was removed because it was rarely used / needed.
> 
>> If so - should shared.loader resources now be loaded via the common.loader 
>> instead?
> 
> You can re-introduce the shared loader (and the catalina loader) if you
> want to.
> 
> Mark
> 
> Thanks for the heads up.
> 
> The motivation for asking this question is that we do have issues with web 
> apps classes not being removed cleanly from memory after they are undeployed, 
> which eventually results Perm Gen errors.
> 
> Is it safe to say that using shared.loader would not contribute to this issue?

That isn't strictly true. Using the shared loader will (probably) mask
the issue but the issue will still be there. In addition, you'll have
some extra problems. Namely:

- all webapps have to use exactly the same version of the JAR in the
  shared loader
- you'll need to exclude those JARs from your build process
- you'll have to stop Tomcat if you want to update the JARs in the
  shared loader

If you can live with those restrictions then you should be OK.

Personally, I'd be looking at fixing the memory leaks but that is just me.

Mark

-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org



Re: Intermittent ClassNotFoundException in Jasper EL evaluation

2016-03-10 Thread Mark Thomas
On 10/03/2016 10:19, Mark Thomas wrote:
> On 10/03/2016 08:12, jimi.hulleg...@svensktnaringsliv.se wrote:



>> Then surely one can look at the other implementations, and what they did to 
>> avoid this problem. But one thing off the top of my head would be to at 
>> least avoid doing that class lookup in cases where it couldn't be a static 
>> field reference (like ${name}, since there is no dot after "name", there is 
>> no reason to check if "name" is a class reference, right?).
> 
> It has been a while since I looked at this. I need to remind myself why
> I thought it needed to be implemented this way. I recall looking at this
> at least once before but another look wouldn't hurt.

Having reviewed this, here is some useful background.

The issue is in ScopedAttributeELResolver.

ScopedAttributeELResolver checks the page/request/session/application
context first and only if it doesn't find the attribute there does it
try loading the class.

If ScopedAttributeELResolver is asked to resolve "foo" it can't
differentiate between ${ foo } which doesn't require a class lookup and
${ foo.bar } which does require a class lookup. Therefore
ScopedAttributeELResolver will always do a class lookup if it didn't
find an a matching attribute.

These lookups pass through various standard APIs (EL and JSP specs) so
we can't simply add a flag to the method to indicate if a class lookup
is required or not.

We can't use ThreadLocals because either:
- the ThreadLocal would need to be defined in a specification class
  which isn't allowed because we can't change the public API; or
- the ThreadLocal would need to be defined in Tomcat and referenced in
  a spec class which isn't allowed because the spec JARs have to be
  independent.

However, all is not lost. I think I have a solution based on ELContext
getContext()/putContext(). Keep an eye on
https://bz.apache.org/bugzilla/show_bug.cgi?id=57583 for progress reports.

-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org



Re: a lot of 502 error with using version 7.0.68-1

2016-03-10 Thread Christopher Schultz
Kenichi,

On 3/10/16 4:52 AM, Kenichi MASUDA wrote:
> Hi,
> 
> I updated the tomcat on CentOS6 from 7.0.39-1 to 7.0.68-1,
> and it seems to that the 502 proxy error was increased than before.
> 
> The error is below:
> - apache log:  proxy: Error reading from remote server returned by
> - mobile phone display: 502 proxy error. The proxy server received an
> invalid response from an upstream server.
> The proxy server could not handle the request POST /foo/bar
> 
> Is there anyone know or encounter the same situation?
> 
> Apache: 2.24-1
> tomcat: 7.0.68-1

Can you provide more information. For example, what do the logs say?
Also, how are you connecting httpd -> Tomcat?

-chris

-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org



Re: Tomcat 8 Shared Classloader

2016-03-10 Thread Theo Sweeny
Hi Mark,

From: Mark Thomas 
Sent: 10 March 2016 10:22
To: Tomcat Users List
Subject: Re: Tomcat 8 Shared cCassloader

On 10/03/2016 10:19, Theo Sweeny wrote:
> Hello - I've recently noticed that there is no reference to shared.loader in 
> this Tomcat How To guide seen here -
>
> https://tomcat.apache.org/tomcat-8.0-doc/class-loader-howto.html
>
> Has the shared.loader property from within catalina.properties been 
> deprecated?

No. It was removed because it was rarely used / needed.

> If so - should shared.loader resources now be loaded via the common.loader 
> instead?

You can re-introduce the shared loader (and the catalina loader) if you
want to.

Mark

Thanks for the heads up.

The motivation for asking this question is that we do have issues with web apps 
classes not being removed cleanly from memory after they are undeployed, which 
eventually results Perm Gen errors.

Is it safe to say that using shared.loader would not contribute to this issue?

Theo


-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org

Avios Group (AGL) Ltd is a limited company registered in England (registered 
number 2260073 and VAT number 512566754) whose registered address is Astral 
Towers, Betts Way, London Road, Crawley, West Sussex RH10 9XY . Avios Group 
(AGL) Limited is part of the IAG group of companies This email and any files 
transmitted with it are confidential and intended solely for the use of the 
individual or entity to whom they are addressed. If you have received this 
email in error please notify the system manager.

-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org



Re: Tomcat 8 Shared cCassloader

2016-03-10 Thread Mark Thomas
On 10/03/2016 10:19, Theo Sweeny wrote:
> Hello - I've recently noticed that there is no reference to shared.loader in 
> this Tomcat How To guide seen here -
> 
> https://tomcat.apache.org/tomcat-8.0-doc/class-loader-howto.html
> 
> Has the shared.loader property from within catalina.properties been 
> deprecated?

No. It was removed because it was rarely used / needed.

> If so - should shared.loader resources now be loaded via the common.loader 
> instead?

You can re-introduce the shared loader (and the catalina loader) if you
want to.

Mark


-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org



Re: Intermittent ClassNotFoundException in Jasper EL evaluation

2016-03-10 Thread Mark Thomas
On 10/03/2016 08:12, jimi.hulleg...@svensktnaringsliv.se wrote:
> On Wednesday, March 9, 2016 8:22 PM, ma...@apache.org wrote:
>> It is a known 'feature' of the new EL requirements added in 3.0. The EL 
>> parser can't differentiate 
>> between an attribute without a scope and a reference to an static field.
>>
>> See https://bz.apache.org/bugzilla/show_bug.cgi?id=57583
> 
> Interesting. I can see how that in a way could explain the the performance 
> regression. But what I don't understand is:
> 
> 1. Why would this cause a ClassNotFoundException? What class exactly is it 
> trying to load?

For each imported package, it is trying to load package.attributeName

> 2. Why is this happening seemingly intermittently, with different EL 
> variables each time?

It isn't. It is happening every time. You are just observing it
intermittently. Probably because class loading is serial by default and
how observable this is depends on how many threads are trying to load
classes in parallel. Note that 9.0.x will use  ParallelWebappClassLoader
by default from the next release and you can always switch to that class
loader in 8.0.x

> 3. Why is the problem not limited to the first request for a jsp page? We see 
> this problem even days after a restart, for jsp-pages that definitely have 
> been used multiple times already, with the same state for it's variables.

Because EL imports may be dynamic so the EL has to be evaluated on
execution.

> 4. Why isn't the ClassNotFoundException logged by the ImportHandler?

Because it is expected and logging it provides no value to the user.

> 5. Why would this change between Tomcat 7 and Tomcat 8, with the exact same 
> webapp war with the exact same web.xml?

Because imports and statics were added in EL 3.0 which was first
implemented in Tomcat 8.

> 6. In our web.xml we specify version="2.5". Wouldn't that mean EL version 
> 2.1? (http://tomcat.apache.org/whichversion.html)

No. The version you specify in web.xml only determines the rules that
Tomcat uses to validate the web.xml. It has zero impact on the behaviour
of the container. The overall JavaEE EG made it very clear that this is
how they expected implementations to behave.

>> The way to avoid it is to always add an explicit scope (page, request, 
>> application, session) to your attributes.
> 
> Is this an official recommendation, stemming from the EL 3.0 specifications? 
> If so, can you point me to that paragraph in the specification document, or 
> some other paper of similar nature? Because when I look at the 3.0 
> specification document, all I see is examples without scope.
> 
> Or is this just a pragmatic response to the Tomcat/Jasper implementation of 
> the specifications?

It is the best recommendation I have right now based on what I know
about the EL spec and Tomcat's implementation of it.

> The reason I ask is that a simple search in our code base show that we have 
> about 10.000 potential candidates for the change you are suggesting, spread 
> out in hundreds of jsp files in our project. And on top of that, the CMS that 
> we use have their own jsp files, and a quick check indicates thousands of 
> potential candidates for the change there as well. So not only would we have 
> to perform a monumental task in our own code (because we would need to 
> determine the scope manually, for each and every variable), we also would 
> need to ask the CMS company to perform the same task.

You aren't the only one in that position.

>> Suggestions for improvements to the default ImportHandler implementation to 
>> make this faster are welcome.
> 
> Well, I am quite pragmatic in my thinking. Is this EL implementation the only 
> implementation with this problem?

I don't know. There aren't that many JSP implementations out there. The
end result should be the same with all of them but how they get there
may vary.

> Then surely one can look at the other implementations, and what they did to 
> avoid this problem. But one thing off the top of my head would be to at least 
> avoid doing that class lookup in cases where it couldn't be a static field 
> reference (like ${name}, since there is no dot after "name", there is no 
> reason to check if "name" is a class reference, right?).

It has been a while since I looked at this. I need to remind myself why
I thought it needed to be implemented this way. I recall looking at this
at least once before but another look wouldn't hurt.

> Otherwise, if more or less all implementations suffer from this problem, then 
> maybe it is the specification that is to blame. Maybe, when introducing the 
> new concept of EL references to static fields, they could use a special 
> prefix, like ${static.Boolean.True}, or they could have had this feature 
> turned off by default, with the possibility to turn it on either per jsp page 
> (using some page directive like <%@ page staticElReferences="true" %>) or 
> webapp-globally in the web.xml. Or, they could simply include the requirement 
> 

Tomcat 8 Shared cCassloader

2016-03-10 Thread Theo Sweeny
Hello - I've recently noticed that there is no reference to shared.loader in 
this Tomcat How To guide seen here -

https://tomcat.apache.org/tomcat-8.0-doc/class-loader-howto.html

Has the shared.loader property from within catalina.properties been deprecated?

If so - should shared.loader resources now be loaded via the common.loader 
instead?


Thank you,


Theo

Avios Group (AGL) Ltd is a limited company registered in England (registered 
number 2260073 and VAT number 512566754) whose registered address is Astral 
Towers, Betts Way, London Road, Crawley, West Sussex RH10 9XY . Avios Group 
(AGL) Limited is part of the IAG group of companies This email and any files 
transmitted with it are confidential and intended solely for the use of the 
individual or entity to whom they are addressed. If you have received this 
email in error please notify the system manager.


a lot of 502 error with using version 7.0.68-1

2016-03-10 Thread Kenichi MASUDA
Hi,

I updated the tomcat on CentOS6 from 7.0.39-1 to 7.0.68-1,
and it seems to that the 502 proxy error was increased than before.

The error is below:
- apache log:  proxy: Error reading from remote server returned by
- mobile phone display: 502 proxy error. The proxy server received an
invalid response from an upstream server.
The proxy server could not handle the request POST /foo/bar

Is there anyone know or encounter the same situation?

Apache: 2.24-1
tomcat: 7.0.68-1


-- 
mailto:masu...@gmail.com


RE: Intermittent ClassNotFoundException in Jasper EL evaluation

2016-03-10 Thread jimi.hullegard
On Wednesday, March 9, 2016 8:22 PM, ma...@apache.org wrote:
> It is a known 'feature' of the new EL requirements added in 3.0. The EL 
> parser can't differentiate 
> between an attribute without a scope and a reference to an static field.
> 
> See https://bz.apache.org/bugzilla/show_bug.cgi?id=57583

Interesting. I can see how that in a way could explain the the performance 
regression. But what I don't understand is:

1. Why would this cause a ClassNotFoundException? What class exactly is it 
trying to load?
2. Why is this happening seemingly intermittently, with different EL variables 
each time?
3. Why is the problem not limited to the first request for a jsp page? We see 
this problem even days after a restart, for jsp-pages that definitely have been 
used multiple times already, with the same state for it's variables.
4. Why isn't the ClassNotFoundException logged by the ImportHandler?
5. Why would this change between Tomcat 7 and Tomcat 8, with the exact same 
webapp war with the exact same web.xml?
6. In our web.xml we specify version="2.5". Wouldn't that mean EL version 2.1? 
(http://tomcat.apache.org/whichversion.html)

> The way to avoid it is to always add an explicit scope (page, request, 
> application, session) to your attributes.

Is this an official recommendation, stemming from the EL 3.0 specifications? If 
so, can you point me to that paragraph in the specification document, or some 
other paper of similar nature? Because when I look at the 3.0 specification 
document, all I see is examples without scope.

Or is this just a pragmatic response to the Tomcat/Jasper implementation of the 
specifications?

The reason I ask is that a simple search in our code base show that we have 
about 10.000 potential candidates for the change you are suggesting, spread out 
in hundreds of jsp files in our project. And on top of that, the CMS that we 
use have their own jsp files, and a quick check indicates thousands of 
potential candidates for the change there as well. So not only would we have to 
perform a monumental task in our own code (because we would need to determine 
the scope manually, for each and every variable), we also would need to ask the 
CMS company to perform the same task.

> Suggestions for improvements to the default ImportHandler implementation to 
> make this faster are welcome.

Well, I am quite pragmatic in my thinking. Is this EL implementation the only 
implementation with this problem? Then surely one can look at the other 
implementations, and what they did to avoid this problem. But one thing off the 
top of my head would be to atleast avoid doing that class lookup in cases where 
it couldn't be a static field reference (like ${name}, since there is no dot 
after "name", there is no reason to check if "name" is a class reference, 
right?). 

Otherwise, if more or less all implementations suffer from this problem, then 
maybe it is the specification that is to blame. Maybe, when introducing the new 
concept of EL references to static fields, they could use a special prefix, 
like ${static.Boolean.True}, or they could have had this feature turned off by 
default, with the possibility to turn it on either per jsp page (using some 
page directive like <%@ page staticElReferences="true" %>) or webapp-globally 
in the web.xml. Or, they could simply include the requirement that the class 
name must start with a capital letter, thus only causing problems for people 
who break the coding standard (ie either having a class name starting with a 
lower case letter, or having a variable name starting with an upper case 
letter).

/Jimi

-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org



Re: Mapping servlet to non English url pattern

2016-03-10 Thread Yuval Schwartz
Christopher,

On Wed, Mar 9, 2016 at 5:38 PM, Christopher Schultz <
ch...@christopherschultz.net> wrote:

> Yuval,
>
> On 3/8/16 4:04 PM, Yuval Schwartz wrote:
> > @WebServlet(name="SomeServlet", urlPatterns={"/help/why-no-work",
> > "/iw/help/למה-לא-עובד",
> > "/iw/help/%D7%9C%D7%9E%D7%94-%D7%9C%D7%90-%D7%A2%D7%95%D7%91%D7%93"}
> > (the last pattern is the same as the second to last, just encoded)
>
> I think that last pattern will not be useful, because Tomcat should be
> doing the encoding internally. For example, if you wanted to map a
> servlet to "/foo bar" you wouldn't have to URL-encode it like /foo%20bar
> in the XML file. But I haven't actually tried it.
>
> (from your other post):
>
> On 3/8/16 5:35 PM, Yuval Schwartz wrote:
> > I did this and it worked:
> > The english patterns show up fine, as expected.
> > The hebrew pattern shows up as a bunch of question marks (eg:
> > -?-)
> > The URLEncoded pattern shows up as wierd symbols (eg: diamond shape,
> > tm symbol).
>
> Hmm, that could certainly be a problem with the tool itself (Netbeans?).
> Can you try running with jconsole just to have a (potentially) clean
> environment?
>
> The encoded pattern showing up as weird symbols sounds .. odd as well.
> Actually, everything about this sounds weird.
>

Do you say this because this is usually something that is straightforward
and simple that doesn't have many issues?


>
> On 3/9/16 4:38 AM, Yuval Schwartz wrote:
> > Are you suggesting I convert all of my source files to UTF-8? Will
> > all new files that are created now be in UTF-8 at least? Because I
> > just created a new servlet for testing purposes (after the
> > -J-Dfile.encoding property was added) and hebrew urls still aren't
> > found.
>
> You definitely don't need to convert your files to UTF-8, but you might
> instead want to use the encoding-proof Unicode escapes instead of the
> raw Hebrew in your source files.
>
> You can use native2ascii to do this for you:
>
> input.txt
> /iw/help/למה-לא-עובד
>
> $ native2ascii -encoding utf8 input.txt
> /iw/help/\u05dc\u05de\u05d4-\u05dc\u05d0-\u05e2\u05d5\u05d1\u05d3
>
> So those are the \u Unicode escapes that you should use in your code.
> Try using those and rebuilding to see if it improves anything.
>
> -chris
>
> -
> To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
> For additional commands, e-mail: users-h...@tomcat.apache.org
>
>