RE: jdbc connction pool issues [EXTERNAL]

[Beard, Shawn]

Well the other tomcat servers had a custom factory specified. That was the only 
difference. (devs wrote their own library for it)
So I looked in tomcat documentation and tried loading a out of the box factory 
that was available. That’s when it was picking up the maxActive setting and I 
started seeing that setting properly reflected in JMX.

​

Shawn   Beard• Sr. Systems Engineer

Middleware Engineering

[cid:image952388.png@65A378B2.7242AA8D]
3840 109th Street   ,   Urbandale   ,   IA  50322

Phone: +1-515-564-2528
Email:  sbe...@wrberkley.com

Website: https://berkleytechnologyservices.com/




[cid:image151023.jpg@DFE8510C.41F52B48]

Technology Leadership Unleashing Business Potential









-Original Message-
From: Christopher Schultz 
Sent: Thursday, December 17, 2020 1:02 PM
To: users@tomcat.apache.org
Subject: Re: jdbc connction pool issues [EXTERNAL]

** CAUTION: External message


Shawn,

On 12/17/20 11:04, Beard, Shawn wrote:
> I was able to fix this. I added this to the config:
>
> factory="org.apache.tomcat.jdbc.pool.DataSourceFactory"
>
> I was then seeing maxConnections of 50 in jmx and not 8.

So Phil was right: you needed to s/maxActive/maxTotal/.

That doesn't sounds right if "all your other servers are working with the same 
config and Tomcat version" so something isn't right about that statement. 
*Something* is different.

Switching to the Tomcat jdbc-pool again uses "maxActive" instead of "maxTotal" 
as the configuration option for the largest number of connections allowed in 
the pool, which is why it works when you change it.

-chris

-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org

CONFIDENTIALITY NOTICE: This e-mail and the transmitted documents contain 
private, privileged and confidential information belonging to the sender. The 
information therein is solely for the use of the addressee. If your receipt of 
this transmission has occurred as the result of an error, please immediately 
notify us so we can arrange for the return of the documents. In such 
circumstances, you are advised that you may not disclose, copy, distribute or 
take any other action in reliance on the information transmitted.

Re: jdbc connction pool issues [EXTERNAL]

[Christopher Schultz]

Shawn,

On 12/17/20 11:04, Beard, Shawn wrote:

I was able to fix this. I added this to the config:

factory="org.apache.tomcat.jdbc.pool.DataSourceFactory"

I was then seeing maxConnections of 50 in jmx and not 8.


So Phil was right: you needed to s/maxActive/maxTotal/.

That doesn't sounds right if "all your other servers are working with 
the same config and Tomcat version" so something isn't right about that 
statement. *Something* is different.


Switching to the Tomcat jdbc-pool again uses "maxActive" instead of 
"maxTotal" as the configuration option for the largest number of 
connections allowed in the pool, which is why it works when you change it.


-chris

-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org

Re: native connector, server problems with "No data received", what could be causing it?

[Christopher Schultz]

Mladen,

On 12/16/20 13:32, Mladen Adamović wrote:
On Wed, Dec 16, 2020 at 7:07 PM Christopher Schultz 
mailto:ch...@christopherschultz.net>> wrote:


I think your scripts will restart Tomcat even when it's not necessary.


Hm, is this 
https://stackoverflow.com/questions/5816239/how-do-i-force-tomcat-to-reload-trusted-certificates 


the way to reload the certificates or is there another suggestion?


The technique showed there seems perfectly fine. Your script completely 
restarts Tomcat instead of just pinging the connector to reload its 
configuration.



I might have found that odd had you posted that in your original
message, but you did not.

You need to show the full stack trace for that thread to make it
meaningful. Sockets are added to the poller all the time. It's not
unusual to see that happening. It they are getting *stuck*, that would
be bad, of course.


I did post full thread dump.


I don't see it in any of your previous posts.


 > Don't you find it weird that all threads are trying to get
synchronized
 > on a Poller instance and no one is in this block or another
synchronized
 > block/method?

I would find it weird if no threads were making any progress. Lots of
threads adding sockets to the poller is not out of the ordinary.

If you suspect a bug in Tomcat's socket handling, upgrading to the
latest 8.5.x release and re-trying would be the best move. There have
been many fixes since your 8.5.5 release which is now 4+ years old.


I have switched today from Apr connector to  Nio2 connector and no 
problem yet.
What I have found strange is that processor usage is lower when using 
Nio2, I have never tested it or tried it before, I have setup APR for 
performances reason back in 2016. But oddly it seems with Nio2 processor 
usage is lower.


Theoretically, NIO2 should be the lowest CPU usage for the same 
workload. APR does real blocking for certain operations while NIO/NIO2 
does "simluated" blocking which usually ends up with slightly higher CPU 
usage (because there is work being actively done wheras blocking usually 
just uses IO interrupts).


I have tried the last OpenJDK but jstack is not working for me, so 
switched back to the previous old JRM.


jstack isn't working? Hmm. Make sure that your jstack and Java binary 
are the same version. If you are a different user you might have to 
elevate privileges and/or use "jstack -F".


-chris

-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org

RE: jdbc connction pool issues [EXTERNAL]

[Beard, Shawn]

I was able to fix this. I added this to the config:

factory="org.apache.tomcat.jdbc.pool.DataSourceFactory"

I was then seeing maxConnections of 50 in jmx and not 8.



Shawn​  Beard

Sr. Systems Engineer |
BTS

Middleware Engineering   |  +1-515-564-2528 |  
sbe...@wrberkley.com









-Original Message-
From: Beard, Shawn 
Sent: Wednesday, December 16, 2020 7:43 AM
To: Tomcat Users List 
Subject: RE: jdbc connction pool issues [EXTERNAL]

** CAUTION: External message


Yes same version of tomcat 9.0.31, however the one having issues is running 
java 8 where the one that shows things correctly is running java 11.



Shawn​ Beard

Sr. Systems Engineer |
BTS

Middleware Engineering | +1-515-564-2528 | 
sbe...@wrberkley.com









-Original Message-
From: Phil Steitz 
Sent: Tuesday, December 15, 2020 3:37 PM
To: users@tomcat.apache.org
Subject: Re: jdbc connction pool issues [EXTERNAL]

** CAUTION: External message


On 12/15/20 1:35 PM, Beard, Shawn wrote:
> No intitialSize is not defined.
>
> Im getting the data to verify from JMX, however we also have an APM called 
> appdynamics loaded. Both verified the 8 max connections.
>
> Other tomcat servers with exact same jdbc connection pool config(only 
> difference is servername, databasename, user and pass) show 50 max 
> connections, which is what maxActive is set to.

Are the other tomcat servers running the same tomcat version? Since TC8, the 
default connection pool is DBCP2 which uses maxTotal in place of maxActive. See 
[1]. If you are running 8+, s/maxActive/maxTotal in the config should work.

Phil

[1] 
https://urldefense.com/v3/__https://tomcat.apache.org/migration-8.html*Database_Connection_Pooling__;Iw!!Li8W9_Um1Taa!orxL7Ci-tTj7jM-90ooXXY8VTcDfIzFLc9YZw4Oh-ybxGw-BbIQv4_O0_vGUlXfF$

>
>
>
> Shawn​ Beard
>
> Sr. Systems Engineer |
> BTS
>
> Middleware Engineering | +1-515-564-2528 |
> sbe...@wrberkley.com
>
>
>
>
>
>
>
>
>
> -Original Message-
> From: Johnson, Jim 
> Sent: Tuesday, December 15, 2020 2:22 PM
> To: Tomcat Users List 
> Subject: RE: jdbc connction pool issues [EXTERNAL]
>
> ** CAUTION: External message
>
>
> Hi Shawn,
>
> No, I don't think that maxActive means that it's defining the maximum
> number of connections for the pool, I think it's strictly referring to
> the "[ .. ] maximum number of *active connections* that can be
> allocated from this pool at the same time. [ .. ]" (emphasis mine on
> active connections)
>
> Here is the doc that I've been referring to - sorry for not linking it 
> earlier:
> https://urldefense.com/v3/__https://tomcat.apache.org/tomcat-9.0-doc/j
> dbc-pool.html__;!!Li8W9_Um1Taa!vJGrxr7yc86joO5r6QoNstpipfempGFqrYRe3_N
> yAIGNs-LenmBWhtjn1fRiAOex$
>
> On those other Tomcat servers is there a initialSize defined?
>
> Jim
>
> -Original Message-
> From: Beard, Shawn 
> Sent: Tuesday, December 15, 2020 2:49 PM
> To: Tomcat Users List 
> Subject: RE: jdbc connction pool issues [EXTERNAL]
>
> But wouldn’t maxActive mean the connection pool has a max of 50 conenctions? 
> On other tomcat servers I have, maxActive is set to 50 and I verified im jmx 
> that there are 50 max connections on the connection pool.
>
> Here though I checked jmx, sure enough, its max connections is 8.
>
>
>
> Shawn​ Beard
>
> Sr. Systems Engineer |
> BTS
>
> Middleware Engineering | +1-515-564-2528 |
> sbe...@wrberkley.com
>
>
>
>
>
>
>
>
>
> -Original Message-
> From: Johnson, Jim 
> Sent: Tuesday, December 15, 2020 1:35 PM
> To: Tomcat Users List 
> Subject: RE: jdbc connction pool issues [EXTERNAL]
>
> ** CAUTION: External message
>
>
> Hi Shawn,
>
> I think you’re missing initialSize
>
> initialSize
> (int)The initial number of connections that are created when the pool
> is started. Default value is 10
>
> maxActive
> (int) The maximum number of active connections that can be allocated
> from this pool at the same time. The default value is 100
>
> It would make sense that 8 connections would be 80% utilized.
>
> I would try replacing maxActive with initialSize and seeing how that works 
> for you.
>
> Good luck!
>
> - Jim
>
> From: Beard, Shawn 
> Sent: Tuesday, December 15, 2020 2:12 PM
> To: Tomcat Users List 
> Subject: jdbc connction pool issues
>
> CAUTION EXTERNAL EMAIL: This email originated from outside of the 
> organization. Do not click links or open attachments unless you recognize the 
> sender and know the content is safe.
>
> We have this jdbc connection pool set up:
>
>  driverClassName="com.microsoft.sqlserver.jdbc.SQLServerDriver"
> url="jdbc:sqlserver://hostname:1433;selectMethod=direct;databaseName=databasename;sendStringParametersAsUnicode=false"
> username="XXX" password="XX" maxIdle="20" minIdle="10" maxActive="50"
> maxWait="3" testWhileIdle="true" testOnBorrow="true"
> validationQuery="SELECT 1"/>
>
> However we 

Re: Tomcat SSO valve implementation

[tomcat/perl]

On 16.12.2020 19:39, Kevin Oxley wrote:

We are trying to support SSO SAML 2.0 for user authentication in Tomcat
(9.0.22).   Can anybody provide a reference to a pre-integrated SAML SSO
valve implementation that you've had a good experience with?



searching Google for "SAML SP for servlet engine" gives a few links, among them 
this one :
https://dzone.com/articles/saml-single-sign-on-with-tomcat-and-picketlink

I haven't tried it myself. In my cases, I always use an Apache httpd front-end, which does 
the authentication prior to proxying to a back-end tomcat (with the Connector attribute ' 
tomcatAuthentication="false" '). In the front-end Apache2 httpd then, we use Shibboleth as 
the SAML SP side.

That works perfectly.

-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org

Re: native connector, server problems with "No data received", what could be causing it?

[Mladen Adamović]

I have just updated to the latest Tomcat 9.0.41

It seems that the problem doesn't appear anymore. If I get it reappeared,
I'll post new details to the Tomcat dev mailing list.


On Thu, Dec 17, 2020 at 8:56 AM Mladen Adamović 
wrote:

> I have these problems again. To narrow it down I have done the following:
> - tested with the latest JDK8 - problem exists
> - it exists in both Apr connector and Nio2 connector
> - did log JVM Garbage Collector details - it's not due to Garbage collector
>
> Christopher (or anyone else), if we had 1570 requests per minute where
> this problem happened (approx. 26 per second), what do you think should our
> Connector params be, it's currently:
>
> protocol="org.apache.coyote.http11.Http11Nio2Protocol"
>   SSLCertificateFile="/etc/letsencrypt/live/
> numbeo.com/cert.pem"
>   SSLCertificateKeyFile="/etc/letsencrypt/live/
> numbeo.com/privkey.pem"
>   SSLCertificateChainFile="/etc/letsencrypt/live/
> numbeo.com/chain.pem"
>   SSLVerifyClient="optional"
> SSLProtocol="TLSv1+TLSv1.1+TLSv1.2"
>   connectionTimeout="2" acceptCount="3"
>   acceptorThreadCount="4"
>   compression="on" maxConnections="5" maxThreads="500"
>
> compressableMimeType="text/html,text/xml,text/plain,text/css,text/javascript,application/javascript,image/svg+xml,image/svg,image/png,image/jpeg"
>
>   useSendfile="false"
>   maxHttpHeaderSize="16392" SSLEnabled="true"
>   enableLookups="false"
>   scheme="https" secure="true"   clientAuth="false"
>  useBodyEncodingForURI="true"
>   URIEncoding="UTF-8"
>   />
>
>
>
>
>
> On Wed, Dec 16, 2020 at 7:32 PM Mladen Adamović 
> wrote:
>
>> On Wed, Dec 16, 2020 at 7:07 PM Christopher Schultz <
>> ch...@christopherschultz.net> wrote:
>>
>>> I think your scripts will restart Tomcat even when it's not necessary.
>>>
>>
>> Hm, is this
>> https://stackoverflow.com/questions/5816239/how-do-i-force-tomcat-to-reload-trusted-certificates
>> the way to reload the certificates or is there another suggestion?
>>
>>
>>> The $? check before sending the email message looks like it should be
>>> checking the result of the certbot command, but it's checking the result
>>> of the chmod command instead. (Or maybe the result of the .sh script,
>>> which will proably be 0.)
>>>
>>
>> Correct, thank you this is fixed now.
>>
>>
>>> I might have found that odd had you posted that in your original
>>> message, but you did not.
>>>
>>> You need to show the full stack trace for that thread to make it
>>> meaningful. Sockets are added to the poller all the time. It's not
>>> unusual to see that happening. It they are getting *stuck*, that would
>>> be bad, of course.
>>>
>>
>> I did post full thread dump.
>>
>>
>>> > Don't you find it weird that all threads are trying to get
>>> synchronized
>>> > on a Poller instance and no one is in this block or another
>>> synchronized
>>> > block/method?
>>>
>>> I would find it weird if no threads were making any progress. Lots of
>>> threads adding sockets to the poller is not out of the ordinary.
>>>
>>> If you suspect a bug in Tomcat's socket handling, upgrading to the
>>> latest 8.5.x release and re-trying would be the best move. There have
>>> been many fixes since your 8.5.5 release which is now 4+ years old.
>>>
>>
>> I have switched today from Apr connector to  Nio2 connector and no
>> problem yet.
>> What I have found strange is that processor usage is lower when using
>> Nio2, I have never tested it or tried it before, I have setup APR for
>> performances reason back in 2016. But oddly it seems with Nio2 processor
>> usage is lower.
>> I have tried the last OpenJDK but jstack is not working for me, so
>> switched back to the previous old JRM.
>> Let's see what will happen...
>>
>>
>>
>>
>>> -chris
>>>
>>> -
>>> To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
>>> For additional commands, e-mail: users-h...@tomcat.apache.org
>>>
>>>