Re: Tomcat "JNDI Datasource How-To" documentation & driver managers

[Andrew Tanton]

Thank you - very helpful & much appreciated.

On Sat, Aug 14, 2021 at 4:24 PM Mark Thomas  wrote:

> On 14/08/2021 01:51, Andrew Tanton wrote:
> > In the Tomcat "JNDI Datasource How-To" documentation page
> > <
> http://tomcat.apache.org/tomcat-9.0-doc/jndi-datasource-examples-howto.html#comments_section
> >,
> > there is an unusually opinionated section, which discusses the Java
> service
> > provider (driver manager) mechanism:
>
> I suspect that was me after spending quite a but of time unpicking
> various issues associated with DriverManager where Tomcat was getting
> blamed. Goes to check the history...
>
> Yep. Here is the bug report:
> https://bz.apache.org/bugzilla/show_bug.cgi?id=52025
> which triggered this doc update:
> https://svn.apache.org/viewvc?view=revision=1184919
>
> > "*However, the implementation is fundamentally broken in all Java
> versions
> > for a servlet container environment. The problem is
> > that java.sql.DriverManager will scan for the drivers only once.*"
> >
> > Can someone help me understand what this means in more practical terms?
>
> This will be a lot simpler to explain with the source code to hand:
>
> https://github.com/openjdk/jdk/blob/master/src/java.sql/share/classes/java/sql/DriverManager.java
>
> > The page goes on to say:
> >
> > "*...web applications that have database drivers in
> > their WEB-INF/lib directory cannot rely on the service provider mechanism
> > and should register the drivers explicitly.*"
> >
> > And, indeed, I have found that placing my JDBC driver in Tomcat's
> > $CATALINA_HOME/lib
> > or $CATALINA_BASE/lib will be loaded correctly, without explicit
> > registration.
> >
> > MY QUESTIONS:
> >
> > (1) I don't understand why the "scan only once" limitation results in
> this
> > behavior - so what am I missing, here, conceptually?
>
> There are several inter-related elements.
>
> Scan once caused problems as if two web apps both have JDBC drivers then
> the DriverManager scan will only load the Driver for the app that
> triggers the scan first.
>
> An added complication is that the memory leak protection code
> essentially ensures that the scan is performed by Tomcat internal code
> so no JDBC drivers in WEB-INF/lib for any wweb application are seen by
> the scan. Hence, the JDBC drivers need to be in CATALINA_BASE/lib.
>   > (2) Where is this "scan only once" behavior documented?
>
> It is sort of implied in the DriverManager Javadoc but you need to read
> the source code to get a clear picture.
>
> > (3) What is it about a servlet container environment which allows this
> > problem to exist?
>
> Dynamic loading and unloading of web applications and use of a dedicated
> class loader per web application.
>
> You can also get various memory leaks associated with DriverManager as
> well (which Tomcat automatically protects you against).
>
> Mark
>
>
> >
> > Thank you.
> >
> > For reference, here is the documentation link I used above:
> >
> >
> >
> http://tomcat.apache.org/tomcat-9.0-doc/jndi-datasource-examples-howto.html
> >
> > (The wording in the documentation has been this way for several Tomcat
> > versions, going back a few years.)
> >
>
>
> -
> To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
> For additional commands, e-mail: users-h...@tomcat.apache.org
>
>

Re: Tomcat "JNDI Datasource How-To" documentation & driver managers

[Mark Thomas]

On 14/08/2021 01:51, Andrew Tanton wrote:

In the Tomcat "JNDI Datasource How-To" documentation page
,
there is an unusually opinionated section, which discusses the Java service
provider (driver manager) mechanism:


I suspect that was me after spending quite a but of time unpicking 
various issues associated with DriverManager where Tomcat was getting 
blamed. Goes to check the history...


Yep. Here is the bug report:
https://bz.apache.org/bugzilla/show_bug.cgi?id=52025
which triggered this doc update:
https://svn.apache.org/viewvc?view=revision=1184919


"*However, the implementation is fundamentally broken in all Java versions
for a servlet container environment. The problem is
that java.sql.DriverManager will scan for the drivers only once.*"

Can someone help me understand what this means in more practical terms?


This will be a lot simpler to explain with the source code to hand:
https://github.com/openjdk/jdk/blob/master/src/java.sql/share/classes/java/sql/DriverManager.java


The page goes on to say:

"*...web applications that have database drivers in
their WEB-INF/lib directory cannot rely on the service provider mechanism
and should register the drivers explicitly.*"

And, indeed, I have found that placing my JDBC driver in Tomcat's
$CATALINA_HOME/lib
or $CATALINA_BASE/lib will be loaded correctly, without explicit
registration.

MY QUESTIONS:

(1) I don't understand why the "scan only once" limitation results in this
behavior - so what am I missing, here, conceptually?


There are several inter-related elements.

Scan once caused problems as if two web apps both have JDBC drivers then 
the DriverManager scan will only load the Driver for the app that 
triggers the scan first.


An added complication is that the memory leak protection code 
essentially ensures that the scan is performed by Tomcat internal code 
so no JDBC drivers in WEB-INF/lib for any wweb application are seen by 
the scan. Hence, the JDBC drivers need to be in CATALINA_BASE/lib.

 > (2) Where is this "scan only once" behavior documented?

It is sort of implied in the DriverManager Javadoc but you need to read 
the source code to get a clear picture.



(3) What is it about a servlet container environment which allows this
problem to exist?


Dynamic loading and unloading of web applications and use of a dedicated 
class loader per web application.


You can also get various memory leaks associated with DriverManager as 
well (which Tomcat automatically protects you against).


Mark




Thank you.

For reference, here is the documentation link I used above:


http://tomcat.apache.org/tomcat-9.0-doc/jndi-datasource-examples-howto.html

(The wording in the documentation has been this way for several Tomcat
versions, going back a few years.)




-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org

Tomcat "JNDI Datasource How-To" documentation & driver managers

[Andrew Tanton]

In the Tomcat "JNDI Datasource How-To" documentation page
,
there is an unusually opinionated section, which discusses the Java service
provider (driver manager) mechanism:

"*However, the implementation is fundamentally broken in all Java versions
for a servlet container environment. The problem is
that java.sql.DriverManager will scan for the drivers only once.*"

Can someone help me understand what this means in more practical terms?

The page goes on to say:

"*...web applications that have database drivers in
their WEB-INF/lib directory cannot rely on the service provider mechanism
and should register the drivers explicitly.*"

And, indeed, I have found that placing my JDBC driver in Tomcat's
$CATALINA_HOME/lib
or $CATALINA_BASE/lib will be loaded correctly, without explicit
registration.

MY QUESTIONS:

(1) I don't understand why the "scan only once" limitation results in this
behavior - so what am I missing, here, conceptually?

(2) Where is this "scan only once" behavior documented?

(3) What is it about a servlet container environment which allows this
problem to exist?

Thank you.

For reference, here is the documentation link I used above:


http://tomcat.apache.org/tomcat-9.0-doc/jndi-datasource-examples-howto.html

(The wording in the documentation has been this way for several Tomcat
versions, going back a few years.)