Re: Tomcat DBCP 8.0.18 issue
Hmm, interesting... Thanks for explanation Mark!
2015-06-03 12:22 GMT+03:00 Mark Thomas ma...@apache.org:
On 03/06/2015 07:24, Tweak Ronaldo wrote:
Thanks Mark, yes I have mixed together Tomcat JDBC 8 and DBCP 7, my bad.
Although I don't understand why Tomcat JDBC don't use DBCP as default
solution for connections pooling.
Tomcat does - and always has - used Commons DBCP for connection pooling
by default.
We do package rename Commons DBCP to avoid conflicts for web
applications that package it in WEB-INF/lib and we have changed how we
have done that repackaging over the years but the source code remains
unchanged from which ever release version / tag / svn revision we select
to pull in.
What follows is a short(ish) history of Commons DBCP and tomcat-jdbc
which attempts to explain why Tomcat ended up shipping two connection
pools.
Commons DBCP 1 has very poor performance in highly concurrent
environments. Things improved during the 1.x series but really didn't
improve much above 'awful'.
As users started to run Tomcat on machines with increasing numbers of
cores, the performance issues with DBCP 1 started to have a noticeable
impact for some web applications. Something needed to be done to address
the performance issues in concurrent environments.
There were two options:
a) fix DBCP (essentially this meant a major re-write for Commons Pool)
b) implement a new connection pool
My preference was for a) although I had little time to do any work along
those lines. Keep in mind that my preference for a) is likely to have
coloured my view of the history of this.
Filip's preference was for b). He also felt that there was room to
further improve performance by removing the overhead inherent in:
i) Commons DBCP being built on Commons Pool
(a single module could be more tightly / efficiently integrated)
ii) Commons DBCP aiming to fully follow the JDBC specs
(making some of the behaviours optional reduced overhead for
users that didn't want/need them)
Filip therefore wrote tomcat-jdbc. He tried to release it several times
without success. Apache releases need 3 +1 votes and there simply wasn't
the interest in this component from the other Tomcat developers (this
was one of my concerns with this approach).
In order to get it released, tomcat-jdbc was bundled with Tomcat from
7.0.19 onwards (July 2011) but it was never configured to be the default.
Filip's focus at work moved away from Tomcat development in 2012. He
does pop back from time to time (mainly to fix issues with tomcat-jdbc)
but it is fair to say that - at the moment - issues in tomcat-jdbc are
not being resolved in a timely manner.
Around the same time tomcat-jdbc was bundled with Tomcat, I (and others
from Commons) started on Commons Pool 2 and Commons DBCP 2. Pool 2 was
finally released in 2013 and DBCP 2 in 2014 (Tomcat 8 always used DBCP 2
and had been depending on snapshot versions for some time before that).
Commons Pool 2 and Commons DBCP 2 continue to be maintained by the
Commons community (there was a Pool 2.4.1 release just last week) and I
aim to keep Tomcat's copy of DBCP 2 (and Pool 2) up to date.
Generally, bugs reported against Commons DBCP2 and Commons Pool 2 are
addressed in a timely manner.
tomcat-jdbc provided (and continues to provide) a simple solution for
those Tomcat 6 and Tomcat 7 users that needed better performance under
concurrent loads. Tomcat 8 and onwards users have DBCP 2 so there is
less need for tomcat-jdbc for those users.
Hope this helps explain how we got to where we are.
As I typed this, I started to wonder if we should even ship tomcat-jdbc
with Tomcat 9. I'll start a discussion along those lines on the dev list.
mark
2015-06-02 16:59 GMT+03:00 Mark Thomas ma...@apache.org:
On 01/06/2015 14:22, Tweak Ronaldo wrote:
Hello guys,
Assuming you don't want to limit your question to men only, you would be
better to use of of the following greetings:
Hello,
Hello all,
Hello folks,
etc.
we have migrated to Tomcat DBCP 8.0.18 from 7.0.37 recently and
faced the following issue:
after database restart (Postgres), our application wasn't been able to
restore connectivity to DB, all connections were closed and every time,
after failed attempt to execute some SQL statement, returned back to
pool.
Pool is configured with all tests (on borrow, on return, on connect,
while
idle) disabled. Pool configuration is the same for 8.0.18 and 7.0.37.
While using 7.0.37 DBCP is able to restore from DB restart, because
PoolableConnection class performs explicit check if underlying SQL
connection is closed:
public synchronized void close() throws SQLException {
if (_closed) {
// already closed
return;
}
boolean isUnderlyingConectionClosed;
try {
isUnderlyingConectionClosed = _conn.isClosed();
} catch
Re: Tomcat DBCP 8.0.18 issue
On 03/06/2015 07:24, Tweak Ronaldo wrote:
Thanks Mark, yes I have mixed together Tomcat JDBC 8 and DBCP 7, my bad.
Although I don't understand why Tomcat JDBC don't use DBCP as default
solution for connections pooling.
Tomcat does - and always has - used Commons DBCP for connection pooling
by default.
We do package rename Commons DBCP to avoid conflicts for web
applications that package it in WEB-INF/lib and we have changed how we
have done that repackaging over the years but the source code remains
unchanged from which ever release version / tag / svn revision we select
to pull in.
What follows is a short(ish) history of Commons DBCP and tomcat-jdbc
which attempts to explain why Tomcat ended up shipping two connection pools.
Commons DBCP 1 has very poor performance in highly concurrent
environments. Things improved during the 1.x series but really didn't
improve much above 'awful'.
As users started to run Tomcat on machines with increasing numbers of
cores, the performance issues with DBCP 1 started to have a noticeable
impact for some web applications. Something needed to be done to address
the performance issues in concurrent environments.
There were two options:
a) fix DBCP (essentially this meant a major re-write for Commons Pool)
b) implement a new connection pool
My preference was for a) although I had little time to do any work along
those lines. Keep in mind that my preference for a) is likely to have
coloured my view of the history of this.
Filip's preference was for b). He also felt that there was room to
further improve performance by removing the overhead inherent in:
i) Commons DBCP being built on Commons Pool
(a single module could be more tightly / efficiently integrated)
ii) Commons DBCP aiming to fully follow the JDBC specs
(making some of the behaviours optional reduced overhead for
users that didn't want/need them)
Filip therefore wrote tomcat-jdbc. He tried to release it several times
without success. Apache releases need 3 +1 votes and there simply wasn't
the interest in this component from the other Tomcat developers (this
was one of my concerns with this approach).
In order to get it released, tomcat-jdbc was bundled with Tomcat from
7.0.19 onwards (July 2011) but it was never configured to be the default.
Filip's focus at work moved away from Tomcat development in 2012. He
does pop back from time to time (mainly to fix issues with tomcat-jdbc)
but it is fair to say that - at the moment - issues in tomcat-jdbc are
not being resolved in a timely manner.
Around the same time tomcat-jdbc was bundled with Tomcat, I (and others
from Commons) started on Commons Pool 2 and Commons DBCP 2. Pool 2 was
finally released in 2013 and DBCP 2 in 2014 (Tomcat 8 always used DBCP 2
and had been depending on snapshot versions for some time before that).
Commons Pool 2 and Commons DBCP 2 continue to be maintained by the
Commons community (there was a Pool 2.4.1 release just last week) and I
aim to keep Tomcat's copy of DBCP 2 (and Pool 2) up to date.
Generally, bugs reported against Commons DBCP2 and Commons Pool 2 are
addressed in a timely manner.
tomcat-jdbc provided (and continues to provide) a simple solution for
those Tomcat 6 and Tomcat 7 users that needed better performance under
concurrent loads. Tomcat 8 and onwards users have DBCP 2 so there is
less need for tomcat-jdbc for those users.
Hope this helps explain how we got to where we are.
As I typed this, I started to wonder if we should even ship tomcat-jdbc
with Tomcat 9. I'll start a discussion along those lines on the dev list.
mark
2015-06-02 16:59 GMT+03:00 Mark Thomas ma...@apache.org:
On 01/06/2015 14:22, Tweak Ronaldo wrote:
Hello guys,
Assuming you don't want to limit your question to men only, you would be
better to use of of the following greetings:
Hello,
Hello all,
Hello folks,
etc.
we have migrated to Tomcat DBCP 8.0.18 from 7.0.37 recently and
faced the following issue:
after database restart (Postgres), our application wasn't been able to
restore connectivity to DB, all connections were closed and every time,
after failed attempt to execute some SQL statement, returned back to
pool.
Pool is configured with all tests (on borrow, on return, on connect,
while
idle) disabled. Pool configuration is the same for 8.0.18 and 7.0.37.
While using 7.0.37 DBCP is able to restore from DB restart, because
PoolableConnection class performs explicit check if underlying SQL
connection is closed:
public synchronized void close() throws SQLException {
if (_closed) {
// already closed
return;
}
boolean isUnderlyingConectionClosed;
try {
isUnderlyingConectionClosed = _conn.isClosed();
} catch (SQLException e) {
try {
_pool.invalidateObject(this); // XXX should be guarded to
happen at most once
} catch(IllegalStateException ise) {
//
Re: Tomcat DBCP 8.0.18 issue
Thanks Mark, yes I have mixed together Tomcat JDBC 8 and DBCP 7, my bad.
Although I don't understand why Tomcat JDBC don't use DBCP as default
solution for connections pooling.
2015-06-02 16:59 GMT+03:00 Mark Thomas ma...@apache.org:
On 01/06/2015 14:22, Tweak Ronaldo wrote:
Hello guys,
Assuming you don't want to limit your question to men only, you would be
better to use of of the following greetings:
Hello,
Hello all,
Hello folks,
etc.
we have migrated to Tomcat DBCP 8.0.18 from 7.0.37 recently and
faced the following issue:
after database restart (Postgres), our application wasn't been able to
restore connectivity to DB, all connections were closed and every time,
after failed attempt to execute some SQL statement, returned back to
pool.
Pool is configured with all tests (on borrow, on return, on connect,
while
idle) disabled. Pool configuration is the same for 8.0.18 and 7.0.37.
While using 7.0.37 DBCP is able to restore from DB restart, because
PoolableConnection class performs explicit check if underlying SQL
connection is closed:
public synchronized void close() throws SQLException {
if (_closed) {
// already closed
return;
}
boolean isUnderlyingConectionClosed;
try {
isUnderlyingConectionClosed = _conn.isClosed();
} catch (SQLException e) {
try {
_pool.invalidateObject(this); // XXX should be guarded to
happen at most once
} catch(IllegalStateException ise) {
// pool is closed, so close the connection
passivate();
getInnermostDelegate().close();
} catch (Exception ie) {
// DO NOTHING the original exception will be rethrown
}
throw (SQLException) new SQLException(Cannot close
connection
(isClosed check failed)).initCause(e);
}
...
My question is: why this check was removed and how can one get the same
behaviour (of 7.0.37) using 8.0.18 (not using on borrow, on return, while
idle validations, which are SQL queries)? I see that there is a property
in
pool configuration which allows user to provide custom Validator, though
I
don't want to go this way because DBCP configuration is performed by 3rd
party library and we don't have direct access to it.
The check to which you refer was not removed. It is still present in the
code base and - as far as I can tell from a quick look at svn - never
been removed.
DBCP has no support for a custom validator. You appear to have confused
something else with Tomcat's port of Commons DBCP (the Tomcat JDBC pool
perhaps?).
Mark
-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org
Re: Tomcat DBCP 8.0.18 issue
Mark, big thanks for a quick historical review, it was very intreresting!
I was thinking that tomcat-jdbc is something new, but wasn't sure about
project maintenance.
2015-06-03 12:22 GMT+03:00 Mark Thomas ma...@apache.org:
On 03/06/2015 07:24, Tweak Ronaldo wrote:
Thanks Mark, yes I have mixed together Tomcat JDBC 8 and DBCP 7, my bad.
Although I don't understand why Tomcat JDBC don't use DBCP as default
solution for connections pooling.
Tomcat does - and always has - used Commons DBCP for connection pooling
by default.
We do package rename Commons DBCP to avoid conflicts for web
applications that package it in WEB-INF/lib and we have changed how we
have done that repackaging over the years but the source code remains
unchanged from which ever release version / tag / svn revision we select
to pull in.
What follows is a short(ish) history of Commons DBCP and tomcat-jdbc
which attempts to explain why Tomcat ended up shipping two connection
pools.
Commons DBCP 1 has very poor performance in highly concurrent
environments. Things improved during the 1.x series but really didn't
improve much above 'awful'.
As users started to run Tomcat on machines with increasing numbers of
cores, the performance issues with DBCP 1 started to have a noticeable
impact for some web applications. Something needed to be done to address
the performance issues in concurrent environments.
There were two options:
a) fix DBCP (essentially this meant a major re-write for Commons Pool)
b) implement a new connection pool
My preference was for a) although I had little time to do any work along
those lines. Keep in mind that my preference for a) is likely to have
coloured my view of the history of this.
Filip's preference was for b). He also felt that there was room to
further improve performance by removing the overhead inherent in:
i) Commons DBCP being built on Commons Pool
(a single module could be more tightly / efficiently integrated)
ii) Commons DBCP aiming to fully follow the JDBC specs
(making some of the behaviours optional reduced overhead for
users that didn't want/need them)
Filip therefore wrote tomcat-jdbc. He tried to release it several times
without success. Apache releases need 3 +1 votes and there simply wasn't
the interest in this component from the other Tomcat developers (this
was one of my concerns with this approach).
In order to get it released, tomcat-jdbc was bundled with Tomcat from
7.0.19 onwards (July 2011) but it was never configured to be the default.
Filip's focus at work moved away from Tomcat development in 2012. He
does pop back from time to time (mainly to fix issues with tomcat-jdbc)
but it is fair to say that - at the moment - issues in tomcat-jdbc are
not being resolved in a timely manner.
Around the same time tomcat-jdbc was bundled with Tomcat, I (and others
from Commons) started on Commons Pool 2 and Commons DBCP 2. Pool 2 was
finally released in 2013 and DBCP 2 in 2014 (Tomcat 8 always used DBCP 2
and had been depending on snapshot versions for some time before that).
Commons Pool 2 and Commons DBCP 2 continue to be maintained by the
Commons community (there was a Pool 2.4.1 release just last week) and I
aim to keep Tomcat's copy of DBCP 2 (and Pool 2) up to date.
Generally, bugs reported against Commons DBCP2 and Commons Pool 2 are
addressed in a timely manner.
tomcat-jdbc provided (and continues to provide) a simple solution for
those Tomcat 6 and Tomcat 7 users that needed better performance under
concurrent loads. Tomcat 8 and onwards users have DBCP 2 so there is
less need for tomcat-jdbc for those users.
Hope this helps explain how we got to where we are.
As I typed this, I started to wonder if we should even ship tomcat-jdbc
with Tomcat 9. I'll start a discussion along those lines on the dev list.
mark
2015-06-02 16:59 GMT+03:00 Mark Thomas ma...@apache.org:
On 01/06/2015 14:22, Tweak Ronaldo wrote:
Hello guys,
Assuming you don't want to limit your question to men only, you would be
better to use of of the following greetings:
Hello,
Hello all,
Hello folks,
etc.
we have migrated to Tomcat DBCP 8.0.18 from 7.0.37 recently and
faced the following issue:
after database restart (Postgres), our application wasn't been able to
restore connectivity to DB, all connections were closed and every time,
after failed attempt to execute some SQL statement, returned back to
pool.
Pool is configured with all tests (on borrow, on return, on connect,
while
idle) disabled. Pool configuration is the same for 8.0.18 and 7.0.37.
While using 7.0.37 DBCP is able to restore from DB restart, because
PoolableConnection class performs explicit check if underlying SQL
connection is closed:
public synchronized void close() throws SQLException {
if (_closed) {
// already closed
return;
}
boolean
Re: Tomcat DBCP 8.0.18 issue
2015-06-01 16:22 GMT+03:00 Tweak Ronaldo twea...@gmail.com: Hello guys, we have migrated to Tomcat DBCP 8.0.18 from 7.0.37 recently and faced the following issue: after database restart (Postgres), our application wasn't been able to restore connectivity to DB, all connections were closed and every time, after failed attempt to execute some SQL statement, returned back to pool. Pool is configured with all tests (on borrow, on return, on connect, while idle) disabled. Pool configuration is the same for 8.0.18 and 7.0.37. the same? Tomcat 7 uses DBCP 1 and Tomcat 8 uses DBCP 2, where some attributes have different names. As mentioned in migration guide: http://tomcat.apache.org/migration-8.html#Database_Connection_Pooling Best regards, Konstantin Kolinko - To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org For additional commands, e-mail: users-h...@tomcat.apache.org
Re: Tomcat DBCP 8.0.18 issue
On 01/06/2015 14:22, Tweak Ronaldo wrote:
Hello guys,
Assuming you don't want to limit your question to men only, you would be
better to use of of the following greetings:
Hello,
Hello all,
Hello folks,
etc.
we have migrated to Tomcat DBCP 8.0.18 from 7.0.37 recently and
faced the following issue:
after database restart (Postgres), our application wasn't been able to
restore connectivity to DB, all connections were closed and every time,
after failed attempt to execute some SQL statement, returned back to pool.
Pool is configured with all tests (on borrow, on return, on connect, while
idle) disabled. Pool configuration is the same for 8.0.18 and 7.0.37.
While using 7.0.37 DBCP is able to restore from DB restart, because
PoolableConnection class performs explicit check if underlying SQL
connection is closed:
public synchronized void close() throws SQLException {
if (_closed) {
// already closed
return;
}
boolean isUnderlyingConectionClosed;
try {
isUnderlyingConectionClosed = _conn.isClosed();
} catch (SQLException e) {
try {
_pool.invalidateObject(this); // XXX should be guarded to
happen at most once
} catch(IllegalStateException ise) {
// pool is closed, so close the connection
passivate();
getInnermostDelegate().close();
} catch (Exception ie) {
// DO NOTHING the original exception will be rethrown
}
throw (SQLException) new SQLException(Cannot close connection
(isClosed check failed)).initCause(e);
}
...
My question is: why this check was removed and how can one get the same
behaviour (of 7.0.37) using 8.0.18 (not using on borrow, on return, while
idle validations, which are SQL queries)? I see that there is a property in
pool configuration which allows user to provide custom Validator, though I
don't want to go this way because DBCP configuration is performed by 3rd
party library and we don't have direct access to it.
The check to which you refer was not removed. It is still present in the
code base and - as far as I can tell from a quick look at svn - never
been removed.
DBCP has no support for a custom validator. You appear to have confused
something else with Tomcat's port of Commons DBCP (the Tomcat JDBC pool
perhaps?).
Mark
-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org
