Tomcat 8.5 Regarding PostResources for Web-Inf\lib and Web-Inf\classes

[Vimil Saju]

This is regarding the Resources feature of tomcat 8.5 that allows us to add 
external jars and class folder to the webapp classpath.
There seems to be an issue with the order in which class folder and jar files 
are added to the classpath.
 
         

With the above configuration
the webapp classpath becomes - 
WEB-INF\classes;WEB-INF\lib\*.jar;D:\Projects\lib\library1.jar;D:\Projects\external\classes
I expect the order to be 
WEB-INF\classes;WEB-INF\lib\*.jar;D:\Projects\external\classes;D:\Projects\lib\library1.jar;
Is there a way to configure Context xml so that the classfolders added through 
PostResources appear before jar files added through PostResources in the 
classpath?
ThanksVimil

Re: Increased memory consumption due to url encoding

[Rainer Jung]

Am 11.08.2016 um 21:16 schrieb Jose María Zaragoza:

2016-08-10 14:29 GMT+02:00 Lazar Kirchev :

Hello Christopher,

I tried with 32 MB and even 24 MB heap and the CPU usage and response time
remained the almost the same (the difference is negligible) as with 1 GB
heap. The cumulative allocated memory for the HeapByteBuffer remains about
400 MB, but of course the frequency of GCs is increased.



A newbie question:

if you tried with 32 MB ( I guess -Xmx size ) , why is not raised an OOM error ?
are those 400MB allocated outside heap ?


Allocation size is not the same as size of objects in memory. An 
allocation of 400 MB during some test means the sum of the sizes of all 
objects that were created was 400 MB. The life time of those objects 
could be very short. So it is possible that only few of them were in 
memory at the same time.


An extreme example: suppose you create 1000 objects per second (or one 
per millisecond) each of size 10 KB. That means you have an allocation 
rate of 10 MB/s or after one minute 600 MB. Assume further that each 
object only lives 5 milliseconds, because it maybe is created and used 
only inside a method that runs very short. Then at each point in time 
about 5 of those objects exist which makes up only 50 KB of memory.


For many java programs the popular slogan "most objects die young" is 
correct, which means you can have high allocation rates with relatively 
small memory. Allocation is relatively cheap in Java. As always details 
go far beyond such simplistic slogans.


Regards,

Rainer

-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org

Re: [jdbc-pool] Validation query leaves transaction open

[Jose María Zaragoza]

2016-08-12 13:46 GMT+02:00 John Huss :
> In every database I have used every query requires a transaction. Often
> they will start them automatically if you don't do it explicitly.

Right , but I guess that Tomcat uses JDBC connections and I wonder why
for executing validationQuery sentence, Tomcat doesn't use autocommit
= true setting


> On Fri, Aug 12, 2016 at 4:13 AM Jose María Zaragoza 
> wrote:
>
>> 2016-08-11 21:33 GMT+02:00 John Huss :
>> > I have verified that in version 8.0.36 of tomcat jdbc the validation
>> query
>> > functionality does not commit the transaction it starts in order to run
>> the
>> > validation query, which can cause an open transaction for a very long
>> time
>> > (possibly until the database crashes).  This seems to be especially
>> > problematic if you have a large pool where the connections are idle for a
>> > large period of time.
>> >
>> > I also tested against version 9.0.0M9, and in that version the problem is
>> > fixed.  However, nothing about this appears in the release notes for any
>> > version (9.0, 8.5, or 8.0).  If this bug has indeed been fixed it should
>> be
>> > documented in the release notes.  Also, back-porting the fix to the 8.0
>> > branch would be very helpful.
>> >
>> > For the time being I am working around the problem by changing my
>> > validation query from "SELECT 1" to be "SELECT 1; COMMIT" (using
>> > postgresql).  This prevents the issue.
>>
>>
>> I wonder why a validationQuery needs to open a new transaction
>>
>>
>> >
>> > This was reported previously (three years ago) here:
>> >
>> https://mail-archives.apache.org/mod_mbox/tomcat-users/201301.mbox/%3CCAPw873vy%2B9kWMQiFPbnTX6g1FSQ8oTVhDJ3Zagp9UkUZ3d5qjw%40mail.gmail.com%3E
>> >
>> > Thanks,
>> > John
>>
>> -
>> To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
>> For additional commands, e-mail: users-h...@tomcat.apache.org
>>
>>

-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org

RE: Native Library Error using Context forward in server.xml

[Caldarale, Charles R]

> From: Pietag, Tobias (Enterprise Security Services) 
> [mailto:tobias.pie...@hpe.com] 
> Subject: RE: Native Library Error using Context forward in server.xml

> We are using the Context in the server.xml to change the root application to 
> a 
> different webapp. 

That's a really, really bad idea.  Name your default webapp ROOT, and be done 
with it.  Alternatively, construct a ROOT webapp that does nothing but redirect 
to iiq (or vice-versa).

>  deployOnStartup="true" >
>    

Read the doc and discover why that path attribute is using an illegal value, 
even when in server.xml.
 
> So that the webapp can be call from https://url:8443/ and 
> https://url:8443/iiq. 

You should do this with a redirect.

> If the webapp is open from both URLs, it will result in the following error.

That's because it's not "the webapp"; your configuration forces it to be 
deployed twice, once as the default, once as iiq.  Each deployment uses a 
separate classloader so the underlying classes (and native library references) 
cannot be mixed.
 
> It seems that the tomcat sees both URLs as different applications and tries 
> to load native library twice.

Exactly.

> After removing the Context element it's working without any error.

Except you cannot reach the webapp without using iiq in the URL.

> Is this the normal behavior with the Context element ?

Yes.

> Is there an option to change this behavior, or does it look like a bug ?

It's a bug in your configuration and deployment strategy.

 - Chuck


THIS COMMUNICATION MAY CONTAIN CONFIDENTIAL AND/OR OTHERWISE PROPRIETARY 
MATERIAL and is thus for use only by the intended recipient. If you received 
this in error, please contact the sender and delete the e-mail and its 
attachments from all computers.


-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org

Re: org.apache.catalina.startup.Bootstrap.getCatalinaHomeFile()Ljava/io/File;

[Violeta Georgieva]

Hi,

2016-08-12 15:01 GMT+03:00 leonidprokopets :
>
> Does anyone know which version of tomcat-catalina.jar contains a
Bootstrap class with getCatalinaHomeFile() method?
>
> I'm using Tomcat 8.0 and Eclipse Neon 4.6.0
>
> I'm getting an error
> java.lang.NoSuchMethodError:
org.apache.catalina.startup.Bootstrap.getCatalinaHomeFile()Ljava/io/File;
> So obviously the Bootstrap class does not have the method.

Check carefully your dependencies. This method is available in Tomcat 8.0
and above, but it is not available in Tomcat 7.


http://svn.apache.org/viewvc/tomcat/tc8.0.x/trunk/java/org/apache/catalina/startup/Bootstrap.java?view=markup#l534
http://svn.apache.org/viewvc/tomcat/tc8.5.x/trunk/java/org/apache/catalina/startup/Bootstrap.java?view=markup#l547
http://svn.apache.org/viewvc/tomcat/trunk/java/org/apache/catalina/startup/Bootstrap.java?view=markup#l547

Regards,
Violeta

> Thanks in advance for any hint.

RE: Native Library Error using Context forward in server.xml

[Pietag, Tobias (Enterprise Security Services)]

Hello,

 

We are using the Context in the server.xml to change the root application to
a different webapp. 

 









 

So that the webapp can be call from https://url:8443/ and
https://url:8443/iiq. 

The webapp is using the sapjco3.jar and the native library libsapjco3.so to
connect to SAP systems.

If the webapp is open from both URLs, it will result in the following error.

 

java.lang.ExceptionInInitializerError: JCo initialization failed with
java.lang.UnsatisfiedLinkError: Native Library
/appl/tomcat/apps/ap/figi/apache-tomcat-latest/webapps/iiq/WEB-INF/lib/libsa
pjco3.so already loaded in another classloader

 

The library only exists once in the tomcat structure.

It seems that the tomcat sees both URLs as different applications and tries
to load native library twice.

After removing the Context element it's working without any error.

 

Is this the normal behavior with the Context element ?

Is there an option to change this behavior, or does it look like a bug ?

 

Thanks

Tobias Pietag



smime.p7s
Description: S/MIME cryptographic signature

org.apache.catalina.startup.Bootstrap.getCatalinaHomeFile()Ljava/io/File;

[leonidprokopets]

Does anyone know which version of tomcat-catalina.jar contains a Bootstrap 
class with getCatalinaHomeFile() method?

I'm using Tomcat 8.0 and Eclipse Neon 4.6.0

I'm getting an error
java.lang.NoSuchMethodError: 
org.apache.catalina.startup.Bootstrap.getCatalinaHomeFile()Ljava/io/File;
So obviously the Bootstrap class does not have the method.

Thanks in advance for any hint.

Re: [jdbc-pool] Validation query leaves transaction open

[John Huss]

In every database I have used every query requires a transaction. Often
they will start them automatically if you don't do it explicitly.
On Fri, Aug 12, 2016 at 4:13 AM Jose María Zaragoza 
wrote:

> 2016-08-11 21:33 GMT+02:00 John Huss :
> > I have verified that in version 8.0.36 of tomcat jdbc the validation
> query
> > functionality does not commit the transaction it starts in order to run
> the
> > validation query, which can cause an open transaction for a very long
> time
> > (possibly until the database crashes).  This seems to be especially
> > problematic if you have a large pool where the connections are idle for a
> > large period of time.
> >
> > I also tested against version 9.0.0M9, and in that version the problem is
> > fixed.  However, nothing about this appears in the release notes for any
> > version (9.0, 8.5, or 8.0).  If this bug has indeed been fixed it should
> be
> > documented in the release notes.  Also, back-porting the fix to the 8.0
> > branch would be very helpful.
> >
> > For the time being I am working around the problem by changing my
> > validation query from "SELECT 1" to be "SELECT 1; COMMIT" (using
> > postgresql).  This prevents the issue.
>
>
> I wonder why a validationQuery needs to open a new transaction
>
>
> >
> > This was reported previously (three years ago) here:
> >
> https://mail-archives.apache.org/mod_mbox/tomcat-users/201301.mbox/%3CCAPw873vy%2B9kWMQiFPbnTX6g1FSQ8oTVhDJ3Zagp9UkUZ3d5qjw%40mail.gmail.com%3E
> >
> > Thanks,
> > John
>
> -
> To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
> For additional commands, e-mail: users-h...@tomcat.apache.org
>
>

Re: [jdbc-pool] Validation query leaves transaction open

[Jose María Zaragoza]

2016-08-11 21:33 GMT+02:00 John Huss :
> I have verified that in version 8.0.36 of tomcat jdbc the validation query
> functionality does not commit the transaction it starts in order to run the
> validation query, which can cause an open transaction for a very long time
> (possibly until the database crashes).  This seems to be especially
> problematic if you have a large pool where the connections are idle for a
> large period of time.
>
> I also tested against version 9.0.0M9, and in that version the problem is
> fixed.  However, nothing about this appears in the release notes for any
> version (9.0, 8.5, or 8.0).  If this bug has indeed been fixed it should be
> documented in the release notes.  Also, back-porting the fix to the 8.0
> branch would be very helpful.
>
> For the time being I am working around the problem by changing my
> validation query from "SELECT 1" to be "SELECT 1; COMMIT" (using
> postgresql).  This prevents the issue.


I wonder why a validationQuery needs to open a new transaction


>
> This was reported previously (three years ago) here:
> https://mail-archives.apache.org/mod_mbox/tomcat-users/201301.mbox/%3CCAPw873vy%2B9kWMQiFPbnTX6g1FSQ8oTVhDJ3Zagp9UkUZ3d5qjw%40mail.gmail.com%3E
>
> Thanks,
> John

-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org

Re: BasicDataSource creation throws SQLException

[Roy Leonardus]

Hi Christopher,

I only called the basicdatasource.getconnection() in the main test class.

but i added TestDriver.addconnection() beforehand.

this is my driver class

public class TestDriver implements Driver {

public static final String TESTDRIVER_PREFIX = "jdbc:not_a_db:";

static {
try {
DriverManager.registerDriver(new TestDriver());
} catch (SQLException e) {
e.printStackTrace();
}
}

private static List connections = new ArrayList();

public static void reset() {
connections.clear();
}

public static void addConnection(Connection c) {
connections.add(c);
}

private static Connection getNextConnectionFromList() {
if (connections.isEmpty()) {
throw new IllegalStateException("No more connections available.");
}
return connections.remove(0);
}


@Override
public Connection connect(String url, Properties info) throws SQLException {
return getNextConnectionFromList();
}

@Override
public boolean acceptsURL(String url) throws SQLException {
return url.contains(TESTDRIVER_PREFIX);
}

@Override
public DriverPropertyInfo[] getPropertyInfo(String url, Properties info)
throws SQLException {
return new DriverPropertyInfo[0];
}

@Override
public int getMajorVersion() {
return 3;
}

@Override
public int getMinorVersion() {
return 42;
}

@Override
public boolean jdbcCompliant() {
return true;
}

// for JDK 7
public Logger getParentLogger() throws SQLFeatureNotSupportedException {
return null;
}

as you can see, there is no resultset returned.

regarding the validation query, i only use it during the test with Tomcat
8, as it complains about it.

I agree with the mocking connection, the biggest PITA is : it does not
prove anything, that is why i am using Derby to speed up the process.

What John Huss wrote in the email chain is also interesting and need to
keep in note, though.

Roy


On Thu, Aug 11, 2016 at 6:02 PM, Christopher Schultz <
ch...@christopherschultz.net> wrote:

> -BEGIN PGP SIGNED MESSAGE-
> Hash: SHA256
>
> Roy,
>
> On 8/11/16 3:43 AM, Roy Leonardus wrote:
> > Hello Christopher,
> >
> > maybe i'm a little bit slow here, but from what i see, we should
> > mock the connection method
> >
> > Connection connection= ds.getConnection();
> >
> > something like
> >
> > Connection connection =(Connection) EasyMock.expect(ds.
> > isvalid()).andReturn(true).anyTimes();
> >
> > Problem is, the error happened during the validation of the
> > PoolableConnection, below is the build error from the surefire
> >
> > Caused by: java.sql.SQLException: isValid() returned false at
> > org.apache.tomcat.dbcp.dbcp2.PoolableConnection.validate(PoolableConne
> ction.java:283)
> >
> >
> at
> > org.apache.tomcat.dbcp.dbcp2.PoolableConnectionFactory.validateConnect
> ion(PoolableConnectionFactory.java:357)
> >
> >
> at
> > org.apache.tomcat.dbcp.dbcp2.BasicDataSource.validateConnectionFactory
> (BasicDataSource.java:2307)
> >
> >
> at
> > org.apache.tomcat.dbcp.dbcp2.BasicDataSource.createPoolableConnectionF
> actory(BasicDataSource.java:2290)
>
>
> It's
> >
> been a while since I've bothered to mock the JDBC interfaces
> because they are such a PITA to do. But it's possible that your mock
> Connection isn't being used by the BasicDataSource. Perhaps you need
> to mock the BasicDataSource (or maybe just DataSource) too.
>
> > so i think we should either override the validate() so the
> > validation is always returning true( which i am reluctant to do, as
> > the point is to test the whole thing)
> >
> > or enhancing the fake driver connection, so it always responding
> > with a result set.
>
> If you have set a validation query, that query should be executed. If
> you don't specify a query, I'd expect Connection.isValid to be used
> instead. I see from the exception that the isValid is failing, but you
> also mentioned that you tried to set a validation query and got a
> different error.
>
> I think you should keep going without a validation query because it
> should simplify things a little bit. Can you post more of your code?
> It's hard to tell how you are setting-up your test-case. If you have
> mock objects, what are you mocking and how are you injecting those
> mock objects into the code you are testing?
>
> - -chris
> -BEGIN PGP SIGNATURE-
> Comment: GPGTools - http://gpgtools.org
> Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/
>
> iQIcBAEBCAAGBQJXrKGFAAoJEBzwKT+lPKRYhfwP/AvTbY+aDWduuHChdoRAoPR2
> bjcA6u6QjCOxmpFT2qzMnp9PYNnUbqx6GpGyY0poSN6eX9lG5jc27r0SN1Hvrbwk
> Rp/+dFySnwMupnLccvhfZnqMZvoyVhPegdEwsbZ4Ep9zzE+wVK0hxAc/8V5nAZpY
> 5SLTYACt63i5pWyYTwxCHpVVYYyeSm+S9OH2nogD4QugDoV93g6K+YSLpM35SVHp
> ul55OBUrWazDQIay6FcFmIf1aQmeNlGejHIPjwdwCsF4QXEwY179s3VFMlKeKkUa
> ur/fnZpJ2ZTGFe9q7UzZW+X0HaZHNDHlu/FJAzjquNIxzHxZBh1vsOHncn0D44F5
> 0yWLRJkuDDT/2e9gwZPQ2civfrAC5c6bKudB03KIZfx5tmCrJMRuneUTt5viIHet
> yCF5F30jeOEbBeag1u6A5MH1f96kwJ6h995ohitSzwH4K9vUqaDhL+UFnkVdjusR
> vjanlOQcLUYu99fce7QtCMJTThlZy00REmrB0kgnM/VKPOU7ovBFQC95vYknsYyx
>