[GUMP@vmgump-vm3]: Project tomcat-trunk-test-nio2 (in module tomcat-trunk) failed

2018-11-22 Thread Bill Barker
To whom it may engage...

This is an automated request, but not an unsolicited one. For 
more information please visit http://gump.apache.org/nagged.html, 
and/or contact the folk at gene...@gump.apache.org.

Project tomcat-trunk-test-nio2 has an issue affecting its community integration.
This issue affects 1 projects.
The current state of this project is 'Failed', with reason 'Build Failed'.
For reference only, the following projects are affected by this:
- tomcat-trunk-test-nio2 :  Tomcat 9.x, a web server implementing the Java 
Servlet 4.0,
...


Full details are available at:
http://vmgump-vm3.apache.org/tomcat-trunk/tomcat-trunk-test-nio2/index.html

That said, some information snippets are provided here.

The following annotations (debug/informational/warning/error messages) were 
provided:
 -DEBUG- Dependency on bnd exists, no need to add for property bndlib.jar.
 -INFO- Failed with reason build failed
 -INFO- Project Reports in: 
/srv/gump/public/workspace/tomcat-trunk/output/logs-NIO2
 -INFO- Project Reports in: 
/srv/gump/public/workspace/tomcat-trunk/output/test-tmp-NIO2/logs
 -WARNING- No directory 
[/srv/gump/public/workspace/tomcat-trunk/output/test-tmp-NIO2/logs]



The following work was performed:
http://vmgump-vm3.apache.org/tomcat-trunk/tomcat-trunk-test-nio2/gump_work/build_tomcat-trunk_tomcat-trunk-test-nio2.html
Work Name: build_tomcat-trunk_tomcat-trunk-test-nio2 (Type: Build)
Work ended in a state of : Failed
Elapsed: 22 mins 45 secs
Command Line: /usr/lib/jvm/java-8-oracle/bin/java -Djava.awt.headless=true 
-Dbuild.sysclasspath=only -Dsun.zip.disableMemoryMapping=true 
org.apache.tools.ant.Main -Dgump.merge=/srv/gump/public/gump/work/merge.xml 
-Djunit.jar=/srv/gump/public/workspace/junit/target/junit-4.13-SNAPSHOT.jar 
-Djava.net.preferIPv4Stack=/srv/gump/public/workspace/tomcat-trunk/true 
-Dobjenesis.jar=/srv/gump/public/workspace/objenesis/main/target/objenesis-3.1-SNAPSHOT.jar
 -Dtest.reports=output/logs-NIO2 -Dexecute.test.nio2=true 
-Dexamples.sources.skip=true 
-Dbase.path=/srv/gump/public/workspace/tomcat-trunk/tomcat-build-libs 
-Djdt.jar=/srv/gump/packages/eclipse/plugins/R-4.7.3a-201803300640/ecj-4.7.3a.jar
 -Dbndlib.jar=/srv/gump/packages/bnd/bndlib-4.0.0/biz.aQute.bndlib-4.0.0.jar 
-Dcommons-daemon.jar=/srv/gump/public/workspace/apache-commons/daemon/target/commons-daemon-1.1.1-SNAPSHOT.jar
 
-Dtest.openssl.path=/srv/gump/public/workspace/openssl-master/dest-20181123/bin/openssl
 -Dtest.temp=output/test-tmp-NIO2
  -Dtest.accesslog=true -Dexecute.test.nio=false 
-Dbnd.jar=/srv/gump/packages/bnd/bnd-4.0.0/biz.aQute.bnd-4.0.0.jar 
-Dexecute.test.apr=false -Dtest.excludePerformance=true -Dtest.relaxTiming=true 
-Deasymock.jar=/srv/gump/public/workspace/easymock/core/target/easymock-4.1-SNAPSHOT.jar
 -Dhamcrest.jar=/srv/gump/packages/hamcrest/hamcrest-core-1.3.jar 
-Dcglib.jar=/srv/gump/packages/cglib/cglib-nodep-2.2.jar test 
[Working Directory: /srv/gump/public/workspace/tomcat-trunk]
CLASSPATH: 
/usr/lib/jvm/java-8-oracle/lib/tools.jar:/srv/gump/public/workspace/tomcat-trunk/output/build/webapps/examples/WEB-INF/classes:/srv/gump/public/workspace/tomcat-trunk/output/testclasses:/srv/gump/public/workspace/ant/dist/lib/ant.jar:/srv/gump/public/workspace/ant/dist/lib/ant-launcher.jar:/srv/gump/public/workspace/ant/dist/lib/ant-jmf.jar:/srv/gump/public/workspace/ant/dist/lib/ant-junit.jar:/srv/gump/public/workspace/ant/dist/lib/ant-junit4.jar:/srv/gump/public/workspace/ant/dist/lib/ant-swing.jar:/srv/gump/public/workspace/ant/dist/lib/ant-apache-resolver.jar:/srv/gump/public/workspace/ant/dist/lib/ant-apache-xalan2.jar:/srv/gump/public/workspace/xml-commons/java/build/resolver.jar:/srv/gump/public/workspace/tomcat-trunk/output/build/bin/bootstrap.jar:/srv/gump/public/workspace/tomcat-trunk/output/build/bin/tomcat-juli.jar:/srv/gump/public/workspace/tomcat-trunk/output/build/lib/annotations-api.jar:/srv/gump/public/workspace/tomcat-trunk/output/build/lib/servlet-api.ja
 
r:/srv/gump/public/workspace/tomcat-trunk/output/build/lib/jsp-api.jar:/srv/gump/public/workspace/tomcat-trunk/output/build/lib/el-api.jar:/srv/gump/public/workspace/tomcat-trunk/output/build/lib/websocket-api.jar:/srv/gump/public/workspace/tomcat-trunk/output/build/lib/jaspic-api.jar:/srv/gump/public/workspace/tomcat-trunk/output/build/lib/catalina.jar:/srv/gump/public/workspace/tomcat-trunk/output/build/lib/catalina-ant.jar:/srv/gump/public/workspace/tomcat-trunk/output/build/lib/catalina-storeconfig.jar:/srv/gump/public/workspace/tomcat-trunk/output/build/lib/tomcat-coyote.jar:/srv/gump/public/workspace/tomcat-trunk/output/build/lib/jasper.jar:/srv/gump/public/workspace/tomcat-trunk/output/build/lib/jasper-el.jar:/srv/gump/public/workspace/tomcat-trunk/output/build/lib/catalina-tribes.jar:/srv/gump/public/workspace/tomcat-trunk/output/build/lib/catalina-ha.jar:/srv/gump/public/workspace/tomcat-trunk/output/build/lib/tomcat-api.jar:/srv/gump/public/workspace/tomcat-trunk/output/bu
 
ild/lib

[GUMP@vmgump-vm3]: Project tomcat-tc8.5.x-test-nio (in module tomcat-8.5.x) failed

2018-11-22 Thread Bill Barker
To whom it may engage...

This is an automated request, but not an unsolicited one. For 
more information please visit http://gump.apache.org/nagged.html, 
and/or contact the folk at gene...@gump.apache.org.

Project tomcat-tc8.5.x-test-nio has an issue affecting its community 
integration.
This issue affects 1 projects.
The current state of this project is 'Failed', with reason 'Build Failed'.
For reference only, the following projects are affected by this:
- tomcat-tc8.5.x-test-nio :  Tomcat 8.x, a web server implementing the Java 
Servlet 3.1,
...


Full details are available at:
http://vmgump-vm3.apache.org/tomcat-8.5.x/tomcat-tc8.5.x-test-nio/index.html

That said, some information snippets are provided here.

The following annotations (debug/informational/warning/error messages) were 
provided:
 -INFO- Failed with reason build failed
 -INFO- Project Reports in: 
/srv/gump/public/workspace/tomcat-8.5.x/output/logs-NIO
 -INFO- Project Reports in: 
/srv/gump/public/workspace/tomcat-8.5.x/output/test-tmp-NIO/logs
 -WARNING- No directory 
[/srv/gump/public/workspace/tomcat-8.5.x/output/test-tmp-NIO/logs]



The following work was performed:
http://vmgump-vm3.apache.org/tomcat-8.5.x/tomcat-tc8.5.x-test-nio/gump_work/build_tomcat-8.5.x_tomcat-tc8.5.x-test-nio.html
Work Name: build_tomcat-8.5.x_tomcat-tc8.5.x-test-nio (Type: Build)
Work ended in a state of : Failed
Elapsed: 19 mins 1 sec
Command Line: /usr/lib/jvm/java-8-oracle/bin/java -Djava.awt.headless=true 
-Dbuild.sysclasspath=only -Dsun.zip.disableMemoryMapping=true 
org.apache.tools.ant.Main -Dgump.merge=/srv/gump/public/gump/work/merge.xml 
-Djunit.jar=/srv/gump/public/workspace/junit/target/junit-4.13-SNAPSHOT.jar 
-Djava.net.preferIPv4Stack=/srv/gump/public/workspace/tomcat-8.5.x/true 
-Dobjenesis.jar=/srv/gump/public/workspace/objenesis/main/target/objenesis-3.1-SNAPSHOT.jar
 -Dtest.reports=output/logs-NIO -Dexecute.test.nio2=false 
-Dexamples.sources.skip=true 
-Dbase.path=/srv/gump/public/workspace/tomcat-8.5.x/tomcat-build-libs 
-Djdt.jar=/srv/gump/packages/eclipse/plugins/R-4.7.3a-201803300640/ecj-4.7.3a.jar
 -Dtest.relaxTiming=true 
-Dcommons-daemon.jar=/srv/gump/public/workspace/apache-commons/daemon/target/commons-daemon-1.1.1-SNAPSHOT.jar
 -Dtest.temp=output/test-tmp-NIO -Dtest.accesslog=true -Dexecute.test.nio=true 
-Dtest.openssl.path=/srv/gump/public/workspace/openssl-1.1.1/dest-20181123/bin/openssl
 -Dexecu
 te.test.bio=false -Dexecute.test.apr=false -Dtest.excludePerformance=true 
-Deasymock.jar=/srv/gump/packages/easymock3/easymock-3.6.jar 
-Dhamcrest.jar=/srv/gump/packages/hamcrest/hamcrest-core-1.3.jar 
-Dcglib.jar=/srv/gump/packages/cglib/cglib-nodep-2.2.jar test 
[Working Directory: /srv/gump/public/workspace/tomcat-8.5.x]
CLASSPATH: 
/usr/lib/jvm/java-8-oracle/lib/tools.jar:/srv/gump/public/workspace/tomcat-8.5.x/output/build/webapps/examples/WEB-INF/classes:/srv/gump/public/workspace/tomcat-8.5.x/output/testclasses:/srv/gump/public/workspace/ant/dist/lib/ant.jar:/srv/gump/public/workspace/ant/dist/lib/ant-launcher.jar:/srv/gump/public/workspace/ant/dist/lib/ant-jmf.jar:/srv/gump/public/workspace/ant/dist/lib/ant-junit.jar:/srv/gump/public/workspace/ant/dist/lib/ant-junit4.jar:/srv/gump/public/workspace/ant/dist/lib/ant-swing.jar:/srv/gump/public/workspace/ant/dist/lib/ant-apache-resolver.jar:/srv/gump/public/workspace/ant/dist/lib/ant-apache-xalan2.jar:/srv/gump/public/workspace/xml-commons/java/build/resolver.jar:/srv/gump/public/workspace/tomcat-8.5.x/output/build/bin/bootstrap.jar:/srv/gump/public/workspace/tomcat-8.5.x/output/build/bin/tomcat-juli.jar:/srv/gump/public/workspace/tomcat-8.5.x/output/build/lib/annotations-api.jar:/srv/gump/public/workspace/tomcat-8.5.x/output/build/lib/servlet-api.ja
 
r:/srv/gump/public/workspace/tomcat-8.5.x/output/build/lib/jsp-api.jar:/srv/gump/public/workspace/tomcat-8.5.x/output/build/lib/el-api.jar:/srv/gump/public/workspace/tomcat-8.5.x/output/build/lib/websocket-api.jar:/srv/gump/public/workspace/tomcat-8.5.x/output/build/lib/jaspic-api.jar:/srv/gump/public/workspace/tomcat-8.5.x/output/build/lib/catalina.jar:/srv/gump/public/workspace/tomcat-8.5.x/output/build/lib/catalina-ant.jar:/srv/gump/public/workspace/tomcat-8.5.x/output/build/lib/catalina-storeconfig.jar:/srv/gump/public/workspace/tomcat-8.5.x/output/build/lib/tomcat-coyote.jar:/srv/gump/public/workspace/tomcat-8.5.x/output/build/lib/jasper.jar:/srv/gump/public/workspace/tomcat-8.5.x/output/build/lib/jasper-el.jar:/srv/gump/public/workspace/tomcat-8.5.x/output/build/lib/catalina-tribes.jar:/srv/gump/public/workspace/tomcat-8.5.x/output/build/lib/catalina-ha.jar:/srv/gump/public/workspace/tomcat-8.5.x/output/build/lib/tomcat-api.jar:/srv/gump/public/workspace/tomcat-8.5.x/output/bu
 
ild/lib/tomcat-jni.jar:/srv/gump/public/workspace/tomcat-8.5.x/output/build/lib/tomcat-util.jar:/srv/gump/public/workspace/tomcat-8.5.x/output/build/lib/tomcat-util-scan.jar:/srv/gump/public/workspace/tomcat-8.5.x/output/build/lib/

Re: svn commit: r1847118 - in /tomcat/trunk: java/org/apache/catalina/tribes/group/interceptors/ test/org/apache/catalina/tribes/group/interceptors/

2018-11-22 Thread Christopher Schultz
-BEGIN PGP SIGNED MESSAGE-
Hash: SHA256

Mark,

On 11/22/18 18:24, Mark Thomas wrote:
> 
> 
> On 22/11/2018 23:05, Christopher Schultz wrote:
>> On 11/22/18 17:52, Mark Thomas wrote:
>>> On 22/11/2018 22:32, Christopher Schultz wrote:
>>>> On 11/22/18 16:43, Mark Thomas wrote:
> 
> 
> 
>>>>> syncs on encrypt() and decrypt() seem to have done the
>>>>> trick. That was just a quick hack to confirm a suspicion -
>>>>> it isn't the right long term fix.
>>>>> 
>>>>> If we want this to be performant under load I'd lean
>>>>> towards using a Queue for encryption ciphers and another
>>>>> for decryption ciphers along the lines of the way
>>>>> SessionIdGeneratorBase handles SecureRandom.
>>>>> 
>>>>> We should probably handle SecureRandom the same way.
>>>>> 
>>>>> I'll start working on a patch.
>>>> 
>>>> Hmm... I was under the impression that the message-sending 
>>>> operations were single-threaded (and similar with the
>>>> receiving operations).
>>> 
>>> There are locks in other Interceptors which are consistent
>>> with them being multi-threaded.
>>> 
>>>> I've read that calling Cipher.init() is expensive, but since 
>>>> it's being done for every outgoing (and incoming) message, 
>>>> perhaps there is no reason to re-use Cipher objects. I'd be 
>>>> interested to see a micro-benchmark showing whether all that 
>>>> complexity (Cipher object pool) is really worth it. It
>>>> probably isn't, given that the code without that complexity
>>>> would be super clear and compact.
>>> 
>>> Good point. I'll try that. It will depend on how expensive
>>> creating the object is.
>>> 
>>> Even with the pools the code is pretty clean but I take the
>>> point that it would be (even) cleaner without.
>>> 
>>> I have a patch ready to go but I'll do some work on the
>>> benchmark first.
>> 
>> I have a patch below. Passes all my unit-tests, but I don't have
>> any multi-threaded ones written at this point.
>> 
>> I'd appreciate a review before I commit. I'm going to change the 
>> cipher-pool-size to be configurable via an XML attribute, etc.
> 
> The patch is hard to read. Can you upload it to people.a.o (or
> maybe we should create a BZ issue for this).
> 
> My benchmark shows similar results to yours. Pooling is certainly
> worth while.
> 
> I have a patch too. Available here: 
> http://people.apache.org/~markt/patches/20181122-encryption-intercepto
r-concurrency-v1.patch
>
> 
> 
> It does a little more than just add pooling.
> 
> It uses the same principle as elsewhere in Tomcat - that the pools
> grows as needed and doesn't shrink so no need for additional
> configuration options.
> 
> I expect we'll pull bits from both patches but I'm going to call it
> a day now and come back to this tomorrow.

I took a slightly different approach, using a fixed-size
ArrayBlockingQueue. I suppose we could use a LinkedBlockingQueue... it
that fits-in better with how Tomcat does things in other scenarios,
that's fine. My code defaulted to using a queue the size of the number
of CPUs, since the encryption code is CPU-bound so a huge pool isn't
really going to help even with thousands of request-processing threads.

In terms of the Queue API... is there a big difference between using
take/put versus poll/offer? I can't think of any case where we'd want
to time-out waiting for an item (either checking-out or checking-in).

For the stop() method... does it matter what "kind" of stop is being
processed? We only call initCiphers() on certain kinds of "start"
events, so we should probably tear everything down under the same
conditions.

I have a multi-threaded unit test that showed me that XButeBuffer is
also not thread-safe :)

- -chris
-BEGIN PGP SIGNATURE-
Comment: Using GnuPG with Thunderbird - https://www.enigmail.net/

iQIzBAEBCAAdFiEEMmKgYcQvxMe7tcJcHPApP6U8pFgFAlv3QywACgkQHPApP6U8
pFgBbg//fX1G6zLFMlsBs9UT6anKzz7vgNt/0z6ZEIJgFQ9zJa3AL+Xbp8FmwIQH
lZy7aqn+uRPRhRYobsHA24Xyz98z6PzQYbOgch6TGzCyWPJ+h0IGNZVHnvTeOGfO
xstiv/G5Y4GCRkdCvEz1TXnArZOoKp7H3Q9ZvOEqj/AShqycEtWYMx4a3Jt44JbC
1nc79P496Gpska3lYhBJSyhWs9IBVWpfKucCSEThEqcZbZrtDw9hpsCNK2gIYYup
IxtMHZmIgwEtNM8luS6rsbD6Pad4fgbysCeiAfM1wfOmjOaaUU+6JCnv1zAEidGW
TuofTCUvMFotVI/A3tAjxzXFVMi8kP329tFJBzzvmR2ka2D2SanGQIabJG+f3cWQ
7wJiV94cUGOhGpI3tYc6EmS98e9VxR24qv9wHV6gPu5dW5pDNQxjjDHALhnR4Uqq
nIYk5L25iG+YYVQejWt30+vlnkLIPmic95nqx2ODNPN+f9g/21mQiAC2RGIMy415
QUKbt7BFv4P8ejSmcwFoy3rw02kJ2bfBU31rR741Tg6F3oeKed6GRc9N/W7XdaNy
UPSoMel5QH5bLHJUXtbLRk05D/C1n55LQe500gvQQ1pUcaROPjbHg4et0y/eiBg8
5GMlN5WlFh4IAhZVrDCzibfqFeBIcX4pTS8rPqBJzhZcS4etgVs=
=uRYu
-END PGP SIGNATURE-

-
To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org
For additional commands, e-mail: dev-h...@tomcat.apache.org



[Solved] Problems running TC 7 with Log4J 1.2 and Java 11

2018-11-22 Thread Rainer Jung

Am 22.11.2018 um 16:45 schrieb Rainer Jung:
I have a problem running TC 7.0.92 with Log4J 1.2.17 and Java 11 when 
trying to load the config from ${catalina.base}/somedir/log4j.properties 
via server.loader=${catalina.base}/somedir in conf/catalina.properties.


It works with Java 9 and 10 and it also works when using the 
common.loader instead of server.loader. Setting -Dlog4j.debug shows, 
that log4j tries to load the files via the class loader but isn't able to:


Trying to find [log4j.xml] using java.net.URLClassLoader@c267ef4 class 
loader.

Trying to find [log4j.xml] using ClassLoader.getSystemResource().
Trying to find [log4j.properties] using java.net.URLClassLoader@c267ef4 
class loader.

Trying to find [log4j.properties] using ClassLoader.getSystemResource().
Could not find resource: [null].
log4j:WARN No appenders could be found for logger 
(org.apache.catalina.startup.Catalina).

log4j:WARN Please initialize the log4j system properly.
log4j:WARN See http://logging.apache.org/log4j/1.2/faq.html#noconfig for 
more info.


I suspect it might have to do with some module system change in JAVA 11, 
but I found nothing obvious. Adding "--illegal-access=debug" doesn't 
produce any output.


I'm using the log4j juli plus adapters. No problems using log4j 1.2 
inside webapps directly.


It turns out it is a version detection bug in log4j 1.2. They read the 
"java.version" system property to check, whether the JVM is Java 1 
(sic!). If so, they do not try to use the TCCL to look up the config 
file, only the class loader that defines the class that looks it up.


In my Tomcat case, it would be the TCCL (system aka catalina loader) 
that would find it, but not the class defining class loader (common loader).


My Java 10 had 10.0.2 as "java.version", but Java 11 (no patch yet 
installed) had a plain "11" and if "java.version" contains no dot, then 
Log4J 1.2 thinks it is Java 1 (plus some more erroneous assumptions).


So no Java bug, no TC bug, just a Log4J 1.2 annoyance that should no 
longer be observable once I have installed 11.0.1.


Thanks and regards,

Rainer

-
To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org
For additional commands, e-mail: dev-h...@tomcat.apache.org



Re: svn commit: r1847118 - in /tomcat/trunk: java/org/apache/catalina/tribes/group/interceptors/ test/org/apache/catalina/tribes/group/interceptors/

2018-11-22 Thread Mark Thomas




On 22/11/2018 23:05, Christopher Schultz wrote:

On 11/22/18 17:52, Mark Thomas wrote:

On 22/11/2018 22:32, Christopher Schultz wrote:

On 11/22/18 16:43, Mark Thomas wrote:





syncs on encrypt() and decrypt() seem to have done the trick.
That was just a quick hack to confirm a suspicion - it isn't
the right long term fix.

If we want this to be performant under load I'd lean towards
using a Queue for encryption ciphers and another for decryption
ciphers along the lines of the way SessionIdGeneratorBase
handles SecureRandom.

We should probably handle SecureRandom the same way.

I'll start working on a patch.


Hmm... I was under the impression that the message-sending
operations were single-threaded (and similar with the receiving
operations).


There are locks in other Interceptors which are consistent with
them being multi-threaded.


I've read that calling Cipher.init() is expensive, but since
it's being done for every outgoing (and incoming) message,
perhaps there is no reason to re-use Cipher objects. I'd be
interested to see a micro-benchmark showing whether all that
complexity (Cipher object pool) is really worth it. It probably
isn't, given that the code without that complexity would be super
clear and compact.


Good point. I'll try that. It will depend on how expensive creating
the object is.

Even with the pools the code is pretty clean but I take the point
that it would be (even) cleaner without.

I have a patch ready to go but I'll do some work on the benchmark
first.


I have a patch below. Passes all my unit-tests, but I don't have any
multi-threaded ones written at this point.

I'd appreciate a review before I commit. I'm going to change the
cipher-pool-size to be configurable via an XML attribute, etc.


The patch is hard to read. Can you upload it to people.a.o (or maybe we 
should create a BZ issue for this).


My benchmark shows similar results to yours. Pooling is certainly worth 
while.


I have a patch too. Available here:
http://people.apache.org/~markt/patches/20181122-encryption-interceptor-concurrency-v1.patch

It does a little more than just add pooling.

It uses the same principle as elsewhere in Tomcat - that the pools grows 
as needed and doesn't shrink so no need for additional configuration 
options.


I expect we'll pull bits from both patches but I'm going to call it a 
day now and come back to this tomorrow.


Mark

-
To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org
For additional commands, e-mail: dev-h...@tomcat.apache.org



Re: svn commit: r1847118 - in /tomcat/trunk: java/org/apache/catalina/tribes/group/interceptors/ test/org/apache/catalina/tribes/group/interceptors/

2018-11-22 Thread Christopher Schultz
-BEGIN PGP SIGNED MESSAGE-
Hash: SHA256

Mark,

On 11/22/18 17:52, Mark Thomas wrote:
> On 22/11/2018 22:32, Christopher Schultz wrote:
>> -BEGIN PGP SIGNED MESSAGE- Hash: SHA256
>> 
>> Mark,
>> 
>> On 11/22/18 16:43, Mark Thomas wrote:
>>> On 22/11/2018 19:17, Christopher Schultz wrote:
 Mark,
 
 On 11/22/18 05:21, Mark Thomas wrote:
> On 21/11/2018 22:39, Christopher Schultz wrote:
>> Mark,
>> 
> 
 
>>> I thought you were using CBC so a missing block (a
>>> message being one or more blocks) means that the next
>>> message can't be decrypted.
>> 
>> CBC *is* being used, but the cipher is reset after each 
>> message, and a new IV is being randomly generated for
>> that purpose. There is no state-carryover between
>> messages. At least, there shouldn't be.
 
> Ah. Thanks for the explanation. I should have looked at
> the code. That should all work then.
 
> I'll try and find some time today to figure out what is 
> causing the error messages I am seeing.
 
 Thanks, I'd appreciate a second set of eyes.
 
 I can't seem to find any problems with it. The only
 "problems" I ended up finding were poorly-written tests :)
>>> 
>>> syncs on encrypt() and decrypt() seem to have done the trick.
>>> That was just a quick hack to confirm a suspicion - it isn't
>>> the right long term fix.
>>> 
>>> If we want this to be performant under load I'd lean towards
>>> using a Queue for encryption ciphers and another for decryption
>>> ciphers along the lines of the way SessionIdGeneratorBase
>>> handles SecureRandom.
>>> 
>>> We should probably handle SecureRandom the same way.
>>> 
>>> I'll start working on a patch.
>> 
>> Hmm... I was under the impression that the message-sending
>> operations were single-threaded (and similar with the receiving
>> operations).
> 
> There are locks in other Interceptors which are consistent with
> them being multi-threaded.
> 
>> I've read that calling Cipher.init() is expensive, but since
>> it's being done for every outgoing (and incoming) message,
>> perhaps there is no reason to re-use Cipher objects. I'd be
>> interested to see a micro-benchmark showing whether all that
>> complexity (Cipher object pool) is really worth it. It probably
>> isn't, given that the code without that complexity would be super
>> clear and compact.
> 
> Good point. I'll try that. It will depend on how expensive creating
> the object is.
> 
> Even with the pools the code is pretty clean but I take the point
> that it would be (even) cleaner without.
> 
> I have a patch ready to go but I'll do some work on the benchmark
> first.

I have a patch below. Passes all my unit-tests, but I don't have any
multi-threaded ones written at this point.

I'd appreciate a review before I commit. I'm going to change the
cipher-pool-size to be configurable via an XML attribute, etc.

- -chris


=== CUT ===

> ### Eclipse Workspace Patch 1.0 #P tomcat-trunk Index:
> java/org/apache/catalina/tribes/group/interceptors/EncryptInterceptor.java
>
> 
===
> ---
> java/org/apache/catalina/tribes/group/interceptors/EncryptInterceptor.java
> (revision 1847118) +++
> java/org/apache/catalina/tribes/group/interceptors/EncryptInterceptor.java
> (working copy) @@ -20,7 +20,7 @@ import
> java.security.InvalidAlgorithmParameterException; import
> java.security.InvalidKeyException; import
> java.security.SecureRandom; - +import
> java.util.concurrent.ArrayBlockingQueue; import
> javax.crypto.BadPaddingException; import javax.crypto.Cipher; 
> import javax.crypto.IllegalBlockSizeException; @@ -63,8 +63,7 @@ 
> private String encryptionKeyString; private SecretKeySpec
> secretKey;
> 
> -private Cipher encryptionCipher; -private Cipher
> decryptionCipher; +private ArrayBlockingQueue
> cipherQueue;
> 
> public EncryptInterceptor() { } @@ -113,6 +112,9 @@ } catch
> (InvalidAlgorithmParameterException iape) { 
> log.error(sm.getString("encryptInterceptor.encrypt.failed")); throw
> new ChannelException(iape); +} catch (InterruptedException
> ie) { +
> log.error(sm.getString("encryptInterceptor.cipher-borrow.failed"),
> ie); +throw new ChannelException(ie); } }
> 
> @@ -138,6 +140,8 @@ 
> log.error(sm.getString("encryptInterceptor.decrypt.failed"), ike); 
> } catch (InvalidAlgorithmParameterException iape) { 
> log.error(sm.getString("encryptInterceptor.decrypt.failed"),
> iape); +} catch (InterruptedException ie) { +
> log.error(sm.getString("encryptInterceptor.cipher-borrow.failed"),
> ie); } }
> 
> @@ -297,12 +301,15 @@ setSecretKey(new
> SecretKeySpec(getEncryptionKeyInternal(), algorithmName));
> 
> String providerName = getProviderName(); -if(null ==
> providerName) { -encryptionCipher =
> Cipher.getInstance(algorithm); -decryptionCipher =
> Cipher.getInstance(algorithm); -} else { -

Re: svn commit: r1847118 - in /tomcat/trunk: java/org/apache/catalina/tribes/group/interceptors/ test/org/apache/catalina/tribes/group/interceptors/

2018-11-22 Thread Mark Thomas

On 22/11/2018 22:32, Christopher Schultz wrote:

-BEGIN PGP SIGNED MESSAGE-
Hash: SHA256

Mark,

On 11/22/18 16:43, Mark Thomas wrote:

On 22/11/2018 19:17, Christopher Schultz wrote:

Mark,

On 11/22/18 05:21, Mark Thomas wrote:

On 21/11/2018 22:39, Christopher Schultz wrote:

Mark,






I thought you were using CBC so a missing block (a message
being one or more blocks) means that the next message can't
be decrypted.


CBC *is* being used, but the cipher is reset after each
message, and a new IV is being randomly generated for that
purpose. There is no state-carryover between messages. At
least, there shouldn't be.



Ah. Thanks for the explanation. I should have looked at the
code. That should all work then.



I'll try and find some time today to figure out what is
causing the error messages I am seeing.


Thanks, I'd appreciate a second set of eyes.

I can't seem to find any problems with it. The only "problems" I
ended up finding were poorly-written tests :)


syncs on encrypt() and decrypt() seem to have done the trick. That
was just a quick hack to confirm a suspicion - it isn't the right
long term fix.

If we want this to be performant under load I'd lean towards using
a Queue for encryption ciphers and another for decryption ciphers
along the lines of the way SessionIdGeneratorBase handles
SecureRandom.

We should probably handle SecureRandom the same way.

I'll start working on a patch.


Hmm... I was under the impression that the message-sending operations
were single-threaded (and similar with the receiving operations).


There are locks in other Interceptors which are consistent with them 
being multi-threaded.



I've read that calling Cipher.init() is expensive, but since it's
being done for every outgoing (and incoming) message, perhaps there is
no reason to re-use Cipher objects. I'd be interested to see a
micro-benchmark showing whether all that complexity (Cipher object
pool) is really worth it. It probably isn't, given that the code
without that complexity would be super clear and compact.


Good point. I'll try that. It will depend on how expensive creating the 
object is.


Even with the pools the code is pretty clean but I take the point that 
it would be (even) cleaner without.


I have a patch ready to go but I'll do some work on the benchmark first.

Mark

-
To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org
For additional commands, e-mail: dev-h...@tomcat.apache.org



Re: svn commit: r1847118 - in /tomcat/trunk: java/org/apache/catalina/tribes/group/interceptors/ test/org/apache/catalina/tribes/group/interceptors/

2018-11-22 Thread Christopher Schultz
-BEGIN PGP SIGNED MESSAGE-
Hash: SHA256

All,

On 11/22/18 17:32, Christopher Schultz wrote:
> Mark,
> 
> On 11/22/18 16:43, Mark Thomas wrote:
>> On 22/11/2018 19:17, Christopher Schultz wrote:
>>> Mark,
>>> 
>>> On 11/22/18 05:21, Mark Thomas wrote:
 On 21/11/2018 22:39, Christopher Schultz wrote:
> Mark,
> 
 
>>> 
>> I thought you were using CBC so a missing block (a
>> message being one or more blocks) means that the next
>> message can't be decrypted.
> 
> CBC *is* being used, but the cipher is reset after each 
> message, and a new IV is being randomly generated for that 
> purpose. There is no state-carryover between messages. At 
> least, there shouldn't be.
>>> 
 Ah. Thanks for the explanation. I should have looked at the 
 code. That should all work then.
>>> 
 I'll try and find some time today to figure out what is 
 causing the error messages I am seeing.
>>> 
>>> Thanks, I'd appreciate a second set of eyes.
>>> 
>>> I can't seem to find any problems with it. The only "problems"
>>> I ended up finding were poorly-written tests :)
> 
>> syncs on encrypt() and decrypt() seem to have done the trick.
>> That was just a quick hack to confirm a suspicion - it isn't the
>> right long term fix.
> 
>> If we want this to be performant under load I'd lean towards
>> using a Queue for encryption ciphers and another for decryption
>> ciphers along the lines of the way SessionIdGeneratorBase
>> handles SecureRandom.
> 
>> We should probably handle SecureRandom the same way.
> 
>> I'll start working on a patch.
> 
> Hmm... I was under the impression that the message-sending
> operations were single-threaded (and similar with the receiving
> operations).
> 
> I've read that calling Cipher.init() is expensive, but since it's 
> being done for every outgoing (and incoming) message, perhaps there
> is no reason to re-use Cipher objects. I'd be interested to see a 
> micro-benchmark showing whether all that complexity (Cipher object 
> pool) is really worth it. It probably isn't, given that the code 
> without that complexity would be super clear and compact.

Okay, I did a1M rounds with:

a) Cipher.getInstance("AES").init(Cipher.ENCRYPT_MODE, key, IV)

b) existingCipher.init(Cipher.ENCRYPT_MODE, key, IV)

And I got these results:

  New Cipher took 31485 ms
  Only init took 307 ms

I ran my benchmark a bunch of times and got very similar results.

So, it looks like maintaining a pool of Cipher objects is really
beneficial.

Did you end up doing any work on this yet, or shall I take a stab at it?

- -chris
-BEGIN PGP SIGNATURE-
Comment: Using GnuPG with Thunderbird - https://www.enigmail.net/

iQIzBAEBCAAdFiEEMmKgYcQvxMe7tcJcHPApP6U8pFgFAlv3MhsACgkQHPApP6U8
pFgD0g//V15YGjhwL0P75Rbk6r0q0W7KfwoOuNsi08AK+mQfZ1WKUnqDSIXKcNvC
G6VcQ4QGuwjXMxD0GI7WrBbnhtYtwgteX19AgoHdGxw1cPDRePJyt2e3MNKHj/7W
TIfgTJYXqPbJTDuBGRha2Y9UVqze7vgi21pRVcXtwlEQez9JJdzsWuufMXCH6RAs
a9BpfbpU48JJlyf8a84vKHT3ccuscsa1rIxQ+l2ldJk1Gf4Y/Rl41dDJyEVEOqaN
j2Zb/Jin3DXapDja9SFOwMO5Do9gbEi9/qDXGTgp53YQgTRX2wyVpbFD6JRmqs2z
czFa6zFf0D5rbw2/M4bPmIezyuQp7pydPUsHzMYwrrISfLGMQJbBZAjEtMC0jWPf
fqVGX/RHU2k1B2x9eFVKPZ8HUKbuum/9iZGK3vIrachncx7OyDQGZLAMsHQBWeU4
pJXnzQV6L83VPMriBymcDKINeVmA/lugUtQq90YpxRlicv1gFsP4gopQWBXL2bAI
uwsbOWYFRkYWMb6Rg+h+e2T6L1uNE5vQZtLN369xOcHnjZFNgJSrPCViwxsCayc6
dQByJH8EYkP96xBisNCzB8rL27J9E4EDeoXdumpr7XKn8BIaMtkrHedft17WUuiO
v/ptPuV7+FBEhj8sqetlxObYrMBmJMyl2JNrJgeJBDzq+ddIujs=
=po3H
-END PGP SIGNATURE-

-
To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org
For additional commands, e-mail: dev-h...@tomcat.apache.org



Re: svn commit: r1847118 - in /tomcat/trunk: java/org/apache/catalina/tribes/group/interceptors/ test/org/apache/catalina/tribes/group/interceptors/

2018-11-22 Thread Christopher Schultz
-BEGIN PGP SIGNED MESSAGE-
Hash: SHA256

Mark,

On 11/22/18 16:43, Mark Thomas wrote:
> On 22/11/2018 19:17, Christopher Schultz wrote:
>> Mark,
>> 
>> On 11/22/18 05:21, Mark Thomas wrote:
>>> On 21/11/2018 22:39, Christopher Schultz wrote:
 Mark,
 
>>> 
>> 
> I thought you were using CBC so a missing block (a message 
> being one or more blocks) means that the next message can't
> be decrypted.
 
 CBC *is* being used, but the cipher is reset after each
 message, and a new IV is being randomly generated for that
 purpose. There is no state-carryover between messages. At
 least, there shouldn't be.
>> 
>>> Ah. Thanks for the explanation. I should have looked at the
>>> code. That should all work then.
>> 
>>> I'll try and find some time today to figure out what is
>>> causing the error messages I am seeing.
>> 
>> Thanks, I'd appreciate a second set of eyes.
>> 
>> I can't seem to find any problems with it. The only "problems" I
>> ended up finding were poorly-written tests :)
> 
> syncs on encrypt() and decrypt() seem to have done the trick. That
> was just a quick hack to confirm a suspicion - it isn't the right
> long term fix.
> 
> If we want this to be performant under load I'd lean towards using
> a Queue for encryption ciphers and another for decryption ciphers
> along the lines of the way SessionIdGeneratorBase handles
> SecureRandom.
> 
> We should probably handle SecureRandom the same way.
> 
> I'll start working on a patch.

Hmm... I was under the impression that the message-sending operations
were single-threaded (and similar with the receiving operations).

I've read that calling Cipher.init() is expensive, but since it's
being done for every outgoing (and incoming) message, perhaps there is
no reason to re-use Cipher objects. I'd be interested to see a
micro-benchmark showing whether all that complexity (Cipher object
pool) is really worth it. It probably isn't, given that the code
without that complexity would be super clear and compact.

- -chris
-BEGIN PGP SIGNATURE-
Comment: Using GnuPG with Thunderbird - https://www.enigmail.net/

iQIzBAEBCAAdFiEEMmKgYcQvxMe7tcJcHPApP6U8pFgFAlv3LnoACgkQHPApP6U8
pFg2ZA/+N1tUfYqTja/rEpgrf3FeM9PSukLit78qK16bFXRjyB7RbkiwaBj696VW
hhOvO5/5FeRPIJWHBPbAL6pwiMND3vGjhZHjCM9HOjoTF1cAL75s+0PUQNYy184O
71T3ozvcxy0TQ/cUKZb0eYOGfleeZWmQ7SZsrozNtGgT9QDSptGLsXi4a+8B5VfJ
nbtpOAibFyCSYguLuBHjCdBzY1xAQXEZnvPOpEfZyYl40aTjEn7o8GmbdqOtcu1t
BrATqi0Dtju5PqPHsnAgdG9PDbw6KyDC+qcCaJ7ljF8arfiGXrc84T5X398ZWEGq
0dzLJeAe4gCfriBDY7EKl62bwVQFQZAOXxA4tvYaSS6kUI+Y1tWxm7pq28qdUXfS
QEKxV+mwglxkhjRdBbiuKW+7TJV6vX81G7hNud6kaaEIh+ycoIXGJfLgir4Q7PKP
uL8CQtQfsTd17lX7nBvfSuV+9/eWLOGPBjcpUrFQzDruH0OYE99rM9SikGlQlS1h
UfKdYuI2H1JxRxMC5Etd9gEDFtiBbencnjMUv295xu4N01UvqklKniHzoFMRwWV/
Z0oGHvAboE40EeGiW1ybiofLteO1ZwYJ83pq1Ma4muN+swvkJqVz7IiQswasKPwP
+HMv9o47IQbEQVyfsHyT+NMFOTgfB1FZWxU3D666Hl+QVREJbyA=
=/Di8
-END PGP SIGNATURE-

-
To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org
For additional commands, e-mail: dev-h...@tomcat.apache.org



Re: svn commit: r1847118 - in /tomcat/trunk: java/org/apache/catalina/tribes/group/interceptors/ test/org/apache/catalina/tribes/group/interceptors/

2018-11-22 Thread Mark Thomas
On 22/11/2018 19:17, Christopher Schultz wrote:
> Mark,
> 
> On 11/22/18 05:21, Mark Thomas wrote:
>> On 21/11/2018 22:39, Christopher Schultz wrote:
>>> Mark,
>>>
>> 
> 
 I thought you were using CBC so a missing block (a message
 being one or more blocks) means that the next message can't be 
 decrypted.
>>>
>>> CBC *is* being used, but the cipher is reset after each message,
>>> and a new IV is being randomly generated for that purpose. There
>>> is no state-carryover between messages. At least, there shouldn't
>>> be.
> 
>> Ah. Thanks for the explanation. I should have looked at the code.
>> That should all work then.
> 
>> I'll try and find some time today to figure out what is causing
>> the error messages I am seeing.
> 
> Thanks, I'd appreciate a second set of eyes.
> 
> I can't seem to find any problems with it. The only "problems" I ended
> up finding were poorly-written tests :)

syncs on encrypt() and decrypt() seem to have done the trick. That was
just a quick hack to confirm a suspicion - it isn't the right long term fix.

If we want this to be performant under load I'd lean towards using a
Queue for encryption ciphers and another for decryption ciphers along
the lines of the way SessionIdGeneratorBase handles SecureRandom.

We should probably handle SecureRandom the same way.

I'll start working on a patch.

Mark

-
To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org
For additional commands, e-mail: dev-h...@tomcat.apache.org



Re: svn commit: r1847118 - in /tomcat/trunk: java/org/apache/catalina/tribes/group/interceptors/ test/org/apache/catalina/tribes/group/interceptors/

2018-11-22 Thread Christopher Schultz
-BEGIN PGP SIGNED MESSAGE-
Hash: SHA256

Mark,

On 11/22/18 05:21, Mark Thomas wrote:
> On 21/11/2018 22:39, Christopher Schultz wrote:
>> Mark,
>> 
> 
> 
>>> I thought you were using CBC so a missing block (a message
>>> being one or more blocks) means that the next message can't be 
>>> decrypted.
>> 
>> CBC *is* being used, but the cipher is reset after each message,
>> and a new IV is being randomly generated for that purpose. There
>> is no state-carryover between messages. At least, there shouldn't
>> be.
> 
> Ah. Thanks for the explanation. I should have looked at the code.
> That should all work then.
> 
> I'll try and find some time today to figure out what is causing
> the error messages I am seeing.

Thanks, I'd appreciate a second set of eyes.

I can't seem to find any problems with it. The only "problems" I ended
up finding were poorly-written tests :)

- -chris
-BEGIN PGP SIGNATURE-
Comment: Using GnuPG with Thunderbird - https://www.enigmail.net/

iQIzBAEBCAAdFiEEMmKgYcQvxMe7tcJcHPApP6U8pFgFAlv3ALEACgkQHPApP6U8
pFgN3w/+JLgVNXCkg+zMNHmqxG4nJeTmByt1QQtJIYWOWMrg7cQYgj4RO+MucGsb
HiapvPQijnP8Po4jh3Xp2gcQ8ZxWPEX/OJTMFItNsP9Bb4/WCYqSy63N8tFo30Zb
7QZ3ZvLY95d8yJOtt0I95d0lgEd00RR++yB8xC/3Km6GvnnX88jwErL1DEWG7K1C
xgX9QQLUrn7PMlko/+gffmlu6yEDuG4UDgis3QvTBeLFDF5OfdBLTAGqsWUwXijr
YqhitGH9LHQtDBY9jiT4k3b/OwTbOpg4KEVGUM5LoseyDDdqhbyYTJW1KoBE0QD2
LPVmac13stfkk9Zu5kpQzPyyo8XyP+3ZJs4Rhgbq/2AmiQ63z9RiGx0WGx3nwDvB
htSxCbvIf3+fWD1d43LIy8ahR2uDS6Ni2bXn1AfQGZyr4nMcsv56DVnBdPldQ7X6
PcEvZTy9/3zx54nBNl3CWe0m2d300ys04piC2eeS0VyICdVhxfQnTJV4w17/lkoJ
7G3QVz+zE5qckLjeroGhhZsU8JJUgL/+fcff6sja7K8wNmFvRzAJYulE+nc//u1y
QnesXMs3tM6+1010guR36Ilaxb1mJAMFvm1ikbUl8MIdm6UmADiA5YwPuSuJwq66
nWCzXCUMZJ20zhaoY86naKczxZgv+g3bnpDKHj4M5BnzgHWQ6wM=
=YD4u
-END PGP SIGNATURE-

-
To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org
For additional commands, e-mail: dev-h...@tomcat.apache.org



Re: Problems running TC 7 with Log4J 1.2 and Java 11

2018-11-22 Thread Romain Manni-Bucau
You mean the config in server and libs in common? Only tested with all in
the same loader.

Le jeu. 22 nov. 2018 19:52, Rainer Jung  a écrit :

> Hi Romain, hi all,
>
> are you sure you tried i as I described? Namely loading the config via a
> server.loader path property newly defined in conf/catalina.properties?
> And of course I was using the juli and juli-adapters jar from the 7.0.92
> extras download.
>
> I have another detail to report: it starts working even with Java 11 and
> using the server loader, when i place the log4j.jar and the
> juli-adapters.jar into the server loader instead of the common loader.
> That's interesting, because it was not needed until and including Java
> 10. It worked there by just putting the config into the server loader
> and keeping the log4j and juli-adapters jar in the common loader.
>
> It seems that either for some reason the server loader is no longer
> parent of the common loader or the getResource() call changed behavior
> w.r.t. class loader hierarchies and delegation.
>
> Will debug further.
>
> Regards,
>
> Rainer
>
> Am 22.11.2018 um 18:31 schrieb Romain Manni-Bucau:
> > Hi Rainer,
> >
> > You are right, missed it was set OOTB.
> >
> > BTW, just tested on tomcat 7.0.92 with the log4j (v1.2.17) and its
> > adapter (of the 8.0.53) and java 11.0.1+13-LTS and it works for me
> > (base=home in my test).
> >
> > Romain Manni-Bucau
> > @rmannibucau  | Blog
> >  | Old Blog
> >  | Github
> >  | LinkedIn
> >  | Book
> > <
> https://www.packtpub.com/application-development/java-ee-8-high-performance
> >
> >
> >
> > Le jeu. 22 nov. 2018 à 17:40, Rainer Jung  > > a écrit :
> >
> > Hi Romain,
> >
> > Am 22.11.2018 um 16:53 schrieb Romain Manni-Bucau:
> >  > Hi Rainer,
> >  >
> >  > did you open some java.base modules? like
> >
> > No, just the add-opens that we ship in our catalina.sh.
> >
> >  > --add-opens java.base/java.lang=log4j
> >  >
> >  > (not sure this is the one to open but I guess you can debug and
> > identified
> >  > missing open this way - debugging
> > java.lang.Module#isOpen(java.lang.String)
> >  > for instance)
> >
> > I hoped to find a more definitive answer here, not needing to debug
> > into
> > code.
> >
> > Note this is Log4j 1.2, not 2. So not sure why there should be a
> module
> > "log4j". And if log4j would instead be part of the unnamed module,
> then
> > the already existing --add-opens=java.base/java.lang=ALL-UNNAMED
> would
> > be the corrected line for what you suggest. But as said: that one is
> > already one of the three add-opens TC contains in catalina.sh by
> > default.
> >
> > Regards,
> >
> > Rainer
> >
> >  > Romain Manni-Bucau
> >  > @rmannibucau  |  Blog
> >  >  | Old Blog
> >  >  | Github
> >  |
> >  > LinkedIn  | Book
> >  >
> > <
> https://www.packtpub.com/application-development/java-ee-8-high-performance
> >
> >  >
> >  >
> >  > Le jeu. 22 nov. 2018 à 16:45, Rainer Jung
> > mailto:rainer.j...@kippdata.de>> a
> >  > écrit :
> >  >
> >  >> Hi all,
> >  >>
> >  >> I have a problem running TC 7.0.92 with Log4J 1.2.17 and Java 11
> > when
> >  >> trying to load the config from
> > ${catalina.base}/somedir/log4j.properties
> >  >> via server.loader=${catalina.base}/somedir in
> > conf/catalina.properties.
> >  >>
> >  >> It works with Java 9 and 10 and it also works when using the
> >  >> common.loader instead of server.loader. Setting -Dlog4j.debug
> shows,
> >  >> that log4j tries to load the files via the class loader but
> > isn't able to:
> >  >>
> >  >> Trying to find [log4j.xml] using java.net.URLClassLoader@c267ef4
> > class
> >  >> loader.
> >  >> Trying to find [log4j.xml] using ClassLoader.getSystemResource().
> >  >> Trying to find [log4j.properties] using
> > java.net.URLClassLoader@c267ef4
> >  >> class loader.
> >  >> Trying to find [log4j.properties] using
> > ClassLoader.getSystemResource().
> >  >> Could not find resource: [null].
> >  >> log4j:WARN No appenders could be found for logger
> >  >> (org.apache.catalina.startup.Catalina).
> >  >> log4j:WARN Please initialize the log4j system properly.
> >  >> log4j:WARN See
> > http://logging.apache.org/log4j/1.2/faq.html#noconfig for
> >  >> more info.
> >  >>
> >  >> I suspect it might have to do with some module system change in
> > JAVA 11,
> >  >> but I found nothing obvious. Adding "--illegal-access=debu

Re: Problems running TC 7 with Log4J 1.2 and Java 11

2018-11-22 Thread Rainer Jung

Hi Romain, hi all,

are you sure you tried i as I described? Namely loading the config via a 
server.loader path property newly defined in conf/catalina.properties? 
And of course I was using the juli and juli-adapters jar from the 7.0.92 
extras download.


I have another detail to report: it starts working even with Java 11 and 
using the server loader, when i place the log4j.jar and the 
juli-adapters.jar into the server loader instead of the common loader. 
That's interesting, because it was not needed until and including Java 
10. It worked there by just putting the config into the server loader 
and keeping the log4j and juli-adapters jar in the common loader.


It seems that either for some reason the server loader is no longer 
parent of the common loader or the getResource() call changed behavior 
w.r.t. class loader hierarchies and delegation.


Will debug further.

Regards,

Rainer

Am 22.11.2018 um 18:31 schrieb Romain Manni-Bucau:

Hi Rainer,

You are right, missed it was set OOTB.

BTW, just tested on tomcat 7.0.92 with the log4j (v1.2.17) and its 
adapter (of the 8.0.53) and java 11.0.1+13-LTS and it works for me 
(base=home in my test).


Romain Manni-Bucau
@rmannibucau  | Blog 
 | Old Blog 
 | Github 
 | LinkedIn 
 | Book 




Le jeu. 22 nov. 2018 à 17:40, Rainer Jung > a écrit :


Hi Romain,

Am 22.11.2018 um 16:53 schrieb Romain Manni-Bucau:
 > Hi Rainer,
 >
 > did you open some java.base modules? like

No, just the add-opens that we ship in our catalina.sh.

 > --add-opens java.base/java.lang=log4j
 >
 > (not sure this is the one to open but I guess you can debug and
identified
 > missing open this way - debugging
java.lang.Module#isOpen(java.lang.String)
 > for instance)

I hoped to find a more definitive answer here, not needing to debug
into
code.

Note this is Log4j 1.2, not 2. So not sure why there should be a module
"log4j". And if log4j would instead be part of the unnamed module, then
the already existing --add-opens=java.base/java.lang=ALL-UNNAMED would
be the corrected line for what you suggest. But as said: that one is
already one of the three add-opens TC contains in catalina.sh by
default.

Regards,

Rainer

 > Romain Manni-Bucau
 > @rmannibucau  |  Blog
 >  | Old Blog
 >  | Github
 |
 > LinkedIn  | Book
 >


 >
 >
 > Le jeu. 22 nov. 2018 à 16:45, Rainer Jung
mailto:rainer.j...@kippdata.de>> a
 > écrit :
 >
 >> Hi all,
 >>
 >> I have a problem running TC 7.0.92 with Log4J 1.2.17 and Java 11
when
 >> trying to load the config from
${catalina.base}/somedir/log4j.properties
 >> via server.loader=${catalina.base}/somedir in
conf/catalina.properties.
 >>
 >> It works with Java 9 and 10 and it also works when using the
 >> common.loader instead of server.loader. Setting -Dlog4j.debug shows,
 >> that log4j tries to load the files via the class loader but
isn't able to:
 >>
 >> Trying to find [log4j.xml] using java.net.URLClassLoader@c267ef4
class
 >> loader.
 >> Trying to find [log4j.xml] using ClassLoader.getSystemResource().
 >> Trying to find [log4j.properties] using
java.net.URLClassLoader@c267ef4
 >> class loader.
 >> Trying to find [log4j.properties] using
ClassLoader.getSystemResource().
 >> Could not find resource: [null].
 >> log4j:WARN No appenders could be found for logger
 >> (org.apache.catalina.startup.Catalina).
 >> log4j:WARN Please initialize the log4j system properly.
 >> log4j:WARN See
http://logging.apache.org/log4j/1.2/faq.html#noconfig for
 >> more info.
 >>
 >> I suspect it might have to do with some module system change in
JAVA 11,
 >> but I found nothing obvious. Adding "--illegal-access=debug" doesn't
 >> produce any output.
 >>
 >> I'm using the log4j juli plus adapters. No problems using log4j 1.2
 >> inside webapps directly.
 >>
 >> Any ideas?
 >>
 >> Regards,
 >>
 >> Rainer


-
To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org
For additional commands, e-mail: dev-h...@tomcat.apache.org



Re: Problems running TC 7 with Log4J 1.2 and Java 11

2018-11-22 Thread Romain Manni-Bucau
Hi Rainer,

You are right, missed it was set OOTB.

BTW, just tested on tomcat 7.0.92 with the log4j (v1.2.17) and its adapter
(of the 8.0.53) and java 11.0.1+13-LTS and it works for me (base=home in my
test).

Romain Manni-Bucau
@rmannibucau  |  Blog
 | Old Blog
 | Github  |
LinkedIn  | Book



Le jeu. 22 nov. 2018 à 17:40, Rainer Jung  a
écrit :

> Hi Romain,
>
> Am 22.11.2018 um 16:53 schrieb Romain Manni-Bucau:
> > Hi Rainer,
> >
> > did you open some java.base modules? like
>
> No, just the add-opens that we ship in our catalina.sh.
>
> > --add-opens java.base/java.lang=log4j
> >
> > (not sure this is the one to open but I guess you can debug and
> identified
> > missing open this way - debugging
> java.lang.Module#isOpen(java.lang.String)
> > for instance)
>
> I hoped to find a more definitive answer here, not needing to debug into
> code.
>
> Note this is Log4j 1.2, not 2. So not sure why there should be a module
> "log4j". And if log4j would instead be part of the unnamed module, then
> the already existing --add-opens=java.base/java.lang=ALL-UNNAMED would
> be the corrected line for what you suggest. But as said: that one is
> already one of the three add-opens TC contains in catalina.sh by default.
>
> Regards,
>
> Rainer
>
> > Romain Manni-Bucau
> > @rmannibucau  |  Blog
> >  | Old Blog
> >  | Github <
> https://github.com/rmannibucau> |
> > LinkedIn  | Book
> > <
> https://www.packtpub.com/application-development/java-ee-8-high-performance
> >
> >
> >
> > Le jeu. 22 nov. 2018 à 16:45, Rainer Jung  a
> > écrit :
> >
> >> Hi all,
> >>
> >> I have a problem running TC 7.0.92 with Log4J 1.2.17 and Java 11 when
> >> trying to load the config from ${catalina.base}/somedir/log4j.properties
> >> via server.loader=${catalina.base}/somedir in conf/catalina.properties.
> >>
> >> It works with Java 9 and 10 and it also works when using the
> >> common.loader instead of server.loader. Setting -Dlog4j.debug shows,
> >> that log4j tries to load the files via the class loader but isn't able
> to:
> >>
> >> Trying to find [log4j.xml] using java.net.URLClassLoader@c267ef4 class
> >> loader.
> >> Trying to find [log4j.xml] using ClassLoader.getSystemResource().
> >> Trying to find [log4j.properties] using java.net.URLClassLoader@c267ef4
> >> class loader.
> >> Trying to find [log4j.properties] using ClassLoader.getSystemResource().
> >> Could not find resource: [null].
> >> log4j:WARN No appenders could be found for logger
> >> (org.apache.catalina.startup.Catalina).
> >> log4j:WARN Please initialize the log4j system properly.
> >> log4j:WARN See http://logging.apache.org/log4j/1.2/faq.html#noconfig
> for
> >> more info.
> >>
> >> I suspect it might have to do with some module system change in JAVA 11,
> >> but I found nothing obvious. Adding "--illegal-access=debug" doesn't
> >> produce any output.
> >>
> >> I'm using the log4j juli plus adapters. No problems using log4j 1.2
> >> inside webapps directly.
> >>
> >> Any ideas?
> >>
> >> Regards,
> >>
> >> Rainer
>


Re: Problems running TC 7 with Log4J 1.2 and Java 11

2018-11-22 Thread Rainer Jung

Hi Romain,

Am 22.11.2018 um 16:53 schrieb Romain Manni-Bucau:

Hi Rainer,

did you open some java.base modules? like


No, just the add-opens that we ship in our catalina.sh.


--add-opens java.base/java.lang=log4j

(not sure this is the one to open but I guess you can debug and identified
missing open this way - debugging java.lang.Module#isOpen(java.lang.String)
for instance)


I hoped to find a more definitive answer here, not needing to debug into 
code.


Note this is Log4j 1.2, not 2. So not sure why there should be a module 
"log4j". And if log4j would instead be part of the unnamed module, then 
the already existing --add-opens=java.base/java.lang=ALL-UNNAMED would 
be the corrected line for what you suggest. But as said: that one is 
already one of the three add-opens TC contains in catalina.sh by default.


Regards,

Rainer


Romain Manni-Bucau
@rmannibucau  |  Blog
 | Old Blog
 | Github  |
LinkedIn  | Book



Le jeu. 22 nov. 2018 à 16:45, Rainer Jung  a
écrit :


Hi all,

I have a problem running TC 7.0.92 with Log4J 1.2.17 and Java 11 when
trying to load the config from ${catalina.base}/somedir/log4j.properties
via server.loader=${catalina.base}/somedir in conf/catalina.properties.

It works with Java 9 and 10 and it also works when using the
common.loader instead of server.loader. Setting -Dlog4j.debug shows,
that log4j tries to load the files via the class loader but isn't able to:

Trying to find [log4j.xml] using java.net.URLClassLoader@c267ef4 class
loader.
Trying to find [log4j.xml] using ClassLoader.getSystemResource().
Trying to find [log4j.properties] using java.net.URLClassLoader@c267ef4
class loader.
Trying to find [log4j.properties] using ClassLoader.getSystemResource().
Could not find resource: [null].
log4j:WARN No appenders could be found for logger
(org.apache.catalina.startup.Catalina).
log4j:WARN Please initialize the log4j system properly.
log4j:WARN See http://logging.apache.org/log4j/1.2/faq.html#noconfig for
more info.

I suspect it might have to do with some module system change in JAVA 11,
but I found nothing obvious. Adding "--illegal-access=debug" doesn't
produce any output.

I'm using the log4j juli plus adapters. No problems using log4j 1.2
inside webapps directly.

Any ideas?

Regards,

Rainer


-
To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org
For additional commands, e-mail: dev-h...@tomcat.apache.org



Re: Problems running TC 7 with Log4J 1.2 and Java 11

2018-11-22 Thread Romain Manni-Bucau
Hi Rainer,

did you open some java.base modules? like

--add-opens java.base/java.lang=log4j

(not sure this is the one to open but I guess you can debug and identified
missing open this way - debugging java.lang.Module#isOpen(java.lang.String)
for instance)

Romain Manni-Bucau
@rmannibucau  |  Blog
 | Old Blog
 | Github  |
LinkedIn  | Book



Le jeu. 22 nov. 2018 à 16:45, Rainer Jung  a
écrit :

> Hi all,
>
> I have a problem running TC 7.0.92 with Log4J 1.2.17 and Java 11 when
> trying to load the config from ${catalina.base}/somedir/log4j.properties
> via server.loader=${catalina.base}/somedir in conf/catalina.properties.
>
> It works with Java 9 and 10 and it also works when using the
> common.loader instead of server.loader. Setting -Dlog4j.debug shows,
> that log4j tries to load the files via the class loader but isn't able to:
>
> Trying to find [log4j.xml] using java.net.URLClassLoader@c267ef4 class
> loader.
> Trying to find [log4j.xml] using ClassLoader.getSystemResource().
> Trying to find [log4j.properties] using java.net.URLClassLoader@c267ef4
> class loader.
> Trying to find [log4j.properties] using ClassLoader.getSystemResource().
> Could not find resource: [null].
> log4j:WARN No appenders could be found for logger
> (org.apache.catalina.startup.Catalina).
> log4j:WARN Please initialize the log4j system properly.
> log4j:WARN See http://logging.apache.org/log4j/1.2/faq.html#noconfig for
> more info.
>
> I suspect it might have to do with some module system change in JAVA 11,
> but I found nothing obvious. Adding "--illegal-access=debug" doesn't
> produce any output.
>
> I'm using the log4j juli plus adapters. No problems using log4j 1.2
> inside webapps directly.
>
> Any ideas?
>
> Regards,
>
> Rainer
>
> -
> To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org
> For additional commands, e-mail: dev-h...@tomcat.apache.org
>
>


Problems running TC 7 with Log4J 1.2 and Java 11

2018-11-22 Thread Rainer Jung

Hi all,

I have a problem running TC 7.0.92 with Log4J 1.2.17 and Java 11 when 
trying to load the config from ${catalina.base}/somedir/log4j.properties 
via server.loader=${catalina.base}/somedir in conf/catalina.properties.


It works with Java 9 and 10 and it also works when using the 
common.loader instead of server.loader. Setting -Dlog4j.debug shows, 
that log4j tries to load the files via the class loader but isn't able to:


Trying to find [log4j.xml] using java.net.URLClassLoader@c267ef4 class 
loader.

Trying to find [log4j.xml] using ClassLoader.getSystemResource().
Trying to find [log4j.properties] using java.net.URLClassLoader@c267ef4 
class loader.

Trying to find [log4j.properties] using ClassLoader.getSystemResource().
Could not find resource: [null].
log4j:WARN No appenders could be found for logger 
(org.apache.catalina.startup.Catalina).

log4j:WARN Please initialize the log4j system properly.
log4j:WARN See http://logging.apache.org/log4j/1.2/faq.html#noconfig for 
more info.


I suspect it might have to do with some module system change in JAVA 11, 
but I found nothing obvious. Adding "--illegal-access=debug" doesn't 
produce any output.


I'm using the log4j juli plus adapters. No problems using log4j 1.2 
inside webapps directly.


Any ideas?

Regards,

Rainer

-
To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org
For additional commands, e-mail: dev-h...@tomcat.apache.org



Container packaging

2018-11-22 Thread Rémy Maucherat
Hi,

After a bit of experimenting, the docker image works well, and is nicer
(IMO) to configure and customize with the embedded updates from 9.0.14. I
have verified that Kubernetes based discovery works without being too
difficult to configure. Custom code and components can be plugged in easily
as anything in src/main/java will get compiled and packaged in the jar.

https://github.com/rmaucher/tomcat-maven

Now, distributing this sort of package is "easy" but is usually done with a
standalone git repo like this one. Where could I put this packaging instead
since we probably cannot do that in Tomcat-land ?

Rémy


Re: svn commit: r1847118 - in /tomcat/trunk: java/org/apache/catalina/tribes/group/interceptors/ test/org/apache/catalina/tribes/group/interceptors/

2018-11-22 Thread Mark Thomas
On 21/11/2018 22:39, Christopher Schultz wrote:
> Mark,
> 


>> I thought you were using CBC so a missing block (a message being
>> one or more blocks) means that the next message can't be
>> decrypted.
> 
> CBC *is* being used, but the cipher is reset after each message, and a
> new IV is being randomly generated for that purpose. There is no
> state-carryover between messages. At least, there shouldn't be.

Ah. Thanks for the explanation. I should have looked at the code. That
should all work then.

I'll try and find some time today to figure out what is causing the
error messages I am seeing.

Mark

-
To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org
For additional commands, e-mail: dev-h...@tomcat.apache.org



[GUMP@vmgump-vm3]: Project tomcat-trunk-test-nio2 (in module tomcat-trunk) failed

2018-11-22 Thread Bill Barker
To whom it may engage...

This is an automated request, but not an unsolicited one. For 
more information please visit http://gump.apache.org/nagged.html, 
and/or contact the folk at gene...@gump.apache.org.

Project tomcat-trunk-test-nio2 has an issue affecting its community integration.
This issue affects 1 projects.
The current state of this project is 'Failed', with reason 'Build Failed'.
For reference only, the following projects are affected by this:
- tomcat-trunk-test-nio2 :  Tomcat 9.x, a web server implementing the Java 
Servlet 4.0,
...


Full details are available at:
http://vmgump-vm3.apache.org/tomcat-trunk/tomcat-trunk-test-nio2/index.html

That said, some information snippets are provided here.

The following annotations (debug/informational/warning/error messages) were 
provided:
 -DEBUG- Dependency on bnd exists, no need to add for property bndlib.jar.
 -INFO- Failed with reason build failed
 -INFO- Project Reports in: 
/srv/gump/public/workspace/tomcat-trunk/output/logs-NIO2
 -INFO- Project Reports in: 
/srv/gump/public/workspace/tomcat-trunk/output/test-tmp-NIO2/logs
 -WARNING- No directory 
[/srv/gump/public/workspace/tomcat-trunk/output/test-tmp-NIO2/logs]



The following work was performed:
http://vmgump-vm3.apache.org/tomcat-trunk/tomcat-trunk-test-nio2/gump_work/build_tomcat-trunk_tomcat-trunk-test-nio2.html
Work Name: build_tomcat-trunk_tomcat-trunk-test-nio2 (Type: Build)
Work ended in a state of : Failed
Elapsed: 23 mins 45 secs
Command Line: /usr/lib/jvm/java-8-oracle/bin/java -Djava.awt.headless=true 
-Dbuild.sysclasspath=only -Dsun.zip.disableMemoryMapping=true 
org.apache.tools.ant.Main -Dgump.merge=/srv/gump/public/gump/work/merge.xml 
-Djunit.jar=/srv/gump/public/workspace/junit/target/junit-4.13-SNAPSHOT.jar 
-Djava.net.preferIPv4Stack=/srv/gump/public/workspace/tomcat-trunk/true 
-Dobjenesis.jar=/srv/gump/public/workspace/objenesis/main/target/objenesis-3.1-SNAPSHOT.jar
 -Dtest.reports=output/logs-NIO2 -Dexecute.test.nio2=true 
-Dexamples.sources.skip=true 
-Dbase.path=/srv/gump/public/workspace/tomcat-trunk/tomcat-build-libs 
-Djdt.jar=/srv/gump/packages/eclipse/plugins/R-4.7.3a-201803300640/ecj-4.7.3a.jar
 -Dbndlib.jar=/srv/gump/packages/bnd/bndlib-4.0.0/biz.aQute.bndlib-4.0.0.jar 
-Dcommons-daemon.jar=/srv/gump/public/workspace/apache-commons/daemon/target/commons-daemon-1.1.1-SNAPSHOT.jar
 
-Dtest.openssl.path=/srv/gump/public/workspace/openssl-master/dest-20181122/bin/openssl
 -Dtest.temp=output/test-tmp-NIO2
  -Dtest.accesslog=true -Dexecute.test.nio=false 
-Dbnd.jar=/srv/gump/packages/bnd/bnd-4.0.0/biz.aQute.bnd-4.0.0.jar 
-Dexecute.test.apr=false -Dtest.excludePerformance=true -Dtest.relaxTiming=true 
-Deasymock.jar=/srv/gump/public/workspace/easymock/core/target/easymock-4.1-SNAPSHOT.jar
 -Dhamcrest.jar=/srv/gump/packages/hamcrest/hamcrest-core-1.3.jar 
-Dcglib.jar=/srv/gump/packages/cglib/cglib-nodep-2.2.jar test 
[Working Directory: /srv/gump/public/workspace/tomcat-trunk]
CLASSPATH: 
/usr/lib/jvm/java-8-oracle/lib/tools.jar:/srv/gump/public/workspace/tomcat-trunk/output/build/webapps/examples/WEB-INF/classes:/srv/gump/public/workspace/tomcat-trunk/output/testclasses:/srv/gump/public/workspace/ant/dist/lib/ant.jar:/srv/gump/public/workspace/ant/dist/lib/ant-launcher.jar:/srv/gump/public/workspace/ant/dist/lib/ant-jmf.jar:/srv/gump/public/workspace/ant/dist/lib/ant-junit.jar:/srv/gump/public/workspace/ant/dist/lib/ant-junit4.jar:/srv/gump/public/workspace/ant/dist/lib/ant-swing.jar:/srv/gump/public/workspace/ant/dist/lib/ant-apache-resolver.jar:/srv/gump/public/workspace/ant/dist/lib/ant-apache-xalan2.jar:/srv/gump/public/workspace/xml-commons/java/build/resolver.jar:/srv/gump/public/workspace/tomcat-trunk/output/build/bin/bootstrap.jar:/srv/gump/public/workspace/tomcat-trunk/output/build/bin/tomcat-juli.jar:/srv/gump/public/workspace/tomcat-trunk/output/build/lib/annotations-api.jar:/srv/gump/public/workspace/tomcat-trunk/output/build/lib/servlet-api.ja
 
r:/srv/gump/public/workspace/tomcat-trunk/output/build/lib/jsp-api.jar:/srv/gump/public/workspace/tomcat-trunk/output/build/lib/el-api.jar:/srv/gump/public/workspace/tomcat-trunk/output/build/lib/websocket-api.jar:/srv/gump/public/workspace/tomcat-trunk/output/build/lib/jaspic-api.jar:/srv/gump/public/workspace/tomcat-trunk/output/build/lib/catalina.jar:/srv/gump/public/workspace/tomcat-trunk/output/build/lib/catalina-ant.jar:/srv/gump/public/workspace/tomcat-trunk/output/build/lib/catalina-storeconfig.jar:/srv/gump/public/workspace/tomcat-trunk/output/build/lib/tomcat-coyote.jar:/srv/gump/public/workspace/tomcat-trunk/output/build/lib/jasper.jar:/srv/gump/public/workspace/tomcat-trunk/output/build/lib/jasper-el.jar:/srv/gump/public/workspace/tomcat-trunk/output/build/lib/catalina-tribes.jar:/srv/gump/public/workspace/tomcat-trunk/output/build/lib/catalina-ha.jar:/srv/gump/public/workspace/tomcat-trunk/output/build/lib/tomcat-api.jar:/srv/gump/public/workspace/tomcat-t