Re: [VOTE] Release Apache Tomcat 11.0.0-M25

[Rainer Jung]

Minor nit, irrelevant vor voting: it seems the previous (M24) changelog 
entry has no release date.


Am 09.09.24 um 16:32 schrieb Rainer Jung:

Am 05.09.24 um 15:08 schrieb Mark Thomas:

The proposed Apache Tomcat 11.0.0-M25 release is now available for
voting.

Apache Tomcat 11.0.0-M25 is a milestone release of the 11.0.x branch 
and has been made to provide users with early access to the new 
features in Apache Tomcat 11.0.x so that they may provide feedback. 
The notable changes compared to 11.0.0-M24 include:


- Implement the recent clarification from the Jakarta Servlet project
   that if a content length is declared then once that many bytes have
   been written to the response, further writes should trigger an
   IOException

- If an HTTP/2 client resets a stream before the request body is fully
   written, ensure that any ReadListener is notified via a call to
   ReadListener.onErrror()

- An Exception being thrown during WebSocket message processing (e.g. in
   a method annotated with @onMessage) should not automatically cause the
   connection to close. The application should handle the exception and
   make the decision whether or not to close the connection.

For full details, see the change log:
https://nightlies.apache.org/tomcat/tomcat-11.0.x/docs/changelog.html

Applications that run on Tomcat 9 and earlier will not run on Tomcat 
11 without changes. Java EE applications designed for Tomcat 9 and 
earlier may be placed in the $CATALINA_BASE/webapps-javaee directory 
and Tomcat will automatically convert them to Jakarta EE and copy them 
to the webapps directory. Applications using deprecated APIs may 
require further changes.


It can be obtained from:
https://dist.apache.org/repos/dist/dev/tomcat/tomcat-11/v11.0.0-M25/

The Maven staging repo is:
https://repository.apache.org/content/repositories/orgapachetomcat-1511

The tag is:
https://github.com/apache/tomcat/tree/11.0.0-M25
fafe3dc7a63c12fbb45aa076c90d6a9e7f77e5c8

The proposed 11.0.0-M25 release is:
[ ] -1 Broken - do not release
[X] +1 Beta   - go ahead and release as 11.0.0-M25


+1

Tested on platforms

- RHEL 6, 7, 8 and 9, SLES 11, 12 and 15

using

- JDK 11, 17, 21, 22, 23 (Release Candidate) and 24 (current EA)

from

- Eclipse Adoptium, Azul Zulu, Amazon Coretto, Oracle, RedHat and 
OpenJDK (for the RC and EA)


where applicable.

Also tested with

- tcnative 1.3.1, tcnative 2.0.8 and panama

based on

- OpenSSL 3.0.15, 3.1.7, 3.2.3 and 3.3.2.

All fine, except for the usual sporadic crashes with tcnative during 
shutdown and also the known bunch of test failures with JDK 24. Of 
course JDK 24 EA problems are not a showstopper in any way.


Thanks for RM!

Best regards,

Rainer


-
To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org
For additional commands, e-mail: dev-h...@tomcat.apache.org

Re: [VOTE] Release Apache Tomcat 11.0.0-M25

[Rainer Jung]

Am 05.09.24 um 15:08 schrieb Mark Thomas:

The proposed Apache Tomcat 11.0.0-M25 release is now available for
voting.

Apache Tomcat 11.0.0-M25 is a milestone release of the 11.0.x branch and 
has been made to provide users with early access to the new features in 
Apache Tomcat 11.0.x so that they may provide feedback. The notable 
changes compared to 11.0.0-M24 include:


- Implement the recent clarification from the Jakarta Servlet project
   that if a content length is declared then once that many bytes have
   been written to the response, further writes should trigger an
   IOException

- If an HTTP/2 client resets a stream before the request body is fully
   written, ensure that any ReadListener is notified via a call to
   ReadListener.onErrror()

- An Exception being thrown during WebSocket message processing (e.g. in
   a method annotated with @onMessage) should not automatically cause the
   connection to close. The application should handle the exception and
   make the decision whether or not to close the connection.

For full details, see the change log:
https://nightlies.apache.org/tomcat/tomcat-11.0.x/docs/changelog.html

Applications that run on Tomcat 9 and earlier will not run on Tomcat 11 
without changes. Java EE applications designed for Tomcat 9 and earlier 
may be placed in the $CATALINA_BASE/webapps-javaee directory and Tomcat 
will automatically convert them to Jakarta EE and copy them to the 
webapps directory. Applications using deprecated APIs may require 
further changes.


It can be obtained from:
https://dist.apache.org/repos/dist/dev/tomcat/tomcat-11/v11.0.0-M25/

The Maven staging repo is:
https://repository.apache.org/content/repositories/orgapachetomcat-1511

The tag is:
https://github.com/apache/tomcat/tree/11.0.0-M25
fafe3dc7a63c12fbb45aa076c90d6a9e7f77e5c8

The proposed 11.0.0-M25 release is:
[ ] -1 Broken - do not release
[X] +1 Beta   - go ahead and release as 11.0.0-M25


+1

Tested on platforms

- RHEL 6, 7, 8 and 9, SLES 11, 12 and 15

using

- JDK 11, 17, 21, 22, 23 (Release Candidate) and 24 (current EA)

from

- Eclipse Adoptium, Azul Zulu, Amazon Coretto, Oracle, RedHat and 
OpenJDK (for the RC and EA)


where applicable.

Also tested with

- tcnative 1.3.1, tcnative 2.0.8 and panama

based on

- OpenSSL 3.0.15, 3.1.7, 3.2.3 and 3.3.2.

All fine, except for the usual sporadic crashes with tcnative during 
shutdown and also the known bunch of test failures with JDK 24. Of 
course JDK 24 EA problems are not a showstopper in any way.


Thanks for RM!

Best regards,

Rainer

-
To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org
For additional commands, e-mail: dev-h...@tomcat.apache.org

Re: Create a Tomcat 12 branch?

[Rainer Jung]

Am 12.08.24 um 20:30 schrieb Mark Thomas:

All,

As I mentioned earlier, I am starting work on some new EL API features 
that will be part of Jakarta EE 12 so implemented in Tomcat 12.


How do we want to handle this?

My current thinking is:

- create a 11.0.x branch from current main
- main becomes 12.0.x


Probably the least surprising solution compared to how TC usually 
develops. Main is for the highest major version.


I'm not expecting releases to start from 12.0.x any time soon. Users 
that want to experiment with the new features can use snapshots. If we 
reach a point where snapshots would be useful we can re-assess.


The other alternative I thought of was creating my own 12.0.x branch in 
my fork but that seems wrong.


Agreed.

Yes, it means a little more back-porting but (for me at least) that is 
all scripted and I'm not expecting many conflicts between 12.0.x and 
11.0.x.


Sounds good.


I'd update the CI systems to build 12.0.x.

Part of me thinks it is a little odd to be thinking about 12.0.x before 
11.0.x is stable but on reflection I think it is a good thing that there 
is momentum already for new features in Jakarta EE 12.


Agreed.


Oh, there is also a small change to the Servlet API we could pick up.

Thoughts?


+1 for 11.0.x branch plus main for 12.0.x.

Best regards,

Rainer

-
To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org
For additional commands, e-mail: dev-h...@tomcat.apache.org

Re: [VOTE] Release Apache Tomcat Connectors (JK) 1.2.50

[Rainer Jung]

Am 08.08.24 um 16:13 schrieb Mark Thomas:

Tag:
https://github.com/apache/tomcat-connectors/tree/JK_1_2_50

Source:
https://github.com/apache/tomcat-connectors/tree/JK_1_2_50

Dist:
https://dist.apache.org/repos/dist/dev/tomcat/tomcat-connectors/jk/


This is a maintenance release with a handful of dependency updates and 
bug fixes (compared to 1.2.49). It also includes Windows binaries for IIS.


The significant changes are:
- Improve shared memory handling on non-Windows

The proposed JK 1.2.50 release is:
[ ] Broken - do not release
[X] Stable - go ahead and release as 1.2.50


+1 for release.

Thanks, especially for RM!

Rainer

-
To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org
For additional commands, e-mail: dev-h...@tomcat.apache.org

Re: [VOTE] Release Apache Tomcat Connectors (JK) 1.2.50

[Rainer Jung]

Am 08.08.24 um 16:13 schrieb Mark Thomas:

Tag:
https://github.com/apache/tomcat-connectors/tree/JK_1_2_50

Source:
https://github.com/apache/tomcat-connectors/tree/JK_1_2_50

Dist:
https://dist.apache.org/repos/dist/dev/tomcat/tomcat-connectors/jk/


This is a maintenance release with a handful of dependency updates and 
bug fixes (compared to 1.2.49). It also includes Windows binaries for IIS.


The significant changes are:
- Improve shared memory handling on non-Windows

The proposed JK 1.2.50 release is:
[ ] Broken - do not release
[ ] Stable - go ahead and release as 1.2.50


I still need to do some more checks but currently looks good except for 
the binary zip files: for 1.2.49 they contained LICENSE and NOTICE, for 
1.2.50 these are missing from the binaries zip files. Maybe there is an 
easy way to add them and redo the sha512 and gpg signature.


Thanks so far!

Rainer

-
To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org
For additional commands, e-mail: dev-h...@tomcat.apache.org

Re: [VOTE] Release Apache Tomcat 10.1.28

[Rainer Jung]

Am 02.08.24 um 19:27 schrieb Christopher Schultz:

The proposed Apache Tomcat 10.1.28 release is now available for
voting.

All committers and PMC members are kindly requested to provide a vote if 
possible. ANY TOMCAT USER MAY VOTE, though only PMC members votes are 
binding. We welcome non-committer votes or comments on release builds.


Note that release 10.1.27 was cancelled due to a regression to HTTP/2 
handling. That regression has been fixed in this release along with 
these additional items:


The notable changes compared to 10.1.26 are:

- Add support for RFC 8297 (Early Hints). Applications can use this
   feature by casting the HttpServletResponse to
   org.apache.catalina.connector.Reponse and then calling the method
   void sendEarlyHints()

- Align HTTP/2 with HTTP/1.1 and recycle the container internal request
   and response processing objects by default. This behaviour can be
   controlled via the new discardRequestsAndResponses attribute on the
   HTTP/2 upgrade protocol.

- Ensure statements returned from Statement methods executeQuery(),
   getResultSet() and getGeneratedKeys() are correctly wrapped before
   being returned to the caller.

For full details, see the change log:
https://nightlies.apache.org/tomcat/tomcat-10.1.x/docs/changelog.html

Applications that run on Tomcat 9 and earlier will not run on Tomcat 10 
without changes. Java EE applications designed for Tomcat 9 and earlier 
may be placed in the $CATALINA_BASE/webapps-javaee directory and Tomcat 
will automatically convert them to Jakarta EE and copy them to the 
webapps directory.


It can be obtained from:
https://dist.apache.org/repos/dist/dev/tomcat/tomcat-10/v10.1.28/

The Maven staging repo is:
https://repository.apache.org/content/repositories/orgapachetomcat-1508

The tag is:
https://github.com/apache/tomcat/tree/10.1.28
https://github.com/apache/tomcat/commit/ 
aae1e30f78ba5ace25848084a500662ecff0b75f


Please reply with a +1 for release or +0/-0/-1 with an explanation.


+1

Tested on platforms

- RHEL 6, 7, 8 and 9, SLES 11, 12 and 15 and Solaris 11 Sparc

using

- JDK 11, 17, 21, 22, 23(EA) and 24(EA)

from

- Eclipse Adoptium, Azul Zulu, Amazon Coretto, Oracle, RedHat and 
OpenJDK (for the EAs)


where applicable.

Also tested with

- tcnative 1.3.1, tcnative 2.0.8 and panama

based on

- OpenSSL 3.0.14, 3.1.6, 3.2.2 and 3.3.1.

All fine, except for the usual sporadic crashes with tcnative during 
shutdown and also the bunch of test failures with JDK 24. They started a 
couple of months ago as noted by me and others and happen for all TC 
branches. For details see my vote mail for TC 11.0.0-M24. Of course JDK 
24 EA problems are not a showstopper in any way.


Thanks for RM!

Best regards,

Rainer

-
To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org
For additional commands, e-mail: dev-h...@tomcat.apache.org

Re: [VOTE] Release Apache Tomcat 11.0.0-M24

[Rainer Jung]

Am 05.08.24 um 20:18 schrieb Coty Sutherland:

I'm testing and see an issue with o.a.c.http2.TestStreamProcessor getting
some unexpected result:

Testcase: testPrepareHeaders[0: loop [0], useAsyncIO[false]] took 0.061 sec
 FAILED
expected:<...-Header-[etag]-[W/"9[34]-1447269522000"]
3-H...> but was:<...-Header-[etag]-[W/"9[57]-1447269522000"]
3-H...>
junit.framework.AssertionFailedError:
expected:<...-Header-[etag]-[W/"9[34]-1447269522000"]
3-H...> but was:<...-Header-[etag]-[W/"9[57]-1447269522000"]
3-H...>
 at
org.apache.coyote.http2.TestStreamProcessor.testPrepareHeaders(TestStreamProcessor.java:167)
 at
java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native
Method)
 at
java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:77)
 at
java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)


The test class contains:

// Different line-endings -> different files size -> different 
weak eTag

if (JrePlatform.IS_WINDOWS) {
expected.append("3-Header-[etag]-[W/\"957-1447269522000\"]\n");
} else {
expected.append("3-Header-[etag]-[W/\"934-1447269522000\"]\n");
}

The number behind the "W" (weak" in the etag header ist the file size, 
the second (correct) number the timestamp. It seems you get the 957 size 
exoected for Windows line endings instead of the 934 for Unix line 
endings. You could check the file test/webapp/index.html for size/line 
endings. Maybe your git settings lead to a DOS checkout instead of a 
unix one?



same output for:

Testcase: testPrepareHeaders[1: loop [0], useAsyncIO[true]] took 0.036 sec

This happens for all three branches. I ran the unit tests for using Fedora
40 with OpenJDK 17 (java-17-openjdk-17.0.11.0.9-1.fc39.x86_64) and 22
(java-22-openjdk-22.0.2.0.9-1.rolling.fc39.x86_64).

I'm also getting a failure in o.a.jasper.runtime.TestJspRuntimeLibrary.


What kind of failure?

Best regards,

Rainer


Any thoughts or know what might be up off the top of anyone's head? Given
that this seems fine for everyone else, I'm comfortable still giving a +1
and chalking it up to something odd in my environment.

On Fri, Aug 2, 2024 at 10:15 AM Mark Thomas  wrote:


The proposed Apache Tomcat 11.0.0-M24 release is now available for
voting.

Apache Tomcat 11.0.0-M24 is a milestone release of the 11.0.x branch and
has been made to provide users with early access to the new features in
Apache Tomcat 11.0.x so that they may provide feedback. The notable
changes compared to 11.0.0-M22 include:

- Align HTTP/2 with HTTP/1.1 and recycle the container internal request
and response processing objects by default. This behaviour can be
controlled via the new discardRequestsAndResponses attribute on the
HTTP/2 upgrade protocol.

- Add FFM compatibility methods for LibreSSL and BoringSSL support.

- Add support for RFC 8297 (Early Hints). Applications can use this
feature by casting the HttpServletResponse to
org.apache.catalina.connector.Reponse and then calling the method
void sendEarlyHints()

For full details, see the change log:
https://nightlies.apache.org/tomcat/tomcat-11.0.x/docs/changelog.html

Applications that run on Tomcat 9 and earlier will not run on Tomcat 11
without changes. Java EE applications designed for Tomcat 9 and earlier
may be placed in the $CATALINA_BASE/webapps-javaee directory and Tomcat
will automatically convert them to Jakarta EE and copy them to the
webapps directory. Applications using deprecated APIs may require
further changes.

It can be obtained from:
https://dist.apache.org/repos/dist/dev/tomcat/tomcat-11/v11.0.0-M24/

The Maven staging repo is:
https://repository.apache.org/content/repositories/orgapachetomcat-1507

The tag is:
https://github.com/apache/tomcat/tree/11.0.0-M24
5301df36454fcf22081108e25199f29904cadc79

The proposed 11.0.0-M24 release is:
[ ] -1 Broken - do not release
[ ] +1 Beta   - go ahead and release as 11.0.0-M24



-
To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org
For additional commands, e-mail: dev-h...@tomcat.apache.org

Re: [VOTE] Release Apache Tomcat 9.0.93

[Rainer Jung]

Am 03.08.24 um 01:02 schrieb Rémy Maucherat:

The proposed Apache Tomcat 9.0.93 release is now available for voting.

The notable changes compared to 9.0.91 are:

- Align HTTP/2 with HTTP/1.1 and recycle the container internal request
and response processing objects by default. This behaviour can be
controlled via the new discardRequestsAndResponses attribute on the
HTTP/2 upgrade protocol.

- Add OpenSSL support for FFM. Using this feature requires Java 22
or newer.

- Add support for RFC 8297 (Early Hints). Applications can use this
feature by casting the HttpServletResponse to
org.apache.catalina.connector.Reponse and then calling the method
void sendEarlyHints()

For full details, see the changelog:
https://nightlies.apache.org/tomcat/tomcat-9.0.x/docs/changelog.html

It can be obtained from:
https://dist.apache.org/repos/dist/dev/tomcat/tomcat-9/v9.0.93/

The Maven staging repo is:
https://repository.apache.org/content/repositories/orgapachetomcat-1510

The tag is:
https://github.com/apache/tomcat/tree/9.0.93
a33d708d9b078e0d7bc8abda91c8634c4f338d99

The proposed 9.0.93 release is:
[ ] -1, Broken - do not release
[ ] +1, Stable - go ahead and release as 9.0.93

Rémy


+1

Tested on platforms

- RHEL 6, 7, 8 and 9, SLES 11, 12 and 15 and Solaris 10+11 Sparc

using

- JDK 8, 11, 17, 21, 22, 23(EA) and 24(EA)

from

- Eclipse Adoptium, Azul Zulu, Amazon Coretto, Oracle, RedHat and 
OpenJDK (for the EAs)


where applicable.

Also tested with

- tcnative 1.3.1, tcnative 2.0.8 and panama

based on

- OpenSSL 3.0.14, 3.1.6, 3.2.2 and 3.3.1.

All fine, except for the usual sporadic crashes with tcnative during 
shutdown and also the bunch of test failures with JDK 24. They started a 
couple of months ago as noted by me and others and happen for all TC 
branches. For details see my vote mail for TC 11.0.0-M24. Of course JDK 
24 EA problems are not a showstopper in any way.


Thanks for RM!

Best regards,

Rainer

-
To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org
For additional commands, e-mail: dev-h...@tomcat.apache.org

Re: [VOTE] Release Apache Tomcat 11.0.0-M24

[Rainer Jung]

Am 02.08.24 um 16:15 schrieb Mark Thomas:

The proposed Apache Tomcat 11.0.0-M24 release is now available for
voting.

Apache Tomcat 11.0.0-M24 is a milestone release of the 11.0.x branch and 
has been made to provide users with early access to the new features in 
Apache Tomcat 11.0.x so that they may provide feedback. The notable 
changes compared to 11.0.0-M22 include:


- Align HTTP/2 with HTTP/1.1 and recycle the container internal request
   and response processing objects by default. This behaviour can be
   controlled via the new discardRequestsAndResponses attribute on the
   HTTP/2 upgrade protocol.

- Add FFM compatibility methods for LibreSSL and BoringSSL support.

- Add support for RFC 8297 (Early Hints). Applications can use this
   feature by casting the HttpServletResponse to
   org.apache.catalina.connector.Reponse and then calling the method
   void sendEarlyHints()

For full details, see the change log:
https://nightlies.apache.org/tomcat/tomcat-11.0.x/docs/changelog.html

Applications that run on Tomcat 9 and earlier will not run on Tomcat 11 
without changes. Java EE applications designed for Tomcat 9 and earlier 
may be placed in the $CATALINA_BASE/webapps-javaee directory and Tomcat 
will automatically convert them to Jakarta EE and copy them to the 
webapps directory. Applications using deprecated APIs may require 
further changes.


It can be obtained from:
https://dist.apache.org/repos/dist/dev/tomcat/tomcat-11/v11.0.0-M24/

The Maven staging repo is:
https://repository.apache.org/content/repositories/orgapachetomcat-1507

The tag is:
https://github.com/apache/tomcat/tree/11.0.0-M24
5301df36454fcf22081108e25199f29904cadc79

The proposed 11.0.0-M24 release is:
[ ] -1 Broken - do not release
[ ] +1 Beta   - go ahead and release as 11.0.0-M24


+1

Tested on platforms

- RHEL 6, 7, 8 and 9 and SLES 11, 12 and 15

using

- JDK 17, 21, 22, 23(EA) and 24(EA)

from

- Eclipse Adoptium, Azul Zulu, Amazon Coretto, Oracle, RedHat and 
OpenJDK (for the EAs)


where applicable.

Also tested with

- tcnative 1.3.1, tcnative 2.0.8 and panama

based on

- OpenSSL 3.0.14, 3.1.6, 3.2.2 and 3.3.1.

All fine, except for the usual sporadic crashes with tcnative during 
shutdown and also the bunch of test failures with JDK 24. Example for a 
JDK 24 test failure:


- org.apache.coyote.TestRequest

Tests run: 4, Failures: 0, Errors: 4, Skipped: 0, Time elapsed: 4.226 sec

Testcase: testNoExpectationWithOnRequestBodyReadPolicy took 2.621 sec
Caused an ERROR
Could not create type
java.lang.IllegalArgumentException: Could not create type
at net.bytebuddy.TypeCache.findOrInsert(TypeCache.java:170)
at 
net.bytebuddy.TypeCache$WithInlineExpunction.findOrInsert(TypeCache.java:399)
at 
org.easymock.internal.ClassProxyFactory.createProxy(ClassProxyFactory.java:151)
at 
org.easymock.internal.MocksControl.createMock(MocksControl.java:110)
at 
org.easymock.internal.MocksControl.createMock(MocksControl.java:83)

at org.easymock.IMocksControl.mock(IMocksControl.java:44)
at org.easymock.EasyMock.niceMock(EasyMock.java:187)
at org.easymock.EasyMock.createNiceMock(EasyMock.java:361)
at org.apache.coyote.TestRequest.(TestRequest.java:39)
at 
java.base/jdk.internal.reflect.DirectConstructorHandleAccessor.newInstance(DirectConstructorHandleAccessor.java:62)
at 
java.base/java.lang.reflect.Constructor.newInstanceWithCaller(Constructor.java:501)
at 
java.base/java.lang.reflect.Constructor.newInstance(Constructor.java:485)
Caused by: java.lang.IllegalArgumentException: 
org.apache.tomcat.util.net.SocketWrapperBase$$$EasyMock$1 must be 
defined in the same package as org.easymock.internal.ClassProxyFactory
at 
net.bytebuddy.dynamic.loading.ClassInjector$UsingLookup.injectRaw(ClassInjector.java:1635)
at 
net.bytebuddy.dynamic.loading.ClassInjector$AbstractBase.inject(ClassInjector.java:118)
at 
net.bytebuddy.dynamic.loading.ClassLoadingStrategy$UsingLookup.load(ClassLoadingStrategy.java:519)
at 
net.bytebuddy.dynamic.TypeResolutionStrategy$Passive.initialize(TypeResolutionStrategy.java:101)
at 
net.bytebuddy.dynamic.DynamicType$Default$Unloaded.load(DynamicType.java:6325)
at 
org.easymock.internal.ClassProxyFactory.lambda$createProxy$0(ClassProxyFactory.java:161)

at net.bytebuddy.TypeCache.findOrInsert(TypeCache.java:168)
...

The failures happen for NIO and NIO2, only for JDK 24 EA 8, for those 
test classes:


- org.apache.catalina.connector.TestSendFile
- org.apache.catalina.realm.TestJNDIRealm
- org.apache.catalina.session.TestPersistentManager
- org.apache.catalina.startup.TestWebappServiceLoader
- org.apache.catalina.valves.TestCrawlerSessionManagerValve
- org.apache.catalina.valves.TestLoadBalancerDrainingValve
- org.apache.catalina.valves.TestSSLValve
- org.apache.coyote.TestRequest
- org.apache.jasper.servlet.TestTldScanner

They started a couple of months ag

Re: Sporadic http2 test failures

[Rainer Jung]

Am 02.08.24 um 10:58 schrieb Mark Thomas:

On 02/08/2024 07:58, Mark Thomas wrote:

On 01/08/2024 23:48, Rainer Jung wrote:



I did not check each occurrence but here are examples which all also 
have a NullPointer in the access log. I don't know whether that 
triggers the failure or is just another symptom triggered by the same 
root cause.




Rainer,

Thanks for the work you have put into this.

The combination of NPE and the timing of these emerging makes me think 
the change to recycle the request and response may be the cause.


I'm holding off closing the 11.0.x vote while I look into this.


I can confirm the request/response recycling for HTTP/2 triggered this. 
Reviewing the code with fresh eyes I see multiple issues. I'll have 
fixes ready shortly but I am going to cancel the 11.0.0-M23 release 
vote. I am also going to change my vote to -1 for 10.1.27 and 9.0.92.


Thanks for investigating so quickly and thoroughly!

Unfortunately my testing zoo always needs a few days before test results 
become reliable enough, so it is hard for me to report very quickly 
after the start of the vote.


Best regards,

Rainer


-
To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org
For additional commands, e-mail: dev-h...@tomcat.apache.org

Sporadic http2 test failures

[Rainer Jung]

Hi there,

I get sporadic http2 test failures when running the many 
JVM/connector/platform combinations against the current tags of TC 9.0, 
10.1 and 11.


It happens most frequently for TC 9, but I looks like it also applies to 
TC 10.1 and 11 but much more rarely.


Here's a list of the combinations, where the test run produced an error:

9.0
- org.apache.coyote.http2.TestHttp2Section_8_1 FAILED
  adopt_jdk1.8.0-rhel6.x86_64-jsse
  amazon_jdk1.8.0-rhel7.x86_64-jsse
  oracle_jdk1.8.0-rhel9.x86_64-jsse
  rh_jdk1.8.0-rhel8.x86_64-jsse
  adopt_jdk11-rhel6.x86_64-jsse
  oracle_jdk11-sles15.x86_64-jsse

- org.apache.coyote.http2.TestStreamProcessor FAILED
  adopt_jdk1.8.0-rhel7.x86_64-jsse
  amazon_jdk1.8.0-rhel6.x86_64-jsse
  amazon_jdk1.8.0-sles12.x86_64-jsse
  oracle_jdk1.8.0-rhel6.x86_64-jsse
  zulu_jdk1.8.0-rhel6.x86_64-jsse
  zulu_jdk1.8.0-rhel7.x86_64-jsse
  zulu_jdk1.8.0-sles12.x86_64-jsse
  adopt_jdk11-sles15.x86_64-jsse
  amazon_jdk11-rhel9.x86_64-jsse
  oracle_jdk11-sles12.x86_64-jsse
  rh_jdk11-rhel9.x86_64-jsse
  zulu_jdk11-rhel9.x86_64-jsse
10.1
- org.apache.coyote.http2.TestStreamProcessor FAILED
  zulu_jdk11-sles11.x86_64-jsse
  zulu_jdk11-sles15.x86_64-jsse
  adopt_jdk17-sles11.x86_64-jsse
  oracle_jdk21-rhel6.x86_64-jsse
11.0
- org.apache.coyote.http2.TestHttp2Section_8_1 FAILED
  jdk23-rhel7.x86_64-jsse

So it seems it can happen for each of the platforms and JVM versions.

I did not check each occurrence but here are examples which all also 
have a NullPointer in the access log. I don't know whether that triggers 
the failure or is just another symptom triggered by the same root cause.


A) org.apache.coyote.http2.TestStreamProcessor
==

  - jsse_amazon_jdk1.8.0-sles12.x86_64 NIO2

Testcase: testValidateRequestURI[0: loop [0], useAsyncIO[false]] took 
0.169 sec

FAILED
expected:<...3-Header-[:status]-[[4]00]
3-Header-[date]-...> but was:<...3-Header-[:status]-[[5]00]
3-Header-[date]-...>
junit.framework.AssertionFailedError: 
expected:<...3-Header-[:status]-[[4]00]

3-Header-[date]-...> but was:<...3-Header-[:status]-[[5]00]
3-Header-[date]-...>
at 
org.apache.coyote.http2.TestStreamProcessor.testValidateRequestURI(TestStreamProcessor.java:347)


...

Testcase: testValidateRequestMethod[0: loop [0], useAsyncIO[false]] took 
0.221 sec

FAILED
expected:<...3-Header-[:status]-[[4]00]
3-Header-[date]-...> but was:<...3-Header-[:status]-[[5]00]
3-Header-[date]-...>
junit.framework.AssertionFailedError: 
expected:<...3-Header-[:status]-[[4]00]

3-Header-[date]-...> but was:<...3-Header-[:status]-[[5]00]
3-Header-[date]-...>
at 
org.apache.coyote.http2.TestStreamProcessor.testValidateRequestMethod(TestStreamProcessor.java:258)


and:

30-Jul-2024 19:33:57.149 INFO [main] 
org.apache.coyote.AbstractProtocol.start Starting ProtocolHandler 
["http-nio2-127.0.0.1-auto-8-37726"]
30-Jul-2024 19:33:57.168 WARNING [http-nio2-127.0.0.1-auto-8-exec-6] 
org.apache.catalina.connector.CoyoteAdapter.log Exception while 
attempting to add an entry to the access log

java.lang.NullPointerException
at 
org.apache.catalina.valves.AbstractAccessLogValve$ByteSentElement.addElement(AbstractAccessLogValve.java:1268)
at 
org.apache.catalina.valves.AbstractAccessLogValve.log(AbstractAccessLogValve.java:686)
at 
org.apache.catalina.core.AccessLogAdapter.log(AccessLogAdapter.java:48)
at 
org.apache.catalina.core.StandardEngine.logAccess(StandardEngine.java:271)
at 
org.apache.catalina.connector.CoyoteAdapter.log(CoyoteAdapter.java:489)
at 
org.apache.coyote.http2.StreamProcessor.service(StreamProcessor.java:443)
at 
org.apache.coyote.AbstractProcessorLight.process(AbstractProcessorLight.java:63)
at 
org.apache.coyote.http2.StreamProcessor.process(StreamProcessor.java:92)
at 
org.apache.coyote.http2.StreamRunnable.run(StreamRunnable.java:35)
at 
org.apache.tomcat.util.threads.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1190)
at 
org.apache.tomcat.util.threads.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:659)
at 
org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:63)

at java.lang.Thread.run(Thread.java:750)
30-Jul-2024 19:33:57.172 INFO [main] 
org.apache.coyote.AbstractProtocol.pause Pausing ProtocolHandler 
["http-nio2-127.0.0.1-auto-8-37726"]

...
30-Jul-2024 19:33:57.795 INFO [main] 
org.apache.coyote.AbstractProtocol.start Starting ProtocolHandler 
["http-nio2-127.0.0.1-auto-12-33381"]
30-Jul-2024 19:33:57.820 WARNING [http-nio2-127.0.0.1-auto-12-exec-5] 
org.apache.catalina.connector.CoyoteAdapter.log Exception while 
attempting to add an entry to the access log

java.lang.NullPointerEx

Re: Short and long term plans for Tomcat Native

[Rainer Jung]

Am 18.07.24 um 16:59 schrieb Rémy Maucherat:

On Thu, Jul 18, 2024 at 2:41 PM Rainer Jung  wrote:


Am 18.07.24 um 14:15 schrieb Michael Osipov:

On 2024/07/17 15:33:02 Mark Thomas wrote:

All,

I've spent some time today trying to build Tomcat Native with Visual
Studio 2022 rather then the current, more involved process without success.

Therefore, my short-term plan for Tomcat Native is to get the next 2.0.x
and 1.3.x releases completed using the existing build process. I'll be
starting that shortly.

Longer term, thinking about Tomcat Native and FFM I have the following
rough plan in mind:

- stop providing x86 binaries for Windows
- require Windows 10 onwards
- provide:
 - libssl.dll
 - libcrypto.dll
 - tomcat-native-2.dll
 - openssl.exe

along with appropriate debug symbols.


This sounds reasonable...but did you rename tcnative on purpose?


I'm not sure if we'd need an apr.dll in there as well.

The idea is that users can then use either the AprLifecycleListener or
OpenSSLLifecycleListener.

This would probably require moving Tomcat Native to 2.1.x and 1.4.x

This really needs someone more familiar with building C libraries
generally and these libraries in particular - i.e. Mladen - to say
whether the above is possible and to provide some pointers on how to do it.


I would prefer that all APR-related stuff in the build is also removed in 
2.1.x. Why does one need it if it is not used? That is weird.


I'll try to clarify a few things: we have a few different cases where
native libraries are used. Please correct me where I am wrong:

- the APR connector; only supported in TC 9.0. Implemented via tcnative
1.3. Needs code from tcnative and the APR libraries.

- doing OpenSSL crypto instead of JSSE crypto in a connector using
tcnative. supported in TC 9+; Implemented via tcnative 1.3 and 2.0.
Needs code from tcnative, the APR library and the OpenSSL library. The
APR library is needed, because tcnative uses the memory pool
implementation of APR for its own memory management.

- doing OpenSSL crypto instead of JSSE crypto in a connector using the
new Java 22+ foreign function interface. Supported in TC 10.1+;
Implemented via FFM plus OpenSSL library. Only needs code from the
OpenSSL library.


I was able to port the FFM support to Tomcat 9.0 (will be in 9.0.92)
with very little actual trouble, along with all the improvements in
TLS testing from 10.1. As suggested by someone (can't remember who,
maybe you ?), Ant's javac task allows specifying another javac
executable and it worked well to solve the blocker issue about the
Java 8 release target.
This really improves the long term support prospects for 9.0 as far as
I am concerned.


That's ultra cool!


Rémy



Concerning the existence of explicit separate libapr, libcrypto and
libssl dll files in the tcnative binary distribution for Windows: I
expect one can decide during copilation, which of these libraries will
be kept separate as dynmic library files and linked in dynamically and
which of them get linked in statically, so that no separate file needs
to be shipped.

It seems until 2.0.7 (8?) and 1.3.0 (1?) we link in the apr, crypto and
ssl libs statically, so no separate dll files are shipped.

Marks suggestion indicates that he suggests to link the two OpenSSL libs
dynamically and ship the crypto and ssl libs them bundled but keep the
apr lib linked statically, so no separate dll file for it.

Best regards,

Rainer


-
To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org
For additional commands, e-mail: dev-h...@tomcat.apache.org

Re: Short and long term plans for Tomcat Native

[Rainer Jung]

Am 18.07.24 um 14:15 schrieb Michael Osipov:

On 2024/07/17 15:33:02 Mark Thomas wrote:

All,

I've spent some time today trying to build Tomcat Native with Visual
Studio 2022 rather then the current, more involved process without success.

Therefore, my short-term plan for Tomcat Native is to get the next 2.0.x
and 1.3.x releases completed using the existing build process. I'll be
starting that shortly.

Longer term, thinking about Tomcat Native and FFM I have the following
rough plan in mind:

- stop providing x86 binaries for Windows
- require Windows 10 onwards
- provide:
- libssl.dll
- libcrypto.dll
- tomcat-native-2.dll
- openssl.exe

along with appropriate debug symbols.


This sounds reasonable...but did you rename tcnative on purpose?


I'm not sure if we'd need an apr.dll in there as well.

The idea is that users can then use either the AprLifecycleListener or
OpenSSLLifecycleListener.

This would probably require moving Tomcat Native to 2.1.x and 1.4.x

This really needs someone more familiar with building C libraries
generally and these libraries in particular - i.e. Mladen - to say
whether the above is possible and to provide some pointers on how to do it.


I would prefer that all APR-related stuff in the build is also removed in 
2.1.x. Why does one need it if it is not used? That is weird.


I'll try to clarify a few things: we have a few different cases where 
native libraries are used. Please correct me where I am wrong:


- the APR connector; only supported in TC 9.0. Implemented via tcnative 
1.3. Needs code from tcnative and the APR libraries.


- doing OpenSSL crypto instead of JSSE crypto in a connector using 
tcnative. supported in TC 9+; Implemented via tcnative 1.3 and 2.0. 
Needs code from tcnative, the APR library and the OpenSSL library. The 
APR library is needed, because tcnative uses the memory pool 
implementation of APR for its own memory management.


- doing OpenSSL crypto instead of JSSE crypto in a connector using the 
new Java 22+ foreign function interface. Supported in TC 10.1+; 
Implemented via FFM plus OpenSSL library. Only needs code from the 
OpenSSL library.


Concerning the existence of explicit separate libapr, libcrypto and 
libssl dll files in the tcnative binary distribution for Windows: I 
expect one can decide during copilation, which of these libraries will 
be kept separate as dynmic library files and linked in dynamically and 
which of them get linked in statically, so that no separate file needs 
to be shipped.


It seems until 2.0.7 (8?) and 1.3.0 (1?) we link in the apr, crypto and 
ssl libs statically, so no separate dll files are shipped.


Marks suggestion indicates that he suggests to link the two OpenSSL libs 
dynamically and ship the crypto and ssl libs them bundled but keep the 
apr lib linked statically, so no separate dll file for it.


Best regards,

Rainer

-
To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org
For additional commands, e-mail: dev-h...@tomcat.apache.org

Re: [VOTE] Release Apache Tomcat 10.1.26

[Rainer Jung]

Am 08.07.24 um 00:07 schrieb Christopher Schultz:

The proposed Apache Tomcat 10.1.26 release is now available for
voting.

All committers and PMC members are kindly requested to provide a vote if 
possible. ANY TOMCAT USER MAY VOTE, though only PMC members votes are 
binding. We welcome non-committer votes or comments on release builds.


The notable changes compared to 10.1.25 are:

- Move OpenSSL support using FFM to a separate JAR named
   tomcat-coyote-ffm.jar that advertises Java 22 in its manifest.

- When using include directives in a tag file packaged in a JAR file,
   ensure that the include directives are processed correctly.

- Expand the implementation of the filter value of the Authenticator
   attribute allowCorsPreflight, so that it applies to all requests that
   match the configured URL patterns for the CORS filter, rather than
   only applying if the CORS filter is mapped to /*

For full details, see the change log:
https://nightlies.apache.org/tomcat/tomcat-10.1.x/docs/changelog.html

Applications that run on Tomcat 9 and earlier will not run on Tomcat 10 
without changes. Java EE applications designed for Tomcat 9 and earlier 
may be placed in the $CATALINA_BASE/webapps-javaee directory and Tomcat 
will automatically convert them to Jakarta EE and copy them to the 
webapps directory.


It can be obtained from:
https://dist.apache.org/repos/dist/dev/tomcat/tomcat-10/v10.1.26/

The Maven staging repo is:
https://repository.apache.org/content/repositories/orgapachetomcat-1502

The tag is:
https://github.com/apache/tomcat/tree/10.1.26
https://github.com/apache/tomcat/commit/43731ff263f74ec9949a3f535fd9254baa932603

Please reply with a +1 for release or +0/-0/-1 with an explanation.


+1, tested with my usual zoo of JVMs and platforms.

Thanks for RM!

Rainer

-
To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org
For additional commands, e-mail: dev-h...@tomcat.apache.org

Unit test failures for 11.0.0-M22 on JDK 24 EA 4

[Rainer Jung]

Hi all,

I get various unit test failures for TC 11.0.0-M22 when running on the 
current JDK 24 (EA 4). As an example:


Testsuite: org.apache.catalina.session.TestPersistentManager
Tests run: 2, Failures: 0, Errors: 1, Skipped: 0, Time elapsed: 6.308 sec

Testcase: testBug62175 took 4.733 sec
Caused an ERROR
Could not create type
java.lang.IllegalArgumentException: Could not create type
at net.bytebuddy.TypeCache.findOrInsert(TypeCache.java:170)
	at 
net.bytebuddy.TypeCache$WithInlineExpunction.findOrInsert(TypeCache.java:399)
	at 
org.easymock.internal.ClassProxyFactory.createProxy(ClassProxyFactory.java:151)

at org.easymock.internal.MocksControl.createMock(MocksControl.java:110)
at org.easymock.internal.MocksControl.createMock(MocksControl.java:83)
at org.easymock.IMocksControl.mock(IMocksControl.java:44)
at org.easymock.EasyMock.niceMock(EasyMock.java:187)
at org.easymock.EasyMock.createNiceMock(EasyMock.java:361)
	at 
org.apache.catalina.session.TestPersistentManager.testBug62175(TestPersistentManager.java:110)
	at 
java.base/jdk.internal.reflect.DirectMethodHandleAccessor.invoke(DirectMethodHandleAccessor.java:103)
Caused by: java.lang.IllegalArgumentException: 
org.apache.catalina.connector.Connector$$$EasyMock$1 must be defined in 
the same package as org.easymock.internal.ClassProxyFactory
	at 
net.bytebuddy.dynamic.loading.ClassInjector$UsingLookup.injectRaw(ClassInjector.java:1635)
	at 
net.bytebuddy.dynamic.loading.ClassInjector$AbstractBase.inject(ClassInjector.java:118)
	at 
net.bytebuddy.dynamic.loading.ClassLoadingStrategy$UsingLookup.load(ClassLoadingStrategy.java:519)
	at 
net.bytebuddy.dynamic.TypeResolutionStrategy$Passive.initialize(TypeResolutionStrategy.java:101)
	at 
net.bytebuddy.dynamic.DynamicType$Default$Unloaded.load(DynamicType.java:6325)
	at 
org.easymock.internal.ClassProxyFactory.lambda$createProxy$0(ClassProxyFactory.java:161)

at net.bytebuddy.TypeCache.findOrInsert(TypeCache.java:168)


Of course JDK 24 failures are not yet critical, just wanted to give an 
early heads up. I was not aware of such failures when testing the 
previous 11.0.0-M21 with JDK 24 EA 2.


Best regards,

Rainer

-
To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org
For additional commands, e-mail: dev-h...@tomcat.apache.org

Re: TestDataSourceUserDatabase and TestDataSourceRealm need Java 17

[Rainer Jung]

Am 20.06.24 um 12:53 schrieb Rémy Maucherat:

On Tue, Jun 18, 2024 at 12:26 PM Rainer Jung  wrote:


Am 18.06.24 um 10:37 schrieb Rémy Maucherat:

...

There are probably more clever ways to achieve this, but that's at least
what I use to be able to run the tests on older JVM versions.

Since I do not apply changes to the tests temselves, this means, that
tests assuming a newer JVM version than the minimal runtime version will
fail when I run them. Of course I can suppress them with test.exclude
(probably only the whole class, not just individual failing tests?). I
will do that, if the derby dependency was intentionally updated.

Thanks and regards,


Thanks a lot ! I tested and committed the changes to all three
branches. I think we're in a much better position now for properly
testing Tomcat 9 and 10.1 with older Java versions.


Wow, thanks! Much appreciated! And we can always improve it when a more 
elegant solution arises.


Thanks for also suppressing the derby-based tests on older JVMs.

Best regards,

Rainer

-
To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org
For additional commands, e-mail: dev-h...@tomcat.apache.org

Re: panama test TestOpenSSLConf.testOpenSSLConfCmdCipher() failing

[Rainer Jung]

Am 18.06.24 um 12:04 schrieb Rémy Maucherat:

On Tue, Jun 18, 2024 at 9:36 AM Rainer Jung  wrote:


Hi all,

when testing 11.0.0-M21 and 10.1.25 I observe new failures in panama:

Testcase:
testOpenSSLConfCmdCipher[org.apache.tomcat.util.net.openssl.panama.OpenSSLImplementation]
took 4.438 sec
  FAILED
Wrong HostConfig ciphers
Expected: is ["AES256-SHA256"]
   but: was ["TLS_AES_256_GCM_SHA384",
"TLS_CHACHA20_POLY1305_SHA256", "TLS_AES_128_GCM_SHA256", "AES256-SHA256"]
junit.framework.AssertionFailedError: Wrong HostConfig ciphers
Expected: is ["AES256-SHA256"]
   but: was ["TLS_AES_256_GCM_SHA384",
"TLS_CHACHA20_POLY1305_SHA256", "TLS_AES_128_GCM_SHA256", "AES256-SHA256"]
  at org.hamcrest.MatcherAssert.assertThat(MatcherAssert.java:20)
  at
org.apache.tomcat.util.net.openssl.TestOpenSSLConf.testOpenSSLConfCmdCipher(TestOpenSSLConf.java:132)
  at
java.base/jdk.internal.reflect.DirectMethodHandleAccessor.invoke(DirectMethodHandleAccessor.java:103)

The test was done with the recent releases of OpenSSL 3.0, 3.1, 3.2 and
3.3. All of them fail in the same way.


I'm using OpenSSL 3.2.1 and the test was not failing for me. However,
it was also not working.

There are two issues that left errors on the stack and making the
command check fail:
- Setting a bad value for the random seed (fixed)
- Then an error supposedly about use of the legacy provider somewhere
(no idea where this happens, it's now logged [error:1E08010C:DECODER
routines::unsupported])
Now the command check passes and I don't see any error processing the command.


Thanks for checking and improving. I will check the updated version and 
investigate deeper in case it still fails for me. I will also check, 
whether the test is new, or behaves diifferently for OpenSSL 3.2.1 (used 
by you and also by me for the previous release) and 3.2.2 (used by me 
now). But probably not before this evening.


No need to wait with closing the release votes, I guess panama is not 
yet a show-stopper for the vote.



Rémy


Best regards,

Rainer


-
To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org
For additional commands, e-mail: dev-h...@tomcat.apache.org

Re: TestDataSourceUserDatabase and TestDataSourceRealm need Java 17

[Rainer Jung]

Am 18.06.24 um 10:37 schrieb Rémy Maucherat:

On Tue, Jun 18, 2024 at 9:45 AM Rainer Jung  wrote:


Hi there,

the test classes org.apache.catalina.users.TestDataSourceUserDatabase
and org.apache.catalina.realm.TestDataSourceRealm have a filing test
which needs Java 17 due to the class version of
org/apache/derby/jdbc/EmbeddedDriver. For TC 10.1 this leads to failures
when testing with JDK 11-16.


More build dependencies now need Java 17 for the testsuite (bnd in
particular, and even the Ant from my Fedora ...). How do you work
around that to reach the test ?


Good point, thanks for asking:

- I provide (recent) ant myselve and add it to the PATH

- I first build everything using the release JDK version, so for 
instance Java 22 (just on one platform, currently RHEL 8).


- for each test platform and JVM version I use a copy of the resulting 
build and output tree and I adjust build.xml to be able to only run the 
tests:


  - I set an additional custom property skip.build.java.version=true in 
build.properties


  - The block "" ist changed from


  


to


  


  


  - the targets setup-bnd and add-osgi get an additional 
unless="skip.build.java.version" to supress them during the pure test run


  - the nio and nio2 test targets are duplicated and slightly adjusted
(dropping the dependencies test-compile and deploy).
Originally:

   

depends="setup-jacoco,test-compile,deploy,test-openssl-exists" 
if="${execute.test.nio}">

 
   

   

depends="setup-jacoco,test-compile,deploy,test-openssl-exists" 
if="${execute.test.nio2}">

 
   

  The added targets:

depends="setup-jacoco,test-openssl-exists" 
f="${execute.test.nio}">


  

depends="setup-jacoco,test-openssl-exists" 
f="${execute.test.nio2}">


  

  - finally the main test target alslo gets a slighty adjusted twin
(drop coverage-report, call the new "only" targets).
Originally

  

   The added target:

  


- call the new "test-only" target.

There are probably more clever ways to achieve this, but that's at least 
what I use to be able to run the tests on older JVM versions.


Since I do not apply changes to the tests temselves, this means, that 
tests assuming a newer JVM version than the minimal runtime version will 
fail when I run them. Of course I can suppress them with test.exclude 
(probably only the whole class, not just individual failing tests?). I 
will do that, if the derby dependency was intentionally updated.


Thanks and regards,

Rainer


Rémy


Details:

Testcase: testBasicUserRoleDatabase took 0.749 sec
  Caused an ERROR
org/apache/derby/jdbc/EmbeddedDriver has been compiled by a more recent
version of the Java Runtime (class file version 61.0), this version of
the Java Runtime only recognizes class file versions up to 55.0
java.lang.UnsupportedClassVersionError:
org/apache/derby/jdbc/EmbeddedDriver has been compiled by a more recent
version of the Java Runtime (class file version 61.0), this version of
the Java Runtime only recognizes class file versions up to 55.0
  at java.base/java.lang.ClassLoader.defineClass1(Native Method)
  at
java.base/java.lang.ClassLoader.defineClass(ClassLoader.java:1022)
  at
java.base/java.security.SecureClassLoader.defineClass(SecureClassLoader.java:174)
  at
java.base/jdk.internal.loader.BuiltinClassLoader.defineClass(BuiltinClassLoader.java:800)
  at
java.base/jdk.internal.loader.BuiltinClassLoader.findClassOnClassPathOrNull(BuiltinClassLoader.java:698)
  at
java.base/jdk.internal.loader.BuiltinClassLoader.loadClassOrNull(BuiltinClassLoader.java:621)
  at
java.base/jdk.internal.loader.BuiltinClassLoader.loadClass(BuiltinClassLoader.java:579)
  at
java.base/jdk.internal.loader.ClassLoaders$AppClassLoader.loadClass(ClassLoaders.java:178)
  at java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:527)
  at java.base/java.lang.Class.forName0(Native Method)
  at java.base/java.lang.Class.forName(Class.java:315)
  at
org.apache.catalina.users.TestDataSourceUserDatabase$DerbyUserDatabase.open(TestDataSourceUserDatabase.java:103)
  at
org.apache.catalina.users.TestDataSourceUserDatabase.testBasicUserRoleDatabase(TestDataSourceUserDatabase.java:131)
  at
java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native
Method)
  at
java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
  at
java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)

and

Testcase: testRealm took 1.298 sec
  Caused an ERROR
org/apache/derby/jdbc/EmbeddedDriver has been compiled by a more recent
version of the Java Runtime (class file version 

jakarta.el.TestImportHandlerStandardPackages fails for JDK 23

[Rainer Jung]

Hi all,

I observe test failures for jakarta.el.TestImportHandlerStandardPackages 
on JDK 23:


Testsuite: jakarta.el.TestImportHandlerStandardPackages
Tests run: 1, Failures: 1, Errors: 0, Skipped: 0, Time elapsed: 0.593 sec

Testcase: testClassListsAreComplete took 0.525 sec
FAILED
java.lang.ScopedValue.CallableOp
junit.framework.AssertionFailedError: java.lang.ScopedValue.CallableOp
at 
jakarta.el.TestImportHandlerStandardPackages.lambda$checkPackageClassList$13(TestImportHandlerStandardPackages.java:76)
at 
java.base/java.util.stream.ForEachOps$ForEachOp$OfRef.accept(ForEachOps.java:184)
at 
java.base/java.util.stream.ReferencePipeline$2$1.accept(ReferencePipeline.java:197)
at 
java.base/java.util.stream.ReferencePipeline$2$1.accept(ReferencePipeline.java:197)
at 
java.base/java.util.stream.ReferencePipeline$2$1.accept(ReferencePipeline.java:197)
at 
java.base/java.util.stream.ReferencePipeline$2$1.accept(ReferencePipeline.java:197)
at 
java.base/java.util.stream.ReferencePipeline$3$1.accept(ReferencePipeline.java:215)
at 
java.base/java.util.stream.ReferencePipeline$3$1.accept(ReferencePipeline.java:215)
at 
java.base/java.util.stream.ReferencePipeline$2$1.accept(ReferencePipeline.java:197)
at 
java.base/java.util.stream.ReferencePipeline$2$1.accept(ReferencePipeline.java:197)
at 
java.base/java.util.stream.ReferencePipeline$3$1.accept(ReferencePipeline.java:215)
at 
java.base/java.util.stream.ReferencePipeline$3$1.accept(ReferencePipeline.java:215)
at 
java.base/java.util.stream.ReferencePipeline$2$1.accept(ReferencePipeline.java:197)
at 
java.base/java.util.stream.ReferencePipeline$2$1.accept(ReferencePipeline.java:197)
at 
java.base/java.util.stream.ReferencePipeline$2$1.accept(ReferencePipeline.java:197)
at 
java.base/jdk.internal.module.SystemModuleFinders$ModuleContentSpliterator.tryAdvance(SystemModuleFinders.java:573)
at 
java.base/java.util.Spliterator.forEachRemaining(Spliterator.java:332)
at 
java.base/java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:570)
at 
java.base/java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:560)
at 
java.base/java.util.stream.ForEachOps$ForEachOp.evaluateSequential(ForEachOps.java:151)
at 
java.base/java.util.stream.ForEachOps$ForEachOp$OfRef.evaluateSequential(ForEachOps.java:174)
at 
java.base/java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:265)
at 
java.base/java.util.stream.ReferencePipeline.forEach(ReferencePipeline.java:636)
at 
jakarta.el.TestImportHandlerStandardPackages.checkPackageClassList(TestImportHandlerStandardPackages.java:76)
at 
jakarta.el.TestImportHandlerStandardPackages.testClassListsAreComplete(TestImportHandlerStandardPackages.java:46)
at 
java.base/jdk.internal.reflect.DirectMethodHandleAccessor.invoke(DirectMethodHandleAccessor.java:103)



I am using the current JDK 23 EA 27.

Best regards,

Rainer

-
To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org
For additional commands, e-mail: dev-h...@tomcat.apache.org

panama test TestOpenSSLConf.testOpenSSLConfCmdCipher() failing

[Rainer Jung]

Hi all,

when testing 11.0.0-M21 and 10.1.25 I observe new failures in panama:

Testcase: 
testOpenSSLConfCmdCipher[org.apache.tomcat.util.net.openssl.panama.OpenSSLImplementation] 
took 4.438 sec

FAILED
Wrong HostConfig ciphers
Expected: is ["AES256-SHA256"]
 but: was ["TLS_AES_256_GCM_SHA384", 
"TLS_CHACHA20_POLY1305_SHA256", "TLS_AES_128_GCM_SHA256", "AES256-SHA256"]

junit.framework.AssertionFailedError: Wrong HostConfig ciphers
Expected: is ["AES256-SHA256"]
 but: was ["TLS_AES_256_GCM_SHA384", 
"TLS_CHACHA20_POLY1305_SHA256", "TLS_AES_128_GCM_SHA256", "AES256-SHA256"]

at org.hamcrest.MatcherAssert.assertThat(MatcherAssert.java:20)
at 
org.apache.tomcat.util.net.openssl.TestOpenSSLConf.testOpenSSLConfCmdCipher(TestOpenSSLConf.java:132)
at 
java.base/jdk.internal.reflect.DirectMethodHandleAccessor.invoke(DirectMethodHandleAccessor.java:103)


The test was done with the recent releases of OpenSSL 3.0, 3.1, 3.2 and 
3.3. All of them fail in the same way.


Best regards,

Rainer

-
To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org
For additional commands, e-mail: dev-h...@tomcat.apache.org

TestDataSourceUserDatabase and TestDataSourceRealm need Java 17

[Rainer Jung]

Hi there,

the test classes org.apache.catalina.users.TestDataSourceUserDatabase 
and org.apache.catalina.realm.TestDataSourceRealm have a filing test 
which needs Java 17 due to the class version of 
org/apache/derby/jdbc/EmbeddedDriver. For TC 10.1 this leads to failures 
when testing with JDK 11-16.


Details:

Testcase: testBasicUserRoleDatabase took 0.749 sec
Caused an ERROR
org/apache/derby/jdbc/EmbeddedDriver has been compiled by a more recent 
version of the Java Runtime (class file version 61.0), this version of 
the Java Runtime only recognizes class file versions up to 55.0
java.lang.UnsupportedClassVersionError: 
org/apache/derby/jdbc/EmbeddedDriver has been compiled by a more recent 
version of the Java Runtime (class file version 61.0), this version of 
the Java Runtime only recognizes class file versions up to 55.0

at java.base/java.lang.ClassLoader.defineClass1(Native Method)
at 
java.base/java.lang.ClassLoader.defineClass(ClassLoader.java:1022)
at 
java.base/java.security.SecureClassLoader.defineClass(SecureClassLoader.java:174)
at 
java.base/jdk.internal.loader.BuiltinClassLoader.defineClass(BuiltinClassLoader.java:800)
at 
java.base/jdk.internal.loader.BuiltinClassLoader.findClassOnClassPathOrNull(BuiltinClassLoader.java:698)
at 
java.base/jdk.internal.loader.BuiltinClassLoader.loadClassOrNull(BuiltinClassLoader.java:621)
at 
java.base/jdk.internal.loader.BuiltinClassLoader.loadClass(BuiltinClassLoader.java:579)
at 
java.base/jdk.internal.loader.ClassLoaders$AppClassLoader.loadClass(ClassLoaders.java:178)

at java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:527)
at java.base/java.lang.Class.forName0(Native Method)
at java.base/java.lang.Class.forName(Class.java:315)
at 
org.apache.catalina.users.TestDataSourceUserDatabase$DerbyUserDatabase.open(TestDataSourceUserDatabase.java:103)
at 
org.apache.catalina.users.TestDataSourceUserDatabase.testBasicUserRoleDatabase(TestDataSourceUserDatabase.java:131)
at 
java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native 
Method)
at 
java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at 
java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)


and

Testcase: testRealm took 1.298 sec
Caused an ERROR
org/apache/derby/jdbc/EmbeddedDriver has been compiled by a more recent 
version of the Java Runtime (class file version 61.0), this version of 
the Java Runtime only recognizes class file versions up to 55.0
java.lang.UnsupportedClassVersionError: 
org/apache/derby/jdbc/EmbeddedDriver has been compiled by a more recent 
version of the Java Runtime (class file version 61.0), this version of 
the Java Runtime only recognizes class file versions up to 55.0

at java.base/java.lang.ClassLoader.defineClass1(Native Method)
at 
java.base/java.lang.ClassLoader.defineClass(ClassLoader.java:1022)
at 
java.base/java.security.SecureClassLoader.defineClass(SecureClassLoader.java:174)
at 
java.base/jdk.internal.loader.BuiltinClassLoader.defineClass(BuiltinClassLoader.java:800)
at 
java.base/jdk.internal.loader.BuiltinClassLoader.findClassOnClassPathOrNull(BuiltinClassLoader.java:698)
at 
java.base/jdk.internal.loader.BuiltinClassLoader.loadClassOrNull(BuiltinClassLoader.java:621)
at 
java.base/jdk.internal.loader.BuiltinClassLoader.loadClass(BuiltinClassLoader.java:579)
at 
java.base/jdk.internal.loader.ClassLoaders$AppClassLoader.loadClass(ClassLoaders.java:178)

at java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:527)
at java.base/java.lang.Class.forName0(Native Method)
at java.base/java.lang.Class.forName(Class.java:315)
at 
org.apache.catalina.realm.TestDataSourceRealm$DerbyDataSourceRealm.open(TestDataSourceRealm.java:61)
at 
org.apache.catalina.realm.TestDataSourceRealm.testRealm(TestDataSourceRealm.java:106)
at 
java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native 
Method)
at 
java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at 
java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)



Best regards,

Rainer



-
To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org
For additional commands, e-mail: dev-h...@tomcat.apache.org

Re: WebDAV and Microsoft clients

[Rainer Jung]

Am 22.05.24 um 19:21 schrieb Mark Thomas:

All,

I've been looking at the WebDav Servlet for the last few days and in 
particular how it interacts with Microsoft clients.


Basic operations including:
- directory listings
- create new file
- create new directory
- rename
- update contents (ie open a file for editing and then saving it)

all work for port 80 and port 8080 when WebDAV is mounted at "/" or a 
specific context.


Drag/drop and copy/paste do not work. This appears to be related to 
Tomcat not implementing PROPPATCH. There is some guess work involved 
since I don't have access to the Microsoft code but I think the client 
is setting timestamps with PROPPATCH and then checking them. Because the 
PROPPATCH fails the overall operation is failed.


I don't think that the WebdavFixFilter is required any more.

I'd like to propose the following:
- deprecate WebdavFixFilter in all current versions and then remove it
   in Tomcat 11
- add the above information on what works and what doesn't to the
   WebdavServlet Javadoc - maybe along with a note to ping the dev list
   if drag/drop and copy/paste are required (or maybe a BZ issue)
- come back to this if there is user interest in getting drag/drop and
   copy/paste working

Thoughts?


You might already know about this, but In the httpd world sometimes the 
litmus test suite was mentioned:


http://www.webdav.org/neon/litmus/

I never have build or even used it, but it might be useful to check for 
improvements or regressions in the course of bigger changes.


Best regards,

Rainer


-
To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org
For additional commands, e-mail: dev-h...@tomcat.apache.org

Re: [VOTE] Release Apache Tomcat 10.1.24

[Rainer Jung]

Am 09.05.24 um 20:12 schrieb Christopher Schultz:

The proposed Apache Tomcat 10.1.24 release is now available for
voting.

The notable changes compared to 10.1.23 are:

- Correct error handling for asynchronous requests

- Refactor HTTP header parsing to use common parsing code and fix
   non-blocking reads of chunked request bodies including trailer fields

- WebDAV locking handling fixes

For full details, see the change log:
https://nightlies.apache.org/tomcat/tomcat-10.1.x/docs/changelog.html

Applications that run on Tomcat 9 and earlier will not run on Tomcat 10 
without changes. Java EE applications designed for Tomcat 9 and earlier 
may be placed in the $CATALINA_BASE/webapps-javaee directory and Tomcat 
will automatically convert them to Jakarta EE and copy them to the 
webapps directory.


It can be obtained from:
https://dist.apache.org/repos/dist/dev/tomcat/tomcat-10/v10.1.24/

The Maven staging repo is:
https://repository.apache.org/content/repositories/orgapachetomcat-1494

The tag is:
https://github.com/apache/tomcat/tree/10.1.24
https://github.com/apache/tomcat/commit/f2a274bc00cf73670a614999561c69a391b5e35f

Please reply with a +1 for release or -0/-1 with an explanation.

The proposed 10.1.24 release is:
[ ] Broken - do not release
[X] Stable - go ahead and release as 10.1.24


+1, builds fine (on RHEL8 using OpenJDK 22.0.1+8-16).

The only test failures are sporadic, happen with tcnative and are 
crashes. I observe these for many years now, but this time I have 
checked and all of them happen with NIO2. I use two threads during 
testing and when I checked a few months ago, it seems they are happening 
during shutdown (duplicate pool destruction).


Tested via unit test suite on RHEL 6, 7, 8 and 9, SLES 11, 12 and 15 and 
Solaris 11 using latest patch levels of 11, 17, 21, 22 from Adoptium 
Temurin, Zulu Azul, Amazon Coretto, Oracle and RedHat (the latter only 
on RHEL) plus JDK 23 EA 18. Solaris tests limited to JDK 11. A total of 
124 test runs for NIO+NIO2.


Also ran the few relevant tests for all of these platforms and JVMs in 
combination with tcnative 1.3.0 and 2.0.7 built with OpenSSL 3.0.13, 
3.1.5, 3.2.1 and 3.3.0. A total of 992 test runs for NIO+NIO2. Tests 
assumed to be relevant were 
org/apache/coyote/http2/TestLargeUpload.java,org/apache/**/Test*SSL*.java,org/apache/**/Test*Ssl*.java,org/apache/**/Test*openssl*.java,org/apache/tomcat/util/net/**/Test*.java,org/apache/tomcat/jni/**/Test*.java


Finally ran the relevant tests with JVMs 22 and 23 plus panama for the 
same four OpenSSL versions. A total of 124 test runs.


No testing done for APR connectors.

Test adjustments:

- TestMapperPerformance increased maxTime from 5000 to 25000
- TesterTagHandlerPoolPerformance lowered loop count from 500, to 50
- SimpleHttpClient increased soTimeout from 1 to 2

No unusual test failures, only sporadic crashes in tcnative (nothing 
new, probably during shutdown, running with 2 test threads; this time 
all with NIO2).


- org.apache.catalina.valves.rewrite.TestResolverSSL
rh_jdk17-rhel7.x86_64-tcnative-1.3.0-300-1

- org.apache.coyote.http2.TestLargeUpload
adopt_jdk22-rhel8.x86_64-tcnative-1.3.0-320-1
oracle_jdk17-rhel8.x86_64-tcnative-2.0.7-300-1

- org.apache.tomcat.util.net.TestClientCert
adopt_jdk21-rhel7.x86_64-tcnative-1.3.0-310-2
amazon_jdk22-rhel8.x86_64-tcnative-2.0.7-320-1
oracle_jdk11-rhel6.x86_64-tcnative-1.3.0-330-1
oracle_jdk22-sles11.x86_64-tcnative-2.0.7-330-1
zulu_jdk17-sles11.x86_64-tcnative-1.3.0-330-1
zulu_jdk21-sles12.x86_64-tcnative-2.0.7-320-1

- org.apache.tomcat.util.net.TestCustomSslTrustManager
zulu_jdk17-rhel8.x86_64-tcnative-2.0.7-330-1

- org.apache.tomcat.util.net.TestSSLHostConfigCompat
adopt_jdk11-sles15.x86_64-tcnative-1.3.0-330-1
amazon_jdk11-rhel9.x86_64-tcnative-2.0.7-300-1
oracle_jdk17-rhel7.x86_64-tcnative-2.0.7-330-1
oracle_jdk17-rhel8.x86_64-tcnative-1.3.0-300-1
oracle_jdk22-rhel7.x86_64-tcnative-2.0.7-300-1
oracle_jdk22-rhel8.x86_64-tcnative-1.3.0-320-1
oracle_jdk22-sles15.x86_64-tcnative-1.3.0-320-1
zulu_jdk11-sles15.x86_64-tcnative-1.3.0-310-2
zulu_jdk11-sles15.x86_64-tcnative-2.0.7-300-1
zulu_jdk17-rhel8.x86_64-tcnative-1.3.0-330-1
zulu_jdk17-sles11.x86_64-tcnative-1.3.0-310-2

- org.apache.tomcat.util.net.TestSsl
amazon_jdk11-rhel7.x86_64-tcnative-1.3.0-320-1
amazon_jdk17-sles15.x86_64-tcnative-1.3.0-330-1
oracle_jdk17-rhel7.x86_64-tcnative-1.3.0-330-1

Best regards,

Rainer

-
To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org
For additional commands, e-mail: dev-h...@tomcat.apache.org

Re: [VOTE] Release Apache Tomcat 9.0.89

[Rainer Jung]

Am 03.05.24 um 22:37 schrieb Rémy Maucherat:

The proposed Apache Tomcat 9.0.89 release is now available for voting.

The notable changes compared to 9.0.88 are:

- Refactor HTTP header parsing to use common parsing code and fix
non-blocking reads of chunked request bodies including trailer fields

- Add more timescale options to AccessLogValve and
ExtendedAccessLogValve

- WebDAV locking handling fixes

For full details, see the changelog:
https://nightlies.apache.org/tomcat/tomcat-9.0.x/docs/changelog.html

It can be obtained from:
https://dist.apache.org/repos/dist/dev/tomcat/tomcat-9/v9.0.89/

The Maven staging repo is:
https://repository.apache.org/content/repositories/orgapachetomcat-1494

The tag is:
https://github.com/apache/tomcat/tree/9.0.89
661a5978828212bbe4a324dd7c854894f34a561b

The proposed 9.0.89 release is:
[ ] -1, Broken - do not release
[X] +1, Stable - go ahead and release as 9.0.89

Rémy


+1, builds fine (on RHEL8 using JDK22).

Tested via unit test suite on RHEL 6, 7, 8 and 9, SLES 11, 12 and 15 and 
Solaris 10 and 11 using latest patch levels of JDK 8, 11, 17, 21, 22 
from Adoptium Temurin, Zulu Azul, Amazon Coretto, Oracle and RedHat (the 
latter only on RHEL) plus JDK 23 EA 18. Solaris tests limited to JDK 
1.8.0 and 11. A total of 161 test runs for NIO+NIO2.


Also ran the few relevant tests for all of these platforms and JVMs in 
combination with tcnative 1.3.0 and 2.0.7 built with OpenSSL 3.0.13, 
3.1.5, 3.2.1 and 3.3.0. A total of 1288 test runs for NIO+NIO2. Tests 
assumed to be relevant were 
org/apache/coyote/http2/TestLargeUpload.java,org/apache/**/Test*SSL*.java,org/apache/**/Test*Ssl*.java,org/apache/**/Test*openssl*.java,org/apache/tomcat/util/net/**/Test*.java,org/apache/tomcat/jni/**/Test*.java


No testing done for APR connectors.

Test adjustments:

- TestMapperPerformance increased maxTime from 5000 to 25000
- TesterTagHandlerPoolPerformance lowered loop count from 500, to 50
- SimpleHttpClient increased soTimeout from 1 to 2

No unusual test failures:

1) sporadic tcnative crashes (probably during shutdown, running with 2 
test threads):


- org.apache.tomcat.util.net.TestClientCert FAILED (crashed)
  1 rh_jdk1.8.0-rhel9.x86_64-tcnative-1.3.0-320-1

- org.apache.tomcat.util.net.TestClientCertTls13 FAILED (crashed)
  1 amazon_jdk1.8.0-rhel9.x86_64-tcnative-2.0.7-300-1

- org.apache.tomcat.util.net.TestSsl FAILED (crashed)
  1 amazon_jdk1.8.0-rhel9.x86_64-tcnative-2.0.7-300-1
  1 oracle_jdk1.8.0-sles12.x86_64-tcnative-2.0.7-310-2
  1 amazon_jdk21-sles12.x86_64-tcnative-2.0.7-320-1
  1 amazon_jdk21-sles15.x86_64-tcnative-1.3.0-310-2
  1 rh_jdk21-rhel8.x86_64-tcnative-2.0.7-320-1

- org.apache.catalina.valves.rewrite.TestResolverSSL FAILED (crashed)
  1 zulu_jdk21-sles12.x86_64-tcnative-2.0.7-300-1


2) very sporadic JSSE failures for Linux:

- org.apache.tomcat.websocket.server.TestSlowClient FAILED
  1 amazon_jdk1.8.0-rhel6.x86_64-jsse
  1 oracle_jdk17-rhel6.x86_64-jsse

- 
org.apache.catalina.tribes.group.interceptors.TestNonBlockingCoordinator 
FAILED

  1 adopt_jdk22-rhel8.x86_64-jsse

3) sporadic JSSE failures for Solaris:

- org.apache.catalina.connector.TestConnector FAILED
  1 oracle_jdk1.8.0-solaris10.sparc-jsse

- org.apache.catalina.core.TestSwallowAbortedUploads FAILED
  1 adopt_jdk1.8.0-solaris11.sparc-jsse
  2 oracle_jdk1.8.0-solaris11.sparc-jsse
  2 zulu_jdk1.8.0-solaris11.sparc-jsse

- org.apache.tomcat.websocket.TestWebSocketFrameClientSSL FAILED
  1 zulu_jdk11-solaris11.sparc-tcnative-1.3.0-300-1

- org.apache.tomcat.websocket.server.TestClose FAILED
  1 zulu_jdk1.8.0-solaris11.sparc-jsse

Best regards,

Rainer

-
To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org
For additional commands, e-mail: dev-h...@tomcat.apache.org

Re: [VOTE] Release Apache Tomcat 11.0.0-M20

[Rainer Jung]

Am 03.05.24 um 18:22 schrieb Mark Thomas:

The proposed Apache Tomcat 11.0.0-M20 release is now available for
voting.

Apache Tomcat 11.0.0-M20 is a milestone release of the 11.0.x branch and 
has been made to provide users with early access to the new features in 
Apache Tomcat 11.0.x so that they may provide feedback. The notable 
changes compared to the previous milestone include:


- Add OpenSSL FFM classes to tomcat-embed-core.jar

- Refactor HTTP header parsing to use common parsing code and fix
   non-blocking reads of chunked request bodies including trailer fields

- Add more timescale options to AccessLogValve and
   ExtendedAccessLogValve


For full details, see the change log:
https://nightlies.apache.org/tomcat/tomcat-11.0.x/docs/changelog.html

Applications that run on Tomcat 9 and earlier will not run on Tomcat 11 
without changes. Java EE applications designed for Tomcat 9 and earlier 
may be placed in the $CATALINA_BASE/webapps-javaee directory and Tomcat 
will automatically convert them to Jakarta EE and copy them to the 
webapps directory. Applications using deprecated APIs may require 
further changes.


It can be obtained from:
https://dist.apache.org/repos/dist/dev/tomcat/tomcat-11/v11.0.0-M20/

The Maven staging repo is:
https://repository.apache.org/content/repositories/orgapachetomcat-1493

The tag is:
https://github.com/apache/tomcat/tree/11.0.0-M20
c400bf727cbc10198d3f52c29849d18660050b0c

The proposed 11.0.0-M20 release is:
[ ] -1 Broken - do not release
[X] +1 Alpha  - go ahead and release as 11.0.0-M20


+1, builds fine (on RHEL8 using JDK22).

Tested via unit test suite on RHEL 6, 7, 8 and 9 and SLES 11, 12 and 15 
using latest patch levels of JDK 17, 21, 22 from Adoptium Temurin, Zulu 
Azul, Amazon Coretto, Oracle and RedHat (the latter only on RHEL) plus 
JDK 23 EA 18. A total of 91 test runs.


Also ran the few relevant tests for all of these platforms and JVMs in 
combination with tcnative 1.3.0 and 2.0.7 built with OpenSSL 3.0.13, 
3.1.5, 3.2.1 and 3.3.0. A total of 728 test runs.


Finally ran the relevant tests with JVMs 22 and 23 plus panama for the 
same four OpenSSL versions. A total of 124 test runs.


Apart from the usual sporadic tcnative crashes during shutdown (running 
with 2 test threads) all was fine:


- org.apache.coyote.http2.TestLargeUpload FAILED (crashed)
  1 oracle_jdk17-rhel8.x86_64-tcnative-2.0.7-300-1
  1 rh_jdk21-rhel7.x86_64-tcnative-1.3.0-330-1

- org.apache.tomcat.util.net.TestClientCert FAILED (crashed)
  1 oracle_jdk21-rhel8.x86_64-tcnative-1.3.0-330-1
  1 zulu_jdk21-sles11.x86_64-tcnative-2.0.7-330-1

- org.apache.tomcat.util.net.TestCustomSslTrustManager FAILED (crashed)
  1 amazon_jdk17-sles15.x86_64-tcnative-1.3.0-300-1
  1 zulu_jdk17-rhel7.x86_64-tcnative-2.0.7-320-1

- org.apache.tomcat.util.net.TestSSLHostConfigCompat FAILED (crashed)
  1 rh_jdk17-rhel8.x86_64-tcnative-1.3.0-310-2
  1 amazon_jdk21-rhel9.x86_64-tcnative-2.0.7-310-2
  1 oracle_jdk21-rhel8.x86_64-tcnative-1.3.0-330-1
  1 oracle_jdk21-sles12.x86_64-tcnative-2.0.7-310-2
  1 rh_jdk21-rhel7.x86_64-tcnative-1.3.0-320-1
  1 adopt_jdk22-sles12.x86_64-tcnative-1.3.0-310-2
  1 oracle_jdk22-rhel8.x86_64-tcnative-1.3.0-320-1

- org.apache.tomcat.util.net.TestSsl FAILED (crashed)
  1 amazon_jdk17-sles11.x86_64-tcnative-2.0.7-320-1
  1 oracle_jdk22-sles12.x86_64-tcnative-2.0.7-320-1

Best regards,

Rainer

-
To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org
For additional commands, e-mail: dev-h...@tomcat.apache.org

Re: [VOTE] Release Apache Tomcat 10.1.23

[Rainer Jung]

Am 21.04.24 um 17:15 schrieb Rémy Maucherat:

On Sun, Apr 21, 2024 at 11:52 AM Rainer Jung  wrote:


Am 16.04.24 um 15:11 schrieb Christopher Schultz:

The proposed Apache Tomcat 10.1.23 release is now available for
voting. Apache Tomcat 10.1.21 was canceled due to a release-build
mistake and Apache Tomcat 10.1.22 was cancelled due to an option in
startup scripts which would have caused Java 11 environments to fail to
start.

The notable changes compared to 10.1.20 are:

- Improve locking strategies in Catalina core

- Update Basic authentication to implement the requirements of RFC 7617

- Updates to Apache Commons dependencies

- Add OpenSSL support when FFM is available

For full details, see the change log:
https://nightlies.apache.org/tomcat/tomcat-10.1.x/docs/changelog.html

Applications that run on Tomcat 9 and earlier will not run on Tomcat 10
without changes. Java EE applications designed for Tomcat 9 and earlier
may be placed in the $CATALINA_BASE/webapps-javaee directory and Tomcat
will automatically convert them to Jakarta EE and copy them to the
webapps directory.

It can be obtained from:
https://dist.apache.org/repos/dist/dev/tomcat/tomcat-10/v10.1.23/

The Maven staging repo is:
https://repository.apache.org/content/repositories/orgapachetomcat-1492

The tag is:
https://github.com/apache/tomcat/tree/10.1.23
https://github.com/apache/tomcat/commit/9062d27dc5122e8241ea62a4c4312af0dc71da49

Please reply with a +1 for release or -0/-1 with an explanation.

The proposed 10.1.23 release is:
[ ] Broken - do not release
[X] Stable - go ahead and release as 10.1.23


+1, builds fine (on RHEL8 using JDK22).

Tested via unit test suite on RHEL 6, 7, 8 and 9, SLES 11, 12 and 15 and
Solaris 11 Sparc using latest patch levels of JDK 11, 17, 21, 22 from
Adoptium Temurin, Zulu Azul, Amazon Coretto, Oracle and RedHat (where
applicable) plus JDK 23 EA 18. That was a total of 124 JSSE based test
suite runs.

Also ran the few relevant tests for all of these platforms and JMVs in
combination with tcnative 1.3.0 and 2.0.7 built with OpenSSL 3.0.13,
3.1.5, 3.2.1 and 3.3.0. That was a total of 992 tcnative based shortened
test suite runs.

Finally ran the relevant tests with JVMs 22 and 23 plus panama for the
same four OpenSSL versions. That was a total of 124 tcnative based
shortened test suite runs.

Apart from the usual sporadic tcnative crashes during shutdown (running
with 2 test threads) all was fine.


That's a lot of tests ! Are the crashes affecting the FFM code too ?

Rémy


No, I started to become an FFM fan because of that ;)

The tcnative crashes happen in apr pool shutdown (not using APR 
connectors, just tcnative openssl), I think due to duplicate library 
termination. Always at the end of a test.


Best regards,

Rainer


-
To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org
For additional commands, e-mail: dev-h...@tomcat.apache.org

Re: [VOTE] Release Apache Tomcat 10.1.23

[Rainer Jung]

Am 16.04.24 um 15:11 schrieb Christopher Schultz:

The proposed Apache Tomcat 10.1.23 release is now available for
voting. Apache Tomcat 10.1.21 was canceled due to a release-build 
mistake and Apache Tomcat 10.1.22 was cancelled due to an option in 
startup scripts which would have caused Java 11 environments to fail to 
start.


The notable changes compared to 10.1.20 are:

- Improve locking strategies in Catalina core

- Update Basic authentication to implement the requirements of RFC 7617

- Updates to Apache Commons dependencies

- Add OpenSSL support when FFM is available

For full details, see the change log:
https://nightlies.apache.org/tomcat/tomcat-10.1.x/docs/changelog.html

Applications that run on Tomcat 9 and earlier will not run on Tomcat 10 
without changes. Java EE applications designed for Tomcat 9 and earlier 
may be placed in the $CATALINA_BASE/webapps-javaee directory and Tomcat 
will automatically convert them to Jakarta EE and copy them to the 
webapps directory.


It can be obtained from:
https://dist.apache.org/repos/dist/dev/tomcat/tomcat-10/v10.1.23/

The Maven staging repo is:
https://repository.apache.org/content/repositories/orgapachetomcat-1492

The tag is:
https://github.com/apache/tomcat/tree/10.1.23
https://github.com/apache/tomcat/commit/9062d27dc5122e8241ea62a4c4312af0dc71da49

Please reply with a +1 for release or -0/-1 with an explanation.

The proposed 10.1.23 release is:
[ ] Broken - do not release
[X] Stable - go ahead and release as 10.1.23


+1, builds fine (on RHEL8 using JDK22).

Tested via unit test suite on RHEL 6, 7, 8 and 9, SLES 11, 12 and 15 and 
Solaris 11 Sparc using latest patch levels of JDK 11, 17, 21, 22 from 
Adoptium Temurin, Zulu Azul, Amazon Coretto, Oracle and RedHat (where 
applicable) plus JDK 23 EA 18. That was a total of 124 JSSE based test 
suite runs.


Also ran the few relevant tests for all of these platforms and JMVs in 
combination with tcnative 1.3.0 and 2.0.7 built with OpenSSL 3.0.13, 
3.1.5, 3.2.1 and 3.3.0. That was a total of 992 tcnative based shortened 
test suite runs.


Finally ran the relevant tests with JVMs 22 and 23 plus panama for the 
same four OpenSSL versions. That was a total of 124 tcnative based 
shortened test suite runs.


Apart from the usual sporadic tcnative crashes during shutdown (running 
with 2 test threads) all was fine.


Best regards,

Rainer

-
To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org
For additional commands, e-mail: dev-h...@tomcat.apache.org

Unit tests using tcnative/panama [Was: [Bug 68910] Improve LibreSSL version check in tcnative.m4]

[Rainer Jung]

Am 18.04.24 um 09:08 schrieb bugzi...@apache.org:

https://bz.apache.org/bugzilla/show_bug.cgi?id=68910

--- Comment #3 from Michael Osipov  ---
(In reply to Christopher Schultz from comment #1)

(In reply to Michael Osipov from comment #0)

since we also do support LibreSSL [...]


Note: Support for LibreSSL is more of an aspiration and less of a
requirement. We don't technically advertise support for LibreSSL, but I
would like to be able to support it.


FYI. Just ran 10.1.x with LibreSSL 3.5.2:
[concat] TEST-org.apache.catalina.valves.rewrite.TestResolverSSL.NIO.txt
[concat] TEST-org.apache.catalina.valves.rewrite.TestResolverSSL.NIO2.txt
[concat] TEST-org.apache.tomcat.util.net.TestClientCert.NIO.txt
[concat] TEST-org.apache.tomcat.util.net.TestClientCert.NIO2.txt
[concat] TEST-org.apache.tomcat.util.net.TestCustomSslTrustManager.NIO.txt
[concat] TEST-org.apache.tomcat.util.net.TestCustomSslTrustManager.NIO2.txt
[concat] TEST-org.apache.tomcat.util.net.openssl.TestOpenSSLConf.NIO.txt
[concat] TEST-org.apache.tomcat.util.net.openssl.TestOpenSSLConf.NIO2.txt

The rest is passing. These are failing for renegotiation or protocol mismatch.
That looks very promising.


Probably not relevant for this specific topic but maybe of general interest:

For other reasons I tried to identify, which unit tests actually load 
and execute with tcnative and/or panama, and those are very few. Most 
tests do not use these. Apart from the ones you mentioned as failing:


org.apache.catalina.valves.rewrite.TestResolverSSL
org.apache.tomcat.util.net.TestClientCert
org.apache.tomcat.util.net.TestCustomSslTrustManager
org.apache.tomcat.util.net.openssl.TestOpenSSLConf

the only other tests I found using tcnative and/or openssl connectors are:

org.apache.coyote.http2.TestLargeUpload
org.apache.tomcat.util.net.TestClientCertTls13
org.apache.tomcat.util.net.TestSSLHostConfigCompat
org.apache.tomcat.util.net.TestSSLHostConfigIntegration
org.apache.tomcat.util.net.TestSsl
org.apache.tomcat.websocket.TestWebSocketFrameClientSSL
org.apache.tomcat.websocket.TestWsWebSocketContainerSSL

So almost all of the tests actually using a connector to run servlets 
etc. only use plain http connectors (or fixed JSSE, but I think such do 
not exist).


A few more might only use the commandline openssl binary. Those are not 
included in the above lists.


Best regards,

Rainer

-
To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org
For additional commands, e-mail: dev-h...@tomcat.apache.org

Re: Some remarks on panama libssl loading

[Rainer Jung]

Hi Konstantin,

Am 17.04.24 um 21:44 schrieb Konstantin Kolinko:

ср, 17 апр. 2024 г. в 17:21, Rainer Jung :


Am 17.04.24 um 15:34 schrieb Michael Osipov:

Rainer, I do not fully understand the problem here. We use libtool to solve 
exactly this problem with versioned SONAMEs. It will create symlinks to the 
SONAME.
Do you expect anyone even with dlopen() to load libfoo.o.{SOVERSION} unless it 
is strictly needed?

E.g.:
lrwxr-xr-x  1 root  wheel26 2024-03-22 10:20 /usr/lib/libcrypto.so@ -> 
../../lib/libcrypto.so.111
lrwxr-xr-x  1 root  wheel   13 2024-03-22 10:20 /usr/lib/libssl.so@ -> 
libssl.so.111
-r--r--r--  1 root  wheel   608008 2024-03-22 10:20 /usr/lib/libssl.so.111
and so on...


Yes, I expect that! anyone is the JVM :(

The problem is, that the Java API does not care about these well thought
native traditions. You can not open libssl.so.3 using
System.loadlibrary(String name), because whatever you give it as "name"
parameter it will always try to open libname.so. It always prepends
"lib" to name and always suffixes it with plain ".so".

Yes, it might exist as the first in your list of symlinks, but on most
linux distributions this link is not installed by default, because it is
only needed when doing compilations. So it is only installed when you
install development packages for libs.


There are two methods,
System.loadLibrary(libname)
System.load(filename)
or the same methods in Runtime.

The second method accepts an absolute path and apparently does no
manipulation on the name.

There is also  System.mapLibraryName().

Note that o.a.t.jni.Library constructor uses all those three methods.
A code that was added several years ago uses mapLibraryName() and
load() to load a library from "catalina.home/bin". It then falls back
to the old algorithm that uses loadLibrary().


https://docs.oracle.com/en/java/javase/22/docs/api/java.base/java/lang/System.html
https://docs.oracle.com/en/java/javase/22/docs/api/java.base/java/lang/Runtime.html


thanks for your valuable input as always. I am referring to 
java/org/apache/tomcat/util/openssl/openssl_h.java, which only contains 
which on Mac does


System.loadLibrary("ssl");
SYMBOL_LOOKUP = 
SymbolLookup.loaderLookup().or(Linker.nativeLinker().defaultLookup());


and else

SYMBOL_LOOKUP = SymbolLookup.libraryLookup(System.mapLibraryName("ssl"), 
LIBRARY_ARENA)

.or(SymbolLookup.loaderLookup())
.or(Linker.nativeLinker().defaultLookup());

I *think* both attempts do not allow to use a versioned SONAME.

I have not experimented with System.load(filename) and how to combine it 
with SymbolLookup. o.a.t.jni.Library does not use SymbolLookup. But it 
seems SymbolLookup is not restricted to the development library names. 
So there's hope we can improve!


https://docs.oracle.com/en/java/javase/22/docs/api/java.base/java/lang/foreign/SymbolLookup.html

Best regards,

Rainer


-
To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org
For additional commands, e-mail: dev-h...@tomcat.apache.org

Re: Some remarks on panama libssl loading

[Rainer Jung]

Am 17.04.24 um 22:46 schrieb Michael Osipov:

On 2024/04/17 14:21:06 Rainer Jung wrote:

Am 17.04.24 um 15:34 schrieb Michael Osipov:

Rainer, I do not fully understand the problem here. We use libtool to solve 
exactly this problem with versioned SONAMEs. It will create symlinks to the 
SONAME.
Do you expect anyone even with dlopen() to load libfoo.o.{SOVERSION} unless it 
is strictly needed?

E.g.:
lrwxr-xr-x  1 root  wheel26 2024-03-22 10:20 /usr/lib/libcrypto.so@ -> 
../../lib/libcrypto.so.111
lrwxr-xr-x  1 root  wheel   13 2024-03-22 10:20 /usr/lib/libssl.so@ -> 
libssl.so.111
-r--r--r--  1 root  wheel   608008 2024-03-22 10:20 /usr/lib/libssl.so.111
and so on...


Yes, I expect that! anyone is the JVM :(

The problem is, that the Java API does not care about these well thought
native traditions. You can not open libssl.so.3 using
System.loadlibrary(String name), because whatever you give it as "name"
parameter it will always try to open libname.so. It always prepends
"lib" to name and always suffixes it with plain ".so".

Yes, it might exist as the first in your list of symlinks, but on most
linux distributions this link is not installed by default, because it is
only needed when doing compilations. So it is only installed when you
install development packages for libs.


Ah, now I see your problem, but it looks like a downstream problem of your 
distro of choice, no? I wonder how you compile then custom software if .so 
isn't present and the linker cannot find it with -L? What if you install the 
devel package to have .so link?


I think it is not a distro problem, but a explicit decision I like. I 
find it normal, that you separate development packages from production 
packages and only install development packages on development systems, 
not on production system. Here a development file is needed for a 
production process :(


In old times for instance it was forbidden to install a compiler on a 
production system. Yeah I know, once Java kicked in this rule was no 
longer possible in the Java world. But in the native world it is often 
still followed. So if there is no native compiler, you don't need the 
devel version of the libraries as well.


Best regards,

Rainer

-
To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org
For additional commands, e-mail: dev-h...@tomcat.apache.org

Re: Some remarks on panama libssl loading

[Rainer Jung]

Am 17.04.24 um 15:34 schrieb Michael Osipov:

Rainer, I do not fully understand the problem here. We use libtool to solve 
exactly this problem with versioned SONAMEs. It will create symlinks to the 
SONAME.
Do you expect anyone even with dlopen() to load libfoo.o.{SOVERSION} unless it 
is strictly needed?

E.g.:
lrwxr-xr-x  1 root  wheel26 2024-03-22 10:20 /usr/lib/libcrypto.so@ -> 
../../lib/libcrypto.so.111
lrwxr-xr-x  1 root  wheel   13 2024-03-22 10:20 /usr/lib/libssl.so@ -> 
libssl.so.111
-r--r--r--  1 root  wheel   608008 2024-03-22 10:20 /usr/lib/libssl.so.111
and so on...


Yes, I expect that! anyone is the JVM :(

The problem is, that the Java API does not care about these well thought 
native traditions. You can not open libssl.so.3 using 
System.loadlibrary(String name), because whatever you give it as "name" 
parameter it will always try to open libname.so. It always prepends 
"lib" to name and always suffixes it with plain ".so".


Yes, it might exist as the first in your list of symlinks, but on most 
linux distributions this link is not installed by default, because it is 
only needed when doing compilations. So it is only installed when you 
install development packages for libs.


Best regards,

Rainer


-
To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org
For additional commands, e-mail: dev-h...@tomcat.apache.org

Some remarks on panama libssl loading

[Rainer Jung]

Hi there,

I did some small tests using OpenSSL/Panama with TC 10.1.23 for running 
the unit tests.


First: they seem to work well using JDK 22 with OpenSSL 3.0.13, 3.1.5, 
3.2.1 and 3.3.0. For JDK 23 the tests are still running, but also look 
good up to now.


But some things around native library loading in the JVM seem to be 
nasty in principle:


1) Library file name


First some (well-known) background info about shared library loading in 
Linux in general:


When one *compiles* a binary or shared object against a library using 
"-lsomelib" the compile time linker looks for a file named 
libsomelib.so. It tries to resolve all symbols needed but not defined in 
the object to compile in these libs. If all goes well, it records the 
library dependency in the resulting object as NEEDED. This record does 
not refer to "libsomelib.so" as a name, but instead the so called SONAME 
of libsomelib.so. That's an internal name of the library often 
reflecting an API stability version. For example for OpenSSL 3.x the 
SONAME of libssl.so is libssl.so.3.


Now during runtime of the compiled object the runtime linker notices the 
need for the recorded SONAME (NEEDED libssl.so.3). It then searches a 
file with *that* name. In the OpenSSL example it is libssl.so.3, not (!) 
libssl.so.


On a "normal" linux system, libssl.so is therefore not installed, only 
libssl.so.3 (if OpenSSL 3 is installed at all). The file libssl.so (or 
better the symlink) is only part of the devel package typically 
installed on development systems, because it is only needed during 
compile time, not during runtime.


Now back to the JVM:

The Java API to load a native library (for example 
System.loadLibrary(String name)) gets the library name as a String 
parameter. What is this library name? Unfortunately the JVM devs did not 
do a good job. If you pass the API the String "somelib", they simply 
prepend it with "lib" and append ".so" and try to load that file. For 
example when you pass "ssl", they look for libssl.so. There is no way to 
tell Java to look for libssl.so.3 using this API. On a normal system, 
the file libssl.so does not exist and should not exist, because it is a 
development file.


As a workaround one has to provide libssl.so, at least as a symlink to 
libssl.so.3.


2) Indirect dependencies


Normally libssl.so(.3) has dependencies itself, e.g. for libcrypto.so.3 
(NEEDED libcrypto.so.3). These dependencies are not longer loaded via 
the Java API but implicitly by the runtime linker. So now, you do not 
need a file libcrypto.so - you can't even make that work - you really 
need the file libcrypto.so.3. Very confusing for newbies and pretty 
inconsistent.


3) java.library.path


The system property java.library.path is only used by the JVM when 
loading the libs requested via the Java API, so here libssl.so. The 
runtime linker looking then for libcrypto.so.3 is not set up to use 
java.library.path. So if the libs are not installed in a default system 
location, you need to set LD_LIBRARY_PATH to point to your libcrypto. If 
libssl is in the same directory (pretty much always), you then won't 
need java.library.path at all, because LD_LIBRARY_PATH also influences 
the search of libssl.so by the Java API. Again confusing.


None of this is Tomcat's fault. They are all deficiencies of the (old) 
Java APIs which were IMHO misdesigned. It might be different on Windows 
or Macs.


Just wanted to mention it in case others run into the same nits. We 
might want to document something though before raising more awareness 
for using OpenSSL via panama.


Best regards,

Rainer

-
To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org
For additional commands, e-mail: dev-h...@tomcat.apache.org

Re: [VOTE] Release Apache Tomcat 11.0.0-M19

[Rainer Jung]

Am 09.04.24 um 15:13 schrieb Rémy Maucherat:

The proposed Apache Tomcat 11.0.0-M19 release is now available for
voting.

Apache Tomcat 11.0.0-M19 is a milestone release of the 11.0.x branch and
has been made to provide users with early access to the new features in
Apache Tomcat 11.0.x so that they may provide feedback. The notable
changes compared to the previous milestone include:

- Finalize update to the Jakarta EE 11 specifications.

- Cookies header generation enhancements.

- Fix regression when reloading TLS configuration and files.

For full details, see the change log:
https://nightlies.apache.org/tomcat/tomcat-11.0.x/docs/changelog.html

Applications that run on Tomcat 9 and earlier will not run on Tomcat 11
without changes. Java EE applications designed for Tomcat 9 and earlier
may be placed in the $CATALINA_BASE/webapps-javaee directory and Tomcat
will automatically convert them to Jakarta EE and copy them to the
webapps directory. Applications using deprecated APIs may require
further changes.

It can be obtained from:
https://dist.apache.org/repos/dist/dev/tomcat/tomcat-11/v11.0.0-M19/

The Maven staging repo is:
https://repository.apache.org/content/repositories/orgapachetomcat-1488

The tag is:
https://github.com/apache/tomcat/tree/11.0.0-M19
19e301275f23056e3c46ab296c87cf6e16fbe68f

The proposed 11.0.0-M19 release is:
[ ] -1 Broken - do not release
[X] +1 Alpha  - go ahead and release as 11.0.0-M19


Thanks for RM!

Rainer

-
To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org
For additional commands, e-mail: dev-h...@tomcat.apache.org

Re: (tomcat) branch 10.1.x updated: Add native access to the scripts

[Rainer Jung]

Thanks Rémy. Things happen, at least we found it before it hit the street.

Am 11.04.24 um 20:52 schrieb Rémy Maucherat:

On Thu, Apr 11, 2024 at 7:21 PM Rainer Jung  wrote:


Hi Rémy,

I think this breaks 10.1 on Java 11, since --enable-native-access was
introduced later.


I was certain I tested it properly with 11, it worked and I thought it
was added as a noop to a later build. Actually, it seems I made a dumb
mistake somewhere while testing, so I will simply remove the patch.

Rémy


Best regards,

Rainer

Am 22.03.24 um 14:13 schrieb r...@apache.org:

This is an automated email from the ASF dual-hosted git repository.

remm pushed a commit to branch 10.1.x
in repository https://gitbox.apache.org/repos/asf/tomcat.git


The following commit(s) were added to refs/heads/10.1.x by this push:
   new 6550e203a8 Add native access to the scripts
6550e203a8 is described below

commit 6550e203a8119683f38c182cbfbb9b56587c54a0
Author: remm 
AuthorDate: Fri Mar 22 13:33:27 2024 +0100

  Add native access to the scripts
---
   bin/catalina.bat | 1 +
   bin/catalina.sh  | 1 +
   2 files changed, 2 insertions(+)

diff --git a/bin/catalina.bat b/bin/catalina.bat
index 9c55ae940e..9a30371013 100755
--- a/bin/catalina.bat
+++ b/bin/catalina.bat
@@ -223,6 +223,7 @@ set "JAVA_OPTS=%JAVA_OPTS% 
--add-opens=java.base/java.io=ALL-UNNAMED"
   set "JAVA_OPTS=%JAVA_OPTS% --add-opens=java.base/java.util=ALL-UNNAMED"
   set "JAVA_OPTS=%JAVA_OPTS% 
--add-opens=java.base/java.util.concurrent=ALL-UNNAMED"
   set "JAVA_OPTS=%JAVA_OPTS% 
--add-opens=java.rmi/sun.rmi.transport=ALL-UNNAMED"
+set "JAVA_OPTS=%JAVA_OPTS% --enable-native-access=ALL-UNNAMED"

   rem - Execute The Requested Command 
---

diff --git a/bin/catalina.sh b/bin/catalina.sh
index 32f87ffb6f..ed647a2dea 100755
--- a/bin/catalina.sh
+++ b/bin/catalina.sh
@@ -296,6 +296,7 @@ JAVA_OPTS="$JAVA_OPTS 
--add-opens=java.base/java.io=ALL-UNNAMED"
   JAVA_OPTS="$JAVA_OPTS --add-opens=java.base/java.util=ALL-UNNAMED"
   JAVA_OPTS="$JAVA_OPTS --add-opens=java.base/java.util.concurrent=ALL-UNNAMED"
   JAVA_OPTS="$JAVA_OPTS --add-opens=java.rmi/sun.rmi.transport=ALL-UNNAMED"
+JAVA_OPTS="$JAVA_OPTS --enable-native-access=ALL-UNNAMED"

   # - Execute The Requested Command 
-


-
To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org
For additional commands, e-mail: dev-h...@tomcat.apache.org

Re: [VOTE] Release Apache Tomcat 10.1.22

[Rainer Jung]

Am 10.04.24 um 20:51 schrieb Christopher Schultz:

The proposed Apache Tomcat 10.1.22 release is now available for
voting. Apache Tomcat 10.1.21 was canceled due to a release-build 
mistake. There are no source-level changes between 10.1.21 and 10.1.22.


The notable changes compared to 10.1.20 are:

- Add OpenSSL support when FFM is available

- Improve locking strategies in Catalina core

- Updates to Apache Commons dependencies

For full details, see the change log:
https://nightlies.apache.org/tomcat/tomcat-10.1.x/docs/changelog.html

Applications that run on Tomcat 9 and earlier will not run on Tomcat 10 
without changes. Java EE applications designed for Tomcat 9 and earlier 
may be placed in the $CATALINA_BASE/webapps-javaee directory and Tomcat 
will automatically convert them to Jakarta EE and copy them to the 
webapps directory.


It can be obtained from:
https://dist.apache.org/repos/dist/dev/tomcat/tomcat-10/v10.1.22/

The Maven staging repo is:
https://repository.apache.org/content/repositories/orgapachetomcat-1491

The tag is:
https://github.com/apache/tomcat/tree/10.1.22
fd9c0bb1f0a7e586daabfda625ef50827bcfddf3

Please reply with a +1 for release or -0/-1 with an explanation.

The proposed 10.1.22 release is:
[X] Broken - do not release
[ ] Stable - go ahead and release as 10.1.22


Unfortunately it seems broken.

Commit 6550e203a8119683f38c182cbfbb9b56587c54a0 introduced the use of 
--enable-native-access to the catalina script, which I think breaks 10.1 
on Java 11, since the flag was introduced later.


Best regards,

Rainer

-
To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org
For additional commands, e-mail: dev-h...@tomcat.apache.org

Re: (tomcat) branch 10.1.x updated: Add native access to the scripts

[Rainer Jung]

Hi Rémy,

I think this breaks 10.1 on Java 11, since --enable-native-access was 
introduced later.


Best regards,

Rainer

Am 22.03.24 um 14:13 schrieb r...@apache.org:

This is an automated email from the ASF dual-hosted git repository.

remm pushed a commit to branch 10.1.x
in repository https://gitbox.apache.org/repos/asf/tomcat.git


The following commit(s) were added to refs/heads/10.1.x by this push:
  new 6550e203a8 Add native access to the scripts
6550e203a8 is described below

commit 6550e203a8119683f38c182cbfbb9b56587c54a0
Author: remm 
AuthorDate: Fri Mar 22 13:33:27 2024 +0100

 Add native access to the scripts
---
  bin/catalina.bat | 1 +
  bin/catalina.sh  | 1 +
  2 files changed, 2 insertions(+)

diff --git a/bin/catalina.bat b/bin/catalina.bat
index 9c55ae940e..9a30371013 100755
--- a/bin/catalina.bat
+++ b/bin/catalina.bat
@@ -223,6 +223,7 @@ set "JAVA_OPTS=%JAVA_OPTS% 
--add-opens=java.base/java.io=ALL-UNNAMED"
  set "JAVA_OPTS=%JAVA_OPTS% --add-opens=java.base/java.util=ALL-UNNAMED"
  set "JAVA_OPTS=%JAVA_OPTS% 
--add-opens=java.base/java.util.concurrent=ALL-UNNAMED"
  set "JAVA_OPTS=%JAVA_OPTS% --add-opens=java.rmi/sun.rmi.transport=ALL-UNNAMED"
+set "JAVA_OPTS=%JAVA_OPTS% --enable-native-access=ALL-UNNAMED"
  
  rem - Execute The Requested Command ---
  
diff --git a/bin/catalina.sh b/bin/catalina.sh

index 32f87ffb6f..ed647a2dea 100755
--- a/bin/catalina.sh
+++ b/bin/catalina.sh
@@ -296,6 +296,7 @@ JAVA_OPTS="$JAVA_OPTS 
--add-opens=java.base/java.io=ALL-UNNAMED"
  JAVA_OPTS="$JAVA_OPTS --add-opens=java.base/java.util=ALL-UNNAMED"
  JAVA_OPTS="$JAVA_OPTS --add-opens=java.base/java.util.concurrent=ALL-UNNAMED"
  JAVA_OPTS="$JAVA_OPTS --add-opens=java.rmi/sun.rmi.transport=ALL-UNNAMED"
+JAVA_OPTS="$JAVA_OPTS --enable-native-access=ALL-UNNAMED"
  
  # - Execute The Requested Command -


-
To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org
For additional commands, e-mail: dev-h...@tomcat.apache.org

Re: Instruct the tomcat source builders before the build process for tomcat source 8.5.100

[Rainer Jung]

You need to look in the published (built) BUILDING.txt file, not the raw 
sources. The raw sources are meant to prouce a build, the yre not for 
human consumption.


You can e.g. download a published form og Tomcat 8.5.100 and look into 
BUILDING.txt there.


Regards,

Rainer

Am 22.03.24 um 13:01 schrieb Koteswararao Gundapaneni:



On Fri, Mar 22, 2024 at 4:09 AM Mark Thomas > wrote:


On 22/03/2024 10:54, Koteswararao Gundapaneni wrote:
 > Hi,
 >
 > While building the tomcat source 8.5.100 its giving an error with
java 7
 >
 > below is the error
 >
 > ANT_OPTS is set to  -Djava.security.manager=allow
 > Error occurred during initialization of VM
 > java.lang.UnsupportedClassVersionError: allow : Unsupported
major.minor
 > version 52.0
 >
 > in Building.txt file it has the following instruction
 >
 >   3. Install the JDK according to the instructions included with
the release.
 >
 > so if am not wrong, we could suggest java11 is required for the build
 > process in the instructions given

Yes, you are wrong.

PLEASE read the documentation first. Specifically this sentence from
BUILDING.txt


Download a version 11 or later of Java Development Kit...


Mark

-
To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org

For additional commands, e-mail: dev-h...@tomcat.apache.org


The above quote is  not available in BUILDING.txt:

FYI: added BUILDING.txt


-
To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org
For additional commands, e-mail: dev-h...@tomcat.apache.org

Re: Observability of virtual threads

[Rainer Jung]

Am 20.03.24 um 18:50 schrieb Romain Manni-Bucau:

Le mer. 20 mars 2024 à 18:43, Rainer Jung  a
écrit :


Am 20.03.24 um 18:34 schrieb Romain Manni-Bucau:

Hi Chris,

Thread dumps being dump of threads - literally os threads - and virtual
threads not being threads at all - they are runnables in a dedicated

thread

pool - it is quite fair to not make them the same and have their

scheduler

- pool - only in the thread dump but not themselves no?
Similarly to why threadlocal are not recommended for virtual thread this
would probably make an useless pressure on the JVM IMHO - why there is an
option to see it but it is mainly for debug purposes.
See virtual threads as continuations (suspendable/resumable "Runnable")

in

a dedicated and not programmatically configurable nor

selectable/proviable

thread pool, they are not in thread dumps and this doesnt bother you ;) -
ultimately if you want it you want all java objects to be monitored and

in

the thread dump which would be weird - but I agree the semantic is
misleading.


Nevertheless it would be helpful to knw, which virtual threads currently
have an associated OS thread or not, and for those having one, what
their stack is. And this not just as a dump into a file, but instead
available via a proper API. It might be worth to ask the JDK devs about
chanced to get such an API.



Hmm, I must miss something but virtual threads with an associated os thread
are in the thread dump since they run code (execution is in a normal
forkjoinpool).
So if you lock - synchronized/mutex for ex issue - then you see it as
before.


Did you actually try? It's my whole point: these threads are not 
contained in the usual thread dump outputs. Even when that virtual 
thread is on CPU. You can write a simple long running CPU hog and run it 
in a request. Every time you do a jstack or "kill -QUIT" this thread is 
not visible, ie. not shown in the dump. Every time you run a "jcmd PID 
Thread.dump_to_file FILE" the thread is part of the dump but without 
infos about locks held.


Example of the new format:

#32 "http-nio-8880-virt-0" virtual
  java.base/java.lang.StrictMath.atan(StrictMath.java:227)
  java.base/java.lang.Math.atan(Math.java:279)
  org.apache.jsp.cpuhog_jsp._jspService(cpuhog_jsp.java:154)
  org.apache.jasper.runtime.HttpJspBase.service(HttpJspBase.java:70)
  jakarta.servlet.http.HttpServlet.service(HttpServlet.java:716)
... 
org.apache.tomcat.util.net.SocketProcessorBase.run(SocketProcessorBase.java:52)

  java.base/java.lang.VirtualThread.run(VirtualThread.java:309)


That said for that they added a pinned system prop enabling to validate
your app is VT friendly or not which is very helpful.



  From my tests virtual threads do their job but they stay slower than a
proper reactive/async impl mainly due to the overhead they add

*everywhere*

compare to reactive programming plus this single thread pool issue which
adds a lot of contention when all the app/lot of tasks is/are done using

it

- vs bulkhead pattern for ex.
But if you come from a plain sync application it can be very interesting

if

it stays compatible and you don't need to add throttling to control the
memory - often in the old style you throttle with the number of threads,
now you need a semaphore or alike.
Will not be a free lunch ;).

Romain Manni-Bucau
@rmannibucau <https://twitter.com/rmannibucau> |  Blog
<https://rmannibucau.metawerx.net/> | Old Blog
<http://rmannibucau.wordpress.com> | Github <

https://github.com/rmannibucau> |

LinkedIn <https://www.linkedin.com/in/rmannibucau> | Book
<

https://www.packtpub.com/application-development/java-ee-8-high-performance




Le mer. 20 mars 2024 à 18:20, Christopher Schultz <
ch...@christopherschultz.net> a écrit :


Rainer,

Thanks for writing this up.

On 3/20/24 07:22, Rainer Jung wrote:

I wanted to share an observation and I hope the things are correct how

I

am describing them. Maybe things have already improved and I am not
aware of it, hints welcome.

Part of JEP 425 (Project Loom, Java virtual threads) discusses how to
handle observability of virtual threads from inside the JVM and

tooling.


The final outcome is, that virtual threads are not included in the
typical JVM APIs which one can use to observe threads, like enumerating
them or accessing the stacks of individual threads. As a consequence,
also jstack and "kill -QUIT" do not show virtual threads at all, not
even when they are attached to a native thread and executing code.


O_O


There is one single method to help with observability of virtual

threads






https://docs.oracle.com/en/java/javase/21/docs/api/jdk.management/com/sun/management/HotSpotDiagnosticMXBean.html#dumpThreads(java.lang.String,com.sun.management.HotSpotDiagnosticMXBean.ThreadDumpFormat)


which dumps a full thread dump to a file. Of course that is by no means
appropriate, if you want to

Re: Observability of virtual threads

[Rainer Jung]

Am 20.03.24 um 18:34 schrieb Romain Manni-Bucau:

Hi Chris,

Thread dumps being dump of threads - literally os threads - and virtual
threads not being threads at all - they are runnables in a dedicated thread
pool - it is quite fair to not make them the same and have their scheduler
- pool - only in the thread dump but not themselves no?
Similarly to why threadlocal are not recommended for virtual thread this
would probably make an useless pressure on the JVM IMHO - why there is an
option to see it but it is mainly for debug purposes.
See virtual threads as continuations (suspendable/resumable "Runnable") in
a dedicated and not programmatically configurable nor selectable/proviable
thread pool, they are not in thread dumps and this doesnt bother you ;) -
ultimately if you want it you want all java objects to be monitored and in
the thread dump which would be weird - but I agree the semantic is
misleading.


Nevertheless it would be helpful to knw, which virtual threads currently 
have an associated OS thread or not, and for those having one, what 
their stack is. And this not just as a dump into a file, but instead 
available via a proper API. It might be worth to ask the JDK devs about 
chanced to get such an API.



 From my tests virtual threads do their job but they stay slower than a
proper reactive/async impl mainly due to the overhead they add *everywhere*
compare to reactive programming plus this single thread pool issue which
adds a lot of contention when all the app/lot of tasks is/are done using it
- vs bulkhead pattern for ex.
But if you come from a plain sync application it can be very interesting if
it stays compatible and you don't need to add throttling to control the
memory - often in the old style you throttle with the number of threads,
now you need a semaphore or alike.
Will not be a free lunch ;).

Romain Manni-Bucau
@rmannibucau <https://twitter.com/rmannibucau> |  Blog
<https://rmannibucau.metawerx.net/> | Old Blog
<http://rmannibucau.wordpress.com> | Github <https://github.com/rmannibucau> |
LinkedIn <https://www.linkedin.com/in/rmannibucau> | Book
<https://www.packtpub.com/application-development/java-ee-8-high-performance>


Le mer. 20 mars 2024 à 18:20, Christopher Schultz <
ch...@christopherschultz.net> a écrit :


Rainer,

Thanks for writing this up.

On 3/20/24 07:22, Rainer Jung wrote:

I wanted to share an observation and I hope the things are correct how I
am describing them. Maybe things have already improved and I am not
aware of it, hints welcome.

Part of JEP 425 (Project Loom, Java virtual threads) discusses how to
handle observability of virtual threads from inside the JVM and tooling.

The final outcome is, that virtual threads are not included in the
typical JVM APIs which one can use to observe threads, like enumerating
them or accessing the stacks of individual threads. As a consequence,
also jstack and "kill -QUIT" do not show virtual threads at all, not
even when they are attached to a native thread and executing code.


O_O


There is one single method to help with observability of virtual threads



https://docs.oracle.com/en/java/javase/21/docs/api/jdk.management/com/sun/management/HotSpotDiagnosticMXBean.html#dumpThreads(java.lang.String,com.sun.management.HotSpotDiagnosticMXBean.ThreadDumpFormat)


which dumps a full thread dump to a file. Of course that is by no means
appropriate, if you want to do a fine grained observation. At least you
can choose to write a json structure instead of a hard to parse text
format, but that's it.


Boy, that's a real miss from the JVM team. I'm surprised that they have
overlooked this. I don't see a real reason that a Virtual Thread would
be treated any differently than a regular thread.


For instance I am often using a tool, that inspects our
RequestProcessors to find long running requests and then retrieves the
list of Java threads as ThreadInfo objects to find the one executing the
request and finally retrieves the stack of that request from the JVM.
Such an approach is no longer possible. Almost all of JMX does not show
any info about virtual threads.

It also seems Tomcat no longer uses a pool when a connector is
configured to use virtual threads. So there are no metrics like
currentThreadCount or currentThreadsBusy.


When using virtual threads, the number of requests is the same as the
number of threads, since each request -> new Virtual Thread though I
think the request-count is only incremented when the request has
completed, so maybe there are threads running that can't be counted (yet).

But I understand that you were hoping to get some information about
these threads and though maybe Tomcat's metrics could help. I guess not
really.

But thread-busy count is the same as in-flight-request count. And
current thread count is the same as in-flight-request count as well.


I guess this also limits the use 

Re: Observability of virtual threads

[Rainer Jung]

Am 20.03.24 um 18:17 schrieb Christopher Schultz:

Rainer,

Thanks for writing this up.

On 3/20/24 07:22, Rainer Jung wrote:
I wanted to share an observation and I hope the things are correct how 
I am describing them. Maybe things have already improved and I am not 
aware of it, hints welcome.


Part of JEP 425 (Project Loom, Java virtual threads) discusses how to 
handle observability of virtual threads from inside the JVM and tooling.


The final outcome is, that virtual threads are not included in the 
typical JVM APIs which one can use to observe threads, like 
enumerating them or accessing the stacks of individual threads. As a 
consequence, also jstack and "kill -QUIT" do not show virtual threads 
at all, not even when they are attached to a native thread and 
executing code.


O_O


There is one single method to help with observability of virtual threads

https://docs.oracle.com/en/java/javase/21/docs/api/jdk.management/com/sun/management/HotSpotDiagnosticMXBean.html#dumpThreads(java.lang.String,com.sun.management.HotSpotDiagnosticMXBean.ThreadDumpFormat)

which dumps a full thread dump to a file. Of course that is by no 
means appropriate, if you want to do a fine grained observation. At 
least you can choose to write a json structure instead of a hard to 
parse text format, but that's it.


Boy, that's a real miss from the JVM team. I'm surprised that they have 
overlooked this. I don't see a real reason that a Virtual Thread would 
be treated any differently than a regular thread.


The JEP hints at "since thre could be millions of virtual threads it 
makes no sense to support them in the existing JMX APIs". I don't by 
this extremely simplistic reasoning.


Furthermore, the "dump to file" method they provided does not contain 
any locking information you usually expect in a thread dump.


For instance I am often using a tool, that inspects our 
RequestProcessors to find long running requests and then retrieves the 
list of Java threads as ThreadInfo objects to find the one executing 
the request and finally retrieves the stack of that request from the 
JVM. Such an approach is no longer possible. Almost all of JMX does 
not show any info about virtual threads.


It also seems Tomcat no longer uses a pool when a connector is 
configured to use virtual threads. So there are no metrics like 
currentThreadCount or currentThreadsBusy.


When using virtual threads, the number of requests is the same as the 
number of threads, since each request -> new Virtual Thread though I 
think the request-count is only incremented when the request has 
completed, so maybe there are threads running that can't be counted (yet).


But I understand that you were hoping to get some information about 
these threads and though maybe Tomcat's metrics could help. I guess not 
really.


But thread-busy count is the same as in-flight-request count. And 
current thread count is the same as in-flight-request count as well.


I guess this also limits the use of some manager features and probably 
also the StuckThreadsDetectionValve when combined with virtual threads.


Most likely. I'm fairly sure that uses the "usual Thread-walker things" 
which you are reporting do not work any more with VTs.


What JVM are you using for your investigations?


Zulu 21, but I expect the other OpenJDK variants to behave the same way. 
It mostly comes out of the JEP.


Of course Tomcat has solved most of the problems, that virtual threads 
want to solve, by doing the hard work of using the NIO and NIO2 APIs. 
So virtual threads are probably not that important for us. But we 
should be aware of the observability deficiencies whenever we would 
discuss whether we could switch from NIO or NIO2 to virtual threads to 
simplify connector code in some distant future.


+1

I've been enthusiastically talking with markt about dropping all that 
nasty NIO stuff and "going back to BIO with virtual threads" which, at 
this point, is mostly a threat and not a promise. But if VTs really 
deliver everything they are claiming to deliver, then it may be possible 
to go back to BIO as long as servlet-async is retired. And I'm not 
holding my breath on that one.


There was an interesting talk at FOSDEM about the current limitations of 
the virtual thread implementation. They have several situations where 
they switch over to blocking. I think for instance file I/O and also 
some types of locks they can not yet handle well.


They have some improvements in the pipeline, but I think one would at 
least wait for those. And hopefully they improve the observability in 
the meantime.


I wish I had known this type of problem at FOSDEM so I could have 
directly talked to the OpenJDK devs there.


Thanks and regards,

Rainer

-
To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org
For additional commands, e-mail: dev-h...@tomcat.apache.org

Re: Observability of virtual threads

[Rainer Jung]

Thanks Romain,

I am well aware of the limits oof this attempt concerning async 
processing. Still lots of applications can still be diagnosed with this 
methodology and I am a bit frustrated, that project loom simply dropped 
support for thread inspection via JMX and as a replacement provided a 
full dump to a file (!).


But of course you are right: for complex async processing this approach 
does not work. Interestingly although Tomcat NIO and NIO2 processors use 
async APIs, they can still be inspected with this simple approach.


Best regards,

Rainer

Am 20.03.24 um 13:58 schrieb Romain Manni-Bucau:

Hi Rainer,

Think this is not related to virtual threads I think and way older (2010?).
Since some years we use AsyncContext and therefore thread dumps don't match
pending requests but just active computation (you can get 8K requests with
8 threads for ex).
Most of the sync instrumentation (assumptions) are broken since Servlet 3.0
and must not be used to conclude anything but generally other metrics and
data enable to monitor the server properly (request counter for example is
decoralated from threads, active connections for database - generic term -
etc).
Thread stacks stay useful for performance related work but not for hanging
investigation (for good since it enables a better scalability in general)
so guess it is mainly a habit switch in terms of tools, no?

Hope it makes sense.

Romain Manni-Bucau
@rmannibucau <https://twitter.com/rmannibucau> |  Blog
<https://rmannibucau.metawerx.net/> | Old Blog
<http://rmannibucau.wordpress.com> | Github <https://github.com/rmannibucau> |
LinkedIn <https://www.linkedin.com/in/rmannibucau> | Book
<https://www.packtpub.com/application-development/java-ee-8-high-performance>


Le mer. 20 mars 2024 à 13:11, Rainer Jung  a
écrit :


Hi hi,

I wanted to share an observation and I hope the things are correct how I
am describing them. Maybe things have already improved and I am not
aware of it, hints welcome.

Part of JEP 425 (Project Loom, Java virtual threads) discusses how to
handle observability of virtual threads from inside the JVM and tooling.

The final outcome is, that virtual threads are not included in the
typical JVM APIs which one can use to observe threads, like enumerating
them or accessing the stacks of individual threads. As a consequence,
also jstack and "kill -QUIT" do not show virtual threads at all, not
even when they are attached to a native thread and executing code.

There is one single method to help with observability of virtual threads


https://docs.oracle.com/en/java/javase/21/docs/api/jdk.management/com/sun/management/HotSpotDiagnosticMXBean.html#dumpThreads(java.lang.String,com.sun.management.HotSpotDiagnosticMXBean.ThreadDumpFormat)

which dumps a full thread dump to a file. Of course that is by no means
appropriate, if you want to do a fine grained observation. At least you
can choose to write a json structure instead of a hard to parse text
format, but that's it.

For instance I am often using a tool, that inspects our
RequestProcessors to find long running requests and then retrieves the
list of Java threads as ThreadInfo objects to find the one executing the
request and finally retrieves the stack of that request from the JVM.
Such an approach is no longer possible. Almost all of JMX does not show
any info about virtual threads.

It also seems Tomcat no longer uses a pool when a connector is
configured to use virtual threads. So there are no metrics like
currentThreadCount or currentThreadsBusy.

I guess this also limits the use of some manager features and probably
also the StuckThreadsDetectionValve when combined with virtual threads.

Of course Tomcat has solved most of the problems, that virtual threads
want to solve, by doing the hard work of using the NIO and NIO2 APIs. So
virtual threads are probably not that important for us. But we should be
aware of the observability deficiencies whenever we would discuss
whether we could switch from NIO or NIO2 to virtual threads to simplify
connector code in some distant future.

Best regards,

Rainer


-
To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org
For additional commands, e-mail: dev-h...@tomcat.apache.org

Observability of virtual threads

[Rainer Jung]

Hi hi,

I wanted to share an observation and I hope the things are correct how I 
am describing them. Maybe things have already improved and I am not 
aware of it, hints welcome.


Part of JEP 425 (Project Loom, Java virtual threads) discusses how to 
handle observability of virtual threads from inside the JVM and tooling.


The final outcome is, that virtual threads are not included in the 
typical JVM APIs which one can use to observe threads, like enumerating 
them or accessing the stacks of individual threads. As a consequence, 
also jstack and "kill -QUIT" do not show virtual threads at all, not 
even when they are attached to a native thread and executing code.


There is one single method to help with observability of virtual threads

https://docs.oracle.com/en/java/javase/21/docs/api/jdk.management/com/sun/management/HotSpotDiagnosticMXBean.html#dumpThreads(java.lang.String,com.sun.management.HotSpotDiagnosticMXBean.ThreadDumpFormat)

which dumps a full thread dump to a file. Of course that is by no means 
appropriate, if you want to do a fine grained observation. At least you 
can choose to write a json structure instead of a hard to parse text 
format, but that's it.


For instance I am often using a tool, that inspects our 
RequestProcessors to find long running requests and then retrieves the 
list of Java threads as ThreadInfo objects to find the one executing the 
request and finally retrieves the stack of that request from the JVM. 
Such an approach is no longer possible. Almost all of JMX does not show 
any info about virtual threads.


It also seems Tomcat no longer uses a pool when a connector is 
configured to use virtual threads. So there are no metrics like 
currentThreadCount or currentThreadsBusy.


I guess this also limits the use of some manager features and probably 
also the StuckThreadsDetectionValve when combined with virtual threads.


Of course Tomcat has solved most of the problems, that virtual threads 
want to solve, by doing the hard work of using the NIO and NIO2 APIs. So 
virtual threads are probably not that important for us. But we should be 
aware of the observability deficiencies whenever we would discuss 
whether we could switch from NIO or NIO2 to virtual threads to simplify 
connector code in some distant future.


Best regards,

Rainer

-
To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org
For additional commands, e-mail: dev-h...@tomcat.apache.org

Re: Very sporadic test failures in org.apache.catalina.core.TestAsyncContextImpl for TC 8.5 and 9

[Rainer Jung]

Thanks again, I will keep an eye on it.

Am 15.03.24 um 20:43 schrieb Mark Thomas:

On 08/03/2024 20:28, Rainer Jung wrote:
Again, this is not a new failure and it happens extremely infrequent. 
Observed with JSSE on various Linux distributions and with various JVM 
versions and vendors. Only observed for TC 8.5 and 9, but for 10.1 and 
11 the unit tests are run with much fewer JVM combinations so I can 
not exclude the problem might happen there as well.


Looks like a timing issue when we have a non-container thread still 
accessing the AsyncContext (it shouldn't do that).


The test acknowledged the problem but you found another fisalure mode 
that wasn't handled. It now is. Hopefully, these tests should be more 
reliable going forwards.


Mark




This info is not intended to prevent a release.

The typical situation is:

Testcase: testBug49567 took 21.162 sec
 FAILED
expected:<...alse2true3true4true5[false]> but 
was:<...alse2true3true4true5[]>
junit.framework.AssertionFailedError: 
expected:<...alse2true3true4true5[false]> but 
was:<...alse2true3true4true5[]>
 at 
org.apache.catalina.core.TestAsyncContextImpl.testBug49567(TestAsyncContextImpl.java:163)
 at 
java.base/jdk.internal.reflect.DirectMethodHandleAccessor.invoke(DirectMethodHandleAccessor.java:103)


The execution log contains:

22-Feb-2024 11:19:03.617 INFO [main] 
org.apache.catalina.startup.LoggingBaseTest.setUp Starting test case 
[testBug49567]
22-Feb-2024 11:19:03.626 INFO [main] 
org.apache.coyote.AbstractProtocol.init Initializing ProtocolHandler 
["http-nio-127.0.0.1-auto-11"]
22-Feb-2024 11:19:03.627 INFO [main] 
org.apache.catalina.core.StandardService.startInternal Starting 
service [Tomcat]
22-Feb-2024 11:19:03.627 INFO [main] 
org.apache.catalina.core.StandardEngine.startInternal Starting Servlet 
engine: [Apache Tomcat/9.0.86]
22-Feb-2024 11:19:03.640 INFO [main] 
org.apache.coyote.AbstractProtocol.start Starting ProtocolHandler 
["http-nio-127.0.0.1-auto-11-46529"]
Exception in thread "Thread-9" java.lang.NullPointerException: Cannot 
invoke "org.apache.catalina.connector.Request.getCoyoteRequest()" 
because "this.request" is null
 at 
org.apache.catalina.core.AsyncContextImpl.isStarted(AsyncContextImpl.java:295)
 at 
org.apache.catalina.connector.Request.isAsyncStarted(Request.java:1715)
 at 
org.apache.catalina.connector.RequestFacade.isAsyncStarted(RequestFacade.java:749)
 at 
org.apache.catalina.core.TestAsyncContextImpl$Bug49567Servlet$1$1.run(TestAsyncContextImpl.java:383)

 at java.base/java.lang.Thread.run(Thread.java:1583)
22-Feb-2024 11:19:24.685 INFO [main] 
org.apache.coyote.AbstractProtocol.pause Pausing ProtocolHandler 
["http-nio-127.0.0.1-auto-11-46529"]
22-Feb-2024 11:19:24.693 INFO [main] 
org.apache.catalina.core.StandardService.stopInternal Stopping service 
[Tomcat]
22-Feb-2024 11:19:24.729 INFO [main] 
org.apache.coyote.AbstractProtocol.stop Stopping ProtocolHandler 
["http-nio-127.0.0.1-auto-11-46529"]
22-Feb-2024 11:19:24.739 INFO [main] 
org.apache.coyote.AbstractProtocol.destroy Destroying ProtocolHandler 
["http-nio-127.0.0.1-auto-11-46529"]



or as a variation:

08-Mar-2024 14:44:59.132 INFO [main] 
org.apache.catalina.startup.LoggingBaseTest.setUp Starting test case 
[testBug49528]
08-Mar-2024 14:44:59.141 INFO [main] 
org.apache.coyote.AbstractProtocol.init Initializing ProtocolHandler 
["http-nio2-127.0.0.1-auto-10"]
08-Mar-2024 14:44:59.151 INFO [main] 
org.apache.catalina.core.StandardService.startInternal Starting 
service [Tomcat]
08-Mar-2024 14:44:59.151 INFO [main] 
org.apache.catalina.core.StandardEngine.startInternal Starting Servlet 
engine: [Apache Tomcat/9.0.86sp1]
08-Mar-2024 14:44:59.170 INFO [main] 
org.apache.coyote.AbstractProtocol.start Starting ProtocolHandler 
["http-nio2-127.0.0.1-auto-10-37101"]
java.lang.NullPointerException: Cannot invoke 
"org.apache.catalina.connector.Request.getCoyoteRequest()" because 
"this.request" is null
 at 
org.apache.catalina.core.AsyncContextImpl.isStarted(AsyncContextImpl.java:295)
 at 
org.apache.catalina.connector.Request.isAsyncStarted(Request.java:1715)
 at 
org.apache.catalina.connector.RequestFacade.isAsyncStarted(RequestFacade.java:749)
 at 
org.apache.catalina.core.TestAsyncContextImpl$Bug49528Servlet$1.run(TestAsyncContextImpl.java:303)
 at 
org.apache.catalina.core.AsyncContextImpl$RunnableWrapper.run(AsyncContextImpl.java:543)
 at 
org.apache.tomcat.util.threads.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1191)
 at 
org.apache.tomcat.util.threads.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:659)
 at 
org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:63)

 at java.base/java.lang.Thread.run(Thread.j

Very sporadic test failures in org.apache.catalina.core.TestAsyncContextImpl for TC 8.5 and 9

[Rainer Jung]

Again, this is not a new failure and it happens extremely infrequent. 
Observed with JSSE on various Linux distributions and with various JVM 
versions and vendors. Only observed for TC 8.5 and 9, but for 10.1 and 
11 the unit tests are run with much fewer JVM combinations so I can not 
exclude the problem might happen there as well.


This info is not intended to prevent a release.

The typical situation is:

Testcase: testBug49567 took 21.162 sec
FAILED
expected:<...alse2true3true4true5[false]> but 
was:<...alse2true3true4true5[]>
junit.framework.AssertionFailedError: 
expected:<...alse2true3true4true5[false]> but 
was:<...alse2true3true4true5[]>
at 
org.apache.catalina.core.TestAsyncContextImpl.testBug49567(TestAsyncContextImpl.java:163)
at 
java.base/jdk.internal.reflect.DirectMethodHandleAccessor.invoke(DirectMethodHandleAccessor.java:103)


The execution log contains:

22-Feb-2024 11:19:03.617 INFO [main] 
org.apache.catalina.startup.LoggingBaseTest.setUp Starting test case 
[testBug49567]
22-Feb-2024 11:19:03.626 INFO [main] 
org.apache.coyote.AbstractProtocol.init Initializing ProtocolHandler 
["http-nio-127.0.0.1-auto-11"]
22-Feb-2024 11:19:03.627 INFO [main] 
org.apache.catalina.core.StandardService.startInternal Starting service 
[Tomcat]
22-Feb-2024 11:19:03.627 INFO [main] 
org.apache.catalina.core.StandardEngine.startInternal Starting Servlet 
engine: [Apache Tomcat/9.0.86]
22-Feb-2024 11:19:03.640 INFO [main] 
org.apache.coyote.AbstractProtocol.start Starting ProtocolHandler 
["http-nio-127.0.0.1-auto-11-46529"]
Exception in thread "Thread-9" java.lang.NullPointerException: Cannot 
invoke "org.apache.catalina.connector.Request.getCoyoteRequest()" 
because "this.request" is null
at 
org.apache.catalina.core.AsyncContextImpl.isStarted(AsyncContextImpl.java:295)
at 
org.apache.catalina.connector.Request.isAsyncStarted(Request.java:1715)
at 
org.apache.catalina.connector.RequestFacade.isAsyncStarted(RequestFacade.java:749)
at 
org.apache.catalina.core.TestAsyncContextImpl$Bug49567Servlet$1$1.run(TestAsyncContextImpl.java:383)

at java.base/java.lang.Thread.run(Thread.java:1583)
22-Feb-2024 11:19:24.685 INFO [main] 
org.apache.coyote.AbstractProtocol.pause Pausing ProtocolHandler 
["http-nio-127.0.0.1-auto-11-46529"]
22-Feb-2024 11:19:24.693 INFO [main] 
org.apache.catalina.core.StandardService.stopInternal Stopping service 
[Tomcat]
22-Feb-2024 11:19:24.729 INFO [main] 
org.apache.coyote.AbstractProtocol.stop Stopping ProtocolHandler 
["http-nio-127.0.0.1-auto-11-46529"]
22-Feb-2024 11:19:24.739 INFO [main] 
org.apache.coyote.AbstractProtocol.destroy Destroying ProtocolHandler 
["http-nio-127.0.0.1-auto-11-46529"]



or as a variation:

08-Mar-2024 14:44:59.132 INFO [main] 
org.apache.catalina.startup.LoggingBaseTest.setUp Starting test case 
[testBug49528]
08-Mar-2024 14:44:59.141 INFO [main] 
org.apache.coyote.AbstractProtocol.init Initializing ProtocolHandler 
["http-nio2-127.0.0.1-auto-10"]
08-Mar-2024 14:44:59.151 INFO [main] 
org.apache.catalina.core.StandardService.startInternal Starting service 
[Tomcat]
08-Mar-2024 14:44:59.151 INFO [main] 
org.apache.catalina.core.StandardEngine.startInternal Starting Servlet 
engine: [Apache Tomcat/9.0.86sp1]
08-Mar-2024 14:44:59.170 INFO [main] 
org.apache.coyote.AbstractProtocol.start Starting ProtocolHandler 
["http-nio2-127.0.0.1-auto-10-37101"]
java.lang.NullPointerException: Cannot invoke 
"org.apache.catalina.connector.Request.getCoyoteRequest()" because 
"this.request" is null
at 
org.apache.catalina.core.AsyncContextImpl.isStarted(AsyncContextImpl.java:295)
at 
org.apache.catalina.connector.Request.isAsyncStarted(Request.java:1715)
at 
org.apache.catalina.connector.RequestFacade.isAsyncStarted(RequestFacade.java:749)
at 
org.apache.catalina.core.TestAsyncContextImpl$Bug49528Servlet$1.run(TestAsyncContextImpl.java:303)
at 
org.apache.catalina.core.AsyncContextImpl$RunnableWrapper.run(AsyncContextImpl.java:543)
at 
org.apache.tomcat.util.threads.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1191)
at 
org.apache.tomcat.util.threads.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:659)
at 
org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:63)

at java.base/java.lang.Thread.run(Thread.java:1583)
08-Mar-2024 14:45:00.196 INFO [main] 
org.apache.coyote.AbstractProtocol.pause Pausing ProtocolHandler 
["http-nio2-127.0.0.1-auto-10-37101"]
08-Mar-2024 14:45:00.198 INFO [main] 
org.apache.catalina.core.StandardService.stopInternal Stopping service 
[Tomcat]
08-Mar-2024 14:45:00.217 INFO [main] 
org.apache.coyote.AbstractProtocol.stop Stopping ProtocolHandler 
["http-nio2-127.0.0.1-auto-10-37101"]
08-Mar-2024 14:45:00.219 INFO [main] 
org.apache.coyote.AbstractProtocol.destroy Destroying ProtocolHandler 
["http-nio2-127.0.0.1-auto-10-37101"]



In addition to these failures I also

Re: Sporadic Failures in org.apache.coyote.http2.TestRfc9218

[Rainer Jung]

Am 07.03.24 um 21:51 schrieb Rainer Jung:

Am 07.03.24 um 18:50 schrieb Mark Thomas:

On 07/03/2024 11:20, Rainer Jung wrote:
As always thanks a lot for the investigation and forthcoming fix. Let 
me know when I can help!


I've committed a fix that worked for me locally but I was focring the 
test failure via the debugger. I'm fairly sure this is the issue you 
were seeing but iof you are able to re-run your tests that would be 
very helpful.


Looks good to me. I patched 9.0.86 and ran the tests 700 times for NIO 
plus NIO2 using Oracke JDK 1.8.0_402 and JSSE on RHEL 9 without failures.


Same setup showed 4 failures in 225 runs. So this looks quite good.

(failures for the original unpatched core)


Great that the improvement was actually in the HTTP/2 code and not just 
a test improvement.


Thanks a bunch,

Rainer


-
To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org
For additional commands, e-mail: dev-h...@tomcat.apache.org

Re: Sporadic Failures in org.apache.coyote.http2.TestRfc9218

[Rainer Jung]

Am 07.03.24 um 18:50 schrieb Mark Thomas:

On 07/03/2024 11:20, Rainer Jung wrote:
As always thanks a lot for the investigation and forthcoming fix. Let 
me know when I can help!


I've committed a fix that worked for me locally but I was focring the 
test failure via the debugger. I'm fairly sure this is the issue you 
were seeing but iof you are able to re-run your tests that would be very 
helpful.


Looks good to me. I patched 9.0.86 and ran the tests 700 times for NIO 
plus NIO2 using Oracke JDK 1.8.0_402 and JSSE on RHEL 9 without failures.


Same setup showed 4 failures in 225 runs. So this looks quite good.

Great that the improvement was actually in the HTTP/2 code and not just 
a test improvement.


Thanks a bunch,

Rainer

-
To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org
For additional commands, e-mail: dev-h...@tomcat.apache.org

Re: Sporadic Failures in org.apache.coyote.http2.TestRfc9218

[Rainer Jung]

As always thanks a lot for the investigation and forthcoming fix. Let me 
know when I can help!


Am 07.03.24 um 16:55 schrieb Mark Thomas:

On 02/03/2024 05:58, Rainer Jung wrote:

Hi there,

for a long time I get sporadic failures in 
org.apache.coyote.http2.TestRfc9218. It happens for all branches of 
TC, for NIO and NIO2 and with JSSE and tcnative. Tested platforms 
Linux and Solaris see this. And JVM versions 1.8.0, 11, 17 and 21 show 
it. It is also not restricted to a specific JDK vendor.


Rainer,

Thanks for raising this. I have worked out what the root cause is. 
Esentially, it is a timing issue.


When stream 17 does its write, it writes 1k of the 8k response. Before 
it tries to write the remaining 7k, streams 19 and/or 21 attempt to 
write 8k, the connection window is extended by a further 1k and that 1k 
is allocated to stream 19 or 21 depending on which streams have 
attempted to write the 8k body by then.


I have a fix in mind which involves adding stream 17 to the backlog as 
soon as it attempts to write more than the connection window will allow. 
I need to do some more work on this as I am seeing some test failures 
with my first attempt at a pacth.


Mark


-
To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org
For additional commands, e-mail: dev-h...@tomcat.apache.org

Sporadic Failures in org.apache.coyote.http2.TestRfc9218

[Rainer Jung]

Hi there,

for a long time I get sporadic failures in 
org.apache.coyote.http2.TestRfc9218. It happens for all branches of TC, 
for NIO and NIO2 and with JSSE and tcnative. Tested platforms Linux and 
Solaris see this. And JVM versions 1.8.0, 11, 17 and 21 show it. It is 
also not restricted to a specific JDK vendor.


It does not happen frequently. For instance I tried running only this 
test in a loop and in 225 test runs for NIO+NIO2, so 450 overall test 
executions, I got 4 failing executions.


The typical failure is like this:

Testcase: testPriority[1: loop [0], useAsyncIO[true]] took 0.309 sec
FAILED
expected:<1[7]-Body-1024
> but was:<1[9]-Body-1024
>
junit.framework.AssertionFailedError: expected:<1[7]-Body-1024
> but was:<1[9]-Body-1024
>
at 
org.apache.coyote.http2.TestRfc9218.testPriority(TestRfc9218.java:80)


but there are also other variations (even more rare) like

Testcase: testPriority[1: loop [0], useAsyncIO[true]] took 0.305 sec
FAILED
null
junit.framework.AssertionFailedError
at 
org.apache.coyote.http2.TestRfc9218.testPriority(TestRfc9218.java:170)



Is there something I can do to provide more information, add more 
logging or so? Full current log output see below.


Thanks and regards,

Rainer

Testsuite: org.apache.coyote.http2.TestRfc9218
Tests run: 2, Failures: 1, Errors: 0, Skipped: 0, Time elapsed: 3.061 sec
- Standard Output ---
3-HeadersStart
3-Header-[:status]-[200]
3-Header-[content-type]-[application/octet-stream]
3-Header-[content-length]-[8192]
3-Header-[date]-[Wed, 11 Nov 2015 19:18:42 GMT]
3-HeadersEnd
3-Body-8192
3-EndOfStream
5-HeadersStart
5-Header-[:status]-[200]
5-Header-[content-type]-[application/octet-stream]
5-Header-[content-length]-[8192]
5-Header-[date]-[Wed, 11 Nov 2015 19:18:42 GMT]
5-HeadersEnd
5-Body-8192
5-EndOfStream
7-HeadersStart
7-Header-[:status]-[200]
7-Header-[content-type]-[application/octet-stream]
7-Header-[content-length]-[8192]
7-Header-[date]-[Wed, 11 Nov 2015 19:18:42 GMT]
7-HeadersEnd
7-Body-8192
7-EndOfStream
9-HeadersStart
9-Header-[:status]-[200]
9-Header-[content-type]-[application/octet-stream]
9-Header-[content-length]-[8192]
9-Header-[date]-[Wed, 11 Nov 2015 19:18:42 GMT]
9-HeadersEnd
9-Body-8192
9-EndOfStream
11-HeadersStart
11-Header-[:status]-[200]
11-Header-[content-type]-[application/octet-stream]
11-Header-[content-length]-[8192]
11-Header-[date]-[Wed, 11 Nov 2015 19:18:42 GMT]
11-HeadersEnd
11-Body-8192
11-EndOfStream
13-HeadersStart
13-Header-[:status]-[200]
13-Header-[content-type]-[application/octet-stream]
13-Header-[content-length]-[8192]
13-Header-[date]-[Wed, 11 Nov 2015 19:18:42 GMT]
13-HeadersEnd
13-Body-8192
13-EndOfStream
15-HeadersStart
15-Header-[:status]-[200]
15-Header-[content-type]-[application/octet-stream]
15-Header-[content-length]-[8192]
15-Header-[date]-[Wed, 11 Nov 2015 19:18:42 GMT]
15-HeadersEnd
15-Body-8192
15-EndOfStream

17-HeadersStart
17-Header-[:status]-[200]
17-Header-[content-type]-[application/octet-stream]
17-Header-[content-length]-[8192]
17-Header-[date]-[Wed, 11 Nov 2015 19:18:42 GMT]
17-HeadersEnd
17-Body-1024

19-HeadersStart
19-Header-[:status]-[200]
19-Header-[content-type]-[application/octet-stream]
19-Header-[content-length]-[8192]
19-Header-[date]-[Wed, 11 Nov 2015 19:18:42 GMT]
19-HeadersEnd
21-HeadersStart
21-Header-[:status]-[200]
21-Header-[content-type]-[application/octet-stream]
21-Header-[content-length]-[8192]
21-Header-[date]-[Wed, 11 Nov 2015 19:18:42 GMT]
21-HeadersEnd

21-Body-1365
17-Body-5266
17-EndOfStream
19-Body-1560

3-HeadersStart
3-Header-[:status]-[200]
3-Header-[content-type]-[application/octet-stream]
3-Header-[content-length]-[8192]
3-Header-[date]-[Wed, 11 Nov 2015 19:18:42 GMT]
3-HeadersEnd
3-Body-8192
3-EndOfStream
5-HeadersStart
5-Header-[:status]-[200]
5-Header-[content-type]-[application/octet-stream]
5-Header-[content-length]-[8192]
5-Header-[date]-[Wed, 11 Nov 2015 19:18:42 GMT]
5-HeadersEnd
5-Body-8192
5-EndOfStream
7-HeadersStart
7-Header-[:status]-[200]
7-Header-[content-type]-[application/octet-stream]
7-Header-[content-length]-[8192]
7-Header-[date]-[Wed, 11 Nov 2015 19:18:42 GMT]
7-HeadersEnd
7-Body-8192
7-EndOfStream
9-HeadersStart
9-Header-[:status]-[200]
9-Header-[content-type]-[application/octet-stream]
9-Header-[content-length]-[8192]
9-Header-[date]-[Wed, 11 Nov 2015 19:18:42 GMT]
9-HeadersEnd
9-Body-8192
9-EndOfStream
11-HeadersStart
11-Header-[:status]-[200]
11-Header-[content-type]-[application/octet-stream]
11-Header-[content-length]-[8192]
11-Header-[date]-[Wed, 11 Nov 2015 19:18:42 GMT]
11-HeadersEnd
11-Body-8192
11-EndOfStream
13-HeadersStart
13-Header-[:status]-[200]
13-Header-[content-type]-[application/octet-stream]
13-Header-[content-length]-[8192]
13-Header-[date]-[Wed, 11 Nov 2015 19:18:42 GMT]
13-HeadersEnd
13-Body-8192
13-EndOfStream
15-HeadersStart
15-Header-[:status]-[200]
15-Header-[content-type]-[application/octet-stream]
15-Header-[content-length]-[8192]
15-He

Re: TestSsl.testClientInitiatedRenegotiation fails for TC 9 and 8.5

[Rainer Jung]

Thank you Remy for fixing it and the explanation.

Am 27.02.24 um 15:33 schrieb Rémy Maucherat:

On Tue, Feb 27, 2024 at 1:32 PM Rainer Jung  wrote:


Hi Mark, hi all,

coming back to this: the reason, why you could not reproduce is, that I
explicitly set

test.sslImplementation=org.apache.tomcat.util.net.jsse.JSSEImplementation

when testing JSSE. It is not necessary but IMHO it should be supported.

The failure was probably triggered by
1064d250c9463d3604d14de6229cf9edd4f04710 changing class
test/org/apache/tomcat/util/net/TestSsl.java in combination with a
difference in TesterSupport between 9.0.x and 10.1.x.

In 9.0.x (and probably 8.5.x) in
test/org/apache/tomcat/util/net/TesterSupport.java method
isClientRenegotiationSupported contains:

...
240 String sslImplementation =
System.getProperty("tomcat.test.sslImplementation");
241 if (sslImplementation != null &&
!"${test.sslImplementation}".equals(sslImplementation)) {
242 // Assume custom SSL is not supporting this
243 return false;
244 }
...

In 10.1.x:

...
  // Disabled for Tomcat Native (part of response to CVE-2009-3555)
  // Only JRE provided JSSE implementation supports this
  String sslImplementation = (String)
tomcat.getConnector().getProperty("sslImplementationName");
  if
(!JSSEImplementation.class.getName().equals(sslImplementation)) {
  return false;
  }
...

So for 9.x setting JSSE explicitly result in returning false. In 10.1.x
the check !JSSEImplementation.class.getName().equals(sslImplementation)
makes sure it is not returning false.

Unfortunately taking over the code from 10.1.x to 9.x does not work,
because at this point
tomcat.getConnector().getProperty("sslImplementationName") returns null.

Possible fix:

--- a/test/org/apache/tomcat/util/net/TesterSupport.java
+++ b/test/org/apache/tomcat/util/net/TesterSupport.java
@@ -64,6 +64,7 @@ import org.apache.tomcat.util.descriptor.web.LoginConfig;
   import org.apache.tomcat.util.descriptor.web.SecurityCollection;
   import org.apache.tomcat.util.descriptor.web.SecurityConstraint;
   import org.apache.tomcat.util.net.SSLHostConfigCertificate.Type;
+import org.apache.tomcat.util.net.jsse.JSSEImplementation;

   public final class TesterSupport {

@@ -238,7 +239,8 @@ public final class TesterSupport {
   return false;
   }
   String sslImplementation =
System.getProperty("tomcat.test.sslImplementation");
-if (sslImplementation != null &&
!"${test.sslImplementation}".equals(sslImplementation)) {
+if (sslImplementation != null &&
!"${test.sslImplementation}".equals(sslImplementation) &&
+
!JSSEImplementation.class.getName().equals(sslImplementation)) {
   // Assume custom SSL is not supporting this
   return false;
   }


Not sure this is the right fix, or it would be better to use a change
that is closer to 10.1.x and find out, why
tomcat.getConnector().getProperty("sslImplementationName") is null
inside isClientRenegotiationSupported.


Ok so I picked up your change.
Things are different in 10.1 because the tests are parametrized.
Actually when you run the test, for example TestSsl, it always does
the same tests regardless of what you configure (so JSSE, OpenSSL,
OpenSSL-FFM). This is mostly due to the APR connector being removed.

Rémy


Best regards,

Rainer

Am 18.01.24 um 21:48 schrieb Mark Thomas:

On 18/01/2024 12:33, Rainer Jung wrote:

Hi all,

after the refactorings for the testing of the forbidden client
initiated renegotiations, these unit tests fail for me for the last
tags of TC 8.5 and 9, but not for 10.1 and 11. I am using JSSE and the
tests fail consistently for all four JDK vendors I am testing against
on all linux distributions I am testing on. I saw it for Java 8 and
11. No info yet about 17 and 21, the test runs will arrive at Java 17
later today.


Very odd. I just ran the 9.0.x test with Java 8 / JSSE and it passed.


Example output (no indication for a reason):






Testcase: testClientInitiatedRenegotiation took 0.383 sec
  FAILED
Renegotiation started when it should have failed
junit.framework.AssertionFailedError: Renegotiation started when it
should have failed
  at
org.apache.tomcat.util.net.TestSsl.testClientInitiatedRenegotiation(TestSsl.java:250)
  at
java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native
Method)
  at
java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
  at
java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)


Odd. With JSSE and NIO, client initiated renegotiation should work.

I can't recreate this. If you can recreate this in a debugger then I'd
look at TesterSupport.isClientRenegotiationSupported

Re: TestSsl.testClientInitiatedRenegotiation fails for TC 9 and 8.5

[Rainer Jung]

Hi Mark, hi all,

coming back to this: the reason, why you could not reproduce is, that I 
explicitly set


test.sslImplementation=org.apache.tomcat.util.net.jsse.JSSEImplementation

when testing JSSE. It is not necessary but IMHO it should be supported.

The failure was probably triggered by 
1064d250c9463d3604d14de6229cf9edd4f04710 changing class 
test/org/apache/tomcat/util/net/TestSsl.java in combination with a 
difference in TesterSupport between 9.0.x and 10.1.x.


In 9.0.x (and probably 8.5.x) in 
test/org/apache/tomcat/util/net/TesterSupport.java method 
isClientRenegotiationSupported contains:


...
240 String sslImplementation = 
System.getProperty("tomcat.test.sslImplementation");
241 if (sslImplementation != null && 
!"${test.sslImplementation}".equals(sslImplementation)) {

242 // Assume custom SSL is not supporting this
243 return false;
244 }
...

In 10.1.x:

...
// Disabled for Tomcat Native (part of response to CVE-2009-3555)
// Only JRE provided JSSE implementation supports this
String sslImplementation = (String) 
tomcat.getConnector().getProperty("sslImplementationName");
if 
(!JSSEImplementation.class.getName().equals(sslImplementation)) {

return false;
}
...

So for 9.x setting JSSE explicitly result in returning false. In 10.1.x 
the check !JSSEImplementation.class.getName().equals(sslImplementation) 
makes sure it is not returning false.


Unfortunately taking over the code from 10.1.x to 9.x does not work, 
because at this point 
tomcat.getConnector().getProperty("sslImplementationName") returns null.


Possible fix:

--- a/test/org/apache/tomcat/util/net/TesterSupport.java
+++ b/test/org/apache/tomcat/util/net/TesterSupport.java
@@ -64,6 +64,7 @@ import org.apache.tomcat.util.descriptor.web.LoginConfig;
 import org.apache.tomcat.util.descriptor.web.SecurityCollection;
 import org.apache.tomcat.util.descriptor.web.SecurityConstraint;
 import org.apache.tomcat.util.net.SSLHostConfigCertificate.Type;
+import org.apache.tomcat.util.net.jsse.JSSEImplementation;

 public final class TesterSupport {

@@ -238,7 +239,8 @@ public final class TesterSupport {
 return false;
 }
 String sslImplementation = 
System.getProperty("tomcat.test.sslImplementation");
-if (sslImplementation != null && 
!"${test.sslImplementation}".equals(sslImplementation)) {
+if (sslImplementation != null && 
!"${test.sslImplementation}".equals(sslImplementation) &&
+ 
!JSSEImplementation.class.getName().equals(sslImplementation)) {

 // Assume custom SSL is not supporting this
 return false;
 }


Not sure this is the right fix, or it would be better to use a change 
that is closer to 10.1.x and find out, why 
tomcat.getConnector().getProperty("sslImplementationName") is null 
inside isClientRenegotiationSupported.


Best regards,

Rainer

Am 18.01.24 um 21:48 schrieb Mark Thomas:

On 18/01/2024 12:33, Rainer Jung wrote:

Hi all,

after the refactorings for the testing of the forbidden client 
initiated renegotiations, these unit tests fail for me for the last 
tags of TC 8.5 and 9, but not for 10.1 and 11. I am using JSSE and the 
tests fail consistently for all four JDK vendors I am testing against 
on all linux distributions I am testing on. I saw it for Java 8 and 
11. No info yet about 17 and 21, the test runs will arrive at Java 17 
later today.


Very odd. I just ran the 9.0.x test with Java 8 / JSSE and it passed.


Example output (no indication for a reason):






Testcase: testClientInitiatedRenegotiation took 0.383 sec
 FAILED
Renegotiation started when it should have failed
junit.framework.AssertionFailedError: Renegotiation started when it 
should have failed
 at 
org.apache.tomcat.util.net.TestSsl.testClientInitiatedRenegotiation(TestSsl.java:250)
 at 
java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native 
Method)
 at 
java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
 at 
java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)


Odd. With JSSE and NIO, client initiated renegotiation should work.

I can't recreate this. If you can recreate this in a debugger then I'd 
look at TesterSupport.isClientRenegotiationSupported()


Mark





I expect it is a new problem in the test after the refactoring, not a 
real TLS issue.


Best regards,

Rainer

-
To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org
For additional commands, e-mail: dev-h...@tomcat.apache.org



-
To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org
For additiona

Re: 10.1.19: sporadic NullPointerException in org.apache.tomcat.websocket.TestWsSessionSuspendResume.testSuspendThenClose()

[Rainer Jung]

OK, great and sorry I didn't notice the fix!

Am 17.02.24 um 17:18 schrieb Mark Thomas:

On 16/02/2024 23:50, Rainer Jung wrote:

Hi there,

I see sporadic NullPointerExceptions when running the new unit test in 
org.apache.tomcat.websocket.TestWsSessionSuspendResume:


Yep. My bad. The fix is already committed:

https://github.com/apache/tomcat/commit/d7daf694cbd773e362567a29a3b518cc1edaa9c1

Mark


-
To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org
For additional commands, e-mail: dev-h...@tomcat.apache.org

10.1.19: sporadic NullPointerException in org.apache.tomcat.websocket.TestWsSessionSuspendResume.testSuspendThenClose()

[Rainer Jung]

Hi there,

I see sporadic NullPointerExceptions when running the new unit test in 
org.apache.tomcat.websocket.TestWsSessionSuspendResume:


Testcase: testSuspendThenClose took 1.952 sec
Caused an ERROR
Cannot invoke "org.apache.tomcat.websocket.WsSession.isClosed()" because 
"org.apache.tomcat.websocket.TestWsSessionSuspendResume$SuspendCloseEndpoint.serverSession" 
is null
java.lang.NullPointerException: Cannot invoke 
"org.apache.tomcat.websocket.WsSession.isClosed()" because 
"org.apache.tomcat.websocket.TestWsSessionSuspendResume$SuspendCloseEndpoint.serverSession" 
is null
at 
org.apache.tomcat.websocket.TestWsSessionSuspendResume$SuspendCloseEndpoint.isServerSessionFullyClosed(TestWsSessionSuspendResume.java:229)
at 
org.apache.tomcat.websocket.TestWsSessionSuspendResume.testSuspendThenClose(TestWsSessionSuspendResume.java:173)
at 
java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native 
Method)
at 
java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:77)
at 
java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)


It happen ed 22 times in 714 different combinations of Java versions, 
JDK vendors, JSSE or tcnative connectors and Linux versions.


They do not seem to depend on the Java version, experiencing them for 
Java 11 and 17 (21 not yet tested). Most of the exceptions happen during 
tests with tcnative 1.3.0 or 2.0.7, but at least once also with JSSE.


The specific test is new in this version.

Best regards,

Rainer


-
To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org
For additional commands, e-mail: dev-h...@tomcat.apache.org

TestSsl.testClientInitiatedRenegotiation fails for TC 9 and 8.5

[Rainer Jung]

Hi all,

after the refactorings for the testing of the forbidden client initiated 
renegotiations, these unit tests fail for me for the last tags of TC 8.5 
and 9, but not for 10.1 and 11. I am using JSSE and the tests fail 
consistently for all four JDK vendors I am testing against on all linux 
distributions I am testing on. I saw it for Java 8 and 11. No info yet 
about 17 and 21, the test runs will arrive at Java 17 later today.


Example output (no indication for a reason):

18-Jan-2024 05:20:09.593 INFO [main] 
org.apache.catalina.startup.LoggingBaseTest.setUp Starting test case 
[testClientInitiatedRenegotiation]
18-Jan-2024 05:20:09.611 INFO [main] 
org.apache.coyote.AbstractProtocol.init Initializing ProtocolHandler 
["https-jsse-nio-127.0.0.1-auto-5"]
18-Jan-2024 05:20:09.615 INFO [main] 
org.apache.tomcat.util.net.AbstractEndpoint.logCertificate Connector 
[https-jsse-nio-127.0.0.1-auto-5], TLS virtual host [_default_], 
certificate type [UNDEFINED] configured from keystore 
[/.../tomcat85_test.sles12.x86_64/test/org/apache/tomcat/util/net/localhost-rsa.jks] 
using alias [tomcat] with trust store 
[/.../tomcat85_test.sles12.x86_64/test/org/apache/tomcat/util/net/ca.jks]
18-Jan-2024 05:20:09.615 INFO [main] 
org.apache.catalina.core.StandardService.startInternal Starting service 
[Tomcat]
18-Jan-2024 05:20:09.616 INFO [main] 
org.apache.catalina.core.StandardEngine.startInternal Starting Servlet 
engine: [Apache Tomcat/8.5.98]
18-Jan-2024 05:20:09.684 INFO [main] 
org.apache.coyote.AbstractProtocol.start Starting ProtocolHandler 
["https-jsse-nio-127.0.0.1-auto-5-45059"]
18-Jan-2024 05:20:09.912 INFO [main] 
org.apache.coyote.AbstractProtocol.pause Pausing ProtocolHandler 
["https-jsse-nio-127.0.0.1-auto-5-45059"]
18-Jan-2024 05:20:09.915 INFO [main] 
org.apache.catalina.core.StandardService.stopInternal Stopping service 
[Tomcat]
18-Jan-2024 05:20:09.928 INFO [main] 
org.apache.coyote.AbstractProtocol.stop Stopping ProtocolHandler 
["https-jsse-nio-127.0.0.1-auto-5-45059"]
18-Jan-2024 05:20:09.943 INFO [main] 
org.apache.coyote.AbstractProtocol.destroy Destroying ProtocolHandler 
["https-jsse-nio-127.0.0.1-auto-5-45059"]

-  ---

Testcase: testSimpleSsl took 7.754 sec
Testcase: testKeyPass took 1.179 sec
Testcase: testPost took 15.93 sec
Testcase: testKeyPassFile took 0.783 sec
Testcase: testClientInitiatedRenegotiation took 0.383 sec
FAILED
Renegotiation started when it should have failed
junit.framework.AssertionFailedError: Renegotiation started when it 
should have failed
	at 
org.apache.tomcat.util.net.TestSsl.testClientInitiatedRenegotiation(TestSsl.java:250)
	at 
java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native 
Method)
	at 
java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at 
java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)



I expect it is a new problem in the test after the refactoring, not a 
real TLS issue.


Best regards,

Rainer

-
To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org
For additional commands, e-mail: dev-h...@tomcat.apache.org

Re: (tomcat) 11/12: Results depend on timing so group this with the performance tests

[Rainer Jung]

Am 03.01.24 um 12:29 schrieb ma...@apache.org:

This is an automated email from the ASF dual-hosted git repository.

markt pushed a commit to branch main
in repository https://gitbox.apache.org/repos/asf/tomcat.git

commit 9c5f303e9aea86564c15ed83491d9a4b6beff829
Author: Mark Thomas 
AuthorDate: Wed Jan 3 10:46:46 2024 +

 Results depend on timing so group this with the performance tests
---
  .../{TestAsyncMessages.java => TestAsyncMessagesPerformance.java} | 4 +---
  1 file changed, 1 insertion(+), 3 deletions(-)

diff --git a/test/org/apache/tomcat/websocket/server/TestAsyncMessages.java 
b/test/org/apache/tomcat/websocket/server/TestAsyncMessagesPerformance.java
similarity index 97%
rename from test/org/apache/tomcat/websocket/server/TestAsyncMessages.java
rename to 
test/org/apache/tomcat/websocket/server/TestAsyncMessagesPerformance.java
index 4a2ef86b39..c169f61a60 100644
--- a/test/org/apache/tomcat/websocket/server/TestAsyncMessages.java
+++ b/test/org/apache/tomcat/websocket/server/TestAsyncMessagesPerformance.java
@@ -27,7 +27,6 @@ import jakarta.websocket.Session;
  import jakarta.websocket.WebSocketContainer;
  
  import org.junit.Assert;

-import org.junit.Ignore;
  import org.junit.Test;
  
  import org.apache.catalina.Context;

@@ -37,8 +36,7 @@ import org.apache.catalina.startup.TomcatBaseTest;
  import org.apache.tomcat.websocket.TesterAsyncTiming;
  import 
org.apache.tomcat.websocket.TesterMessageCountClient.TesterProgrammaticEndpoint;
  
-@Ignore // Test passes but GC delays can introduce false failures.

-public class TestAsyncMessages extends TomcatBaseTest {
+public class TestAsyncMessagesPerformance extends TomcatBaseTest {
  
  @Test

  public void testAsyncTiming() throws Exception {


This test - which was probably not executed before - now fails for me 
always. Most of the times it complains about times longer than the 500 
microsecs limit (going up to a few milliseconds in a few cases), but 
sometimes it even hangs. I can try to get a stack dump when it hangs again.


Best regards,

Rainer

-
To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org
For additional commands, e-mail: dev-h...@tomcat.apache.org

Re: (tomcat) 09/12: This is a performance test - use correct naming

[Rainer Jung]

Am 03.01.24 um 12:29 schrieb ma...@apache.org:

This is an automated email from the ASF dual-hosted git repository.

markt pushed a commit to branch main
in repository https://gitbox.apache.org/repos/asf/tomcat.git

commit 8fe3db67fdd69c160367982e85b23bfe01689c5c
Author: Mark Thomas 
AuthorDate: Wed Jan 3 10:43:07 2024 +

 This is a performance test - use correct naming
---
  .../{TestConnectionLimit.java => TestConnectionLimitPerformance.java} | 4 +---
  1 file changed, 1 insertion(+), 3 deletions(-)

diff --git a/test/org/apache/tomcat/websocket/TestConnectionLimit.java 
b/test/org/apache/tomcat/websocket/TestConnectionLimitPerformance.java
similarity index 96%
rename from test/org/apache/tomcat/websocket/TestConnectionLimit.java
rename to test/org/apache/tomcat/websocket/TestConnectionLimitPerformance.java
index 463ee95b01..6fc045622c 100644
--- a/test/org/apache/tomcat/websocket/TestConnectionLimit.java
+++ b/test/org/apache/tomcat/websocket/TestConnectionLimitPerformance.java
@@ -25,7 +25,6 @@ import jakarta.websocket.DeploymentException;
  import jakarta.websocket.WebSocketContainer;
  
  import org.junit.Assert;

-import org.junit.Ignore;
  import org.junit.Test;
  
  import org.apache.catalina.Context;

@@ -34,8 +33,7 @@ import org.apache.catalina.startup.Tomcat;
  import org.apache.catalina.startup.TomcatBaseTest;
  import 
org.apache.tomcat.websocket.TesterMessageCountClient.TesterProgrammaticEndpoint;
  
-@Ignore // Not for use in normal unit test runs

-public class TestConnectionLimit extends TomcatBaseTest {
+public class TestConnectionLimitPerformance extends TomcatBaseTest {
  
  /*

   * Simple test to see how many outgoing connections can be created on a 
single machine.


This test - which probably wasn't executed before - now fails frequently 
for me. The failure is with java.lang.NoClassDefFoundError in various 
places. An example:


Testcase: testSingleMachine took 9.285 sec
Caused an ERROR
org/apache/catalina/startup/ExpandWar
java.lang.NoClassDefFoundError: org/apache/catalina/startup/ExpandWar
at 
org.apache.catalina.startup.LoggingBaseTest.tearDown(LoggingBaseTest.java:138)
at 
org.apache.catalina.startup.TomcatBaseTest.tearDown(TomcatBaseTest.java:243)
at 
java.base/jdk.internal.reflect.DirectMethodHandleAccessor.invoke(DirectMethodHandleAccessor.java:103)
Caused by: java.lang.ClassNotFoundException: 
org.apache.catalina.startup.ExpandWar
at 
java.base/jdk.internal.loader.BuiltinClassLoader.loadClass(BuiltinClassLoader.java:641)
at 
java.base/jdk.internal.loader.ClassLoaders$AppClassLoader.loadClass(ClassLoaders.java:188)

at java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:526)

Testcase: org.apache.tomcat.websocket.TestConnectionLimitPerformance 
took 0 sec

Caused an ERROR
org/apache/catalina/startup/ExpandWar
java.lang.NoClassDefFoundError: org/apache/catalina/startup/ExpandWar
at 
org.apache.catalina.startup.LoggingBaseTest.tearDownPerTestClass(LoggingBaseTest.java:154)
at 
java.base/jdk.internal.reflect.DirectMethodHandleAccessor.invoke(DirectMethodHandleAccessor.java:103)
Caused by: java.lang.ClassNotFoundException: 
org.apache.catalina.startup.ExpandWar



The stacks can differ and also the classes in the 
java.lang.NoClassDefFoundError:


jakarta/websocket/SendResult
org/apache/catalina/Lifecycle$SingleUse
org/apache/catalina/startup/ExpandWar
org/apache/tomcat/PeriodicEventListener
org/apache/tomcat/util/net/SocketWrapperBase$BlockingMode
org/apache/tomcat/util/net/SocketWrapperBase$CompletionState
org/apache/tomcat/websocket/MessagePart
org/apache/tomcat/websocket/server/WsRemoteEndpointImplServer$1
org/apache/tomcat/websocket/WsRemoteEndpointImplBase$BlockingSendHandler


Wild guess: it might be, because the tested process runs out of file 
descriptors before:


06-Jan-2024 09:59:21.358 SEVERE [http-nio-127.0.0.1-auto-1-Acceptor] 
org.apache.tomcat.util.net.Acceptor.run Socket accept failed

java.io.IOException: Too many open files
at java.base/sun.nio.ch.Net.accept(Native Method)
		at 
java.base/sun.nio.ch.ServerSocketChannelImpl.implAccept(ServerSocketChannelImpl.java:433)
		at 
java.base/sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:399)
		at 
org.apache.tomcat.util.net.NioEndpoint.serverSocketAccept(NioEndpoint.java:518)
		at 
org.apache.tomcat.util.net.NioEndpoint.serverSocketAccept(NioEndpoint.java:80)

at org.apache.tomcat.util.net.Acceptor.run(Acceptor.java:128)
at java.base/java.lang.Thread.run(Thread.java:1583)


Best regards,

Rainer

-
To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org
For additional commands, e-mail: dev-h...@tomcat.apache.org

Re: [tomcat] branch main updated: Add message when not using Java 22 for release

[Rainer Jung]

Am 08.12.23 um 15:04 schrieb Rémy Maucherat:

On Fri, Dec 8, 2023 at 3:01 PM Rainer Jung  wrote:


Hi Remy,

I tried to build TC 11 with Java 22, but cuirrenttly it fails with
errors like:

  [javac]
.../java/org/apache/tomcat/util/openssl/openssl_h_Macros.java:469:
error: MemorySegment is a preview API and is disabled by default.
  [javac] public static void OCSP_RESPONSE_free(MemorySegment a) {
  [javac]   ^
  [javac]   (use --enable-preview to enable preview APIs)

The following classes fail like that:

.../java/org/apache/tomcat/util/net/openssl/panama/OpenSSLContext.java
.../java/org/apache/tomcat/util/net/openssl/panama/OpenSSLSessionContext.java
.../java/org/apache/tomcat/util/net/openssl/panama/OpenSSLSessionStats.java
.../java/org/apache/tomcat/util/openssl/openssl_h_Compatibility.java
.../java/org/apache/tomcat/util/openssl/openssl_h.java
.../java/org/apache/tomcat/util/openssl/openssl_h_Macros.java

I didn't see anything prepared for "--enable-preview" in our ant scripts.


What does java -version says ?

The FFM API is no longer a preview API in the up to date Java 22
builds, so the flag is counter productive.
Downloads are here: https://jdk.java.net/22/

Rémy


Thanks Remy. I am at EA 19, current is EA 26. I guess it was included 
around 24 or 25. Unfortunately it is not mentioned in the release notes. 
Will try next time with the current EA build of jdk 22.




Best regards,

Rainer

Am 24.10.23 um 11:16 schrieb r...@apache.org:

This is an automated email from the ASF dual-hosted git repository.

remm pushed a commit to branch main
in repository https://gitbox.apache.org/repos/asf/tomcat.git


The following commit(s) were added to refs/heads/main by this push:
   new 1185ad1154 Add message when not using Java 22 for release
1185ad1154 is described below

commit 1185ad1154cdbb8003efd29eeb1ccf95c87bdc56
Author: remm 
AuthorDate: Tue Oct 24 11:15:44 2023 +0200

  Add message when not using Java 22 for release

  Filter out packages with FFM API from javadoc.
---
   build.xml | 6 ++
   1 file changed, 6 insertions(+)

diff --git a/build.xml b/build.xml
index 12c720846e..e3cca8f964 100644
--- a/build.xml
+++ b/build.xml
@@ -2366,6 +2366,8 @@
   
   
   
+
+
 
 
 
@@ -2654,6 +2656,10 @@ skip.installer property in build.properties" />
   
 
   -->
+
+  
+
+ JAVA VERSION 22 OR NEWER IS REQUIRED FOR 
RELEASE
 

 


-
To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org
For additional commands, e-mail: dev-h...@tomcat.apache.org

Re: [tomcat] branch main updated: Add message when not using Java 22 for release

[Rainer Jung]

Hi Remy,

I tried to build TC 11 with Java 22, but cuirrenttly it fails with 
errors like:


[javac] 
.../java/org/apache/tomcat/util/openssl/openssl_h_Macros.java:469: 
error: MemorySegment is a preview API and is disabled by default.

[javac] public static void OCSP_RESPONSE_free(MemorySegment a) {
[javac]   ^
[javac]   (use --enable-preview to enable preview APIs)

The following classes fail like that:

.../java/org/apache/tomcat/util/net/openssl/panama/OpenSSLContext.java
.../java/org/apache/tomcat/util/net/openssl/panama/OpenSSLSessionContext.java
.../java/org/apache/tomcat/util/net/openssl/panama/OpenSSLSessionStats.java
.../java/org/apache/tomcat/util/openssl/openssl_h_Compatibility.java
.../java/org/apache/tomcat/util/openssl/openssl_h.java
.../java/org/apache/tomcat/util/openssl/openssl_h_Macros.java

I didn't see anything prepared for "--enable-preview" in our ant scripts.

Best regards,

Rainer

Am 24.10.23 um 11:16 schrieb r...@apache.org:

This is an automated email from the ASF dual-hosted git repository.

remm pushed a commit to branch main
in repository https://gitbox.apache.org/repos/asf/tomcat.git


The following commit(s) were added to refs/heads/main by this push:
  new 1185ad1154 Add message when not using Java 22 for release
1185ad1154 is described below

commit 1185ad1154cdbb8003efd29eeb1ccf95c87bdc56
Author: remm 
AuthorDate: Tue Oct 24 11:15:44 2023 +0200

 Add message when not using Java 22 for release
 
 Filter out packages with FFM API from javadoc.

---
  build.xml | 6 ++
  1 file changed, 6 insertions(+)

diff --git a/build.xml b/build.xml
index 12c720846e..e3cca8f964 100644
--- a/build.xml
+++ b/build.xml
@@ -2366,6 +2366,8 @@
  
  
  
+
+



@@ -2654,6 +2656,10 @@ skip.installer property in build.properties" />
  

  -->
+
+  
+
+   JAVA VERSION 22 OR NEWER IS REQUIRED FOR 
RELEASE

  



-
To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org
For additional commands, e-mail: dev-h...@tomcat.apache.org

Re: Running TC 10.1 unit tests with Java 11?

[Rainer Jung]

I just noticed the skip.build.java.version part, sorry for the noise.

Am 11.11.23 um 01:42 schrieb Rainer Jung:

It seems we can now only run TC 10.1 unit tests using Java 17?
I'd like to also run them with Java versions for which TC itself is 
supported, so for TC 10.1 with Java 11. Is that still possible? It seems 
I get "Java version 17 or newer is required" from:


   

   
   

     
   
     
   

Of course I could patch that away or overwrite 
build.java.versionbuild.java.version before running the tests, but is 
that expected to work if all compilation and test compilation steps are 
done before?


Best regards,

Rainer





-
To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org
For additional commands, e-mail: dev-h...@tomcat.apache.org



--
kippdata
informationstechnologie GmbH   Tel: 0228 98549 -0
Bornheimer Str. 33aFax: 0228 98549 -50
53111 Bonn www.kippdata.de

HRB 8018 Amtsgericht Bonn / USt.-IdNr. DE 196 457 417
Geschäftsführer: Dr. Thomas Höfer, Rainer Jung, Sven Maurmann

-
To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org
For additional commands, e-mail: dev-h...@tomcat.apache.org

Re: (tomcat-connectors) branch main updated: BZ 68117: Fix typo and escaping in libtool flag introduced in 1.2.49.

[Rainer Jung]

I am not sure.

For the escaping: I did not observe a problem with and without the 
escaping part. But I liked the additional safety net.


Concerning the wrong flag name in Makefile: unfortunately of the two 
makefiles apache-2.0/Makefile and apache-2.0/Makefile.apxs the first one 
is the one used by default. That's the one with the broken flag name. 
libtool - at least the version used here - seems to ignore the wrong 
flag name. So the effect we wanted - symbol reduction - does not happen.


So concerning the Makefile changes 1.2.49 is not worse than 1.2.48, but 
unfortunately also not really better.


I added a comment to the original PR 66005.

I am fine with doing a new release, but not sure if it is worth it. 
66005 is a nasty bug but is seems it has not hit too many people.


What do others think?

Best regards,

Rainer

Am 10.11.23 um 11:46 schrieb Mark Thomas:

On 10/11/2023 10:38, rj...@apache.org wrote:

This is an automated email from the ASF dual-hosted git repository.

rjung pushed a commit to branch main
in repository https://gitbox.apache.org/repos/asf/tomcat-connectors.git


The following commit(s) were added to refs/heads/main by this push:
  new ecd005d07 BZ 68117: Fix typo and escaping in libtool flag 
introduced in 1.2.49.

ecd005d07 is described below

commit ecd005d0792441c4510dc4c2d9348979ab71ddcc
Author: Rainer Jung 
AuthorDate: Fri Nov 10 11:38:04 2023 +0100

 BZ 68117: Fix typo and escaping in libtool flag introduced in 
1.2.49.


Do we need a new release because of this?

Asking as I have no clue how much of a problem this is.

Mark


-
To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org
For additional commands, e-mail: dev-h...@tomcat.apache.org

Concerning OpenSSL main (raised in BZ 67628)

[Rainer Jung]

Hi there,

in BZ 67628 (OpenSSLCipherConfigurationParser#parse() produces 
misleading false positive cipher warnings) the following question was 
raised:


Am 03.11.23 um 09:30 schrieb bugzi...@apache.org:

https://bz.apache.org/bugzilla/show_bug.cgi?id=67628

--- Comment #9 from Mark Thomas  ---
(In reply to Michael Osipov from comment #8)


Just tested. From a documentation PoV, this is fine now, though I wonder how
many people will run OpenSSL from main instead of the LTS branch.


I suspect very few, if any. I did consider aligning with the most recent LTS
release but concluded it was better to align with main as that reflects the
latest thinking regarding how secure a cipher or family of ciphers is. In
reality, the differences have been minimal in the last few years. If they
become problematic, we can always review which branch we track.


It seems OpenSSL 3.0 has performance degradation in some aspects (due to 
changes in locking). It might not be relevant for what our users do with 
Tomcat+OpenSSL, but the OpenSSL team invested quite some time to improve 
this. Parts are in 3.1, more parts might come in 3.2. Although the 
OpenSSL team declared 3.0 an LTS release, due to the performance 
situation, some downstreams might switch to a later branch.


I am not directly involved in downstream discussions (except for my 
own), but I followed a bit the performance/locking discussion.


Best regards,

Rainer


-
To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org
For additional commands, e-mail: dev-h...@tomcat.apache.org

Re: Tagging mod_jk

[Rainer Jung]

Should have gone to the list.

I think the mod_jk build process does not make it available online like 
the Tomcat one does, but here's the current section:



 Changes between 1.2.48 and 1.2.49


   Apache

 * Update: Retrieve default request id from mod_unique_id. It can also
   be taken from an arbitrary environment variable by configuring
   "JkRequestIdIndicator". (rjung)
 * Fix: 65901 :
   Don't delegate the generatation of the response body to httpd when
   the status code represents an error if the request used the HEAD
   method. (markt)
 * Fix: 66005 :
   Only export the main module symbol. Visibility of module internal
   symbols led to crashes when conflicting with library symbols. Based
   on a patch provided by Josef Čejka. (rjung)


   IIS

 * Update: Set default request id as a GUID. It can also be taken from
   an arbitrary request header by configuring "request_id_header". (rjung)
 * Fix: Fix non-empty check for the Translate header. (rjung)


   Common

 * Fix: Fix compiler warning when initializing and copying fixed length
   strings. (rjung)
 * Update: Add a request id to mod_jk log lines. (rjung)
 * Fix: 64878 :
   Enable configure to find the correct sizes for pid_t and pthread_t
   when building on MacOS. (markt)
 * Fix: Fix Clang 15/16 compatability. Pull request #6
    provided by Sam
   James. (markt)
 * Update: Improve XSS hardening in status worker. (rjung)


   Docs

 * Update: Remove support for the Netscape / Sun ONE / Oracle iPlanet
   Web Server as the product has been retired. (markt)
 * Update: Remove links to the old JK2 documentation. The JK2
   documentation is still available, it is just no longer linked from
   the current JK documentation. (markt)
 * Update: Restructure subsections in changelog starting with version
   1.2.45. (rjung)

Best regards,
Rainer

Am 07.09.23 um 09:43 schrieb Martin Knoblauch:

Hi Mark,

  is there a Changelog available?

Thanks
Martin

On Wed, Sep 6, 2023 at 4:29 PM Mark Thomas  wrote:


Hi all,

After a long period of inactivity we have had some bug fixes for mod_jk
so I am planning a Tomcat Connectors tag shortly.

I need to run through my pre-tag checks and I'll tag after that -
hopefully in a couple of hours.

If you have any final changes for Tomcat Connectors now is the time to
commit them or let me know to delay the tag.

Mark

Re: [tomcat-connectors] tag JK_1_2_49 created (now 3ad622409)

[Rainer Jung]

I don't have a windows build setup for mod_jk at hand right now, but if 
ou want more eyes to look at the error just post it. I am probably 
responsible for it, because I changes quite a bit if code structure to 
introduce the logging context.


Best regards,

Rainer

-
To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org
For additional commands, e-mail: dev-h...@tomcat.apache.org

Re: Syntax mod_jk changelog

[Rainer Jung]

Am 05.09.23 um 13:18 schrieb Rainer Jung:

Am 05.09.23 um 12:50 schrieb Mark Thomas:

On 05/09/2023 11:40, Rainer Jung wrote:

I saw we use for older versions

   

in the changelog, but since the last version this is missing. Is this 
an oversight, or did we change the style sheet? Should we have the 
"subsection" everywhere or remove everywhere? Any ideas?


Probably an oversight but I'm not seeing the point of the Native 
sub-section. I'm leaning towards removing it everywhere.


If we wanted to add Common, IIS, httpd sections that could be useful 
but it is also a LOT of work. Nice to have but certainly not essential.


Thanks for the quick feedback. Since the subsection use is not 
consistent anyways, we could also switch over to using the more 
meaningful subsections you mentioned. I would like that, especially 
since we already do something similar but not as useful by prefixing the 
entry texts.


I read between the lines, that the style sheet should be agnotic on what 
subsection names we actually use. I'll have a try with adjusting the 
subsections for the last and the forthcoming release and see how well it 
works.


Done for a few versions back to show it is feasible for the future.

I can revert, if people don't like it.

Thanks!

Rainer

-
To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org
For additional commands, e-mail: dev-h...@tomcat.apache.org

Re: Syntax mod_jk changelog

[Rainer Jung]

Am 05.09.23 um 12:50 schrieb Mark Thomas:

On 05/09/2023 11:40, Rainer Jung wrote:

I saw we use for older versions

   

in the changelog, but since the last version this is missing. Is this 
an oversight, or did we change the style sheet? Should we have the 
"subsection" everywhere or remove everywhere? Any ideas?


Probably an oversight but I'm not seeing the point of the Native 
sub-section. I'm leaning towards removing it everywhere.


If we wanted to add Common, IIS, httpd sections that could be useful but 
it is also a LOT of work. Nice to have but certainly not essential.


Thanks for the quick feedback. Since the subsection use is not 
consistent anyways, we could also switch over to using the more 
meaningful subsections you mentioned. I would like that, especially 
since we already do something similar but not as useful by prefixing the 
entry texts.


I read between the lines, that the style sheet should be agnotic on what 
subsection names we actually use. I'll have a try with adjusting the 
subsections for the last and the forthcoming release and see how well it 
works.


Thanks!

Rainer

-
To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org
For additional commands, e-mail: dev-h...@tomcat.apache.org

Syntax mod_jk changelog

[Rainer Jung]

I saw we use for older versions

  

in the changelog, but since the last version this is missing. Is this an 
oversight, or did we change the style sheet? Should we have the 
"subsection" everywhere or remove everywhere? Any ideas?


Thanks and regards,

Rainer

-
To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org
For additional commands, e-mail: dev-h...@tomcat.apache.org

Re: [VOTE] Release Apache Tomcat 8.5.93

[Rainer Jung]

Hi Han,

Am 24.08.23 um 04:31 schrieb Han Li:




On Aug 24, 2023, at 10:11, Rainer Jung  wrote:

Am 24.08.23 um 01:31 schrieb Mark Thomas:

The proposed Apache Tomcat 8.5.93 release is now available for voting.
The notable changes compared to 8.5.92 are:
- If an application or library sets both a non-500 error code and the
   jakarta.servlet.error.exception request attribute, use the
   provided error code during error page processing rather than assuming
   an error code of 500.
- Fix for FORM authentication open redirect - CVE-2023-41080
Along with lots of other bug fixes and improvements.
For full details, see the changelog:
https://nightlies.apache.org/tomcat/tomcat-8.5.x/docs/changelog.html
It can be obtained from:
https://dist.apache.org/repos/dist/dev/tomcat/tomcat-8/v8.5.93/
The Maven staging repo is:
https://repository.apache.org/content/repositories/orgapachetomcat-1454
The tag is:
https://github.com/apache/tomcat/tree/8.5.93/
9d9aea65c435a38c737c1e600e6513f9d0980cf1
The proposed 8.5.93 release is:
[ ] Broken - do not release
[ ] Stable - go ahead and release as 8.5.93 (stable)


Tests ongoing, but just a short note: The changelog still has the previous 
version .92 above the newest section instead of .93. For me this is not a show 
stopper and we can fix the online one after the release. The tags for TC 9, 
10.1 and 11 do not have this problem.

Everything looks fine to me, it's the latest .93 version. Maybe it's caused by 
your browser's local cache, you can try again after clearing local cache.

BUT there is a typo `rlease` in version status.


thanks for also checking. I meant the version of the changelog, that is 
shipped as part of the release, so the tagged one:


https://github.com/apache/tomcat/blob/8.5.93/webapps/docs/changelog.xml

or if you prefer the one that is included in the release files we 
produce (source tarballs and zips etc.).


You  probably looked at the one linked in the release vote mail above. 
That is a nightly changelog version and contains the post-release 
commit, which prepars for the next (upcoming) version. In this commit 
Mark had already fixed the wrong version number.


Best regards,

Rainer

-
To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org
For additional commands, e-mail: dev-h...@tomcat.apache.org

Re: [VOTE] Release Apache Tomcat 8.5.93

[Rainer Jung]

Am 24.08.23 um 01:31 schrieb Mark Thomas:

The proposed Apache Tomcat 8.5.93 release is now available for voting.

The notable changes compared to 8.5.92 are:

- If an application or library sets both a non-500 error code and the
   jakarta.servlet.error.exception request attribute, use the
   provided error code during error page processing rather than assuming
   an error code of 500.

- Fix for FORM authentication open redirect - CVE-2023-41080


Along with lots of other bug fixes and improvements.

For full details, see the changelog:
https://nightlies.apache.org/tomcat/tomcat-8.5.x/docs/changelog.html

It can be obtained from:
https://dist.apache.org/repos/dist/dev/tomcat/tomcat-8/v8.5.93/

The Maven staging repo is:
https://repository.apache.org/content/repositories/orgapachetomcat-1454

The tag is:
https://github.com/apache/tomcat/tree/8.5.93/
9d9aea65c435a38c737c1e600e6513f9d0980cf1

The proposed 8.5.93 release is:
[ ] Broken - do not release
[ ] Stable - go ahead and release as 8.5.93 (stable)


Tests ongoing, but just a short note: The changelog still has the 
previous version .92 above the newest section instead of .93. For me 
this is not a show stopper and we can fix the online one after the 
release. The tags for TC 9, 10.1 and 11 do not have this problem.


Best regards,

Rainer

-
To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org
For additional commands, e-mail: dev-h...@tomcat.apache.org

Re: [tomcat-native] 01/01: Tag 2.0.5

[Rainer Jung]

Hi Mark,

thanks for the tag. Do you intend to tag 1.2 as well?

Thanks and regards,

Rainer

-
To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org
For additional commands, e-mail: dev-h...@tomcat.apache.org

Re: [VOTE] Release Apache Tomcat 8.5.91

[Rainer Jung]

Am 10.07.23 um 20:15 schrieb Christopher Schultz:

All,

On 7/6/23 11:31, Christopher Schultz wrote:

The proposed Apache Tomcat 8.5.91 release is now available for voting.

The notable changes compared to 8.5.90 are:

- Add ContextNamingInfoListener, a listener which creates context naming
   information environment entries.

- Add PropertiesRoleMappingListener, a listener which populates the
   context's role mapping from a properties file.

Along with lots of other bug fixes and improvements.

For full details, see the changelog:
https://nightlies.apache.org/tomcat/tomcat-8.5.x/docs/changelog.html

It can be obtained from:
https://dist.apache.org/repos/dist/dev/tomcat/tomcat-8/v8.5.91/

The Maven staging repo is:
https://repository.apache.org/content/repositories/orgapachetomcat-1446

The tag is:
https://github.com/apache/tomcat/tree/8.5.91/
d5dc384e752d54fcb531c0e86b2ee1ee73b1404e

The proposed 8.5.91 release is:
[ ] Broken - do not release
[ ] Stable - go ahead and release as 8.5.91 (stable)


+1 to release 8.5.91 as stable.

Note that upgrading to JRE 8u372 has resolved the unit test failures in 
TestPEMFile.


Note that fixing a typo in testing configuration has resolved unit test 
failures in both TestCipher and TestOpenSSLCipherConfigurationParser.


The unit-test failures in TestWsWebSocketContainerSSL are currently a 
mystery, but (a) are not new for me in this release and (b) are likely 
to be environment-specific. I do not see a reason to pause or veto the 
release for this reason.


Best I can say: I run te test suite for new tags for quite a few 
combinations of Linux distro plus JVM (version and vendor) plus 
JSSE/tcnative/OpenSSL versions and the test TestWsWebSocketContainerSSL 
for me only fails (sporadically but often) on Solaris 11 with tcnative.


I know this doesn't help much, but wanted to let you know that it is 
stable for me on Linux for 8.5.91 but also for previous 8.5 versions and 
other major versions than 8.5.


Best regards,

Rainer


-
To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org
For additional commands, e-mail: dev-h...@tomcat.apache.org

Re: [VOTE] Release Apache Tomcat 11.0.0-M8

[Rainer Jung]

I already did the fix but forgot the changelog item, because otherwise 
the changelog for the version would be empty ...


Am 04.07.23 um 12:00 schrieb Mark Thomas:

On 04/07/2023 10:45, Rainer Jung wrote:
I see, the checksum and the version number in the comment was updated, 
but not the one used for the download:


Thanks for catching this. I'll cancel the release, fix it and re-tag.

Mark




diff --git a/build.properties.default b/build.properties.default
index 172a08a083..aa8179820c 100644
--- a/build.properties.default
+++ b/build.properties.default
@@ -318,10 +318,10 @@ 
bnd.loc=${base-maven.loc}/biz/aQute/bnd/biz.aQute.bnd/${bnd.version}/biz.aQute.b

  # - JSign, version 4.1 or later -
  jsign.version=4.2

-# checksums for JSign 4.2
+# checksums for JSign 5.0
  jsign.checksum.enabled=true
  jsign.checksum.algorithm=MD5|SHA-1
-jsign.checksum.value=10fb38a1182515d583a1e252c8219eae|1e3b44e0114d599b05be243513b85a51b0c45401
+jsign.checksum.value=79c4f9bdff74a4ccee3d72f020ad45b7|5a6677625413e0d8acb52f80fa6fbb9031a6a9d0

  jsign.home=${base.path}/jsign-${jsign.version}
  jsign.jar=${jsign.home}/jsign-${jsign.version}.jar
diff --git a/webapps/docs/changelog.xml b/webapps/docs/changelog.xml
index c254896add..0dbdaaa8d3 100644
--- a/webapps/docs/changelog.xml
+++ b/webapps/docs/changelog.xml
@@ -197,6 +197,9 @@
    
  Update BND to 6.4.1. (markt)
    
+  
+    Update JSign to 5.0. (markt)
+  
  
    
  

The same is true for 9.0.

Regards,

Rainer


-
To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org
For additional commands, e-mail: dev-h...@tomcat.apache.org

Re: [VOTE] Release Apache Tomcat 11.0.0-M8

[Rainer Jung]

I see, the checksum and the version number in the comment was updated, 
but not the one used for the download:


diff --git a/build.properties.default b/build.properties.default
index 172a08a083..aa8179820c 100644
--- a/build.properties.default
+++ b/build.properties.default
@@ -318,10 +318,10 @@ 
bnd.loc=${base-maven.loc}/biz/aQute/bnd/biz.aQute.bnd/${bnd.version}/biz.aQute.b

 # - JSign, version 4.1 or later -
 jsign.version=4.2

-# checksums for JSign 4.2
+# checksums for JSign 5.0
 jsign.checksum.enabled=true
 jsign.checksum.algorithm=MD5|SHA-1
-jsign.checksum.value=10fb38a1182515d583a1e252c8219eae|1e3b44e0114d599b05be243513b85a51b0c45401
+jsign.checksum.value=79c4f9bdff74a4ccee3d72f020ad45b7|5a6677625413e0d8acb52f80fa6fbb9031a6a9d0

 jsign.home=${base.path}/jsign-${jsign.version}
 jsign.jar=${jsign.home}/jsign-${jsign.version}.jar
diff --git a/webapps/docs/changelog.xml b/webapps/docs/changelog.xml
index c254896add..0dbdaaa8d3 100644
--- a/webapps/docs/changelog.xml
+++ b/webapps/docs/changelog.xml
@@ -197,6 +197,9 @@
   
 Update BND to 6.4.1. (markt)
   
+  
+Update JSign to 5.0. (markt)
+  
 
   
 

The same is true for 9.0.

Regards,

Rainer

-
To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org
For additional commands, e-mail: dev-h...@tomcat.apache.org

Re: [VOTE] Release Apache Tomcat 11.0.0-M8

[Rainer Jung]

Am 03.07.23 um 21:33 schrieb Mark Thomas:

The proposed Apache Tomcat 11.0.0-M8 release is now available for
voting.

Apache Tomcat 11.0.0-M8 is a milestone release of the 11.0.x branch and 
has been made to provide users with early access to the new features in 
Apache Tomcat 11.0.x so that they may provide feedback. The notable 
changes compared to the previous milestone include:


- Add ContextNamingInfoListener, a listener which creates context naming
   information environment entries

- Add PropertiesRoleMappingListener, a listener which populates the
   context's role mapping from a properties file.

- Update the Jakarta EL and Jakarta WebSocket implementations to align
   with the latest changes planned for Jakarta EE 11

For full details, see the change log:
https://nightlies.apache.org/tomcat/tomcat-11.0.x/docs/changelog.html

Applications that run on Tomcat 9 and earlier will not run on Tomcat 11 
without changes. Java EE applications designed for Tomcat 9 and earlier 
may be placed in the $CATALINA_BASE/webapps-javaee directory and Tomcat 
will automatically convert them to Jakarta EE and copy them to the 
webapps directory. Applications using deprecated APIs may require 
further changes.


It can be obtained from:
https://dist.apache.org/repos/dist/dev/tomcat/tomcat-11/v11.0.0-M8/

The Maven staging repo is:
https://repository.apache.org/content/repositories/orgapachetomcat-1441

The tag is:
https://github.com/apache/tomcat/tree/11.0.0-M8
434400d882a20e12ea03855f4bd93451bd3362c6


The proposed 11.0.0-M8 release is:
[ ] -1 Broken - do not release
[ ] +1 Alpha  - go ahead and release as 11.0.0-M8


During

downloadfile:
  [get] Getting: 
https://repo.maven.apache.org/maven2/net/jsign/jsign/4.2/jsign-4.2.jar
  [get] To: 
/shared/build/autobuild/workdirs/current/20230704_112956/bld/tomcat110_incoming/deps/download-1593579538.tmp



I get

Checksum check failure for jsign-4.2.jar (.../deps/download-1593579538.tmp).
  Algorithm: MD5|SHA-1
  Expected value: 
79c4f9bdff74a4ccee3d72f020ad45b7|5a6677625413e0d8acb52f80fa6fbb9031a6a9d0

  Actual values:
  SHA-512: 
0a5e3a96a99b6736fed913da201c0003d2009690b4157ad9894107b1b12b5e76befd0b0f5bf016b475c71b09e97f3d1eac8c4fab138a968fa64beec527dee16d
  SHA-384: 
763de16603fc1921f86643e1c1103f40f5d8c82db4c6a2f90c89aa4baeabd02fd17cf35d587ff00540ffef369a57fb8e

  SHA-256: 290377fc4f593256200b3ea4061b7409e8276255f449d4c6de7833faf0850cc1
  SHA-1: 1e3b44e0114d599b05be243513b85a51b0c45401
  MD5: 10fb38a1182515d583a1e252c8219eae

Was there some dependecy version upfdate without adjusting checksums? Or 
should I look for a problem on my side?


Thanks ands regards,

Rainer

-
To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org
For additional commands, e-mail: dev-h...@tomcat.apache.org

Re: [tomcat-native] branch main updated: native: Fix the build with rlibtool

[Rainer Jung]

Am 06.06.23 um 12:57 schrieb Mark Thomas:

On 05/06/2023 21:20, Rainer Jung wrote:
Something is wrong with our regeneration of configure in the release 
process, at least for the 2.x branch (main). The configure script 
contains "LT_INIT" verbatim instead of LT_INIT being replaced by its 
script implementation. I can't actually say what is wrong :(


If I run "autoreconf --force --install" on my system LT_INIT gets 
resolved but configure also get much bigger (more than double the size).


Let me know if you have no good idea and I should investigate deeper.


Sorry, no idea here. I tested the PR to the extent I check I could still 
build with the PR applied but went no deeper. Linux build systems are 
mostly a mystery to me.


I hope I fixed it today without breaking other stuff. I tested with 
generating a release tarball from the main branch and the tarball deltas 
to 2.0.4 looked reasonable. Also the Makefile generated by configure 
looks consistent. As always things ight vary a bit depending on the 
system used for releases.


Best regards,

Rainer


Am 31.10.22 um 21:02 schrieb ma...@apache.org:

This is an automated email from the ASF dual-hosted git repository.

markt pushed a commit to branch main
in repository https://gitbox.apache.org/repos/asf/tomcat-native.git


The following commit(s) were added to refs/heads/main by this push:
  new 54dccd3a4 native: Fix the build with rlibtool
  new 4f7fb7f44 Merge pull request #14 from orbea/slibtool
54dccd3a4 is described below

commit 54dccd3a4dc01801d9311b3160808305ec9fc2cf
Author: orbea 
AuthorDate: Thu Jul 21 17:59:14 2022 -0700

 native: Fix the build with rlibtool
 When building tomcat-native with slibtool using the rlibtool 
symlink the
 build will fail. This is because rlibtool requires the generated 
libtool

 script to determine if the build is shared, static or both.
 Gentoo bug: https://bugs.gentoo.org/778914
---
  native/configure.in | 3 +++
  1 file changed, 3 insertions(+)

diff --git a/native/configure.in b/native/configure.in
index 567894b10..e082ae6d2 100644
--- a/native/configure.in
+++ b/native/configure.in
@@ -50,6 +50,9 @@ AC_SUBST(TCN_CONFIG_LOCATION)
  AC_CANONICAL_TARGET
  AC_PROG_INSTALL
+dnl Generate the libtool script which is needed for rlibtool
+LT_INIT
+
  dnl
  dnl compute the top directory of the build
  dnl note: this is needed for LIBTOOL and exporting the bundled Expat


-
To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org
For additional commands, e-mail: dev-h...@tomcat.apache.org

Re: [tomcat] branch 9.0.x updated: Fix BZ 66548 - Add validation of Sec-Websocket-Key header

[Rainer Jung]

Am 08.06.23 um 05:50 schrieb Rainer Jung:
The new TestKeyHeader fails for me very frequently for TC 9 and TC 8.5, 
but not for 10.1 or 11.


For TC 9 it fails roughly 30-50% of the times I run it. It fails for 
jsse and for tcnative and for a wide range of JDKs and RHEL/SLES Linux 
versions.


The failure happens only for NIO2, not for NIO. When it fails, the test 
case takes about 60 seconds longer than when it is OK. The failing test 
case is often testValid - the first test case that runs - and sometime 
the second one testTooLong01.


It seems the pattern for TC 8.5 is the same but the tests are still 
running.


You probably already fixed it for 10.1 and 11, but for 9.0 and 8.5 the 
backport of 44e7282b54 seems to be missing.



Best regards,

Rainer

Am 24.05.23 um 12:29 schrieb ma...@apache.org:

This is an automated email from the ASF dual-hosted git repository.

markt pushed a commit to branch 9.0.x
in repository https://gitbox.apache.org/repos/asf/tomcat.git


The following commit(s) were added to refs/heads/9.0.x by this push:
  new 37f979762b Fix BZ 66548 - Add validation of 
Sec-Websocket-Key header

37f979762b is described below

commit 37f979762b6ce28031e8e14a3d258cb1349e05a1
Author: Mark Thomas 
AuthorDate: Tue Apr 11 08:18:43 2023 +0100

 Fix BZ 66548 - Add validation of Sec-Websocket-Key header
 Note that the validation isn't perfect. It aims to be good enough.
 https://bz.apache.org/bugzilla/show_bug.cgi?id=66548
---
  .../apache/tomcat/util/codec/binary/Base64.java    |  9 +++
  .../tomcat/websocket/server/UpgradeUtil.java   | 39 +-
  .../tomcat/websocket/server/TestKeyHeader.java | 87 
++

  .../tomcat/websocket/server/TesterWsClient.java    | 18 -
  webapps/docs/changelog.xml | 11 +++
  5 files changed, 159 insertions(+), 5 deletions(-)

diff --git a/java/org/apache/tomcat/util/codec/binary/Base64.java 
b/java/org/apache/tomcat/util/codec/binary/Base64.java

index c3b4fa9c16..884c3190d0 100644
--- a/java/org/apache/tomcat/util/codec/binary/Base64.java
+++ b/java/org/apache/tomcat/util/codec/binary/Base64.java
@@ -434,6 +434,15 @@ public class Base64 extends BaseNCodec {
  }
+    public static boolean isInAlphabet(char c) {
+    // Fast for valid data. May be slow for invalid data.
+    try {
+    return STANDARD_DECODE_TABLE[c] != -1;
+    } catch (ArrayIndexOutOfBoundsException ex) {
+    return false;
+    }
+    }
+
  /**
   * Encode table to use: either STANDARD or URL_SAFE. Note: the 
DECODE_TABLE above remains static because it is able
   * to decode both STANDARD and URL_SAFE streams, but the 
encodeTable must be a member variable so we can switch
diff --git a/java/org/apache/tomcat/websocket/server/UpgradeUtil.java 
b/java/org/apache/tomcat/websocket/server/UpgradeUtil.java

index eec6698f75..96d7aaf943 100644
--- a/java/org/apache/tomcat/websocket/server/UpgradeUtil.java
+++ b/java/org/apache/tomcat/websocket/server/UpgradeUtil.java
@@ -95,7 +95,7 @@ public class UpgradeUtil {
  return;
  }
  key = req.getHeader(Constants.WS_KEY_HEADER_NAME);
-    if (key == null) {
+    if (!validateKey(key)) {
  resp.sendError(HttpServletResponse.SC_BAD_REQUEST);
  return;
  }
@@ -224,6 +224,43 @@ public class UpgradeUtil {
  }
+    /*
+ * Validate the key. It should be the base64 encoding of a random 
16-byte value. 16-bytes are encoded in 24 base64

+ * characters, the last two of which must be ==.
+ *
+ * The validation isn't perfect:
+ *
+ * - it doesn't check the final non-'=' character is valid in the 
context of the number of bits it is meant to be

+ * encoding.
+ *
+ * - it doesn't check that the value is random and changes for 
each connection.

+ *
+ * Given that this header is for the benefit of the client, not 
the server, this should be good enough.

+ */
+    private static boolean validateKey(String key) {
+    if (key == null) {
+    return false;
+    }
+
+    if (key.length() != 24) {
+    return false;
+    }
+
+    char[] keyChars = key.toCharArray();
+    if (keyChars[22] != '=' || keyChars[23] != '=') {
+    return false;
+    }
+
+    for (int i = 0; i < 22; i++) {
+    if (!Base64.isInAlphabet(keyChars[i])) {
+    return false;
+    }
+    }
+
+    return true;
+    }
+
+
  private static List 
createTransformations(List negotiatedExtensions) {
  TransformationFactory factory = 
TransformationFactory.getInstance();
diff --git 
a/test/org/apache/tomcat/websocket/server/TestKeyHeader.java 
b/test/org/apache/tomcat/websocket/server/TestKeyHeader.java

new file mode 100644
index 00..fa05e44304
--- /dev/null
+++ b/test/org/apache/tomcat/websocket/server/TestKey

Re: [tomcat] branch 9.0.x updated: Fix BZ 66548 - Add validation of Sec-Websocket-Key header

[Rainer Jung]

The new TestKeyHeader fails for me very frequently for TC 9 and TC 8.5, 
but not for 10.1 or 11.


For TC 9 it fails roughly 30-50% of the times I run it. It fails for 
jsse and for tcnative and for a wide range of JDKs and RHEL/SLES Linux 
versions.


The failure happens only for NIO2, not for NIO. When it fails, the test 
case takes about 60 seconds longer than when it is OK. The failing test 
case is often testValid - the first test case that runs - and sometime 
the second one testTooLong01.


It seems the pattern for TC 8.5 is the same but the tests are still running.

Best regards,

Rainer

Am 24.05.23 um 12:29 schrieb ma...@apache.org:

This is an automated email from the ASF dual-hosted git repository.

markt pushed a commit to branch 9.0.x
in repository https://gitbox.apache.org/repos/asf/tomcat.git


The following commit(s) were added to refs/heads/9.0.x by this push:
  new 37f979762b Fix BZ 66548 - Add validation of Sec-Websocket-Key header
37f979762b is described below

commit 37f979762b6ce28031e8e14a3d258cb1349e05a1
Author: Mark Thomas 
AuthorDate: Tue Apr 11 08:18:43 2023 +0100

 Fix BZ 66548 - Add validation of Sec-Websocket-Key header
 
 Note that the validation isn't perfect. It aims to be good enough.

 https://bz.apache.org/bugzilla/show_bug.cgi?id=66548
---
  .../apache/tomcat/util/codec/binary/Base64.java|  9 +++
  .../tomcat/websocket/server/UpgradeUtil.java   | 39 +-
  .../tomcat/websocket/server/TestKeyHeader.java | 87 ++
  .../tomcat/websocket/server/TesterWsClient.java| 18 -
  webapps/docs/changelog.xml | 11 +++
  5 files changed, 159 insertions(+), 5 deletions(-)

diff --git a/java/org/apache/tomcat/util/codec/binary/Base64.java 
b/java/org/apache/tomcat/util/codec/binary/Base64.java
index c3b4fa9c16..884c3190d0 100644
--- a/java/org/apache/tomcat/util/codec/binary/Base64.java
+++ b/java/org/apache/tomcat/util/codec/binary/Base64.java
@@ -434,6 +434,15 @@ public class Base64 extends BaseNCodec {
  }
  
  
+public static boolean isInAlphabet(char c) {

+// Fast for valid data. May be slow for invalid data.
+try {
+return STANDARD_DECODE_TABLE[c] != -1;
+} catch (ArrayIndexOutOfBoundsException ex) {
+return false;
+}
+}
+
  /**
   * Encode table to use: either STANDARD or URL_SAFE. Note: the 
DECODE_TABLE above remains static because it is able
   * to decode both STANDARD and URL_SAFE streams, but the encodeTable must 
be a member variable so we can switch
diff --git a/java/org/apache/tomcat/websocket/server/UpgradeUtil.java 
b/java/org/apache/tomcat/websocket/server/UpgradeUtil.java
index eec6698f75..96d7aaf943 100644
--- a/java/org/apache/tomcat/websocket/server/UpgradeUtil.java
+++ b/java/org/apache/tomcat/websocket/server/UpgradeUtil.java
@@ -95,7 +95,7 @@ public class UpgradeUtil {
  return;
  }
  key = req.getHeader(Constants.WS_KEY_HEADER_NAME);
-if (key == null) {
+if (!validateKey(key)) {
  resp.sendError(HttpServletResponse.SC_BAD_REQUEST);
  return;
  }
@@ -224,6 +224,43 @@ public class UpgradeUtil {
  }
  
  
+/*

+ * Validate the key. It should be the base64 encoding of a random 16-byte 
value. 16-bytes are encoded in 24 base64
+ * characters, the last two of which must be ==.
+ *
+ * The validation isn't perfect:
+ *
+ * - it doesn't check the final non-'=' character is valid in the context 
of the number of bits it is meant to be
+ * encoding.
+ *
+ * - it doesn't check that the value is random and changes for each 
connection.
+ *
+ * Given that this header is for the benefit of the client, not the 
server, this should be good enough.
+ */
+private static boolean validateKey(String key) {
+if (key == null) {
+return false;
+}
+
+if (key.length() != 24) {
+return false;
+}
+
+char[] keyChars = key.toCharArray();
+if (keyChars[22] != '=' || keyChars[23] != '=') {
+return false;
+}
+
+for (int i = 0; i < 22; i++) {
+if (!Base64.isInAlphabet(keyChars[i])) {
+return false;
+}
+}
+
+return true;
+}
+
+
  private static List createTransformations(List 
negotiatedExtensions) {
  
  TransformationFactory factory = TransformationFactory.getInstance();

diff --git a/test/org/apache/tomcat/websocket/server/TestKeyHeader.java 
b/test/org/apache/tomcat/websocket/server/TestKeyHeader.java
new file mode 100644
index 00..fa05e44304
--- /dev/null
+++ b/test/org/apache/tomcat/websocket/server/TestKeyHeader.java
@@ -0,0 +1,87 @@
+/*
+ *  Licensed to the Apache Software Foundation (ASF) under one or more
+ *  contributor license agreements.  See the NOTICE file distributed with
+ *  this work for additional

Re: [tomcat-native] branch main updated: native: Fix the build with rlibtool

[Rainer Jung]

Something is wrong with our regeneration of configure in the release 
process, at least for the 2.x branch (main). The configure script 
contains "LT_INIT" verbatim instead of LT_INIT being replaced by its 
script implementation. I can't actually say what is wrong :(


If I run "autoreconf --force --install" on my system LT_INIT gets 
resolved but configure also get much bigger (more than double the size).


Let me know if you have no good idea and I should investigate deeper.

Thanks and regards,

Rainer

Am 31.10.22 um 21:02 schrieb ma...@apache.org:

This is an automated email from the ASF dual-hosted git repository.

markt pushed a commit to branch main
in repository https://gitbox.apache.org/repos/asf/tomcat-native.git


The following commit(s) were added to refs/heads/main by this push:
  new 54dccd3a4 native: Fix the build with rlibtool
  new 4f7fb7f44 Merge pull request #14 from orbea/slibtool
54dccd3a4 is described below

commit 54dccd3a4dc01801d9311b3160808305ec9fc2cf
Author: orbea 
AuthorDate: Thu Jul 21 17:59:14 2022 -0700

 native: Fix the build with rlibtool
 
 When building tomcat-native with slibtool using the rlibtool symlink the

 build will fail. This is because rlibtool requires the generated libtool
 script to determine if the build is shared, static or both.
 
 Gentoo bug: https://bugs.gentoo.org/778914

---
  native/configure.in | 3 +++
  1 file changed, 3 insertions(+)

diff --git a/native/configure.in b/native/configure.in
index 567894b10..e082ae6d2 100644
--- a/native/configure.in
+++ b/native/configure.in
@@ -50,6 +50,9 @@ AC_SUBST(TCN_CONFIG_LOCATION)
  AC_CANONICAL_TARGET
  AC_PROG_INSTALL
  
+dnl Generate the libtool script which is needed for rlibtool

+LT_INIT
+
  dnl
  dnl compute the top directory of the build
  dnl note: this is needed for LIBTOOL and exporting the bundled Expat


-
To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org
For additional commands, e-mail: dev-h...@tomcat.apache.org


-
To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org
For additional commands, e-mail: dev-h...@tomcat.apache.org

Re: [VOTE] Release Apache Tomcat 8.5.89

[Rainer Jung]

Am 09.05.23 um 19:38 schrieb Christopher Schultz:

The proposed Apache Tomcat 8.5.89 release is now available for voting.

The notable changes compared to 8.5.88 are:

- Many improvements to the JSON access log valve.

- Deprecate support for the HTTP Connector settings rejectIllegalHeader
    and allowHostHeaderMismatch and reject HTTP headers without names.

- Add a RateLimitFilter which can be used to mitigate DoS and Brute
    Force attacks.

Along with lots of other bug fixes and improvements.

For full details, see the changelog:
https://nightlies.apache.org/tomcat/tomcat-8.5.x/docs/changelog.html

It can be obtained from:
https://dist.apache.org/repos/dist/dev/tomcat/tomcat-8/v8.5.89/

The Maven staging repo is:
https://repository.apache.org/content/repositories/orgapachetomcat-1436

The tag is:
https://github.com/apache/tomcat/tree/8.5.89/
da91bd19ef2cb34a96e4ad04749dfc97c941db87

The proposed 8.5.89 release is:
[ ] Broken - do not release
[X] Stable - go ahead and release as 8.5.88 (stable)


+1 to release.

Tested on RHEL 6, 7 and 8, SLES 11, 12 and 15, Solaris 10 and 11 with a 
variety of JVM versions (1.7.0, 1.8.0, 11, 17, 20, 21) and vendors 
(Adoptium, Azul Zulu, Oracle, RedHat). Also tested with tcnative 1.2.36 
and 2.0.3 using OpenSSL 1.1.1t, 3.0.8 and 3.1.0.


The new RateLimitFilter fails its test, but there was a post-8.5.89 
commit that changes the test. Haven't checked, whether that fixed it.


Thanks for RM,

Rainer

-
To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org
For additional commands, e-mail: dev-h...@tomcat.apache.org

Re: [junit] WARNING: Using incubator modules: jdk.incubator.foreign

[Rainer Jung]

Replying to self: it seems that warning is part of the incubator JEP

https://openjdk.org/jeps/11

and can not be suppressed. The "--add-modules" makes the API available, 
but since incubator APIs are supposed to be non-final there is a 
mandatory warning that an applications uses them whenever an incubator 
module is resolved.


I guess we cań live with that.

Sorry for the noise,

Rainer

Am 04.05.23 um 12:40 schrieb Rainer Jung:

Hi there,

when testing TC 10.1 and 11 I get lots of

[junit] WARNING: Using incubator modules: jdk.incubator.foreign

lines in the output. This happens when testing with Java 17. Not with 11 
and not with 20 or 21. Don't know about 12-16 and 18-19.


We do have a block in build.xml that adds an --add-modules and the 
running java processes during the unit tests do have the following flags 
set:


--add-opens=java.base/java.lang=ALL-UNNAMED
--add-opens=java.base/java.io=ALL-UNNAMED
--add-opens=java.rmi/sun.rmi.transport=ALL-UNNAMED
--add-opens=java.base/java.util=ALL-UNNAMED
--add-opens=java.base/java.util.concurrent=ALL-UNNAMED
--enable-native-access=ALL-UNNAMED
--add-modules jdk.incubator.foreign

Only the java launcher process, that is the parent of those two does not 
have these flags set.


The count of such lines is two more than the number of test classes run.

Best regards,

Rainer


-
To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org
For additional commands, e-mail: dev-h...@tomcat.apache.org

[junit] WARNING: Using incubator modules: jdk.incubator.foreign

[Rainer Jung]

Hi there,

when testing TC 10.1 and 11 I get lots of

[junit] WARNING: Using incubator modules: jdk.incubator.foreign

lines in the output. This happens when testing with Java 17. Not with 11 
and not with 20 or 21. Don't know about 12-16 and 18-19.


We do have a block in build.xml that adds an --add-modules and the 
running java processes during the unit tests do have the following flags 
set:


--add-opens=java.base/java.lang=ALL-UNNAMED
--add-opens=java.base/java.io=ALL-UNNAMED
--add-opens=java.rmi/sun.rmi.transport=ALL-UNNAMED
--add-opens=java.base/java.util=ALL-UNNAMED
--add-opens=java.base/java.util.concurrent=ALL-UNNAMED
--enable-native-access=ALL-UNNAMED
--add-modules jdk.incubator.foreign

Only the java launcher process, that is the parent of those two does not 
have these flags set.


The count of such lines is two more than the number of test classes run.

Best regards,

Rainer

-
To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org
For additional commands, e-mail: dev-h...@tomcat.apache.org

Unit tests TestAccessLogValve.java

[Rainer Jung]

Hi all,

after I added some missing functionality to the JsonAccessLogValve I 
added new tests for the AbstractAccessLogValve and JsonAccessLogValve.


The tests use access log valve classes derived from the two, which write 
their output to a string. The tests check the log lines resulting from 
various access log config patterns against an expected regexp.


Feel free to add more test cases to the parameter list in the test 
class. I added at list one test for each pattern identifier plus for 
common and combined. Every pattern is tested once with the standard 
AccessLogValve and once with the Json one.


It would be nice, if some of you with a more unusual setup could run the 
tests. It shouldn't really matter, whether you run them for TC 8.5, 9, 
10.1 or 11, they are almost the same. If most of you will know, you can 
restartict the tests to the single test class by adding


  test.name=**/TestAccessLogValve.java

to build.properties.

The tests succeed in my environment and now also on the CI environment, 
but it could be, that there are eg. locale or JVM specific subtleties 
that I might have overlooked.


Thanks and regards,

Rainer

-
To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org
For additional commands, e-mail: dev-h...@tomcat.apache.org

JsonAccessLogValve

[Rainer Jung]

Hi all,

I am looking at the new access log valve to add support for the pattern 
fields with sub keys (headers, attributes etc.).


Many of those have free text values, so ensuring correct encodig is 
important. I noticed, that the AbstractAccessLogValve already does 
almost correct JSON encoding and the Json one ensures correct JSON 
encoding by again encoding the values coming out of the 
AbstractAccessLogValve. So an encoded char in a header ends up as \\u... 
in the log, not as \u


Since the encoding of the base valve is already so close to JSON, I 
wonder, whether we could remove the only difference that's there and 
then get rid of the double encoding: a vertical tab gets encoded as "\v" 
which is not known in JSON. For JSON it needs to get encoded using an \u 
sequence. We do that already for most other rarely used control 
characters. The only other characters that are simply "\"-escaped are 
also allowed in that form in JSON, so are fine.


So what about switching the encoding of vertical tab in the Abstract 
valve to \u00? Of course for TC 11 that should not be a problem, but do 
people think it would be OK to backport? It doesnt sound like a common 
character to expect, \v is not so well-known as a representation and 
using \u-style would be more consistent with any other characters. \v is 
not even a valid Java character escape (though that's not a strong 
argument here).


WDYT?

Thanks and regards,

Rainer


-
To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org
For additional commands, e-mail: dev-h...@tomcat.apache.org

Re: [tomcat] branch 8.5.x updated: Align with 9.0.x

[Rainer Jung]

Hi Mark,

Am 08.03.23 um 20:15 schrieb ma...@apache.org:
...


diff --git a/test/org/apache/tomcat/websocket/TestWsWebSocketContainerSSL.java 
b/test/org/apache/tomcat/websocket/TestWsWebSocketContainerSSL.java
new file mode 100644
index 00..eb44ac6aed
--- /dev/null
+++ b/test/org/apache/tomcat/websocket/TestWsWebSocketContainerSSL.java
@@ -0,0 +1,76 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements.  See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License.  You may obtain a copy of the License at
+ *
+ *  http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.tomcat.websocket;
+
+import java.net.URI;
+import java.util.Queue;
+import java.util.concurrent.CountDownLatch;
+import java.util.concurrent.TimeUnit;
+
+import javax.websocket.ClientEndpointConfig;
+import javax.websocket.ContainerProvider;
+import javax.websocket.Session;
+import javax.websocket.WebSocketContainer;
+
+import org.junit.Assert;
+import org.junit.Test;
+
+import org.apache.catalina.Context;
+import org.apache.catalina.servlets.DefaultServlet;
+import org.apache.catalina.startup.Tomcat;
+import org.apache.tomcat.util.net.TesterSupport;
+import org.apache.tomcat.websocket.TesterMessageCountClient.BasicText;
+import 
org.apache.tomcat.websocket.TesterMessageCountClient.TesterProgrammaticEndpoint;
+
+public class TestWsWebSocketContainerSSL extends WebSocketBaseTest {
+
+private static final String MESSAGE_STRING_1 = "qwerty";
+
+@Test
+public void testConnectToServerEndpointSSL() throws Exception {
+
+Tomcat tomcat = getTomcatInstance();
+// No file system docBase required
+Context ctx = tomcat.addContext("", null);
+ctx.addApplicationListener(TesterEchoServer.Config.class.getName());
+Tomcat.addServlet(ctx, "default", new DefaultServlet());
+ctx.addServletMappingDecoded("/", "default");
+
+TesterSupport.initSsl(tomcat);
+
+tomcat.start();
+
+WebSocketContainer wsContainer = 
ContainerProvider.getWebSocketContainer();
+ClientEndpointConfig clientEndpointConfig = 
ClientEndpointConfig.Builder.create().build();
+
clientEndpointConfig.getUserProperties().put(org.apache.tomcat.websocket.Constants.SSL_TRUSTSTORE_PROPERTY,
+TesterSupport.CA_JKS);
+Session wsSession = 
wsContainer.connectToServer(TesterProgrammaticEndpoint.class, 
clientEndpointConfig,
+new URI("wss://localhost" + ":" + getPort() + 
TesterEchoServer.Config.PATH_ASYNC));
+CountDownLatch latch = new CountDownLatch(1);
+BasicText handler = new BasicText(latch);
+wsSession.addMessageHandler(handler);
+wsSession.getBasicRemote().sendText(MESSAGE_STRING_1);
+
+boolean latchResult = handler.getLatch().await(10, TimeUnit.SECONDS);
+
+Assert.assertTrue(latchResult);
+
+Queue messages = handler.getMessages();
+Assert.assertEquals(1, messages.size());
+Assert.assertEquals(MESSAGE_STRING_1, messages.peek());
+}
+}

...

this new test fails for TC 8.5.88 with Java 1.7.0 and tcnative with 
OpenSSL 3.x.


Not failing with newer JVM version or JSSE.

tcnative is 1.2.36 OpenSSL 3.0.8, tcnative 2.0.3 build with OpenSSL 
3.0.8 or OpenSSL 3.1.0beta1. Using 1.2.36 with OpenSSL 1.1.1t is not 
failing.


Error output is:

Testsuite: org.apache.tomcat.websocket.TestWsWebSocketContainerSSL
Tests run: 1, Failures: 0, Errors: 1, Skipped: 0, Time elapsed: 2.853 sec
- Standard Error -
17-Apr-2023 00:28:31.122 INFO [main] 
org.apache.catalina.startup.LoggingBaseTest.setUp Starting test case 
[testConnectToServerEndpointSSL]
17-Apr-2023 00:28:31.506 INFO [main] 
org.apache.catalina.core.AprLifecycleListener.lifecycleEvent Loaded 
Apache Tomcat Native library [1.2.36] using APR version [1.7.2sp1].
17-Apr-2023 00:28:31.506 INFO [main] 
org.apache.catalina.core.AprLifecycleListener.lifecycleEvent APR 
capabilities: IPv6 [true], sendfile [true], accept filters [false], 
random [true], UDS [{4}].
17-Apr-2023 00:28:31.506 INFO [main] 
org.apache.catalina.core.AprLifecycleListener.lifecycleEvent APR/OpenSSL 
configuration: useAprConnector [false], useOpenSSL [true]
17-Apr-2023 00:28:31.514 INFO [main] 
org.apache.catalina.core.AprLifecycleListener.initializeSSL OpenSSL 
successfully initialized [OpenSSL 3.0.8 7 Feb 2023]
17-Apr-2023 00:28:32.406 INFO 

Re: [tomcat] branch 8.5.x updated: Support RFC 7616. Add support for multiple algorithms.

[Rainer Jung]

Hi Mark,

only with Java 1.7.0 and TC 8.5 the new test throws an ERROR. It occurs 
many times, all almost looking the same, here's an example:


16-Apr-2023 22:47:40.891 INFO [main] 
org.apache.catalina.startup.LoggingBaseTest.setUp Starting test case 
[testDigestAuthentication[12: Algorithms[MD5,SHA-256,SHA-512-256], 
Algorithm[MD5], PwdDigest[false], AuthExpected[user]]]
16-Apr-2023 22:47:40.893 WARNING [main] 
org.apache.catalina.authenticator.DigestAuthenticator.initAlgorithms 
Unable to configure DIGEST authentication to use the algorithms 
[SHA-512/256] as [{1}] is not supported by the JRE.
java.security.NoSuchAlgorithmException: SHA-512/256 
MessageDigest not available
at 
sun.security.jca.GetInstance.getInstance(GetInstance.java:159)

at java.security.Security.getImpl(Security.java:695)
at 
java.security.MessageDigest.getInstance(MessageDigest.java:167)
at 
org.apache.tomcat.util.security.ConcurrentMessageDigest.init(ConcurrentMessageDigest.java:118)
at 
org.apache.catalina.authenticator.DigestAuthenticator.initAlgorithms(DigestAuthenticator.java:248)
at 
org.apache.catalina.authenticator.DigestAuthenticator.setAlgorithms(DigestAuthenticator.java:235)
at 
org.apache.catalina.authenticator.TestDigestAuthenticatorAlgorithms.testDigestAuthentication(TestDigestAuthenticatorAlgorithms.java:171)


Probably on of the troubles supporting the ols JVM version provides.

Thanks and regards,

Rainer

-
To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org
For additional commands, e-mail: dev-h...@tomcat.apache.org

New unit test failure for JDK21 EA (new charset)

[Rainer Jung]

Hi there,

I get a (not very important) unit test failure for all tags of this 
month when running with


openjdk version "21-ea" 2023-09-19
OpenJDK Runtime Environment (build 21-ea+16-1326)
OpenJDK 64-Bit Server VM (build 21-ea+16-1326, mixed mode, sharing)

Test output is:

Testsuite: org.apache.tomcat.util.buf.TestCharsetCache
Tests run: 1, Failures: 1, Errors: 0, Skipped: 0, Time elapsed: 0.36 sec
- Standard Output ---
"gb18030-2022"
-  ---

Testcase: testAllKnownCharsets took 0.326 sec
FAILED
null
junit.framework.AssertionFailedError
	at 
org.apache.tomcat.util.buf.TestCharsetCache.testAllKnownCharsets(TestCharsetCache.java:76)
	at 
java.base/jdk.internal.reflect.DirectMethodHandleAccessor.invoke(DirectMethodHandleAccessor.java:103)



According to https://jdk.java.net/21/release-notes, this new charset was 
introduced in JVM21 EA build 12. My failure is for build 16, I have not 
tested it with the current build 18.


Thanks and regards,

Rainer

-
To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org
For additional commands, e-mail: dev-h...@tomcat.apache.org

Re: [tomcat] branch 10.1.x updated: Have 'pre-release' indicate the the release is in progress.

[Rainer Jung]

Am 28.02.23 um 09:49 schrieb Mark Thomas:

On 28/02/2023 07:22, Rémy Maucherat wrote:

 wrote:

On 2/27/23 16:09, Rémy Maucherat wrote:

On Mon, Feb 27, 2023 at 9:41 PM Mark Thomas  wrote:

On 27/02/2023 20:36, schu...@apache.org wrote:






   Have 'pre-release' indicate the the release is in progress.

   This saves one more step in the release process.


I'm not sure we want this in a release tag. Better for it to be empty.


+1, this was on purpose.


This changes the changelog when you do "ant pre-release".

You don't want the release-tag to include "Release in progress"?
Otherwise it is just blank which is confusing IMHO.


No, it should be blank. Why would a tagged release be "in progress" ?
When people pull it later from git, it would be done.


To add some background to this.

I started to add "Release in progress" to the change log for two reasons:
- Primarily so there was something in the change log in the docs
   produced by the CI for the version that was being voted on rather
   than leaving it empty as that looked odd.
- Secondarily to act as a reminder to me to update the change log with
   the actual release date once the vote finished. A blank release date
   didn't stand out whereas this text did.

My main concern with this change is that when users download a release 
and look at the change log they will see "Release in progress" for a 
release that is complete. I think that is much more confusing than the 
release date being left blank.


+1, it is.

The official release date is the date that the vote closes. That makes 
it tricky to include in the tag as we never know exactly how long 
release votes are going to last. They are typically three working days 
but that is far from fixed.


I try guessing the release date for Commons Daemon and don't think I 
have ever got it right. I end up correcting the release date of the N-1 
release in the N release which I think is potentially even more confusing.


Overall, my preference is that:
- we leave the release date blank in the tag for that release
- we update HEAD to "Release in progress" for that version while voting
   (probably at the same time as we increment the version number for the
   next development cycle)
- we update HEAD to the actual release date (or "Not released") for that
   version once the vote completes


+1

There would certainly be some benefit to automate the second step in the 
process as a post release task in Ant.


Sure.

Regards,

Rainer

-
To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org
For additional commands, e-mail: dev-h...@tomcat.apache.org

Test certs expired?

[Rainer Jung]

I think the test certs in test/org/apache/tomcat/util/net just expired. 
At least I start to get failures with


Caused by: java.security.cert.CertificateExpiredException: NotAfter: Fri 
Feb 17 15:28:35 CET 2023


and the files in git are two years old.

Best regards,

Rainer

-
To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org
For additional commands, e-mail: dev-h...@tomcat.apache.org

Re: [VOTE] Release Apache Tomcat Native 2.0.3

[Rainer Jung]

Am 10.02.23 um 22:04 schrieb Christopher Schultz:

Rainer,

On 2/10/23 15:08, Rainer Jung wrote:

Am 10.02.23 um 17:05 schrieb Christopher Schultz:

Mark,

I'm having trouble building on MacOS Ventura.

I downloaded the source tarball file from 
https://dist.apache.org/repos/dist/dev/tomcat/tomcat-connectors/native/2.0.3/source/tomcat-native-2.0.3-src.tar.gz, untarred it, and then ran the following commands:


$ ./configure --with-apr=/usr/local/Cellar/apr/1.7.2/bin/apr-1-config 
--with-ssl=/usr/local/Cellar/openssl\@3/3.0.8

checking build system type... x86_64-apple-darwin22.3.0
checking host system type... x86_64-apple-darwin22.3.0
checking target system type... x86_64-apple-darwin22.3.0
checking for a BSD-compatible install... /usr/bin/install -c
./configure: line 2754: LT_INIT: command not found
checking for working mkdir -p... yes
Tomcat Native Version: 2.0.3
checking for chosen layout... tcnative
checking for APR... yes
configure: APR 1.7.2 detected.
./configure: line 3138: cd: 
/usr/local/Cellar/apr/1.7.2/bin/apr-1-config//usr/local/opt/apr/build-1: Not a directory

   setting CC to "clang"
   setting CPP to "clang -E"
   adding 
"-I/Library/Java/JavaVirtualMachines/jdk-16.0.2.jdk/Contents/Home/include" to TCNATIVE_PRIV_INCLUDES

checking for JDK os include directory...  darwin
   adding 
"-I/Library/Java/JavaVirtualMachines/jdk-16.0.2.jdk/Contents/Home/include/darwin" to TCNATIVE_PRIV_INCLUDES

checking for gcc... clang
checking whether the C compiler works... yes
checking for C compiler default output file name... a.out
checking for suffix of executables...
checking whether we are cross compiling... no
checking for suffix of object files... o
checking whether the compiler supports GNU C... yes
checking whether clang accepts -g... yes
checking for clang option to enable C11 features... none needed
checking for OpenSSL library... /usr/local/Cellar/openssl@3/3.0.8
using openssl from /usr/local/Cellar/openssl@3/3.0.8/lib and 
/usr/local/Cellar/openssl@3/3.0.8/include

checking OpenSSL library version >= 3.0.0... ok
   adding "-I/usr/local/Cellar/openssl@3/3.0.8/include" to 
TCNATIVE_PRIV_INCLUDES
   setting TCNATIVE_LDFLAGS to 
"-L/usr/local/Cellar/openssl@3/3.0.8/lib -lssl -lcrypto"

   setting TCNATIVE_LIBS to ""
   setting TCNATIVE_LIBS to " 
-L/usr/local/Cellar/apr/1.7.2/bin/apr-1-config//usr/local/opt/apr/lib 
-lapr-1 -lpthread"

checking for apr_pollset_wakeup in -lapr-1... yes
   adding "-DHAVE_POLLSET_WAKEUP" to CFLAGS
cp: /apr_rules.mk: No such file or directory
configure: creating ./config.status
config.status: creating Makefile
config.status: executing default commands

$ make
Makefile:46: 
[redacted]/tomcat-native-2.0.3-src/native/build/rules.mk: No such 
file or directory
make: *** No rule to make target 
`[redacted]/tomcat-native-2.0.3-src/native/build/rules.mk'.  Stop.


The configure command appears to have succeeded but obviously 
something didn't work as expected.


-chris


I haven't actually looked at it and no Mac experience, but at least 
these lines look wrong:


 > ./configure: line 3138: cd:
 > 
/usr/local/Cellar/apr/1.7.2/bin/apr-1-config//usr/local/opt/apr/build-1:

 > Not a directory
...
 > -L/usr/local/Cellar/apr/1.7.2/bin/apr-1-config//usr/local/opt/apr/lib
 > -lapr-1 -lpthread"

The first should probably something like 
"/usr/local/Cellar/apr/1.7.2/build-1" the ath in the second 
"/usr/local/Cellar/apr/1.7.2/lib"


Could it be, that your /usr/local/Cellar/apr/1.7.2/bin/apr-1-config is 
broken and provides wrong paths as output?


Have a look at calling "/usr/local/Cellar/apr/1.7.2/bin/apr-1-config" 
with:


--version

--installbuilddir

--includes

--link-libtool --libs

--link-ld --libs

I run configure with "--with-apr=/usr/local/Cellar/apr/1.7.2" instead 
of "--with-apr=/usr/local/Cellar/apr/1.7.2/bin/apr-1-config", but that 
should not be the cause of your trouble.


I tried that too and it behaved in the same way.

But if I instead use the "installed" copy of apr under 
/usr/local/opt/apr then the configure command does not choke on all of 
that and I get a usable makefile.


Ah OK, taht could mean, that the one under /usr/local/Cellar/apr/1.7.2 
is just a build tree, and configure does not want a build tree but an 
installed APR. So maybe that's expected and using /usr/local/opt/apr ist 
just right.


My build is successful with quite a few warnings, mostly about 
deprecation of stuff related to OpenSSL.


Deprecation warnings currently are still expected. They have not yet 
been fixed but do not harm functionality.



I also get warnings that my libraries are too new(?):

ld: warning: dylib (/usr/local/Cellar/openssl@3/3.0.8/lib/libssl.dylib) 
was built for newer macOS version (13.0) than being linked (11.3)
ld: warning: dylib (/usr/local/opt/apr/lib/libapr-1.dyli

Re: [VOTE] Release Apache Tomcat Native 2.0.3

[Rainer Jung]

Am 10.02.23 um 17:05 schrieb Christopher Schultz:

Mark,

I'm having trouble building on MacOS Ventura.

I downloaded the source tarball file from 
https://dist.apache.org/repos/dist/dev/tomcat/tomcat-connectors/native/2.0.3/source/tomcat-native-2.0.3-src.tar.gz, untarred it, and then ran the following commands:


$ ./configure --with-apr=/usr/local/Cellar/apr/1.7.2/bin/apr-1-config 
--with-ssl=/usr/local/Cellar/openssl\@3/3.0.8

checking build system type... x86_64-apple-darwin22.3.0
checking host system type... x86_64-apple-darwin22.3.0
checking target system type... x86_64-apple-darwin22.3.0
checking for a BSD-compatible install... /usr/bin/install -c
./configure: line 2754: LT_INIT: command not found
checking for working mkdir -p... yes
Tomcat Native Version: 2.0.3
checking for chosen layout... tcnative
checking for APR... yes
configure: APR 1.7.2 detected.
./configure: line 3138: cd: 
/usr/local/Cellar/apr/1.7.2/bin/apr-1-config//usr/local/opt/apr/build-1: 
Not a directory

   setting CC to "clang"
   setting CPP to "clang -E"
   adding 
"-I/Library/Java/JavaVirtualMachines/jdk-16.0.2.jdk/Contents/Home/include" to TCNATIVE_PRIV_INCLUDES

checking for JDK os include directory...  darwin
   adding 
"-I/Library/Java/JavaVirtualMachines/jdk-16.0.2.jdk/Contents/Home/include/darwin" to TCNATIVE_PRIV_INCLUDES

checking for gcc... clang
checking whether the C compiler works... yes
checking for C compiler default output file name... a.out
checking for suffix of executables...
checking whether we are cross compiling... no
checking for suffix of object files... o
checking whether the compiler supports GNU C... yes
checking whether clang accepts -g... yes
checking for clang option to enable C11 features... none needed
checking for OpenSSL library... /usr/local/Cellar/openssl@3/3.0.8
using openssl from /usr/local/Cellar/openssl@3/3.0.8/lib and 
/usr/local/Cellar/openssl@3/3.0.8/include

checking OpenSSL library version >= 3.0.0... ok
   adding "-I/usr/local/Cellar/openssl@3/3.0.8/include" to 
TCNATIVE_PRIV_INCLUDES
   setting TCNATIVE_LDFLAGS to "-L/usr/local/Cellar/openssl@3/3.0.8/lib 
-lssl -lcrypto"

   setting TCNATIVE_LIBS to ""
   setting TCNATIVE_LIBS to " 
-L/usr/local/Cellar/apr/1.7.2/bin/apr-1-config//usr/local/opt/apr/lib 
-lapr-1 -lpthread"

checking for apr_pollset_wakeup in -lapr-1... yes
   adding "-DHAVE_POLLSET_WAKEUP" to CFLAGS
cp: /apr_rules.mk: No such file or directory
configure: creating ./config.status
config.status: creating Makefile
config.status: executing default commands

$ make
Makefile:46: [redacted]/tomcat-native-2.0.3-src/native/build/rules.mk: 
No such file or directory
make: *** No rule to make target 
`[redacted]/tomcat-native-2.0.3-src/native/build/rules.mk'.  Stop.


The configure command appears to have succeeded but obviously something 
didn't work as expected.


-chris


I haven't actually looked at it and no Mac experience, but at least 
these lines look wrong:


> ./configure: line 3138: cd:
> /usr/local/Cellar/apr/1.7.2/bin/apr-1-config//usr/local/opt/apr/build-1:
> Not a directory
...
> -L/usr/local/Cellar/apr/1.7.2/bin/apr-1-config//usr/local/opt/apr/lib
> -lapr-1 -lpthread"

The first should probably something like 
"/usr/local/Cellar/apr/1.7.2/build-1" the ath in the second 
"/usr/local/Cellar/apr/1.7.2/lib"


Could it be, that your /usr/local/Cellar/apr/1.7.2/bin/apr-1-config is 
broken and provides wrong paths as output?


Have a look at calling "/usr/local/Cellar/apr/1.7.2/bin/apr-1-config" with:

--version

--installbuilddir

--includes

--link-libtool --libs

--link-ld --libs

I run configure with "--with-apr=/usr/local/Cellar/apr/1.7.2" instead of 
"--with-apr=/usr/local/Cellar/apr/1.7.2/bin/apr-1-config", but that 
should not be the cause of your trouble.


HTH

Rainer

-
To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org
For additional commands, e-mail: dev-h...@tomcat.apache.org

Re: [tomcat-native] branch main updated: Update recommended APR version to 1.7.1

[Rainer Jung]

APR 1.7.1 will be replaced by 1.7.2, because in 1.7.1 the tarball was 
rolled with a directory containing an RC verion in its name.


So 1.7.2 will be the same source code as 1.7.1, but with a packaging 
failure fixed.


The release is expected to happen either later today or tomorrow.

Best regards,

Rainer


-
To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org
For additional commands, e-mail: dev-h...@tomcat.apache.org

TC 8.5.85 TestSsl.testPost() fails for Java 1.7.0 with OpenSSL 3.1.0beta1

[Rainer Jung]

It is probably not the most common combination, but when running TC 
8.5.85 unit tests with Java 1.7.0 plus tcnative 2.0.2 build against 
OpenSSL 3.1.0beta1, the test testPost() fails all 8 iterations it uses with:


javax.net.ssl.SSLException: ciphertext sanity check failed
at sun.security.ssl.Alerts.getSSLException(Alerts.java:208)
at sun.security.ssl.SSLSocketImpl.fatal(SSLSocketImpl.java:1904)
at sun.security.ssl.SSLSocketImpl.readRecord(SSLSocketImpl.java:981)
at sun.security.ssl.SSLSocketImpl.readDataRecord(SSLSocketImpl.java:891)
at sun.security.ssl.AppInputStream.read(AppInputStream.java:102)
at sun.security.ssl.AppInputStream.read(AppInputStream.java:69)
at org.apache.tomcat.util.net.TestSsl$1.run(TestSsl.java:131)
Caused by: javax.crypto.BadPaddingException: ciphertext sanity check failed
at sun.security.ssl.InputRecord.decrypt(InputRecord.java:147)
at sun.security.ssl.SSLSocketImpl.readRecord(SSLSocketImpl.java:976)
... 4 more

I guess this is nothing to fix for us, just wanted to mention it in case 
someones searches for it.


The test does not fail with other OpenSSL versions like 3.0.7 and also 
not with higher Java versions, just that combination. But this 
combination does fail consistently on all 6 Linux platforms plus Solaris.


Apart from that testing with OpenSSL 3.1.0beta1 looks pretty good.

Best regards,

Rainer

-
To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org
For additional commands, e-mail: dev-h...@tomcat.apache.org

Re: [tomcat] branch 10.1.x updated: Disable test for Java 16 onwards since performance is comparable

[Rainer Jung]

Any plans to backport this for TC 9? JreCompat seems to provide 
isJre16Available() for TC 9, so backport should work. I can confirm I 
still see the failures for 9.0.71, but only for Java 17 and 21 (most of 
the runs with Java 17 but not every run; Java 16 not tested, no failures 
for 8 and 11).


TC 8.5 does not have isJre16Available(), so backporting there would be a 
bit bigger.


Thanks and regards,

Rainer

Am 14.11.22 um 12:43 schrieb ma...@apache.org:

This is an automated email from the ASF dual-hosted git repository.

markt pushed a commit to branch 10.1.x
in repository https://gitbox.apache.org/repos/asf/tomcat.git


The following commit(s) were added to refs/heads/10.1.x by this push:
  new 1058eed2b6 Disable test for Java 16 onwards since performance is 
comparable
1058eed2b6 is described below

commit 1058eed2b6e94f09b8f3ecdcac3be634baa01f76
Author: Mark Thomas 
AuthorDate: Mon Nov 14 11:42:53 2022 +

 Disable test for Java 16 onwards since performance is comparable
---
  test/org/apache/tomcat/util/buf/TestMessageBytes.java | 9 -
  1 file changed, 8 insertions(+), 1 deletion(-)

diff --git a/test/org/apache/tomcat/util/buf/TestMessageBytes.java 
b/test/org/apache/tomcat/util/buf/TestMessageBytes.java
index 3311996394..4abc1b6374 100644
--- a/test/org/apache/tomcat/util/buf/TestMessageBytes.java
+++ b/test/org/apache/tomcat/util/buf/TestMessageBytes.java
@@ -23,8 +23,11 @@ import java.nio.charset.CodingErrorAction;
  import java.nio.charset.StandardCharsets;
  
  import org.junit.Assert;

+import org.junit.Assume;
  import org.junit.Test;
  
+import org.apache.tomcat.util.compat.JreCompat;

+
  public class TestMessageBytes {
  
  private static final String CONVERSION_STRING =

@@ -100,6 +103,10 @@ public class TestMessageBytes {
   */
  @Test
  public void testConversionPerformance() {
+
+// ISO_8859_1 conversion appears to be optimised in Java 16 onwards
+Assume.assumeFalse(JreCompat.isJre16Available());
+
  long optimized = -1;
  long nonOptimized = -1;
  
@@ -110,7 +117,7 @@ public class TestMessageBytes {

   * once to run the test and once more in case of unexpected CI /GC
   * slowness. The test will exit early if possible.
   *
- * MeesageBytes only optimises conversion for ISO_8859_1
+ * MessageBytes only optimises conversion for ISO_8859_1
   */
  for (int i = 0; i < 3; i++) {
  optimized = doTestOptimisedConversionPerformance();


-
To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org
For additional commands, e-mail: dev-h...@tomcat.apache.org

Re: [VOTE] Release Apache Tomcat 11.0.0-M2

[Rainer Jung]

This tag and the ones in the other branches can't be cleanly build from 
scratch due to a inconsistency in build.properties.default. See details 
in "Re: [tomcat] branch main updated: Update testing/validation 
libraries". The fix is not complicated though.


Thanks and regards,

Rainer

Am 09.01.23 um 19:20 schrieb Mark Thomas:

The proposed Apache Tomcat 11.0.0-M2 release is now available for
voting.

Apache Tomcat 11.0.0-M2 is a milestone release of the 11.0.x branch and 
has been made to provide users with early access to the new features in 
Apache Tomcat 11.0.x so that they may provide feedback. The notable 
changes compared to the previous milestone include:


- Add ByteBuffer support to ServletInputStream and ServletOutputStream

- Update Cookie parsing and handling to treat the quotes in a quoted
   cookie value as part of the value as required by RFC 6265 and
   explicitly clarified in RFC 6265bis.

- When resetting an HTTP/2 stream because the final response has been
   generated before the request has been fully read, use the HTTP/2 error
   code NO_ERROR so that client does not discard the response. Based on a
   suggestion by Lorenzo Dalla Vecchia.

For full details, see the change log:
https://nightlies.apache.org/tomcat/tomcat-11.0.x/docs/changelog.html

Applications that run on Tomcat 9 and earlier will not run on Tomcat 11 
without changes. Java EE applications designed for Tomcat 9 and earlier 
may be placed in the $CATALINA_BASE/webapps-javaee directory and Tomcat 
will automatically convert them to Jakarta EE and copy them to the 
webapps directory. Applications using deprecated APIs may require 
further changes.


It can be obtained from:
https://dist.apache.org/repos/dist/dev/tomcat/tomcat-11/v11.0.0-M2/
4b03c23ad60e678c1d1a85df815fb6cd8d14ca67

The Maven staging repo is:
https://repository.apache.org/content/repositories/orgapachetomcat-1413

The tag is:
https://github.com/apache/tomcat/tree/11.0.0-M2


The proposed 11.0.0-M2 release is:
[ ] Broken - do not release
[ ] Stable - go ahead and release as 11.0.0-M2


-
To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org
For additional commands, e-mail: dev-h...@tomcat.apache.org

Re: [tomcat] branch main updated: Update testing/validation libraries

[Rainer Jung]

Hi Mark,

Am 06.01.23 um 16:25 schrieb ma...@apache.org:

This is an automated email from the ASF dual-hosted git repository.

markt pushed a commit to branch main
in repository https://gitbox.apache.org/repos/asf/tomcat.git


The following commit(s) were added to refs/heads/main by this push:
  new 1c75ed6ec3 Update testing/validation libraries
1c75ed6ec3 is described below

commit 1c75ed6ec354e0a0b50ababc4616fdb8f922cb17
Author: Mark Thomas 
AuthorDate: Fri Jan 6 15:25:20 2023 +

 Update testing/validation libraries
---
  build.properties.default   | 14 +++---
  webapps/docs/changelog.xml |  9 +
  2 files changed, 16 insertions(+), 7 deletions(-)

diff --git a/build.properties.default b/build.properties.default
index be2c5e4851..d0c86a291a 100644
--- a/build.properties.default
+++ b/build.properties.default

...


@@ -298,7 +298,7 @@ bnd.version=6.3.1
  # checksums for biz.aQute.bnd-6.3.1.jar
  bnd.checksum.enabled=true
  bnd.checksum.algorithm=MD5|SHA-1
-bnd.checksum.value=06bb0a51c208fc2f24c8cd54216b4b6f|c00993c55ccee432dd43f540e6f7c194fe946d20
+bnd.checksum.value=a812b31a81f05767ea11896a81cd8eab|8a359edbb02aad9138b7824a0d7bbe22f02cf990
  
  bnd.home=${base.path}/bnd-${bnd.version}

  bnd.jar=${bnd.home}/biz.aQute.bnd-${bnd.version}.jar


this commit and its backports seems to have broken the build. The 
checksums for BND have been updated from the ones for 6.3.1 to the ones 
for 6.4.0, but the version itself has not changed (bnd is also not 
mentioned in the changelog snippet). So we still use bnd 6.3.1 but test 
the download against the checksums for 6.4.0, which fails - unless you 
already have 6.3.1 in place from earlier builds.


Not sure what was intended, a version update to 6.4.0 or staying at 
6.3.1 including checksums.


Thanks and regards,

Rainer


diff --git a/webapps/docs/changelog.xml b/webapps/docs/changelog.xml
index 268b2cb7d7..6a80447ebf 100644
--- a/webapps/docs/changelog.xml
+++ b/webapps/docs/changelog.xml
@@ -209,6 +209,15 @@

  Update to the Eclipse JDT compiler 4.26. (markt)

+  
+Update Checkstyle to 10.6.0. (markt)
+  
+  
+Update Unboundid to 6.0.7. (markt)
+  
+  
+Update SpotBugs to 4.7.3. (markt)
+  
  

  


-
To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org
For additional commands, e-mail: dev-h...@tomcat.apache.org

Test TestNonBlockingAPI (testNonBlockingWriteError02NoSwallow, testNonBlockingWriteError02Swallow) fail on Solaris 11

[Rainer Jung]

The tests testNonBlockingWriteError02NoSwallow() and 
testNonBlockingWriteError02Swallow() of TestNonBlockingAPI fail for me 
on Solaris 11. I am using Java 11 and TC 10.1.4 but it seems it is not 
specific to these. The test loops unterminated doing flush on Solaris 
until after 60 seconds the test server gets shutdown.


The access log entry for Solaris is

127.0.0.1 - - [12/Dec/2022:23:36:18 +0100] "POST / HTTP/1.1" 500 - 
http-nio-127.0.0.1-auto-1-exec-2 1012864


for Linux

127.0.0.1 - - [12/Dec/2022:23:28:29 +0100] "POST / HTTP/1.1" 500 - 
http-nio-127.0.0.1-auto-1-exec-2 208015


so the Solaris line does not reflect this looping for 60 seconds.

I added some debug logging to the TestWriteListener02.onWritePossible(), 
which follows. If there is more I can do to analyze please let me know. 
I assume a BZ would be good?


First the patch that generates the output lines:

diff --git 
a/test/org/apache/catalina/nonblocking/TestNonBlockingAPI.java 
b/test/org/apache/catalina/nonblocking/TestNonBlockingAPI.java

index 0db6faa900..cc8ba0d0b0 100644
--- a/test/org/apache/catalina/nonblocking/TestNonBlockingAPI.java
+++ b/test/org/apache/catalina/nonblocking/TestNonBlockingAPI.java
@@ -1659,27 +1659,36 @@ public class TestNonBlockingAPI extends 
TomcatBaseTest {

 @Override
 public void onWritePossible() throws IOException {
 try {
+log.info("TestWriteListener02 onWritePossible");
 ServletOutputStream sos = 
ac.getResponse().getOutputStream();

 do {
+log.info("sos.isReady() stage.get()==" + stage.get());
 if (stage.get() == 0) {
+log.info("Committing");
 // Commit the response
 ac.getResponse().flushBuffer();
 responseCommitLatch.countDown();
 stage.incrementAndGet();
 } else if (stage.get() == 1) {
+log.info("Waiting for client drop");
 // Wait for the client to drop the connection
 try {
+log.info("Calling clientCloseLatch.await()");
 clientCloseLatch.await();
 } catch (InterruptedException e) {
+log.info("Got InterruptedException", e);
 // Ignore
 }
+log.info("Writing TEST");
 sos.print("TEST");
 stage.incrementAndGet();
 } else if (stage.get() == 2) {
+log.info("Flushing");
 sos.flush();
 }
 } while (sos.isReady());
 } catch (IOException ioe) {
+log.info("IOException", ioe);
 if (!swallowIoException) {
 throw ioe;
 }

This shows the following behavior on Solaris:

[junit] 12-Dec-2022 23:36:22.753 FINE 
[http-nio-127.0.0.1-auto-3-exec-1] 
org.apache.coyote.AbstractProtocol$ConnectionHandler.process Created new 
processor [org.apache.coyote.http11.Http11Processor@71be62ce]
[junit] 12-Dec-2022 23:36:22.754 FINE 
[http-nio-127.0.0.1-auto-3-exec-1] 
org.apache.coyote.http11.Http11InputBuffer.fill Before fill(): 
parsingHeader: [true], parsingRequestLine: [true], 
parsingRequestLinePhase: [0], parsingRequestLineStart: [0], 
byteBuffer.position(): [0], byteBuffer.limit(): [0], end: [0]
[junit] 12-Dec-2022 23:36:22.755 FINE 
[http-nio-127.0.0.1-auto-3-exec-1] 
org.apache.tomcat.util.net.SocketWrapperBase.populateReadBuffer Socket: 
[org.apache.tomcat.util.net.NioEndpoint$NioSocketWrapper@308368f5:org.apache.tomcat.util.net.NioChannel@7f798351:java.nio.channels.SocketChannel[connected 
local=/127.0.0.1:48778 remote=/127.0.0.1:52071]], Read from buffer: [0]
[junit] 12-Dec-2022 23:36:22.757 FINE 
[http-nio-127.0.0.1-auto-3-exec-1] 
org.apache.tomcat.util.net.NioEndpoint$NioSocketWrapper.read Socket: 
[org.apache.tomcat.util.net.NioEndpoint$NioSocketWrapper@308368f5:org.apache.tomcat.util.net.NioChannel@7f798351:java.nio.channels.SocketChannel[connected 
local=/127.0.0.1:48778 remote=/127.0.0.1:52071]], Read direct from 
socket: [36]
[junit] 12-Dec-2022 23:36:22.757 FINE 
[http-nio-127.0.0.1-auto-3-exec-1] 
org.apache.coyote.http11.Http11InputBuffer.fill Received [GET / HTTP/1.1

[junit] Host: localhost:
[junit]
[junit] ]
[junit] 12-Dec-2022 23:36:22.759 FINE 
[http-nio-127.0.0.1-auto-3-exec-1] 
org.apache.tomcat.util.http.Parameters.setQueryStringCharset Set query 
string encoding to UTF-8
[junit] 12-Dec-2022 23:36:22.762 FINE 
[http-nio-127.0.0.1-auto-3-exec-1] 
org.apache.catalina.authenticator.AuthenticatorBase.invoke Security 
checking request GET /
[junit] 12-Dec-2022 23:36:22.763 FINE 
[http-nio-127.0.0.1-auto-3-exec-1] 
org.apache.cat

NIO2 with HTTP/2 wrong Timeout behaviour?

[Rainer Jung]

I observe the following behavior for 10.1.4, but it might neither be a 
regression nor version specific at all.


I am calling a JSP which writes a few bytes, then does 
"Thread.sleep(2);" and then writes the remaining things.


When calling it with HTTP/1.1 it works for all connector types.

When calling it with HTTP/2 it works for NIO, but not for NIO2. After a 
shorter time than the 20 seconds, I get an "empty reply from server" in 
curl and the browser. The request processing actually continues, but of 
course the stream is closed so the trying to write the result after 20 
seconds fails.


It happens for JSSE and tcnative.

The behavior seems to be related to the NIO2 readTimeout setting. The 
empty repky error happens exactly after the amount of time set there and 
when increasing it slightly above the sleep time in the JSP, I get the 
correct result.


Using trace logging and looking at the http2 log lines, all of those are 
the same for NIO and for NIO2.


Should I open an Issue (BZ or GH)?

At the end of the mail you can find the full log for one failing 
request. The problem happens at the SocketTimeout at 12:50:40.


Best regards,

Rainer

12:51:30,382 [exec-0] util.threads.LimitLatch (LimitLatch.java:115) - 
Counting up[exec-0] latch=1
12:51:30,407 [exec-0] util.net.SecureNio2Channel 
(SecureNio2Channel.java:432) - The SNI host name extracted for 
connection [sun.nio.ch.UnixAsynchronousSocketChannelImpl[connected 
local=/0.2.3.4:8244 remote=/1.2.3.1:58186]] was [myserver]
12:51:30,471 [exec-1] http11.Http11Nio2Protocol 
(AbstractProtocol.java:761) - Processing socket 
[org.apache.tomcat.util.net.SecureNio2Channel@39a7a1e8:sun.nio.ch.UnixAsynchronousSocketChannelImpl[connected 
local=/1.2.3.4:8244 remote=/1.2.3.1:58186]] with status [OPEN_READ]
12:51:30,472 [exec-1] http11.Http11Nio2Protocol 
(AbstractProtocol.java:777) - Found processor [null] for socket 
[org.apache.tomcat.util.net.SecureNio2Channel@39a7a1e8:sun.nio.ch.UnixAsynchronousSocketChannelImpl[connected 
local=/1.2.3.4:8244 remote=/1.2.3.1:58186]]
12:51:30,473 [exec-1] http2.ConnectionSettingsBase 
(ConnectionSettingsBase.java:66) - Connection [1], Endpoint 
[Local(client->server)], Parameter type [3] set to [100]
12:51:30,473 [exec-1] http2.ConnectionSettingsBase 
(ConnectionSettingsBase.java:66) - Connection [1], Endpoint 
[Local(client->server)], Parameter type [4] set to [65535]
12:51:30,473 [exec-1] http11.Http11Nio2Protocol 
(AbstractProtocol.java:812) - Created new processor 
[org.apache.coyote.http11.upgrade.UpgradeProcessorInternal@376048ac]
12:51:30,473 [exec-1] http2.Http2UpgradeHandler 
(Http2UpgradeHandler.java:331) - Entry, Connection [1], SocketStatus 
[OPEN_READ]
12:51:30,473 [exec-1] http2.Http2UpgradeHandler 
(Http2UpgradeHandler.java:202) - Connection [1], State [NEW]
12:51:30,480 [exec-1] http2.Http2Parser (Http2Parser.java:669) - 
Connection [1], Stream [0], Frame type [SETTINGS], Flags [0], Payload 
size [18]
12:51:30,480 [exec-1] http2.Http2UpgradeHandler 
(Http2UpgradeHandler.java:1520) - Connection [1], Stream [0], Frame type 
[SETTINGS] resulted in new overhead count of [-90]
12:51:30,480 [exec-1] http2.ConnectionSettingsBase 
(ConnectionSettingsBase.java:66) - Connection [1], Endpoint 
[Remote(server->client)], Parameter type [3] set to [100]
12:51:30,481 [exec-1] http2.Http2UpgradeHandler 
(Http2UpgradeHandler.java:1520) - Connection [1], Stream [0], Frame type 
[SETTINGS] resulted in new overhead count of [-80]
12:51:30,481 [exec-1] http2.ConnectionSettingsBase 
(ConnectionSettingsBase.java:66) - Connection [1], Endpoint 
[Remote(server->client)], Parameter type [4] set to [1073741824]
12:51:30,481 [exec-1] http2.Http2UpgradeHandler 
(Http2UpgradeHandler.java:1520) - Connection [1], Stream [0], Frame type 
[SETTINGS] resulted in new overhead count of [-70]
12:51:30,481 [exec-1] http2.ConnectionSettingsBase 
(ConnectionSettingsBase.java:66) - Connection [1], Endpoint 
[Remote(server->client)], Parameter type [2] set to [0]
12:51:30,482 [exec-1] http2.Http2UpgradeHandler 
(Http2UpgradeHandler.java:331) - Entry, Connection [1], SocketStatus 
[OPEN_READ]
12:51:30,482 [exec-1] http2.Http2UpgradeHandler 
(Http2UpgradeHandler.java:202) - Connection [1], State [CONNECTED]
12:51:30,483 [exec-1] http2.Http2Parser (Http2Parser.java:669) - 
Connection [1], Stream [0], Frame type [WINDOW_UPDATE], Flags [0], 
Payload size [4]
12:51:30,483 [exec-1] http2.Http2Parser (Http2Parser.java:430) - 
Connection [1], Stream [0], Window size increment [1073676289]
12:51:30,483 [exec-1] http2.AbstractStream (AbstractStream.java:130) - 
Connection [1], Stream [0], increase flow control window by [1073676289] 
to [1073741824]
12:51:30,483 [exec-1] http2.Http2Parser (Http2Parser.java:669) - 
Connection [1], Stream [1], Frame type [HEADERS], Flags [5], Payload 
size [52]
12:51:30,484 [exec-1] http2.StreamStateMachine 
(StreamStateMachine.java:115) - Connection [1], Stream [1], State 
changed from [null] to [IDLE]
12:51:30,484 [exec-

Re: Unit Test TestOpenSSLCipherConfigurationParser fails for OpenSSL 3.1.0-alpha1

[Rainer Jung]

Thank you Mark!

Am 05.12.22 um 16:01 schrieb Mark Thomas:

Fixed.

The changes are in the current OpenSSL main development branch which 
(currently) is configured to be 3.2.x.


Mark


On 05/12/2022 11:16, Rainer Jung wrote:

Hi there,

the following tests fail for me when using tcnative 2.0.2 build 
against OpenSSL 3.1.0-alpha1:


testHIGH: Expected 127 ciphers but got 137 for the specification 'HIGH'

testMEDIUM: Expected 10 ciphers but got 0 for the specification 'MEDIUM'

testSpecification01: Expected 115 ciphers but got 125 for the 
specification 'HIGH:!aNULL:!eNULL:!EXPORT:!DES:!RC4:!MD5'


In all three cases the reason seems to be, that the following ciphers 
have moved from MEDIUM to HIGH:


TLS_DHE_RSA_WITH_AES_128_CCM_8
TLS_DHE_RSA_WITH_AES_256_CCM_8
TLS_ECDHE_ECDSA_WITH_AES_128_CCM_8
TLS_ECDHE_ECDSA_WITH_AES_256_CCM_8
TLS_PSK_DHE_WITH_AES_128_CCM_8
TLS_PSK_DHE_WITH_AES_256_CCM_8
TLS_PSK_WITH_AES_128_CCM_8
TLS_PSK_WITH_AES_256_CCM_8
TLS_RSA_WITH_AES_128_CCM_8
TLS_RSA_WITH_AES_256_CCM_8

I could verify this by running:

/path/to/openssl301/bin/openssl ciphers -v HIGH|grep 'CCM8'|sort

AES128-CCM8    TLSv1.2 Kx=RSA  Au=RSA 
Enc=AESCCM8(128)   Mac=AEAD
AES256-CCM8    TLSv1.2 Kx=RSA  Au=RSA 
Enc=AESCCM8(256)   Mac=AEAD
DHE-PSK-AES128-CCM8    TLSv1.2 Kx=DHEPSK   Au=PSK 
Enc=AESCCM8(128)   Mac=AEAD
DHE-PSK-AES256-CCM8    TLSv1.2 Kx=DHEPSK   Au=PSK 
Enc=AESCCM8(256)   Mac=AEAD
DHE-RSA-AES128-CCM8    TLSv1.2 Kx=DH   Au=RSA 
Enc=AESCCM8(128)   Mac=AEAD
DHE-RSA-AES256-CCM8    TLSv1.2 Kx=DH   Au=RSA 
Enc=AESCCM8(256)   Mac=AEAD
ECDHE-ECDSA-AES128-CCM8    TLSv1.2 Kx=ECDH Au=ECDSA 
Enc=AESCCM8(128)   Mac=AEAD
ECDHE-ECDSA-AES256-CCM8    TLSv1.2 Kx=ECDH Au=ECDSA 
Enc=AESCCM8(256)   Mac=AEAD
PSK-AES128-CCM8    TLSv1.2 Kx=PSK  Au=PSK 
Enc=AESCCM8(128)   Mac=AEAD
PSK-AES256-CCM8    TLSv1.2 Kx=PSK  Au=PSK 
Enc=AESCCM8(256)   Mac=AEAD


It seems these are the same ciphers that changed from MEDIUM to high.

Our change lowering those ciphers to MEDIUM in our code was

https://github.com/apache/tomcat/commit/4e20c36e399a61ad173f850fcb7acc863ea4b076

which seems to have been triggered by the OpenSSL changes

https://github.com/openssl/openssl/commit/1a473d1cc67e04ae9fea517b36dc332143250cf5

https://github.com/openssl/openssl/commit/56ffcce492ffc6f36b2f0d9431e23febe054dd04

https://github.com/openssl/openssl/commit/e07102220afe4059bc45aa3d7073b7678329e26e

But these changes on the OpenSSL main branch are not part of the 3.1 
branch.


Best regards,

Rainer


-
To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org
For additional commands, e-mail: dev-h...@tomcat.apache.org

Unit Test TestOpenSSLCipherConfigurationParser fails for OpenSSL 3.1.0-alpha1

[Rainer Jung]

Hi there,

the following tests fail for me when using tcnative 2.0.2 build against 
OpenSSL 3.1.0-alpha1:


testHIGH: Expected 127 ciphers but got 137 for the specification 'HIGH'

testMEDIUM: Expected 10 ciphers but got 0 for the specification 'MEDIUM'

testSpecification01: Expected 115 ciphers but got 125 for the 
specification 'HIGH:!aNULL:!eNULL:!EXPORT:!DES:!RC4:!MD5'


In all three cases the reason seems to be, that the following ciphers 
have moved from MEDIUM to HIGH:


TLS_DHE_RSA_WITH_AES_128_CCM_8
TLS_DHE_RSA_WITH_AES_256_CCM_8
TLS_ECDHE_ECDSA_WITH_AES_128_CCM_8
TLS_ECDHE_ECDSA_WITH_AES_256_CCM_8
TLS_PSK_DHE_WITH_AES_128_CCM_8
TLS_PSK_DHE_WITH_AES_256_CCM_8
TLS_PSK_WITH_AES_128_CCM_8
TLS_PSK_WITH_AES_256_CCM_8
TLS_RSA_WITH_AES_128_CCM_8
TLS_RSA_WITH_AES_256_CCM_8

I could verify this by running:

/path/to/openssl301/bin/openssl ciphers -v HIGH|grep 'CCM8'|sort

AES128-CCM8TLSv1.2 Kx=RSA  Au=RSA 
Enc=AESCCM8(128)   Mac=AEAD
AES256-CCM8TLSv1.2 Kx=RSA  Au=RSA 
Enc=AESCCM8(256)   Mac=AEAD
DHE-PSK-AES128-CCM8TLSv1.2 Kx=DHEPSK   Au=PSK 
Enc=AESCCM8(128)   Mac=AEAD
DHE-PSK-AES256-CCM8TLSv1.2 Kx=DHEPSK   Au=PSK 
Enc=AESCCM8(256)   Mac=AEAD
DHE-RSA-AES128-CCM8TLSv1.2 Kx=DH   Au=RSA 
Enc=AESCCM8(128)   Mac=AEAD
DHE-RSA-AES256-CCM8TLSv1.2 Kx=DH   Au=RSA 
Enc=AESCCM8(256)   Mac=AEAD
ECDHE-ECDSA-AES128-CCM8TLSv1.2 Kx=ECDH Au=ECDSA 
Enc=AESCCM8(128)   Mac=AEAD
ECDHE-ECDSA-AES256-CCM8TLSv1.2 Kx=ECDH Au=ECDSA 
Enc=AESCCM8(256)   Mac=AEAD
PSK-AES128-CCM8TLSv1.2 Kx=PSK  Au=PSK 
Enc=AESCCM8(128)   Mac=AEAD
PSK-AES256-CCM8TLSv1.2 Kx=PSK  Au=PSK 
Enc=AESCCM8(256)   Mac=AEAD


It seems these are the same ciphers that changed from MEDIUM to high.

Our change lowering those ciphers to MEDIUM in our code was

https://github.com/apache/tomcat/commit/4e20c36e399a61ad173f850fcb7acc863ea4b076

which seems to have been triggered by the OpenSSL changes

https://github.com/openssl/openssl/commit/1a473d1cc67e04ae9fea517b36dc332143250cf5

https://github.com/openssl/openssl/commit/56ffcce492ffc6f36b2f0d9431e23febe054dd04

https://github.com/openssl/openssl/commit/e07102220afe4059bc45aa3d7073b7678329e26e

But these changes on the OpenSSL main branch are not part of the 3.1 branch.

Best regards,

Rainer

-
To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org
For additional commands, e-mail: dev-h...@tomcat.apache.org

TC 10.1 TestMessageBytes.testConversionPerformance() fails sometimes

[Rainer Jung]

FYI: the performance check TestMessageBytes.testConversionPerformance() 
seems to fail sometimes for various Linux variants and JVM versions. It 
fails in 13 out of 116 combinations of Linux distro, JVM version and 
provider and connector.


When failing, it complains, that the non-optimised version is faster, 
then the optimised one. The speed difference is between one and 24 percent.


The system on which the tests run is CPU limited. Most failures seem to 
happen with JDK 17, but that could be just because during those runs 
might have been most concurrent things happening on the underlying 
virtualization host.


Maybe one could move the testConversionPerformance test into a separate 
class handled by the existing test.excludePerformance?


Thanks and regards,

Rainer

-
To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org
For additional commands, e-mail: dev-h...@tomcat.apache.org

Re: [DISCUSS] EOL date for 8.5.x

[Rainer Jung]

Am 20.10.2022 um 11:01 schrieb Rémy Maucherat:

On Fri, Oct 7, 2022 at 11:28 AM Mark Thomas  wrote:


Hi all,

I don't think there is a need to make a decision on this quickly, but
based on past experience and the current discussions about Jakarta EE 11
I think this is something we need to start thinking about.

Some key facts:

- Tomcat 7.0.x reached EOL on 31 March 2021
- EOL dates for major versions tend to be 3-4 years apart
- We aim to support 3 major versions in parallel - currently 8.5.x,
9.0.x and 10.1.x.
- Tomcat 11 will implement Jakarta EE 11
- Current Jakarta EE discussions are around a release in ~1 year
- Ideally, Tomcat 8.5.x EOL would be just after Tomcat 11 is declared
stable

Based on the above I think EOL for 8.5.x should be either 31 March 2024
or 30 Sept 2024 depending on when we think Jakarta EE 11 will be released.

Jakarta EE releases have tendency to slip so I think the 30 Sept 2024 is
probably the more likely. However, it is much easier to delay an EOL
date than to bring to bring it forward so my current thinking is to
announce 31 March 2024 as the EOL date for 8.5.x and keep in mind that
we can extend that if we want to.

Thoughts?


My slides said Tomcat 8.5 will be EOL when Tomcat 11 is released, so
it seems that was the plan all along.
So easy +1 :)

About the exact date, I think it is ok to set a date even if Tomcat 11
happens to slip, Tomcat 9.0 is there.


I am also fine with 31 March 2024.

Regards,

Rainer

-
To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org
For additional commands, e-mail: dev-h...@tomcat.apache.org

Re: HTTP2 with NIO2 broken?

[Rainer Jung]

https://bz.apache.org/bugzilla/show_bug.cgi?id=66281

Thanks,

Rainer

-
To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org
For additional commands, e-mail: dev-h...@tomcat.apache.org