Re: [VOTE] Release Apache Tomcat 9.0.48

2021-06-15 Thread Rémy Maucherat
The following votes were cast:

Binding:
+1: jclere, makt, isapir, mgrigorov, remm

No other votes were cast. The vote therefore passes.

Thanks to everyone who contributed to this release.

Rémy

On Thu, Jun 10, 2021 at 12:19 PM Rémy Maucherat  wrote:

> The proposed Apache Tomcat 9.0.48 release is now available for voting.
>
> The notable changes compared to 9.0.46 are:
>
> - Improve robustness of HTTP/2 HPACK decoding
>
> - Improvements to the handling of the Transfer-Encoding header
>
> - Review code used to generate Java source from JSPs and tags and remove
>code found to be unnecessary.
>
> - Backport the updated blocking NIO code and optimizations from Tomcat
> 10.0.
>
> Along with lots of other bug fixes and improvements.
>
> For full details, see the changelog:
> https://ci.apache.org/projects/tomcat/tomcat-9.0.x/docs/changelog.html
>
> It can be obtained from:
> https://dist.apache.org/repos/dist/dev/tomcat/tomcat-9/v9.0.48/
> The Maven staging repo is:
> https://repository.apache.org/content/repositories/orgapachetomcat-1315
> The tag is:
> https://github.com/apache/tomcat/tree/9.0.48
> 327fb6ff6de541e3d3e321a434d13585cea07875
>
> The proposed 9.0.48 release is:
> [ ] Broken - do not release
> [ ] Stable - go ahead and release as 9.0.48 (stable)
>
> Rémy
>


Re: [VOTE] Release Apache Tomcat 9.0.48

2021-06-15 Thread Rémy Maucherat
On Thu, Jun 10, 2021 at 12:19 PM Rémy Maucherat  wrote:

> The proposed Apache Tomcat 9.0.48 release is now available for voting.
>
> The notable changes compared to 9.0.46 are:
>
> - Improve robustness of HTTP/2 HPACK decoding
>
> - Improvements to the handling of the Transfer-Encoding header
>
> - Review code used to generate Java source from JSPs and tags and remove
>code found to be unnecessary.
>
> - Backport the updated blocking NIO code and optimizations from Tomcat
> 10.0.
>
> Along with lots of other bug fixes and improvements.
>
> For full details, see the changelog:
> https://ci.apache.org/projects/tomcat/tomcat-9.0.x/docs/changelog.html
>
> It can be obtained from:
> https://dist.apache.org/repos/dist/dev/tomcat/tomcat-9/v9.0.48/
> The Maven staging repo is:
> https://repository.apache.org/content/repositories/orgapachetomcat-1315
> The tag is:
> https://github.com/apache/tomcat/tree/9.0.48
> 327fb6ff6de541e3d3e321a434d13585cea07875
>
> The proposed 9.0.48 release is:
> [ ] Broken - do not release
> [X] Stable - go ahead and release as 9.0.48 (stable)
>
> Rémy


Re: [VOTE] Release Apache Tomcat 8.5.68

2021-06-11 Thread Rémy Maucherat
On Fri, Jun 11, 2021 at 4:01 PM Christopher Schultz <
ch...@christopherschultz.net> wrote:

> The proposed Apache Tomcat 8.5.68 release is now available for voting.
>
> The notable changes compared to the (cancelled) 8.5.67 release is:
>
> - Windows installer / uninstaller are now properly-signed.
>
> The notable changes compared to the 8.5.66 release are:
>
> - Improve robustness of HTTP/2 HPACK decoding
>
> - Improvements to the handling of the Transfer-Encoding header
>
> - Review code used to generate Java source from JSPs and tags and remove
> code found to be unnecessary.
>
> Along with lots of other bug fixes and improvements.
>
> For full details, see the changelog:
> https://ci.apache.org/projects/tomcat/tomcat-8.5.x/docs/changelog.html
>
> It can be obtained from:
> https://dist.apache.org/repos/dist/dev/tomcat/tomcat-8/v8.5.68/
>
> The Maven staging repo is:
> https://repository.apache.org/content/repositories/orgapachetomcat-1317/
>
> The tag is:
> https://github.com/apache/tomcat/tree/8.5.68
> 06846f94cda226d1ed0ff7fe05faff177d5b20b6
>
> The proposed 8.5.68 release is:
> [ ] Broken - do not release
> [X] Stable - go ahead and release as 8.5.68
>
> Rémy


Re: Cleaning up the KEYS file

2021-06-11 Thread Rémy Maucherat
On Thu, Jun 10, 2021 at 5:28 PM Mark Thomas  wrote:

> Hi all,
>
> This came up while discussing the current set of releases with the
> new/returning release managers.
>
> The official ASF recommendation is a single KEYS file under
> https://downloads.apache.org/tomcat/
> (source https://dist.apache.org/repos/dist/release/tomcat/)
>
> We have multiple KEYS files one level down. i.e., Tomcat 10, Tomcat 9, etc.
>
> First question is are we happy with having multiple KEYS files one level
> down?
> On the plus side, it means smaller KEYS files.
> On the down side, it means a little more management may be required.
>

Ok why not. I agree it's a mess right now ;)


>
> Personally, I like the one level down approach. It means we don't have
> to have an every growing KEYS file which for a project the age of Tomcat
> is a good thing.
>
> The second question (assuming we are happy with the one level down
> approach) is do we want to clean-up some/all of the existing KEYS files?
> This would mean aligning:
> - https://downloads.apache.org/tomcat/tomcat-X/KEYS file
> - the KEYS file in the release branch in the source repo
> - the list of RMs for that specific branch (or sub-project)
>
> I'd be happy to see this alignment happen but it isn't particularly high
> up my list of priorities at the moment.
>

Rémy


Re: [VOTE] Release Apache Tomcat 8.5.67

2021-06-11 Thread Rémy Maucherat
On Fri, Jun 11, 2021 at 12:35 AM Christopher Schultz <
ch...@christopherschultz.net> wrote:

> The proposed Apache Tomcat 8.5.67 release is now available for voting.
>
> The notable changes compared to the 8.5.66 release are:
>
> - Improve robustness of HTTP/2 HPACK decoding
>
> - Improvements to the handling of the Transfer-Encoding header
>
> - Review code used to generate Java source from JSPs and tags and remove
> code found to be unnecessary.
>
> Along with lots of other bug fixes and improvements.
>
> For full details, see the changelog:
> https://ci.apache.org/projects/tomcat/tomcat-8.5.x/docs/changelog.html
>
> It can be obtained from:
> https://dist.apache.org/repos/dist/dev/tomcat/tomcat-8/v8.5.67/
>
> The Maven staging repo is:
> https://repository.apache.org/content/repositories/orgapachetomcat-1316/
>
> The tag is:
> https://github.com/apache/tomcat/tree/8.5.67
> 64f53975de68f57a1aef663d3ff05d4cca7d9a5b
>
> The proposed 8.5.67 release is:
> [ ] Broken - do not release
> [ ] Stable - go ahead and release as 8.5.67
>

I have an issue with the signature of the Windows installer, see the slack
channel for more details. Not sure what's going on, there is a signature,
but ...

Rémy


Re: [VOTE] Release Apache Tomcat 10.0.7

2021-06-10 Thread Rémy Maucherat
On Tue, Jun 8, 2021 at 7:47 PM Mark Thomas  wrote:

> The proposed Apache Tomcat 10.0.7 release is now available for
> voting.
>
> Apache Tomcat 10.x implements Jakarta EE 9 and, as such, the primary
> package for all the specification APIs has changed from javax.* to
> jakarta.*
>
> Applications that run on Tomcat 9 will not run on Tomcat 10 without
> changes. Java EE applications designed for Tomcat 9 and earlier may be
> placed in the $CATALINA_BASE/webapps-javaee directory and Tomcat will
> automatically convert them to Jakarta EE and copy them to the webapps
> directory
>
> The notable changes compared to 10.0.6 are:
>
> - Improve robustness of HTTP/2 HPACK decoding
>
> - Improvements to the handling of the Transfer-Encoding header
>
> - Review code used to generate Java source from JSPs and tags and remove
>code found to be unnecessary.
>
> Along with lots of other bug fixes and improvements.
>
> For full details, see the changelog:
> https://ci.apache.org/projects/tomcat/tomcat-10.0.x/docs/changelog.html
>
> It can be obtained from:
> https://dist.apache.org/repos/dist/dev/tomcat/tomcat-10/v10.0.7/
> The Maven staging repo is:
> https://repository.apache.org/content/repositories/orgapachetomcat-1313
> The tag is:
> https://github.com/apache/tomcat/tree/10.0.7
> f0213194669c0d6ac9d60d564f198e3fcf47cbf9
>
> The proposed 10.0.7 release is:
> [ ] Broken - do not release
> [X] Stable - go ahead and release as 10.0.7 (stable)
>

Rémy

>
> -
> To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org
> For additional commands, e-mail: dev-h...@tomcat.apache.org
>
>


Re: [VOTE] Release Apache Tomcat 10.1.0-M1

2021-06-10 Thread Rémy Maucherat
On Tue, Jun 8, 2021 at 5:37 PM Mark Thomas  wrote:

> The proposed Apache Tomcat 10.1.0-M1 release is now available for
> voting.
>
> Applications that run on Tomcat 9 and earlier will not run on Tomcat 10
> without changes. Java EE applications designed for Tomcat 9 and earlier
> may be placed in the $CATALINA_BASE/webapps-javaee directory and Tomcat
> will automatically convert them to Jakarta EE and copy them to the
> webapps directory.
>
> This is the first milestone release of the 10.1.x branch.
>
> The notable changes compared to 10.0.x are:
>
> - Remove code (but not the APR/Native Connector) previously marked for
>removal in 10.1.x The APR/Native Connector will almost certainly be
>removed in a future milestone.
>
> - Align the Servlet API implementation with the current Servlet API
>development branch.
>
> - Align the EL API implementation with the current El API development
>branch.
>
> For full details, see the changelog:
> https://ci.apache.org/projects/tomcat/tomcat-10.1.x/docs/changelog.html
>
> It can be obtained from:
> f2ab9ac8bc3f40ee9b2cb50b030c99df927f0429
> The Maven staging repo is:
> https://repository.apache.org/content/repositories/orgapachetomcat-1312
> The tag is:
> https://github.com/apache/tomcat/tree/10.1.0-M1
> f2ab9ac8bc3f40ee9b2cb50b030c99df927f0429
>
> The proposed 10.1.0-M1 release is:
> [ ] Broken - do not release
> [X] Alpha - go ahead and release as 10.1.0-M1 (alpha)
>

Seems perfect for a first WIP build.

Rémy


>
> -
> To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org
> For additional commands, e-mail: dev-h...@tomcat.apache.org
>
>


[VOTE] Release Apache Tomcat 9.0.48

2021-06-10 Thread Rémy Maucherat
The proposed Apache Tomcat 9.0.48 release is now available for voting.

The notable changes compared to 9.0.46 are:

- Improve robustness of HTTP/2 HPACK decoding

- Improvements to the handling of the Transfer-Encoding header

- Review code used to generate Java source from JSPs and tags and remove
   code found to be unnecessary.

- Backport the updated blocking NIO code and optimizations from Tomcat 10.0.

Along with lots of other bug fixes and improvements.

For full details, see the changelog:
https://ci.apache.org/projects/tomcat/tomcat-9.0.x/docs/changelog.html

It can be obtained from:
https://dist.apache.org/repos/dist/dev/tomcat/tomcat-9/v9.0.48/
The Maven staging repo is:
https://repository.apache.org/content/repositories/orgapachetomcat-1315
The tag is:
https://github.com/apache/tomcat/tree/9.0.48
327fb6ff6de541e3d3e321a434d13585cea07875

The proposed 9.0.48 release is:
[ ] Broken - do not release
[ ] Stable - go ahead and release as 9.0.48 (stable)

Rémy


Re: [tomcat] branch main updated: Integrate JSign for cross-platform builds with signed Windows binaries

2021-06-10 Thread Rémy Maucherat
On Thu, Jun 10, 2021 at 12:08 AM Emmanuel Bourg  wrote:

> Le 2021-06-09 21:09, Rémy Maucherat a écrit :
>
> > Caused by: sun.security.pkcs11.wrapper.PKCS11Exception:
> > CKR_FUNCTION_FAILED
> > at
> >
> jdk.crypto.cryptoki/sun.security.pkcs11.wrapper.PKCS11.C_SignFinal(Native
> > Method)
> > at
> >
> jdk.crypto.cryptoki/sun.security.pkcs11.P11Signature.engineSign(P11Signature.java:635)
> > ... 12 more
> > Try `java -jar jsign.jar --help' for more information.
> >
> > The cfg file is:
> > name=DigiCertONE
> > library="/home/remm/.digicertone/smpkcs11.so"
> > slotListIndex=0
> >
> > The .so is there (otherwise it would complain earlier). Also the smctl
> > tool
> > shows the key. I tried other algorithms but no success so far.
>
>
> You can try adding -Djava.security.debug=sunpkcs11, it should provide
> more info.
>

https://pastebin.com/nqNUix6j
So I think it shows the security provider [why was this hacked in as a fake
token card ??] works on init, but I didn't get any extra details on the
error.

Now I will try again with a clean environment instead of my bleeding edge
Fedora stuff.

Rémy


>
> Emmanuel Bourg
>
> -
> To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org
> For additional commands, e-mail: dev-h...@tomcat.apache.org
>
>


Re: [tomcat] branch main updated: Integrate JSign for cross-platform builds with signed Windows binaries

2021-06-09 Thread Rémy Maucherat
On Wed, Jun 9, 2021 at 6:40 PM Mark Thomas  wrote:

> On 09/06/2021 17:36, ma...@apache.org wrote:
> > This is an automated email from the ASF dual-hosted git repository.
> >
> > markt pushed a commit to branch main
> > in repository https://gitbox.apache.org/repos/asf/tomcat.git
> >
> >
> > The following commit(s) were added to refs/heads/main by this push:
> >   new 9f391c9  Integrate JSign for cross-platform builds with signed
> Windows binaries
> > 9f391c9 is described below
> >
> > commit 9f391c998ee9adbc22acce2bbabbc2c6b8fc4172
> > Author: Mark Thomas 
> > AuthorDate: Wed Jun 9 17:36:25 2021 +0100
> >
> >  Integrate JSign for cross-platform builds with signed Windows
> binaries
>
> The signing works on Linux. I'm just testing it on Windows before
> back-porting.
>

-installer-sign-uninstaller:
[jsign] Adding Authenticode signature to
/home/remm/Work/releases/tomcat-9.0.47/output/dist/Uninstall.exe

BUILD FAILED
/home/remm/Work/releases/tomcat-9.0.47/build.xml:2615: Couldn't sign
/home/remm/Work/releases/tomcat-9.0.47/output/dist/Uninstall.exe

With the command line and after getting a real standalone JVM, I'm still
getting:
[remm@omni releases]$ java -jar libs/jsign-3.1/jsign-3.1.jar --keystore
~/.digicertone/pkcs11properties.cfg --storepass NONE --storetype PKCS11
--alias "Tomcat-PMC-key-2021-04" --alg SHA-512 --tsaurl
http://timestamp.digicert.com tomcat-9.0.47/output/dist/Uninstall.exe
Adding Authenticode signature to tomcat-9.0.47/output/dist/Uninstall.exe
jsign: Couldn't sign tomcat-9.0.47/output/dist/Uninstall.exe
java.security.ProviderException:
sun.security.pkcs11.wrapper.PKCS11Exception: CKR_FUNCTION_FAILED
at
jdk.crypto.cryptoki/sun.security.pkcs11.P11Signature.engineSign(P11Signature.java:685)
at
java.base/java.security.Signature$Delegate.engineSign(Signature.java:1404)
at java.base/java.security.Signature.sign(Signature.java:713)
at
net.jsign.bouncycastle.operator.jcajce.JcaContentSignerBuilder$1.getSignature(Unknown
Source)
at net.jsign.bouncycastle.cms.SignerInfoGenerator.generate(Unknown Source)
at net.jsign.bouncycastle.cms.CMSSignedDataGenerator.generate(Unknown
Source)
at net.jsign.bouncycastle.cms.CMSSignedDataGenerator.generate(Unknown
Source)
at
net.jsign.asn1.authenticode.AuthenticodeSignedDataGenerator.generate(AuthenticodeSignedDataGenerator.java:50)
at
net.jsign.AuthenticodeSigner.createSignedData(AuthenticodeSigner.java:368)
at net.jsign.AuthenticodeSigner.sign(AuthenticodeSigner.java:339)
at net.jsign.SignerHelper.sign(SignerHelper.java:424)
at net.jsign.JsignCLI.execute(JsignCLI.java:111)
at net.jsign.JsignCLI.main(JsignCLI.java:40)
Caused by: sun.security.pkcs11.wrapper.PKCS11Exception: CKR_FUNCTION_FAILED
at
jdk.crypto.cryptoki/sun.security.pkcs11.wrapper.PKCS11.C_SignFinal(Native
Method)
at
jdk.crypto.cryptoki/sun.security.pkcs11.P11Signature.engineSign(P11Signature.java:635)
... 12 more
Try `java -jar jsign.jar --help' for more information.

The cfg file is:
name=DigiCertONE
library="/home/remm/.digicertone/smpkcs11.so"
slotListIndex=0

The .so is there (otherwise it would complain earlier). Also the smctl tool
shows the key. I tried other algorithms but no success so far.

Rémy


> Mark
>
> -
> To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org
> For additional commands, e-mail: dev-h...@tomcat.apache.org
>
>


Re: [VOTE][CANCELLED] Release Apache Tomcat 9.0.47

2021-06-09 Thread Rémy Maucherat
On Wed, Jun 9, 2021 at 12:20 PM Mark Thomas  wrote:

> On 09/06/2021 08:09, Rémy Maucherat wrote:
> > TLDR: Please review the packaging / signatures / whatever even though the
> > 9.0.47 release is cancelled !
>
> I can't start Tomcat on Java 8. Lots of:
>
> 09-Jun-2021 11:12:46.489 SEVERE [http-nio-8080-Acceptor]
> org.apache.tomcat.util.net.NioEndpoint.setSocketOptions Error setting
> socket options
> java.lang.NoSuchMethodError:
> java.nio.ByteBuffer.clear()Ljava/nio/ByteBuffer;
> at
> org.apache.tomcat.util.net
> .SocketBufferHandler.reset(SocketBufferHandler.java:213)
> at org.apache.tomcat.util.net
> .NioChannel.reset(NioChannel.java:59)
> at
> org.apache.tomcat.util.net
> .NioEndpoint.setSocketOptions(NioEndpoint.java:488)
> at
> org.apache.tomcat.util.net
> .NioEndpoint.setSocketOptions(NioEndpoint.java:79)
> at org.apache.tomcat.util.net
> .Acceptor.run(Acceptor.java:126)
> at java.lang.Thread.run(Thread.java:748)
>
>
> It appears the release was built with Java 11. That won't work. It needs
> to be built with Java 8. Well, strictly, it needs to be compiled with
> Java 8.
>

Oops. That's easy to fix thankfully. The JVM default changed not that long
ago to Java 11 on my Linux, that makes the compile step error prone.

>
> I also noticed that the Tomcat Installer for Windows was not signed. I
> think you said you were building on Linux. I haven't tested the Windows
> exe signing working on Linux but the docs suggest it is possible.
> Investigating this has been on my TODO list for a while. I'll take a look.
>

I'll continue trying but the smctl tool (even the Linux version) doesn't
display the certificate.

Rémy


>
> Mark
>
> -
> To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org
> For additional commands, e-mail: dev-h...@tomcat.apache.org
>
>


[VOTE][CANCELLED] Release Apache Tomcat 9.0.47

2021-06-09 Thread Rémy Maucherat
TLDR: Please review the packaging / signatures / whatever even though the
9.0.47 release is cancelled !

The proposed Apache Tomcat 9.0.47 release is now available for
voting.

The notable changes compared to 9.0.46 are:

- Improve robustness of HTTP/2 HPACK decoding

- Improvements to the handling of the Transfer-Encoding header

- Review code used to generate Java source from JSPs and tags and remove
   code found to be unnecessary.

- Backport the updated blocking NIO code and optimizations from Tomcat 10.0.

Along with lots of other bug fixes and improvements.

For full details, see the changelog:
https://ci.apache.org/projects/tomcat/tomcat-9.0.x/docs/changelog.html


It can be obtained from:
https://dist.apache.org/repos/dist/dev/tomcat/tomcat-9/v9.0.47/
The Maven staging repo is:
https://repository.apache.org/content/repositories/orgapachetomcat-1314/
The tag is:
https://github.com/apache/tomcat/tree/9.0.47
e2febebd2fffc10764f70bdd7a3c879f571b3795

The proposed 9.0.47 release is:
[X] Broken - do not release

Broken due to an issue with the reflection code generator that did not get
updated after the NIO backport (one introspected class is now missing).
Unfortunately this part is not usually run so I never noticed it until I
did the release target.

Rémy


Re: Tagging 10.1.x, 10.0.x, 9.0.x and 8.5.x

2021-06-08 Thread Rémy Maucherat
On Fri, Jun 4, 2021 at 8:14 PM Mark Thomas  wrote:

> On 04/06/2021 18:35, Rémy Maucherat wrote:
> > On Fri, Jun 4, 2021 at 7:09 PM Mark Thomas  wrote:
> >
> >> Hi all,
> >>
> >> It looks like the mirrors are going to need a little more time for
> >> 1.2.30 to replicate before I can update the release branches to use the
> >> new release. I also still have a few odds and ends I want to finish off
> >> before tagging so it is looking like the tags will happen on Monday 7th
> >> June.
> >>
> >> Chris, how are you getting on with getting set up to release 8.5.x? Can
> >> I help at all?
> >>
> >> Remy, did you make a decision on being RM for 9.0.x?
> >>
> >
> > Let's do this. Do I get a support hotline for the first one ? My last one
> > was 6.0.20 ...
>
> Absolutely. Let me just get a premium rate phone number set up... ;)
>
> Seriously, ping me off-list or on the Tomcat slack channel if you have
> any questions.
>

Since you seem to be about to tag, can you also tag the 9.0 branch ? I
don't want it to be treated differently from the three others this time
around, and I'll (try to) handle it from there.

Rémy


>
> Mark
>
> -
> To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org
> For additional commands, e-mail: dev-h...@tomcat.apache.org
>
>


Re: Tagging 10.1.x, 10.0.x, 9.0.x and 8.5.x

2021-06-04 Thread Rémy Maucherat
On Fri, Jun 4, 2021 at 7:09 PM Mark Thomas  wrote:

> Hi all,
>
> It looks like the mirrors are going to need a little more time for
> 1.2.30 to replicate before I can update the release branches to use the
> new release. I also still have a few odds and ends I want to finish off
> before tagging so it is looking like the tags will happen on Monday 7th
> June.
>
> Chris, how are you getting on with getting set up to release 8.5.x? Can
> I help at all?
>
> Remy, did you make a decision on being RM for 9.0.x?
>

Let's do this. Do I get a support hotline for the first one ? My last one
was 6.0.20 ...

Rémy


>
> Thanks,
>
> Mark
>
> -
> To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org
> For additional commands, e-mail: dev-h...@tomcat.apache.org
>
>


Re: [VOTE] Release Apache Tomcat Native 1.2.30

2021-06-03 Thread Rémy Maucherat
On Tue, Jun 1, 2021 at 11:53 AM Mark Thomas  wrote:

> Resending with correct subject line...
>
> Version 1.2.30 includes the following changes compared to 1.2.28
>
> - Fix an issue where some Windows systems in some configurations would
>only listen on IPv6 addresses on dual stack systems even though
>configured to listen on both IPv6 and IPv4 addresses.
>
> - Complete the fix for BZ 65181
>
> - Correct constants used for Windows versions. Expand versions
>supported.
>
> The proposed release artefacts can be found at [1],
> and the build was done using tag [2].
>
> The Apache Tomcat Native 1.2.30 release is
>   [X] Stable, go ahead and release
>   [ ] Broken because of ...
>

Rémy

>
> Thanks,
>
> Mark
>
>
> [1]
>
> https://dist.apache.org/repos/dist/dev/tomcat/tomcat-connectors/native/1.2.30
> [2]
>
> https://gitbox.apache.org/repos/asf?p=tomcat-native.git;a=commit;h=8b1baaa1af07c07a9f44f9adb31b7dbdabd6ff39
>
> -
> To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org
> For additional commands, e-mail: dev-h...@tomcat.apache.org
>
>


Re: Renamings with APR removal in 10.1.x

2021-05-26 Thread Rémy Maucherat
On Wed, May 26, 2021 at 3:29 PM Mark Thomas  wrote:

> On 25/05/2021 17:27, Michael Osipov wrote:
> > Mark,
> >
> > since you are going to remove all bits soon, will you rename Java items
> > with still carry the APR substring? E.g., AprLifecycleListener will be a
> > misleading name for obvious reasons.
>
> I hadn't considered that but it seems like a good opportunity to remove
> some of the confusion that that naming has caused. I think a switch to
> TomcatNativeLifecylcleListener would work.
>
> My expectation is that in the early milestone releases for 10.1.x that
> the listener looks for Tomcat Native 1.2.x but at some point before the
> first stable release it switches to Tomcat Native 2.0.x. That assumes we
> find the time to create Tomcat Native 2.0.x. Given that it will mostly
> be deleting code, that should be doable.
>

Nice plan.

Rémy


>
> Mark
>
> -
> To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org
> For additional commands, e-mail: dev-h...@tomcat.apache.org
>
>


Re: Buildbot renaming

2021-05-25 Thread Rémy Maucherat
On Tue, May 25, 2021 at 11:33 AM Mark Thomas  wrote:

> All,
>
> Looking ahead to when we will have a 9.10.x branch, the naming of the
> builds and output directories in buildbot isn't ideal. I'd like to move
> everything over to the n.n.x naming convention. That would mean the
> following changes:
>
> tomcat-trunk-> tomcat-10.1.x
> outputs to
> tomcat10-> tomcat-10.1.x
>
> tomcat-9-trunk  -> tomcat-9.0.x
> outputs to
> tomcat9 -> tomcat-9.0.x
>
> tomcat-85-trunk -> tomcat-8.5.x
> outputs to
> tomcat85-> tomcat-8.5.x
>
> tomcat-7-trunk  -> tomcat-7.0.x
> outputs to
> tomcat7 -> tomcat-7.0.x
>
> Changing the builder name means the build counter will restart at 0.
> We can retain the old output directories until we have a reasonable
> history of builds in the new locations. I've been periodically cleaning
> out old builds and keeping ~100 old ones and no-one has noticed - well,
> no-one has complained ;) - so removing the old dirs once we have ~100
> builds in the new locations seems reasonable.
>
> Thoughts?
>

+1, it's more consistent.
Too bad, I knew the output URLs quite well by now ;)

Rémy


>
> Mark
>
> -
> To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org
> For additional commands, e-mail: dev-h...@tomcat.apache.org
>
>


Re: [tomcat] branch main updated: Remove code deprecated in 10.1.x apart from the APR Endpoint

2021-05-24 Thread Rémy Maucherat
On Mon, May 24, 2021 at 6:34 PM  wrote:

> This is an automated email from the ASF dual-hosted git repository.
>
> markt pushed a commit to branch main
> in repository https://gitbox.apache.org/repos/asf/tomcat.git
>
>
> The following commit(s) were added to refs/heads/main by this push:
>  new 620e06c  Remove code deprecated in 10.1.x apart from the APR
> Endpoint
> 620e06c is described below
>
> commit 620e06c468a02ca98dc4beacf556dd12d2440877
> Author: Mark Thomas 
> AuthorDate: Mon May 24 17:34:09 2021 +0100
>
> Remove code deprecated in 10.1.x apart from the APR Endpoint
>
> Ah, it's *that* APR time again :)

Also another question is if this branch moves to Java 11. Has it been
decided officially for Jakarta 10 ?

Rémy


Re: [tomcat] branch main updated: Correction. Currently, next iteration of the Servlet spec is 5.1

2021-05-24 Thread Rémy Maucherat
On Mon, May 24, 2021 at 1:26 PM  wrote:

> This is an automated email from the ASF dual-hosted git repository.
>
> markt pushed a commit to branch main
> in repository https://gitbox.apache.org/repos/asf/tomcat.git
>
>
> The following commit(s) were added to refs/heads/main by this push:
>  new 502b6fe  Correction. Currently, next iteration of the Servlet
> spec is 5.1
> 502b6fe is described below
>
> commit 502b6fe021fbc85164401cf7edd81542a62b9260
> Author: Mark Thomas 
> AuthorDate: Mon May 24 12:25:14 2021 +0100
>
> Correction. Currently, next iteration of the Servlet spec is 5.1
>

Seems 6.0 to me here: https://projects.eclipse.org/projects/ee4j.servlet
Right ?

But I don't see any actual content or changes.

Rémy


> ---
>  BUILDING.txt  |  2 +-
>  RELEASE-NOTES |  2 +-
>  build.xml |  2 +-
>  java/jakarta/el/ImportHandler.java|  4 ++--
>  java/jakarta/servlet/ServletContext.java  | 12 ++--
>  java/org/apache/catalina/connector/CoyoteAdapter.java |  2 +-
>  webapps/docs/class-loader-howto.xml   |  2 +-
>  webapps/docs/index.xml|  2 +-
>  webapps/docs/project.xml  |  2 +-
>  9 files changed, 15 insertions(+), 15 deletions(-)
>
> diff --git a/BUILDING.txt b/BUILDING.txt
> index 9bcd4b8..9c6d883 100644
> --- a/BUILDING.txt
> +++ b/BUILDING.txt
> @@ -20,7 +20,7 @@
>  
>
>  This project contains the source code for Tomcat @VERSION_MAJOR_MINOR@,
> a container that
> -implements the Jakarta Servlet 6.0, JSP 3.0, EL 4.0, WebSocket 2.0 and
> +implements the Jakarta Servlet 5.1, JSP 3.0, EL 4.0, WebSocket 2.0 and
>  Authentication 2.0 specifications from the Jakarta EE project at Eclipse
>  .
>
> diff --git a/RELEASE-NOTES b/RELEASE-NOTES
> index 1a02fb7..b90c48b 100644
> --- a/RELEASE-NOTES
> +++ b/RELEASE-NOTES
> @@ -78,7 +78,7 @@ for use by web applications (by placing them in "lib"):
>  * jasper.jar (Jasper 2 Compiler and Runtime)
>  * jasper-el.jar (Jasper 2 EL implementation)
>  * jsp-api.jar (JSP 3.0 API)
> -* servlet-api.jar (Servlet 6.0 API)
> +* servlet-api.jar (Servlet 5.1 API)
>  * tomcat-api.jar (Interfaces shared by Catalina and Jasper)
>  * tomcat-coyote.jar (Tomcat connectors and utility classes)
>  * tomcat-dbcp.jar (package renamed database connection pool based on
> Commons DBCP 2)
> diff --git a/build.xml b/build.xml
> index 48f726b..f244afa 100644
> --- a/build.xml
> +++ b/build.xml
> @@ -52,7 +52,7 @@
>
>
>
> -  
> +  
>
>
>
> diff --git a/java/jakarta/el/ImportHandler.java
> b/java/jakarta/el/ImportHandler.java
> index 018f53d..74858d2 100644
> --- a/java/jakarta/el/ImportHandler.java
> +++ b/java/jakarta/el/ImportHandler.java
> @@ -38,7 +38,7 @@ public class ImportHandler {
>  private static final Map> standardPackages = new
> HashMap<>();
>
>  static {
> -// Servlet 6.0
> +// Servlet 5.1
>  Set servletClassNames = new HashSet<>();
>  // Interfaces
>  servletClassNames.add("AsyncContext");
> @@ -91,7 +91,7 @@ public class ImportHandler {
>  servletClassNames.add("UnavailableException");
>  standardPackages.put("jakarta.servlet", servletClassNames);
>
> -// Servlet 6.0
> +// Servlet 5.1
>  Set servletHttpClassNames = new HashSet<>();
>  // Interfaces
>  servletHttpClassNames.add("HttpServletMapping");
> diff --git a/java/jakarta/servlet/ServletContext.java
> b/java/jakarta/servlet/ServletContext.java
> index 3c8894c..b66891a 100644
> --- a/java/jakarta/servlet/ServletContext.java
> +++ b/java/jakarta/servlet/ServletContext.java
> @@ -92,19 +92,19 @@ public interface ServletContext {
>
>  /**
>   * Returns the major version of the Java Servlet API that this servlet
> - * container supports. All implementations that comply with Version
> 6.0 must
> - * have this method return the integer 6.
> + * container supports. All implementations that comply with Version
> 5.1 must
> + * have this method return the integer 5.
>   *
> - * @return 6
> + * @return 5
>   */
>  public int getMajorVersion();
>
>  /**
>   * Returns the minor version of the Servlet API that this servlet
> container
> - * supports. All implementations that comply with Version 6.0 must
> have this
> - * method return the integer 0.
> + * supports. All implementations that comply with Version 5.1 must
> have this
> + * method return the integer 1.
>   *
> - * @return 0
> + * @return 1
>   */
>  public int getMinorVersion();
>
> diff --git a/java/org/apache/catalina/connector/CoyoteAdapter.java
> b/java/org/apache/catalina/connector/CoyoteAdapter.java
> index 3367425..818b881 100644
> --- 

Re: [NOTICE] Branch renaming starting shortly

2021-05-21 Thread Rémy Maucherat
On Fri, May 21, 2021 at 10:51 PM Mark Thomas  wrote:

> On 21/05/2021 21:38, Rémy Maucherat wrote:
> > On Fri, May 21, 2021 at 10:30 PM Mark Thomas  wrote:
> >
> >> On 21/05/2021 21:20, Rémy Maucherat wrote:
> >>> On Fri, May 21, 2021 at 10:17 PM Mark Thomas  wrote:
> >>>
> >>>> On 21/05/2021 21:06, Mark Thomas wrote:
> >>>>> On 21/05/2021 20:55, Mark Thomas wrote:
> >>>>>> I'll post to this thread as I complete each repo.
> >>>>>>
> >>>>>> After migration, everyone with a local clone will need to run this
> >>>>>> from the root of each clone:
> >>>>>>
> >>>>>> git branch -m master main
> >>>>>> git fetch origin
> >>>>>> git branch -u origin/main main
> >>>>>
> >>>>> Note you may also want to call
> >>>>> git remote prune origin
> >>>>>
> >>>>> Completed https://github.com/apache/tomcat-jakartaee-migration
> >>>>
> >>>> Along with
> >>>> tomcat-training and tomcat-taglibs-*
> >>>>
> >>>
> >>> Why the email notifications ? Will migrating Tomcat blow up our emails
> /
> >>> lists / etc ?
> >>
> >> It looks like gitbox isn't perfect. It sometimes misses notifications
> >> and sometimes duplicates them.
> >>
> >> It looks to be a one-off issue with taglibs-parent so far. I'm not sure
> >> of the exact root cause but I'm relived not to have seen similar issues
> >> with the other repos.
> >>
> >> tomcat-native and tomcat-connectors are done now too.
> >>
> >
> > All ok for me so far, fingers crossed for the big one ...
>
> :)
>
> All done.
>
> I've updated the CI systems (I hope). I suspect I've missed some
> references somewhere. We can fix those as we find them.
>

Ok, so we won at that git-russian roulette thing, it works perfect for me
and no unexpected email bombing.

Rémy


>
> Mark
>
> -
> To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org
> For additional commands, e-mail: dev-h...@tomcat.apache.org
>
>


Re: [NOTICE] Branch renaming starting shortly

2021-05-21 Thread Rémy Maucherat
On Fri, May 21, 2021 at 10:30 PM Mark Thomas  wrote:

> On 21/05/2021 21:20, Rémy Maucherat wrote:
> > On Fri, May 21, 2021 at 10:17 PM Mark Thomas  wrote:
> >
> >> On 21/05/2021 21:06, Mark Thomas wrote:
> >>> On 21/05/2021 20:55, Mark Thomas wrote:
> >>>> I'll post to this thread as I complete each repo.
> >>>>
> >>>> After migration, everyone with a local clone will need to run this
> >>>> from the root of each clone:
> >>>>
> >>>> git branch -m master main
> >>>> git fetch origin
> >>>> git branch -u origin/main main
> >>>
> >>> Note you may also want to call
> >>> git remote prune origin
> >>>
> >>> Completed https://github.com/apache/tomcat-jakartaee-migration
> >>
> >> Along with
> >> tomcat-training and tomcat-taglibs-*
> >>
> >
> > Why the email notifications ? Will migrating Tomcat blow up our emails /
> > lists / etc ?
>
> It looks like gitbox isn't perfect. It sometimes misses notifications
> and sometimes duplicates them.
>
> It looks to be a one-off issue with taglibs-parent so far. I'm not sure
> of the exact root cause but I'm relived not to have seen similar issues
> with the other repos.
>
> tomcat-native and tomcat-connectors are done now too.
>

All ok for me so far, fingers crossed for the big one ...

Rémy


>
> Mark
>
> -
> To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org
> For additional commands, e-mail: dev-h...@tomcat.apache.org
>
>


Re: [NOTICE] Branch renaming starting shortly

2021-05-21 Thread Rémy Maucherat
On Fri, May 21, 2021 at 10:17 PM Mark Thomas  wrote:

> On 21/05/2021 21:06, Mark Thomas wrote:
> > On 21/05/2021 20:55, Mark Thomas wrote:
> >> I'll post to this thread as I complete each repo.
> >>
> >> After migration, everyone with a local clone will need to run this
> >> from the root of each clone:
> >>
> >> git branch -m master main
> >> git fetch origin
> >> git branch -u origin/main main
> >
> > Note you may also want to call
> > git remote prune origin
> >
> > Completed https://github.com/apache/tomcat-jakartaee-migration
>
> Along with
> tomcat-training and tomcat-taglibs-*
>

Why the email notifications ? Will migrating Tomcat blow up our emails /
lists / etc ?

Rémy


>
> Mark
>
> -
> To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org
> For additional commands, e-mail: dev-h...@tomcat.apache.org
>
>


Re: JDK 17 Early Access build 23 is available

2021-05-21 Thread Rémy Maucherat
On Fri, May 21, 2021 at 9:38 AM Rory O'Donnell 
wrote:

> Hi Mark, **
>
> *OpenJDK 17 Early Access build 23 is now available at
> *_*https://jdk.java.net/17* _
>
>   * These early-access , open-source builds are provided under the
>   o GNU General Public License, version 2, with the Classpath
> Exception 
>   * JEPs targeted to JDK 17, so far:
>   o JEP 356: _Enhanced Pseudo-Random Number Generators
> _
>   o JEP 382: _New macOS Rendering Pipeline
> _
>   o JEP 391: _macOS/AArch64 Port _
>   o JEP 398: _Deprecate the Applet API for Removal
> _
>   o JEP 409: _Sealed Classes _
>   o JEP 410: _Remove the Experimental AOT and JIT Compiler
> _
>   o JEP 412: _Foreign Function & Memory API (Incubator)
> _
>   o JEP 414: _Vector API (Second Incubator)
> _
>   * Release Notes are available at
> _https://jdk.java.net/17/release-notes
> _
>   * Changes in recent builds that maybe of interest:
>   o Build 23
>   + JDK-8243287: Removal of Unsafe::defineAnonymousClass.
>   o Build 22
>   + *JDK-8266369: New implementation of
> java.nio.channels.Selector on Microsoft Windows. *
>

So, ok, that's quite the bomb for Windows users.
https://bugs.openjdk.java.net/browse/JDK-8266369

Rémy


>   o Build 21
>   + *JDK-8196415: JARs signed with SHA-1 algorithms are
> restricted by default.*
>   + *JDK-8266858: macOS on ARM early access available.*
>   # The ARM port should behave similarly to the Intel port.
> There are no known feature differences.
>   # When reporting issues on macOS please specify if using
> ARM or x64.
>
> *We need your help in testing new Selector implementation on Windows [1]:*
>
>   * The implementation of the Selector API on Windows has been replaced
> in JDK 17 b22 with a new more scalable implementation [2].
>   * The old select based Selector implementation has been the default
> since Java 1.4 (2002) so replacing it is a significant change.
>   * It would be really helpful to get more testing of the new
> implementation before the fork for Rampdown Phase One on June 10th.
>
> *Other Topics which might be of Interest:*
>
>   * Updates to JEP 411: Deprecate the Security Manager for Removal |
> _Link_
> 
>   * "The meaning, or not, of “LTS” | _Link_
> 
>   * JFR Remote Recording Stream | _Link_
> 
>
> *Project Loom Early-Access Build: **_Build 17-loom+7-342_*
> *(2021/5/11)*
>
>   * These early-access builds are provided under the _GNU General Public
> License, version 2, with the Classpath Exception_
> .
>   * These builds are produced for the purpose of gathering feedback. Use
> for any other purpose is at your own risk.
>   * Please send feedback via e-mail to _loom-...@openjdk.java.net
> _.To send e-mail to this address
> you must first _subscribe to the mailing list_
> .
>
> *Project Panama Early-Access Build: *_*Build 17-panama+3-167*
> _*(2021/5/18)*
>
>   * These early-access builds are provided under the _GNU General Public
> License, version 2, with the Classpath Exception_
> .
>   * This build is aimed at testing a prototype implementation of the
> foreign memory support, foreign function support and native
> extraction tooling from the "foreign-jextract" branch of the Panama
> repo.
>   * Please send feedback via e-mail to _panama-...@openjdk.java.net
> _. To send e-mail to this
> address you must first _subscribe to the mailing list_
> .
>
>
> Rgds,Rory
>
>
> [1]
> _https://mail.openjdk.java.net/pipermail/nio-dev/2021-May/008988.html_
> 
> [2] _https://bugs.openjdk.java.net/browse/JDK-8266369_
> 
>
> --
> Rgds, Rory O'Donnell
> Quality Engineering Manager
> Oracle EMEA, Dublin, Ireland
>
>


Re: Time to create Tomcat 10.1.x and master->main migration

2021-05-18 Thread Rémy Maucherat
On Tue, May 18, 2021 at 1:34 PM Mark Thomas  wrote:

> All,
>
> Things are starting to move forward for Jakarta EE 10 so I think it is
> time for us to create the 10.1.x branch. At the same time, I'd like to
> switch our primary development branches from master to main for all our
> repos.
>

Ah good news. I sort of wasn't expecting anything anymore ;)


>
> We would, therefore, end up with the following for the Tomcat repo:
>
> main   - 10.1.x development
> 10.0.x - 10.0.x development/maintenance
> 9.0.x  - 9.0.x development/maintenance
> 8.5.x  - 8.5.x development/maintenance
> 7.0.x  - 7.0.x development/maintenance
>

+1

>
> There are some git commands each committer will need to run locally for
> each repo to switch from master to main.
>
> I have also been looking into how we can "retire" the 7.0.x branch when
> the time comes (after end of June). I'd like to suggest the following:
> - tag the HEAD of the 7.0.x branch as "7.0.x-archive"
> - delete the 7.0.x branch
>
> That way it won't appear in the list of branches but it is trivial to
> recreate if we need it.
>

Good idea !


>
> I'd like to get the master->main rename completed and the 10.1.x
> development branch created towards the end of this week (unless there
> are objections or things we need to discuss further).
>
> Comments?
>

Rémy

>
> Mark
>
> -
> To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org
> For additional commands, e-mail: dev-h...@tomcat.apache.org
>
>


Re: JSP code generation optimizations

2021-05-17 Thread Rémy Maucherat
On Mon, May 17, 2021 at 8:47 PM Mark Thomas  wrote:

> Hi all,
>
> I am looking at some of the optimizations requested in [1]
>
> One of the examples is show below. Code like this is used when the JSP
> calls a tag.
>
>
> private boolean _jspx_meth_mytag_005fhelloWorld_005f0(
>  jakarta.servlet.jsp.PageContext _jspx_page_context)
>  throws java.lang.Throwable {
>  jakarta.servlet.jsp.PageContext pageContext = _jspx_page_context;
>  //  mytag:helloWorld
>  
> }
>
> The question is, can the generated code above be changed to something
> like this:
>
> private boolean _jspx_meth_mytag_005fhelloWorld_005f0(
>  final jakarta.servlet.jsp.PageContext pageContext)
>  throws java.lang.Throwable {
>  //  mytag:helloWorld
>  
> }
>
>
> I am assuming that the aliasing of _jspx_page_context to pageContext is
> so that _jspx_page_context always references the original even if the
> tag decides to do something like:
>
> pageContext = new MyCustomPageContextImpl();
>
> The question is, is behaviour like this allowed?
>
> I can't see anything in the spec that disallows this.
>
> I don't think the aliasing is a defence against malicious apps as it is
> only the implicit objects that are protected this way. Internal objects
> (those name _jsp*) are unprotected.
>
> Given that the original authors (who were on the JSP EG at the time)
> added this aliasing for the implicit objects, it looks like the
> intention was to allow applications to manipulate/replace the implicit
> objects if they wish - without breaking Jasper.
>
> If the above assumptions are correct, what do we want to do about [1]?
> The aliasing is often unnecessary but will sometimes be essential. Do we
> reject this optimization request? Do we add a configuration option? Do
> we try and make this aspect of the code generation pluggable somehow?
> Something else?
>
> Thoughts very welcome.
>

Well, although I do see how this makes the generated code a little bit
smaller, this is a rather small optimization (even with 1000s of
occurrences). Since you've identified a risk, and it seems correct to me
(I'm almost certain this was on purpose and because the idea is not
forbidden in the specification), I would say we should skip that one.

Rémy


>
> Mark
>
>
> [1] https://bz.apache.org/bugzilla/show_bug.cgi?id=65124
>
> -
> To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org
> For additional commands, e-mail: dev-h...@tomcat.apache.org
>
>


Re: Release Managers wanted

2021-05-14 Thread Rémy Maucherat
On Fri, May 14, 2021 at 12:06 PM Mark Thomas  wrote:

> On 14/05/2021 09:39, Rémy Maucherat wrote:
> > On Thu, May 13, 2021 at 12:10 AM Mark Thomas  wrote:
>
> 
>
> >> So, who'd like to volunteer?
> >>
> >
> > Well, I used to RM Tomcat a while ago, but I don't think I was very good
> at
> > it. Is it time to start again ??
>
> That would be great. I'm certainly not going to complain if are able and
> willing to start RM'ing again. As Chris has volunteered for 8.5.x can I
> tempt you with 9.0.x?
>

Possibly. I'll need to get my signing key situation sorted out first though.

Rémy


Re: Release Managers wanted

2021-05-14 Thread Rémy Maucherat
On Thu, May 13, 2021 at 12:10 AM Mark Thomas  wrote:

> All,
>
> Assuming 7.0.109 is the last Tomcat 7 release I am the current release
> manager for 10.0.x, 9.0.x, 8.5.x, migration tool, native and mod_jk. I'd
> like to share the load and the knowledge a little.
>
> The step-by-step release process was documented for Tomcat 7:
> https://cwiki.apache.org/confluence/display/TOMCAT/ReleaseProcess
>
> It needs a few minor updates but that should give you an idea of what is
> involved.
>
> I suggest that the major Tomcat versions are actually easier to release as:
> - the process is more automated
> - because the happen on a roughly monthly cycle you don't forget too
>much of the detail between releases
> - there is greater flexibility in the build environment (anything that
>needs a Windows binary needs a very specific environment to avoid
>unwanted system dependencies)
>
> You should be able to build on Linux or Windows (or MacOS assuming
> cygwin works well enough with NSIS) although anything other than Windows
> will require a little more setup in the first instance
>
> I'd be happy to mentor volunteers through their first couple of releases.
>
> Once you get into the swing of things, the process is fairly quick.
>
> The only pre-requisite is that you need to be a committer.
>
> So, who'd like to volunteer?
>

Well, I used to RM Tomcat a while ago, but I don't think I was very good at
it. Is it time to start again ??

Rémy


>
> Mark
>
> -
> To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org
> For additional commands, e-mail: dev-h...@tomcat.apache.org
>
>


Re: Tagging May releases towards the end of this week

2021-05-12 Thread Rémy Maucherat
On Tue, May 4, 2021 at 12:55 PM Mark Thomas  wrote:

> On 04/05/2021 08:43, Rémy Maucherat wrote:
> > On Tue, May 4, 2021 at 9:36 AM Mark Thomas  wrote:
> >
> >> On 04/05/2021 08:29, Rémy Maucherat wrote:
> >>> On Tue, May 4, 2021 at 9:26 AM Mark Thomas  wrote:
> >>>
> >>>> Hi all,
> >>>>
> >>>> It is the start of a new month so time to tag and release. I have a
> few
> >>>> things I'd like to get fixed first:
> >>>>
> >>>> 1. BZ 65272 (allow CR as HTTP line separator).
> >>>>   I'm planning on applying PR 417 to address this before I tag
> unless
> >>>>   there are objections.
> >>>>
> >>>> 2. BZ 65262 (WebSocket & InstanceManager)
> >>>>   I have a patch for this about 80% complete. I intend to complete
> >> and
> >>>>   apply it before tagging.
> >>>>
> >>>> Given these (and no more issues raised this week) I'm currently
> >>>> expecting the tags to be towards the end of the week.
> >>>>
> >>>
> >>> Can we release a stable Jakarta migration tool before that release ?
> >> There
> >>> are a couple changes that could be picked up and since it's a
> >> subcomponent
> >>> it would need to be declared stable anyway.
> >>
> >> Sure. 0.3.0 or 1.0.0 ?
> >>
> >
> > I think it should move to stable 1.0, unless there's some mandatory
> feature
> > that got forgotten.
>
> I can't think of anything. Folks are starting to use it (and Tomcat 10)
> and the biggest problem seems to be that users aren't aware of the Java
> EE -> Jakarta EE changes.
>
> Is there merit in attempting to detect an application trying to load a
> Java EE 9 class (maybe limit it to Servlet, JSP, EL & WebSocket) and
> fail the deployment (assuming we catch the problem early enough) with a
> clear error message? Maybe pointing to a (maybe expanded) section in the
> migration guide?
>

That could be a good idea but it goes back to the deployment problem:
ideally legacy EE webapps would be detected and migrated automatically
(unless configured otherwise). But there is a problem since it's hard to
do. Even if the intent is to limit that to classloading, it's only a bit
easier. Most webapps do load stuff on startup so that's a great spot to
detect this kind of problem, but for the others the error will be lost in
the shuffle.

Rémy


>
> Mark
>
> -
> To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org
> For additional commands, e-mail: dev-h...@tomcat.apache.org
>
>


Re: [VOTE] Release Apache Tomcat 8.5.66

2021-05-10 Thread Rémy Maucherat
On Sun, May 9, 2021 at 1:12 AM Mark Thomas  wrote:

> The proposed Apache Tomcat 8.5.66 release is now available for voting.
>
> The notable changes compared to the 8.5.65 release are:
>
> - Ensure the correct escaping of attribute values and search filters in
>the JNDIRealm.
>
> - HandlesTypes should include classes that use the specified annotation
>types on fields or methods.
>
> - Refactor the creation of WebSocket end point, decoder and encoder
>instances to be more IoC friendly. Instances are now created via the
>InstanceManager where possible.
>
> Along with lots of other bug fixes and improvements.
>
> For full details, see the changelog:
> https://ci.apache.org/projects/tomcat/tomcat85/docs/changelog.html
>
> It can be obtained from:
> https://dist.apache.org/repos/dist/dev/tomcat/tomcat-8/v8.5.66/
>
> The Maven staging repo is:
> https://repository.apache.org/content/repositories/orgapachetomcat-1311/
>
> The tag is:
> https://github.com/apache/tomcat/tree/8.5.66
> 12afa2cd11ffa9522cd98acc228ecb1bad6b8006
>
> The proposed 8.5.66 release is:
> [ ] Broken - do not release
> [X] Stable - go ahead and release as 8.5.66
>
> Rémy


Re: [VOTE] Release Apache Tomcat 9.0.46

2021-05-10 Thread Rémy Maucherat
On Sat, May 8, 2021 at 8:14 PM Mark Thomas  wrote:

> The proposed Apache Tomcat 9.0.46 release is now available for voting.
>
> The notable changes compared to the 9.0.45 release are:
>
> - Ensure the correct escaping of attribute values and search filters in
>the JNDIRealm.
>
> - HandlesTypes should include classes that use the specified annotation
>types on fields or methods.
>
> - Refactor the creation of WebSocket end point, decoder and encoder
>instances to be more IoC friendly. Instances are now created via the
>InstanceManager where possible.
>
> Along with lots of other bug fixes and improvements.
>
> For full details, see the changelog:
> https://ci.apache.org/projects/tomcat/tomcat9/docs/changelog.html
>
> It can be obtained from:
> https://dist.apache.org/repos/dist/dev/tomcat/tomcat-9/v9.0.46/
> The Maven staging repo is:
> https://repository.apache.org/content/repositories/orgapachetomcat-1310/
> The tag is:
> https://github.com/apache/tomcat/tree/9.0.46
> 37ae42a2996911b9ba6b88e7b0828f855b9d38f6
>
> The proposed 9.0.46 release is:
> [ ] Broken - do not release
> [X] Stable - go ahead and release as 9.0.46
>
> Rémy


Re: [VOTE] Release Apache Tomcat 10.0.6

2021-05-10 Thread Rémy Maucherat
On Sat, May 8, 2021 at 6:18 PM Mark Thomas  wrote:

> The proposed Apache Tomcat 10.0.6 release is now available for
> voting.
>
> Apache Tomcat 10.x implements Jakarta EE 9 and, as such, the primary
> package for all the specification APIs has changed from javax.* to
> jakarta.*
>
> Applications that run on Tomcat 9 will not run on Tomcat 10 without
> changes. Java EE applications designed for Tomcat 9 and earlier may be
> placed in the $CATALINA_BASE/webapps-javaee directory and Tomcat will
> automatically convert them to Jakarta EE and copy them to the webapps
> directory
>
> The notable changes compared to 10.0.5 are:
>
> - Ensure the correct escaping of attribute values and search filters in
>the JNDIRealm.
>
> - HandlesTypes should include classes that use the specified annotation
>types on fields or methods.
>
> - Refactor the creation of WebSocket end point, decoder and encoder
>instances to be more IoC friendly. Instances are now created via the
>InstanceManager where possible.
>
> Along with lots of other bug fixes and improvements.
>
>
> For full details, see the changelog:
> https://ci.apache.org/projects/tomcat/tomcat10/docs/changelog.html
>
> It can be obtained from:
> https://dist.apache.org/repos/dist/dev/tomcat/tomcat-10/v10.0.6/
> The Maven staging repo is:
> https://repository.apache.org/content/repositories/orgapachetomcat-1309
> The tag is:
> https://github.com/apache/tomcat/tree/10.0.6
> f725f57f195de035a5cbc6602a1f7a3ad43ee5b5
>
> The proposed 10.0.6 release is:
> [ ] Broken - do not release
> [X] Stable - go ahead and release as 10.0.6 (stable)
>
> Rémy


Re: mod_headers as a Filter

2021-05-07 Thread Rémy Maucherat
On Wed, Apr 28, 2021 at 10:45 AM Rémy Maucherat  wrote:

> On Wed, Apr 28, 2021 at 9:07 AM Mark Thomas  wrote:
>
>> I'm wondering if there is merit in a Valve-like mechanism for Coyote.
>> Name TBD but would look something like:
>> - callbacks
>>- after request headers are parsed / before the request is prepared
>>- after the request is prepared
>>- before response headers are prepared
>>- after response headers are prepared / before they are written
>> - allow multiple "Valves" to be configured
>> - provide a "default" that doesn't require explicit config
>> - explicit config can add custom "valves", the default "valve" or any
>>combination
>>
>
> I thought about it quite a bit in the past, an interceptor [resurrecting
> the old name] could be a solution to a lot of problems here. So why not.
>

Thinking about it some more, I would propose investigating either:
- Adding a new interface in addition to ActionHook to coyote.Request, maybe
called EventHook (gets a code and the request object).
- Allowing more than one ActionHook [the codes include plenty of the
interesting events already especially commit, flush]. User configured hooks
would be added and called *before* the main coyote hook (obviously).
Although this sounds easier to implement, this is tricker and error prone
as the call patterns are funky (also the hook field is duplicated between
request and response likely for no good reason); basically I would say all
hook action calls would now go through a helper method on the Request.
- Something else ;)

Rémy


>
> Rémy
>
>
>>
>> I'm leaning towards the latter approach as it has much greater
>> flexibility and I can see different users having subtly different
>> requirements and this avoids an explosion of configuration attributes on
>> a single class.
>>
>> Mark
>>
>> -
>> To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org
>> For additional commands, e-mail: dev-h...@tomcat.apache.org
>>
>>


Re: Implementing web.xml default-context-path

2021-05-05 Thread Rémy Maucherat
On Wed, May 5, 2021 at 2:42 PM Jean-Louis MONTEIRO 
wrote:

> Thanks for the feedback.
>
> Actually I was not looking for Mark or the Tomcat community to change their
> minds. But for some guidance on how to hack something to get this test
> to pass.
>
> I've been able to do it in TomEE which was the goal.
> Now, I'm totally with you guys. If this is in your opinion a bad use case
> with a bad specification solution, we should challenge the feature and then
> the test.
>

The feature is optional, so it's "fine" with me. It allows suggesting a
path to map the webapp, like Jira saying it should be mapped to /jira
because "why not". This is pointless in practice, but since it's optional,
there's nothing to do and I'm fine with it. The problem is then only the
TCK testing for a mandatory optional feature.

Good news on you working around it.

Rémy


>
> Le mer. 5 mai 2021 à 11:22, Romain Manni-Bucau  a
> écrit :
>
> > Le mer. 5 mai 2021 à 11:19, Mark Thomas  a écrit :
> >
> > > On 05/05/2021 09:58, Rémy Maucherat wrote:
> > > > On Sun, May 2, 2021 at 3:17 PM Jean-Louis MONTEIRO <
> jeano...@gmail.com
> > >
> > > > wrote:
> > > >
> > > >> Hi,
> > > >>
> > > >> Still working on getting TomEE certified for Jakarta EE 9.1
> > > >> We are using latest Tomcat 10.x and we indeed see now only one
> failure
> > > as
> > > >> described here
> > > >>
> > > >> https://cwiki.apache.org/confluence/display/TOMCAT/Servlet+TCK+5.0
> > > >>
> > > >> I understand this is not really a critical thing to be certified for
> > > >> Tomcat.
> > > >> And I also understand this is not something the community agrees on.
> > > >>
> > > >> But I'd like, for the sake of getting TomEE certified, to pass this
> > test
> > > >> and therefore I'm looking at a way to patch it on our side.
> > > >>
> > > >> Would you be able to help in giving pointers on how to do it?
> > > >>
> > > >> Any help is appreciated at this point.
> > > >>
> > > >
> > > > We already talked about this (bad) "feature" quite a bit in the past.
> > > > Unless there's a big surprise:
> > > > - This needs intrusive and annoying change to the deployer
> > > > - The benefit to actual users seems to be zero
> > > > - The feature is optional
> > > > As a result, I doubt Mark will change his mind [I am not enthusiastic
> > > > either] and I still don't understand how it is a mandatory TCK test.
> > >
> > > +1 allowing this feature opens up a huge mess of deployment edge cases
> > > that the Tomcat deployment process was designed very carefully to
> avoid.
> > >
> > > > So ... I think you can come up with a hack instead: a Catalina
> listener
> > > > could be added to the Context, then it could configure the rewrite
> > valve
> > > to
> > > > do the url mapping. It should be enough to make this work.
> > >
> > > Another option would be to challenge the TCK test on the grounds the
> > > spec allows containers to override any default-context-path setting but
> > > the TCK assumes this isn't going to happen (in Tomcat it will always
> > > happen).
> > >
> >
> > +1 to challenge this test, if there is a consensus it is wrong no point
> to
> > keep it year after year IMHO.
> >
> >
> > >
> > > Mark
> > >
> > > -
> > > To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org
> > > For additional commands, e-mail: dev-h...@tomcat.apache.org
> > >
> > >
> >
>
>
> --
> Jean-Louis
>


Re: Implementing web.xml default-context-path

2021-05-05 Thread Rémy Maucherat
On Sun, May 2, 2021 at 3:17 PM Jean-Louis MONTEIRO 
wrote:

> Hi,
>
> Still working on getting TomEE certified for Jakarta EE 9.1
> We are using latest Tomcat 10.x and we indeed see now only one failure as
> described here
>
> https://cwiki.apache.org/confluence/display/TOMCAT/Servlet+TCK+5.0
>
> I understand this is not really a critical thing to be certified for
> Tomcat.
> And I also understand this is not something the community agrees on.
>
> But I'd like, for the sake of getting TomEE certified, to pass this test
> and therefore I'm looking at a way to patch it on our side.
>
> Would you be able to help in giving pointers on how to do it?
>
> Any help is appreciated at this point.
>

We already talked about this (bad) "feature" quite a bit in the past.
Unless there's a big surprise:
- This needs intrusive and annoying change to the deployer
- The benefit to actual users seems to be zero
- The feature is optional
As a result, I doubt Mark will change his mind [I am not enthusiastic
either] and I still don't understand how it is a mandatory TCK test.

So ... I think you can come up with a hack instead: a Catalina listener
could be added to the Context, then it could configure the rewrite valve to
do the url mapping. It should be enough to make this work.

Rémy


>
>
> --
> Jean-Louis
>


Re: [VOTE] Apache Tomcat migration tool for Jakarta EE 1.0.0

2021-05-04 Thread Rémy Maucherat
On Tue, May 4, 2021 at 2:06 PM Mark Thomas  wrote:

> The proposed Apache Tomcat migration tool for Jakarta EE 1.0.0 is now
> available for voting.
>
> The significant changes since 0.2.0 are:
>
> - Further fixes to exclude javax.xml packages that are not part of
>Java EE from the migration
>
> - The class transformer now validates that the target classes in the
>Jakarta namespace exist in the runtime environment
>
>
> It can be obtained from:
> https://dist.apache.org/repos/dist/dev/tomcat/jakartaee-migration/v1.0.0/
>
> The Maven staging repo is:
> https://repository.apache.org/content/repositories/orgapachetomcat-1308/
>
> The tag is:
> https://github.com/apache/tomcat-jakartaee-migration/tree/1.0.0
> 2a27b633f456710dc5d51680344ca7f047642a60
>
> The proposed 1.0.0 release is:
>
> [ ] -1: Broken. Do not release because...
> [X] +1: Acceptable. Go ahead and release.
>
> Rémy


Re: Tagging May releases towards the end of this week

2021-05-04 Thread Rémy Maucherat
On Tue, May 4, 2021 at 9:36 AM Mark Thomas  wrote:

> On 04/05/2021 08:29, Rémy Maucherat wrote:
> > On Tue, May 4, 2021 at 9:26 AM Mark Thomas  wrote:
> >
> >> Hi all,
> >>
> >> It is the start of a new month so time to tag and release. I have a few
> >> things I'd like to get fixed first:
> >>
> >> 1. BZ 65272 (allow CR as HTTP line separator).
> >>  I'm planning on applying PR 417 to address this before I tag unless
> >>  there are objections.
> >>
> >> 2. BZ 65262 (WebSocket & InstanceManager)
> >>  I have a patch for this about 80% complete. I intend to complete
> and
> >>  apply it before tagging.
> >>
> >> Given these (and no more issues raised this week) I'm currently
> >> expecting the tags to be towards the end of the week.
> >>
> >
> > Can we release a stable Jakarta migration tool before that release ?
> There
> > are a couple changes that could be picked up and since it's a
> subcomponent
> > it would need to be declared stable anyway.
>
> Sure. 0.3.0 or 1.0.0 ?
>

I think it should move to stable 1.0, unless there's some mandatory feature
that got forgotten.

Rémy


>
> Mark
>
> -
> To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org
> For additional commands, e-mail: dev-h...@tomcat.apache.org
>
>


Re: Tagging May releases towards the end of this week

2021-05-04 Thread Rémy Maucherat
On Tue, May 4, 2021 at 9:26 AM Mark Thomas  wrote:

> Hi all,
>
> It is the start of a new month so time to tag and release. I have a few
> things I'd like to get fixed first:
>
> 1. BZ 65272 (allow CR as HTTP line separator).
> I'm planning on applying PR 417 to address this before I tag unless
> there are objections.
>
> 2. BZ 65262 (WebSocket & InstanceManager)
> I have a patch for this about 80% complete. I intend to complete and
> apply it before tagging.
>
> Given these (and no more issues raised this week) I'm currently
> expecting the tags to be towards the end of the week.
>

Can we release a stable Jakarta migration tool before that release ? There
are a couple changes that could be picked up and since it's a subcomponent
it would need to be declared stable anyway.

Rémy


>
> Mark
>
> -
> To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org
> For additional commands, e-mail: dev-h...@tomcat.apache.org
>
>


Re: mod_headers as a Filter

2021-04-28 Thread Rémy Maucherat
On Wed, Apr 28, 2021 at 9:07 AM Mark Thomas  wrote:

> I'm wondering if there is merit in a Valve-like mechanism for Coyote.
> Name TBD but would look something like:
> - callbacks
>- after request headers are parsed / before the request is prepared
>- after the request is prepared
>- before response headers are prepared
>- after response headers are prepared / before they are written
> - allow multiple "Valves" to be configured
> - provide a "default" that doesn't require explicit config
> - explicit config can add custom "valves", the default "valve" or any
>combination
>

I thought about it quite a bit in the past, an interceptor [resurrecting
the old name] could be a solution to a lot of problems here. So why not.

Rémy


>
> I'm leaning towards the latter approach as it has much greater
> flexibility and I can see different users having subtly different
> requirements and this avoids an explosion of configuration attributes on
> a single class.
>
> Mark
>
> -
> To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org
> For additional commands, e-mail: dev-h...@tomcat.apache.org
>
>


Re: mod_headers as a Filter

2021-04-27 Thread Rémy Maucherat
On Tue, Apr 27, 2021 at 7:05 PM Mark Thomas  wrote:

> Hi all,
>
> I've started to  look at this and I am struggling to see a way to
> implement something that looks like mod_headers as a Filter.
>
> Request headers are fairly simple. The process looks something like:
> a) take a copy of all the headers received
> b) apply all the rules for request headers
> c) wrap the request, overriding the various getHeader... methods and
> return values appropriate for the modified set of headers
>
> Response headers are where I am currently stuck.
> A similar model to request headers might look like:
> a) wrap the response
> b) intercept all the headers set by the application
> c) apply all the rules for response headers
> d) call the appropriate setHeader... methods on the wrapped response
> for the modified set of headers
>
> The problem is that d) (and hence c) needs to be done immediately before
> the response is committed and - short of buffering the entire response
> body - there is no way to know when that is going to happen.
>
> Is the answer we need to buffer the entire response body?
>
> Any cunning ideas on how to detect (in a Filter or wrapped response)
> that the response is about to be committed?
>
> I guess we could try and track bytes (about to be) written and compare
> that to the known buffer size. That seems a little fragile on first
> impression.
>
> Another option is to abandon the mod_headers clone aim and do something
> simpler along the lines of blocking applications from setting specific
> headers and/or header/value combinations.
>
> Thoughts?
>

I remember after doing the rewrite valve I got asked a bit about
mod_headers because "why not". However, now I recall I found out it would
be far less practical. So I very quickly moved on since it was also less
useful than rewrite. I would still probably not do it.

Rémy


>
> Mark
>
> -
> To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org
> For additional commands, e-mail: dev-h...@tomcat.apache.org
>
>


Re: [VOTE] Release Apache Tomcat 7.0.109

2021-04-26 Thread Rémy Maucherat
On Thu, Apr 22, 2021 at 9:13 PM Violeta Georgieva 
wrote:

> The proposed Apache Tomcat 7.0.109 release is now available for voting.
> Please note that this is the last Tomcat 7 release.
>
> It can be obtained from:
> https://dist.apache.org/repos/dist/dev/tomcat/tomcat-7/v7.0.109/
> The Maven staging repo is:
> https://repository.apache.org/content/repositories/orgapachetomcat-1307/
> The git tag is:
> https://github.com/apache/tomcat/tree/7.0.109
> 2cdef2c0241cdf70b5edd88d3733a52e6b675047
>
> The proposed 7.0.109 release is:
> [ ] Broken - do not release
> [X] Stable - go ahead and release as 7.0.109 Stable
>

Congratulations on wrapping up the branch :)

Rémy


>
> Regards,
> Violeta
>


Re: Annotations aren't working with tomcat 9

2021-04-12 Thread Rémy Maucherat
On Mon, Apr 12, 2021 at 12:51 PM Shrivastava, Vijay
 wrote:

> Hi,
>
> @webservlet & @webListner annotations aren't working.
>
> Classes in which we are using these annotations are present in jar which
> is getting scanned but registration of servlet with tomcat is not happening.
>
> What we have tried to fix:
>
>   1.  To confirm the issue we downgraded Tomcat version in Mercury 10.5
> from Tomcat 9 to Tomcat 7. With Tomcat 7,  servlets get registered with
> Tomcat and hence annotation scanning is happening as desired.
>   2.  Tried setting properties values mentioned in the
> $INFA_HOME/tomcat/conf/catalina.properties file related to "jarSkip", but
> no luck.
>   3.  Tried adding "metadata-complete" tag to false/true in web.xml as per
> Servlet 4.0 specification, but no luck.
>
>
> Things that worked-
> 1 - We put these classes at web-inf\classes folder,
> 2 - Added entry for the servlet in we.xml file.
>
> Since in our case these servlets are dynamically added via installer. We
> can't use any of the above method.
>
> Can anyone please help us with this?
>

We probably could help if you had posted in the Tomcat user mailing list
instead.
http://tomcat.apache.org/lists.html#tomcat-users

Rémy


>
> We are using below configuration:
> Tomcat version details:
> Server built:   Dec 3 2020 11:43:00 UTC
> Server number:  9.0.41.0
> OS Name:Linux
> OS Version: 3.10.0-693.el7.x86_64
> Architecture:   amd64
> JVM Version:1.8.0_191-b12
> JVM Vendor: Oracle Corporation
>
> annotation-api.jar - version - 1.3
> Servlet-api.jar -  version -4.0
>
> Thanks & Regards,
> Vijay Shrivastava
>


Re: AprLifecycleListener broken since 9.0.38?

2021-04-08 Thread Rémy Maucherat
On Thu, Apr 8, 2021 at 1:01 PM Rainer Jung  wrote:

> Hi Remy,
>
> Filip wrote in the git commit message "Avoid having to load APR classes
> in the Connector". If this is an important goal for TC 9, we would need
> some other place during TC startup that loads the APR classes before the
> connector gets instantiated. I don't know, what a good place would be.
> Currently it seems, the connector gets instantiated while the digester
> processes server.xml, and the Lifecycle events that would init the
> AprLifecycleListener start after that phase, so too late for the
> connector. Not sure whether it would be feasible to use a digester end
> method in a rule for the AprLifecycleListener to call its static init()
> after it has configured the class.
>
> My patch simply reverts the part of Filip's change in the Connector
> class related to isAprAvailable() back to the 9.0.37 situation.
>

I think it should be fine now since AprStatus allows avoiding using
AprLIfecycleListener if it is not configured.


>
> Using explicit className surely is possible, but I think we shouldn't
> break the well known and documented auto-behavior in TC up until 9. For
> example our Windows binary distribution contains tcnative, so each TC 9
> installed with that download and having edited server.xml with
> useAprConnector="true" automatically switches to the APR connectoer, If
> people update after 9.0.37, it will no longer use APR without being made
> aware.
>
> Concerning the three places in Connector, I agree, that only the first
> is needed.
>

All good then.

Rémy

>
> Regards,
>
> Rainer
>
> Am 08.04.2021 um 09:29 schrieb Rémy Maucherat:
> > On Wed, Apr 7, 2021 at 2:42 PM Rainer Jung 
> wrote:
> >
> >> The only direct calls to AprStatus.isAprAvailable() outside of the
> >> APRLifecycleListener itself are actually in Connector.java. Replacing
> >> those by calls to the listener seems to work for me:
> >>
> >> diff --git a/java/org/apache/catalina/connector/Connector.java
> >> b/java/org/apache/catalina/connector/Connector.java
> >> index 1cc15802eb..53a210e6b2 100644
> >> --- a/java/org/apache/catalina/connector/Connector.java
> >> +++ b/java/org/apache/catalina/connector/Connector.java
> >> @@ -29,6 +29,7 @@ import org.apache.catalina.Globals;
> >>import org.apache.catalina.LifecycleException;
> >>import org.apache.catalina.LifecycleState;
> >>import org.apache.catalina.Service;
> >> +import org.apache.catalina.core.AprLifecycleListener;
> >>import org.apache.catalina.core.AprStatus;
> >>import org.apache.catalina.util.LifecycleMBeanBase;
> >>import org.apache.coyote.AbstractProtocol;
> >> @@ -80,7 +81,7 @@ public class Connector extends LifecycleMBeanBase  {
> >>
> >>
> >>public Connector(String protocol) {
> >> -boolean apr = AprStatus.isAprAvailable() &&
> >> +boolean apr = AprLifecycleListener.isAprAvailable() &&
> >>
> >
> > Ok, so I thought only the first change was significant, the others occur
> > after AprLifecycleListener runs its init so it should work just fine.
> >
> > Rémy
> >
> >
> >>AprStatus.getUseAprConnector();
> >>ProtocolHandler p = null;
> >>try {
> >> @@ -1020,11 +1021,11 @@ public class Connector extends
> LifecycleMBeanBase
> >> {
> >>throw new
> >>
> >>
> LifecycleException(sm.getString("coyoteConnector.protocolHandlerNoAprListener",
> >>getProtocolHandlerClassName()));
> >>}
> >> -if (protocolHandler.isAprRequired() &&
> >> !AprStatus.isAprAvailable()) {
> >> +if (protocolHandler.isAprRequired() &&
> >> !AprLifecycleListener.isAprAvailable()) {
> >>throw new
> >>
> >>
> LifecycleException(sm.getString("coyoteConnector.protocolHandlerNoAprLibrary",
> >>getProtocolHandlerClassName()));
> >>}
> >> -if (AprStatus.isAprAvailable() && AprStatus.getUseOpenSSL() &&
> >> +if (AprLifecycleListener.isAprAvailable() &&
> >> AprStatus.getUseOpenSSL() &&
> >>protocolHandler instanceof
> AbstractHttp11JsseProtocol) {
> >>AbstractHttp11JsseProtocol jsseProtocolHandler =
> >>(AbstractHttp11JsseProtocol) protocolHandler;
> &

Re: AprLifecycleListener broken since 9.0.38?

2021-04-08 Thread Rémy Maucherat
On Wed, Apr 7, 2021 at 2:42 PM Rainer Jung  wrote:

> The only direct calls to AprStatus.isAprAvailable() outside of the
> APRLifecycleListener itself are actually in Connector.java. Replacing
> those by calls to the listener seems to work for me:
>
> diff --git a/java/org/apache/catalina/connector/Connector.java
> b/java/org/apache/catalina/connector/Connector.java
> index 1cc15802eb..53a210e6b2 100644
> --- a/java/org/apache/catalina/connector/Connector.java
> +++ b/java/org/apache/catalina/connector/Connector.java
> @@ -29,6 +29,7 @@ import org.apache.catalina.Globals;
>   import org.apache.catalina.LifecycleException;
>   import org.apache.catalina.LifecycleState;
>   import org.apache.catalina.Service;
> +import org.apache.catalina.core.AprLifecycleListener;
>   import org.apache.catalina.core.AprStatus;
>   import org.apache.catalina.util.LifecycleMBeanBase;
>   import org.apache.coyote.AbstractProtocol;
> @@ -80,7 +81,7 @@ public class Connector extends LifecycleMBeanBase  {
>
>
>   public Connector(String protocol) {
> -boolean apr = AprStatus.isAprAvailable() &&
> +boolean apr = AprLifecycleListener.isAprAvailable() &&
>

Ok, so I thought only the first change was significant, the others occur
after AprLifecycleListener runs its init so it should work just fine.

Rémy


>   AprStatus.getUseAprConnector();
>   ProtocolHandler p = null;
>   try {
> @@ -1020,11 +1021,11 @@ public class Connector extends LifecycleMBeanBase
> {
>   throw new
>
> LifecycleException(sm.getString("coyoteConnector.protocolHandlerNoAprListener",
>   getProtocolHandlerClassName()));
>   }
> -if (protocolHandler.isAprRequired() &&
> !AprStatus.isAprAvailable()) {
> +if (protocolHandler.isAprRequired() &&
> !AprLifecycleListener.isAprAvailable()) {
>   throw new
>
> LifecycleException(sm.getString("coyoteConnector.protocolHandlerNoAprLibrary",
>   getProtocolHandlerClassName()));
>   }
> -if (AprStatus.isAprAvailable() && AprStatus.getUseOpenSSL() &&
> +if (AprLifecycleListener.isAprAvailable() &&
> AprStatus.getUseOpenSSL() &&
>   protocolHandler instanceof AbstractHttp11JsseProtocol) {
>   AbstractHttp11JsseProtocol jsseProtocolHandler =
>   (AbstractHttp11JsseProtocol) protocolHandler;
>
> Feel free to apply.
>
> Regards,
>
> Rainer
>
>
> Am 07.04.2021 um 14:32 schrieb Rainer Jung:
> > I think the reason is:
> >
> > o.a.c.connector.Connector checks AprStatus.isAprAvailable(). It
> > previously checked AprLifecycleListener.isAprAvailable(). The difference
> > is, that the old AprLifecycleListener.isAprAvailable() triggers the
> > init() of AprLifecycleListener, whereas AprStatus.isAprAvailable() does
> > not.
> >
> > I think isAprAvailable() is the only method with side effects that was
> > moved from AprLifecycleListener to AprStatus. Since it is still
> > available in AprLifecycleListener, I think it would be safest to
> > typically not call it in AprStatus but instead in AprLifecycleListener.
> >
> > Regards,
> >
> > Rainer
> >
> > Am 07.04.2021 um 14:02 schrieb Rainer Jung:
> >> Maybe related to f4dac6846c548144799b1c3f33aba4eb320a3413.
> >>
> >> Am 07.04.2021 um 13:53 schrieb Rainer Jung:
> >>> Hi there,
> >>>
> >>> a colleague of mine observed a change in behavior starting with TC
> >>> 9.0.38: until 9.0.37 the default server.xml with the connector
> >>> protocol="HTTP/1.1", installed tcnative and attribute
> >>> useAprConnector="true" for the AprLifecycleListener actually starts
> >>> an APR connector as documented and expected:
> >>>
> >>> INFO [main] org.apache.coyote.AbstractProtocol.init Initializing
> >>> ProtocolHandler ["http-apr-8080"]
> >>>
> >>> Same situation starting with 9.0.38 (and at least until 9.0.44):
> >>>
> >>> INFO [main] org.apache.coyote.AbstractProtocol.init Initializing
> >>> ProtocolHandler ["http-nio-8080"]
> >>>
> >>> I increased log level for a coupe of packages to FINEST and see the
> >>> following differences during startup:
> >>>
> >>> FINE [main] org.apache.tomcat.util.IntrospectionUtils.setProperty
> >>> IntrospectionUtils: setProperty(class
> >>> org.apache.catalina.core.AprLifecycleListener SSLEngine=on)
> >>> FINE [main] org.apache.tomcat.util.digester.SetPropertiesRule.begin
> >>> [SetPropertiesRule]{Server/Listener} Setting property
> >>> 'useAprConnector' to 'true'
> >>> FINE [main] org.apache.tomcat.util.IntrospectionUtils.setProperty
> >>> IntrospectionUtils: setProperty(class
> >>> org.apache.catalina.core.AprLifecycleListener useAprConnector=true)
> >>> ### ONLY ADDRESS DIFFERENCE
> >>> -FINE [main] org.apache.tomcat.util.digester.SetNextRule.end
> >>> [SetNextRule]{Server/Listener} Call
> >>>
> org.apache.catalina.core.StandardServer.addLifecycleListener(org.apache.catalina.core.AprLifecycleListener@383534aa)
>
> >>>
> >>> +FINE [main] 

Re: AprLifecycleListener broken since 9.0.38?

2021-04-07 Thread Rémy Maucherat
On Wed, Apr 7, 2021 at 2:42 PM Rainer Jung  wrote:

> The only direct calls to AprStatus.isAprAvailable() outside of the
> APRLifecycleListener itself are actually in Connector.java. Replacing
> those by calls to the listener seems to work for me:
>

I get the idea. So I think the purpose of the change was to avoid APR and
it worked out great for that. Normally the AprStatus.isInstanceCreated
should be enough to achieve what was originally needed and it is possible
to check AprLifecycleListener.isAprAvailable, since as you correctly
noticed AprStatus.isAprAvailable does nothing. I´m still a bit worried I
could be causing problems.

OTOH, it could be a good move to stop using this since it is removed in
Tomcat 10 and use the APR connector classname instead.

Rémy


>
> diff --git a/java/org/apache/catalina/connector/Connector.java
> b/java/org/apache/catalina/connector/Connector.java
> index 1cc15802eb..53a210e6b2 100644
> --- a/java/org/apache/catalina/connector/Connector.java
> +++ b/java/org/apache/catalina/connector/Connector.java
> @@ -29,6 +29,7 @@ import org.apache.catalina.Globals;
>   import org.apache.catalina.LifecycleException;
>   import org.apache.catalina.LifecycleState;
>   import org.apache.catalina.Service;
> +import org.apache.catalina.core.AprLifecycleListener;
>   import org.apache.catalina.core.AprStatus;
>   import org.apache.catalina.util.LifecycleMBeanBase;
>   import org.apache.coyote.AbstractProtocol;
> @@ -80,7 +81,7 @@ public class Connector extends LifecycleMBeanBase  {
>
>
>   public Connector(String protocol) {
> -boolean apr = AprStatus.isAprAvailable() &&
> +boolean apr = AprLifecycleListener.isAprAvailable() &&
>   AprStatus.getUseAprConnector();
>   ProtocolHandler p = null;
>   try {
> @@ -1020,11 +1021,11 @@ public class Connector extends LifecycleMBeanBase
> {
>   throw new
>
> LifecycleException(sm.getString("coyoteConnector.protocolHandlerNoAprListener",
>   getProtocolHandlerClassName()));
>   }
> -if (protocolHandler.isAprRequired() &&
> !AprStatus.isAprAvailable()) {
> +if (protocolHandler.isAprRequired() &&
> !AprLifecycleListener.isAprAvailable()) {
>   throw new
>
> LifecycleException(sm.getString("coyoteConnector.protocolHandlerNoAprLibrary",
>   getProtocolHandlerClassName()));
>   }
> -if (AprStatus.isAprAvailable() && AprStatus.getUseOpenSSL() &&
> +if (AprLifecycleListener.isAprAvailable() &&
> AprStatus.getUseOpenSSL() &&
>   protocolHandler instanceof AbstractHttp11JsseProtocol) {
>   AbstractHttp11JsseProtocol jsseProtocolHandler =
>   (AbstractHttp11JsseProtocol) protocolHandler;
>
> Feel free to apply.
>
> Regards,
>
> Rainer
>
>
> Am 07.04.2021 um 14:32 schrieb Rainer Jung:
> > I think the reason is:
> >
> > o.a.c.connector.Connector checks AprStatus.isAprAvailable(). It
> > previously checked AprLifecycleListener.isAprAvailable(). The difference
> > is, that the old AprLifecycleListener.isAprAvailable() triggers the
> > init() of AprLifecycleListener, whereas AprStatus.isAprAvailable() does
> > not.
> >
> > I think isAprAvailable() is the only method with side effects that was
> > moved from AprLifecycleListener to AprStatus. Since it is still
> > available in AprLifecycleListener, I think it would be safest to
> > typically not call it in AprStatus but instead in AprLifecycleListener.
> >
> > Regards,
> >
> > Rainer
> >
> > Am 07.04.2021 um 14:02 schrieb Rainer Jung:
> >> Maybe related to f4dac6846c548144799b1c3f33aba4eb320a3413.
> >>
> >> Am 07.04.2021 um 13:53 schrieb Rainer Jung:
> >>> Hi there,
> >>>
> >>> a colleague of mine observed a change in behavior starting with TC
> >>> 9.0.38: until 9.0.37 the default server.xml with the connector
> >>> protocol="HTTP/1.1", installed tcnative and attribute
> >>> useAprConnector="true" for the AprLifecycleListener actually starts
> >>> an APR connector as documented and expected:
> >>>
> >>> INFO [main] org.apache.coyote.AbstractProtocol.init Initializing
> >>> ProtocolHandler ["http-apr-8080"]
> >>>
> >>> Same situation starting with 9.0.38 (and at least until 9.0.44):
> >>>
> >>> INFO [main] org.apache.coyote.AbstractProtocol.init Initializing
> >>> ProtocolHandler ["http-nio-8080"]
> >>>
> >>> I increased log level for a coupe of packages to FINEST and see the
> >>> following differences during startup:
> >>>
> >>> FINE [main] org.apache.tomcat.util.IntrospectionUtils.setProperty
> >>> IntrospectionUtils: setProperty(class
> >>> org.apache.catalina.core.AprLifecycleListener SSLEngine=on)
> >>> FINE [main] org.apache.tomcat.util.digester.SetPropertiesRule.begin
> >>> [SetPropertiesRule]{Server/Listener} Setting property
> >>> 'useAprConnector' to 'true'
> >>> FINE [main] org.apache.tomcat.util.IntrospectionUtils.setProperty
> >>> IntrospectionUtils: setProperty(class
> >>> 

Re: OpenJDK 17 Early Access build 16 is now available

2021-04-02 Thread Rémy Maucherat
On Fri, Apr 2, 2021 at 10:09 AM Rory O'Donnell 
wrote:

>
> Hi Mark,
>
> *OpenJDK 17 Early Access build 16 is now available at
> **http://jdk.java.net/17* 
>
>   * These early-access , open-source builds are provided under the
>   o GNU General Public License, version 2, with the Classpath
> Exception 
>
>   * Schedule (proposed)
>   o 2021/06/10 Rampdown Phase One
>   o 2021/07/15 Rampdown Phase Two
>   o 2021/08/05 Initial Release Candidate
>   o 2021/08/19 Final Release Candidate
>   o 2021/09/14 General Availability
>
>   * Features:*Heads-up on an important Candidate JEP
> *
>   o *Candidate - JEP 403: **Strongly Encapsulate JDK Internals
> *
>   o successor to JEP 396: Strongly Encapsulate JDK Internals by
> Default 
>   o strongly encapsulate all internal elements of the JDK by default
>   o exception for Critical Internal APIs such as /sun.misc.Unsafe/
>

Our use of Unsafe does indeed seem to still work, and the testsuite and
OpenSSL TLS support is still functional.
It would be good to get a better method to deallocate a direct ByteBuffer,
eventually ...

Rémy


>
>   * JEPs targeted to JDK 17, so far:
>   o JEP 356: Enhanced Pseudo-Random Number Generators
> 
>   o JEP 382: New macOS Rendering Pipeline
> 
>   o JEP 391: macOS/AArch64 Port 
>   o JEP 398: Deprecate the Applet API for Removal
> 
>
>   * Release Notes are available at http://jdk.java.net/17/release-notes
> 
>   * Changes in recent builds that maybe of interest:
>   o Build 16
>   + JDK-8263898: (fs) Files.newOutputStream on the "NUL" special
> device throws FileSystemException: "nul: Incorrect function"
> (win)
>   # Reported by Apache Ant
>   o Build 15
>   + JDK-8263575: Conflict between JDK rpms and OL8 Modularity
> prevents dnf install/updates
>   o Build 14
>   + JDK-8262277: URLClassLoader.getResource throws undocumented
> IllegalArgumentException
>   + JDK-8262351: Extra '0' in java.util.Formatter for '%012a'
> conversion with a sign character
>
> *Project Loom Early-Access Build: **Build 17-loom+5-191*
> *(2021/3/19)*
>
>   * These early-access builds are provided under the GNU General Public
> License, version 2, with the Classpath Exception
> .
>   * These builds are produced for the purpose of gathering feedback. Use
> for any other purpose is at your own risk.
>   * Please send feedback via e-mail to loom-...@openjdk.java.net
> . To send e-mail to this address
> you must first subscribe to the mailing list
> .
>
> *Quality Report for March 2021 was published here [1]*.
>
>   * Thanks to everyone who contributed by creating features or
> enhancements, logging  bugs, or downloading and testing the
> early-access builds.
>
> *Worth reading - **The Arrival of Java 16!
> *
>
>   * JDK 16 Migration guide -
> https://docs.oracle.com/en/java/javase/16/migrate/getting-started.html
>   * #AllTestsGreenOnJDK16 - If your tests are green on JDK 16 please
> respond to this *tweet
> *.
>   * Oracle Developer Live event - Individual sessions:
>  1. *Java 16: Consistency and Innovation* (Aurelio Garcia-Ribeyro):
> https://youtu.be/1acKCBbd6f4 
>  2. *Java Language Futures: Spring 2021* (Gavin Bierman):
> https://youtu.be/K9SVV0XNIP8 
>  3. *Ask the Java Architects* (Mark Reinhold, Brian Goetz, Mikael
> Vidstedt, Ron Pressler): https://youtu.be/CVE4bWvuD3o
> 
>  4. *Learn Java 16 with IntelliJ IDEA *(Trisha Gee[JetBrains])*:
> *https://youtu.be/1hyWJTjxeGM **
>  5. *How Records Can Improve Serialization* (Julia Boes, Chris
> Hegarty): https://youtu.be/44D8M6ZxuLU
> 
>  6. *Vector API: SIMD Programming in Java* (Paul Sandoz, Sandhya
> Viswanathan[Intel]): https://youtu.be/VYo3p4R66N8
> 
>  7. *Your Guide to OpenJDK Development* (Jesper Wilhelmsson):
> https://youtu.be/bHcKTYy_Nec 
>  8. *Project Skara: Migrating OpenJDK to Git and 

Re: [VOTE] Release Apache Tomcat Native 1.2.28

2021-04-01 Thread Rémy Maucherat
On Thu, Apr 1, 2021 at 3:57 PM Mark Thomas  wrote:

> Version 1.2.28 includes the following changes compared to 1.2.27
>
> - Correct regression in previous fix for BZ 65181
>
> The proposed release artefacts can be found at [1],
> and the build was done using tag [2].
>
> The Apache Tomcat Native 1.2.28 release is
>   [X] Stable, go ahead and release
>   [ ] Broken because of ...
>

Basic compilation and testing.

Rémy


>
> Thanks,
>
> Mark
>
>
> [1]
>
> https://dist.apache.org/repos/dist/dev/tomcat/tomcat-connectors/native/1.2.28
> [2]
>
> https://gitbox.apache.org/repos/asf?p=tomcat-native.git;a=commit;h=5566385ab63361d8d707613508d803964a15a1f8
>
> -
> To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org
> For additional commands, e-mail: dev-h...@tomcat.apache.org
>
>


Re: [tomcat] branch 9.0.x updated: Fix merge error

2021-04-01 Thread Rémy Maucherat
On Thu, Apr 1, 2021 at 3:14 PM  wrote:

> This is an automated email from the ASF dual-hosted git repository.
>
> markt pushed a commit to branch 9.0.x
> in repository https://gitbox.apache.org/repos/asf/tomcat.git
>
>
> The following commit(s) were added to refs/heads/9.0.x by this push:
>  new 97e6911  Fix merge error
> 97e6911 is described below
>
> commit 97e6911953f21815a0e2f9e774dd0b35e622c902
> Author: Mark Thomas 
> AuthorDate: Thu Apr 1 14:13:34 2021 +0100
>
> Fix merge error
>

I hate it too ... Right after a new release, git cherry pick will always
cause a conflict since there are no lines matching for the auto merge ...

Rémy


> ---
>  webapps/docs/changelog.xml | 4 
>  1 file changed, 4 deletions(-)
>
> diff --git a/webapps/docs/changelog.xml b/webapps/docs/changelog.xml
> index 98899d4..55e5ff8 100644
> --- a/webapps/docs/changelog.xml
> +++ b/webapps/docs/changelog.xml
> @@ -103,10 +103,7 @@
>They eventually become mixed with the numbered issues (i.e., numbered
>issues do not "pop up" wrt. others).
>  -->
> -<<< HEAD
>  
> -===
> -
>
>  
>
> @@ -117,7 +114,6 @@
>
>  
>
> ->>> 5651dfad39... Remove Bnd annotation dependency from Jakarta API
> JARs
>  
>  
>
>
> -
> To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org
> For additional commands, e-mail: dev-h...@tomcat.apache.org
>
>


Re: [VOTE] Release Apache Tomcat 8.5.65

2021-03-31 Thread Rémy Maucherat
On Tue, Mar 30, 2021 at 3:13 PM Mark Thomas  wrote:

> The proposed Apache Tomcat 8.5.65 release is now available for voting.
>
> The notable changes compared to the 8.5.64 release are:
>
> - Fix a regression in 8.5.64 that meant that an error during an
>asynchronous read broke all future asynchronous reads associated with
>the same request instance.
>
> - Prevent concurrent calls to ServletInputStream.isReady() corrupting
>the input buffer.
>
> - Update the packaged version of Tomcat Native to 1.2.27 to pick up
>binaries built with OpenSSL 1.1.1k
>
> Along with lots of other bug fixes and improvements.
>
> For full details, see the changelog:
> https://ci.apache.org/projects/tomcat/tomcat85/docs/changelog.html
>
> It can be obtained from:
> https://dist.apache.org/repos/dist/dev/tomcat/tomcat-8/v8.5.65/
>
> The Maven staging repo is:
> https://repository.apache.org/content/repositories/orgapachetomcat-1306/
>
> The tag is:
> https://github.com/apache/tomcat/tree/8.5.65
> 752c1b9221f7d51a9f0f13d5ce83540589e228e4
>
> The proposed 8.5.65 release is:
> [ ] Broken - do not release
> [X] Stable - go ahead and release as 8.5.65
>
> Rémy


Re: [VOTE] Release Apache Tomcat 9.0.45

2021-03-31 Thread Rémy Maucherat
On Tue, Mar 30, 2021 at 12:52 PM Mark Thomas  wrote:

> The proposed Apache Tomcat 9.0.45 release is now available for voting.
>
> The notable changes compared to the 9.0.44 release are:
>
> - Fix a regression in 9.0.44 that meant that an error during an
>asynchronous read broke all future asynchronous reads associated with
>the same request instance.
>
> - Prevent concurrent calls to ServletInputStream.isReady() corrupting
>the input buffer.
>
> - Update the packaged version of Tomcat Native to 1.2.27 to pick up
>binaries built with OpenSSL 1.1.1k
>
> Along with lots of other bug fixes and improvements.
>
> For full details, see the changelog:
> https://ci.apache.org/projects/tomcat/tomcat9/docs/changelog.html
>
> It can be obtained from:
> https://dist.apache.org/repos/dist/dev/tomcat/tomcat-9/v9.0.45/
> The Maven staging repo is:
> https://repository.apache.org/content/repositories/orgapachetomcat-1305/
> The tag is:
> https://github.com/apache/tomcat/tree/9.0.45
> 4dcf07fd1b53d3934d408060c6ef1ea13894c16f
>
> The proposed 9.0.45 release is:
> [ ] Broken - do not release
> [X] Stable - go ahead and release as 9.0.45
>
> Rémy


Re: [VOTE] Release Apache Tomcat 10.0.5

2021-03-31 Thread Rémy Maucherat
On Tue, Mar 30, 2021 at 10:46 AM Mark Thomas  wrote:

> The proposed Apache Tomcat 10.0.5 release is now available for
> voting.
>
> Apache Tomcat 10.x implements Jakarta EE 9 and, as such, the primary
> package for all the specification APIs has changed from javax.* to
> jakarta.*
>
> Applications that run on Tomcat 9 will not run on Tomcat 10 without
> changes. Java EE applications designed for Tomcat 9 and earlier may be
> placed in the $CATALINA_BASE/webapps-javaee directory and Tomcat will
> automatically convert them to Jakarta EE and copy them to the webapps
> directory
>
> The notable changes compared to 10.0.4 are:
>
> - Fix a regression in 10.0.4 that meant that an error during an
>asynchronous read broke all future asynchronous reads associated with
>the same request instance.
>
> - Prevent concurrent calls to ServletInputStream.isReady() corrupting
>the input buffer.
>
> - Update the packaged version of Tomcat Native to 1.2.27 to pick up
>binaries built with OpenSSL 1.1.1k
>
> Along with lots of other bug fixes and improvements.
>
>
> For full details, see the changelog:
> https://ci.apache.org/projects/tomcat/tomcat10/docs/changelog.html
>
> It can be obtained from:
> https://dist.apache.org/repos/dist/dev/tomcat/tomcat-10/v10.0.5/
> The Maven staging repo is:
> https://repository.apache.org/content/repositories/orgapachetomcat-1304
> The tag is:
> https://github.com/apache/tomcat/tree/10.0.5
> 328d87e3d1ef41c46b5173114e30d37394bd68b9
>
> The proposed 10.0.5 release is:
> [ ] Broken - do not release
> [X] Stable - go ahead and release as 10.0.5 (stable)
>
> We should probably consider moving the migration tool to 1.0 if it "works".

Rémy


Re: [VOTE] Release Apache Tomcat Native 1.2.27

2021-03-26 Thread Rémy Maucherat
On Fri, Mar 26, 2021 at 12:10 PM Mark Thomas  wrote:

> Version 1.2.27 includes the following changes compared to 1.2.26
>
> - Windows binaries built using 1.1.1k
>
> - BZ 65181 - Improved support for OpenSSL engines with proprietary key
>formats
>
> The proposed release artefacts can be found at [1],
> and the build was done using tag [2].
>
> The Apache Tomcat Native 1.2.27 release is
>   [X] Stable, go ahead and release
>   [ ] Broken because of ...
>

Rémy


>
> Thanks,
>
> Mark
>
>
> [1]
>
> https://dist.apache.org/repos/dist/dev/tomcat/tomcat-connectors/native/1.2.27
> [2]
>
> https://gitbox.apache.org/repos/asf?p=tomcat-native.git;a=commit;h=c7c22b9c7f96869aa8f04e6692518aa5323a467b
>
> -
> To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org
> For additional commands, e-mail: dev-h...@tomcat.apache.org
>
>


Re: [tomcat] branch master updated: Avoid reflection use for classloader configuration

2021-03-22 Thread Rémy Maucherat
On Mon, Mar 22, 2021 at 12:53 PM  wrote:

> -
> setClassLoaderProperty("clearReferencesObjectStreamClassCaches",
> -getClearReferencesObjectStreamClassCaches());
> -
> setClassLoaderProperty("clearReferencesObjectStreamClassCaches",
> -getClearReferencesObjectStreamClassCaches());
>
>
When writing this change, I noticed this. So I don't understand what
happened here, especially since the duplication is not in 8.5.

Rémy


Re: Back-port asyncIO support for HTTP/2 to 8.5.x

2021-03-15 Thread Rémy Maucherat
On Thu, Mar 11, 2021 at 5:39 PM Mark Thomas  wrote:

> On 11/03/2021 16:11, Rémy Maucherat wrote:
> > On Thu, Mar 11, 2021 at 3:17 PM Mark Thomas  wrote:
> >
> >> A question mainly for Rémy I guess.
> >>
> >> Any reason not to back-port the asyncIO HTTP/2 implementation to 8.5.x?
> >>
> >
> > Today's my lucky day it seems :)
> >
> > Ok, so there's a balance between the usefulness of the feature, the
> > complexity of the changes, the risk introduced and the stability of more
> > ancient branches. The older the branch, the higher the expected stability
> > IMO. There's often the scenario of backporting too early: the feature
> seems
> > fine, but actually it's just not been tested enough yet.
> >
> > Here, I was thinking the feature is not a must have, and 8.5 is supposed
> to
> > be really stable now, so I limited the backport to 9 and never considered
> > 8.5. I understand the increased usefulness it has if the idea is to
> really
> > harmonize the h2 code.
>
> Yes, it is a balance.
>
> Given the recent flurry of HTTP/2 issues that have affected 8.5.x, 9.0.x
> and 10.0.x, I think there is more benefit in harmonization than there is
> risk in introducing the async code.
>
> I think I have most of the alignment complete apart from the async
> stuff. I'll try and keep that in a separate commit in case it causes
> problems.
>

After checking, +1 for fully harmonizing the HTTP/2 code, the bug reports
are now quite limited and less serious, while we can assume there's
significant use of HTTP/2.
I looked at the connector code and the backports seem to be there.

Rémy


Re: Back-port asyncIO support for HTTP/2 to 8.5.x

2021-03-11 Thread Rémy Maucherat
On Thu, Mar 11, 2021 at 3:17 PM Mark Thomas  wrote:

> A question mainly for Rémy I guess.
>
> Any reason not to back-port the asyncIO HTTP/2 implementation to 8.5.x?
>

Today's my lucky day it seems :)

Ok, so there's a balance between the usefulness of the feature, the
complexity of the changes, the risk introduced and the stability of more
ancient branches. The older the branch, the higher the expected stability
IMO. There's often the scenario of backporting too early: the feature seems
fine, but actually it's just not been tested enough yet.

Here, I was thinking the feature is not a must have, and 8.5 is supposed to
be really stable now, so I limited the backport to 9 and never considered
8.5. I understand the increased usefulness it has if the idea is to really
harmonize the h2 code.

Rémy


> Mark
>
> -
> To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org
> For additional commands, e-mail: dev-h...@tomcat.apache.org
>
>


Re: Refactoring the HTTP/2 API

2021-03-11 Thread Rémy Maucherat
On Thu, Mar 11, 2021 at 11:03 AM Mark Thomas  wrote:

> Hi all,
>
> I've been spending a lot of time looking at the HTTP/2 code lately. It
> has become apparent to me that the naming of various methods is not as
> clear as it code be. For example, it is not clear if a method is:
> - triggered by receiving a given frame
> - will send a given frame
> - is called before/after receiving/sending a given frame to update state
> - somethign else
> I'd like to fix this. Mostly, this means renaming a lot of the methods.
>
> In 10.0.x and 9.0.x much of the API is package private so we are
> relatively free to make changes. That is not the case with 8.5.x.
> However, I'd *really* like to keep the HTTP/2 code aligned between all
> versions as much as possible.
>
> We have made changes to the 8.5.x public API in the past (adding
> methods, adding parameters to methods) and we haven't received any
> negative feedback afterwards. Also, the HTTP/2 code is unlikely to be
> extended for some custom purpose.
>
> Given all of the above, I'd like to bring the current HTTP/2 code into
> alignment across all three versions. The majority of the changes will be
> in the public API in 8.5.x.
>
> Are there any objections to me doing this?
>

You can try it, but I suppose if anyone complains you'll have to take it
out.

Rémy

>
> Mark
>
> -
> To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org
> For additional commands, e-mail: dev-h...@tomcat.apache.org
>
>


Re: JRE networking bug - JDK-8263243

2021-03-09 Thread Rémy Maucherat
On Tue, Mar 9, 2021 at 1:06 PM Mark Thomas  wrote:

> Hi Rory,
>
> I have spent a lot of time of the last few days investigating some
> networking issues with Tomcat and it looks like the root cause is in the
> JRE. I have opened the following issue:
>
> https://bugs.openjdk.java.net/browse/JDK-8263243
>
> TL;DR it appears that ServerSocketChannel.accept() can return multiple,
> valid SocketChannel instances for the same client connection. Start
> trying to use those duplicate SocketChannel instances with a Selector
> and stuff starts failing.
>
> If the bug report is valid, it could be the root cause of all sorts of
> strange networking bugs. I'd appreciate anything you can do to get the
> right folks looking at the report. I am happy to provide any additional
> information that may be required and/or test nay proposed fixes.
>

I don't get what the "wrk" test client is. Anyway, impressive stuff.

I wonder if it happens with NIO2 as well.

Rémy


>
> Mark
>
> -
> To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org
> For additional commands, e-mail: dev-h...@tomcat.apache.org
>
>


Re: [VOTE] Release Apache Tomcat 10.0.4

2021-03-05 Thread Rémy Maucherat
On Fri, Mar 5, 2021 at 12:26 PM Mark Thomas  wrote:

> The proposed Apache Tomcat 10.0.4 release is now available for
> voting.
>
> Apache Tomcat 10.x implements Jakarta EE 9 and, as such, the primary
> package for all the specification APIs has changed from javax.* to
> jakarta.*
>
> Applications that run on Tomcat 9 will not run on Tomcat 10 without
> changes. Java EE applications designed for Tomcat 9 and earlier may be
> placed in the $CATALINA_BASE/webapps-javaee directory and Tomcat will
> automatically convert them to Jakarta EE and copy them to the webapps
> directory
>
> The notable changes compared to 10.0.2 are:
>
> - Integration of the Apache Tomcat Migration Tool for Jakarta EE via
>the webapps-javaee directory
>
> - Improvements to Async and non-blocking IO error handling
>
> - Add support for Unix Domain Sockets to the APR/Native connector
>
> Along with lots of other bug fixes and improvements.
>
>
> For full details, see the changelog:
> https://ci.apache.org/projects/tomcat/tomcat10/docs/changelog.html
>
> It can be obtained from:
> https://dist.apache.org/repos/dist/dev/tomcat/tomcat-10/v10.0.4/
> The Maven staging repo is:
> https://repository.apache.org/content/repositories/orgapachetomcat-1303
> The tag is:
> https://github.com/apache/tomcat/tree/10.0.4
> ec47d24b49bc176f47b786bc96248e94d1823685
>
> The proposed 10.0.4 release is:
> [ ] Broken - do not release
> [X] Stable - go ahead and release as 10.0.4 (stable)
>
> Rémy


Re: [VOTE] Release Apache Tomcat 8.5.64

2021-03-05 Thread Rémy Maucherat
On Fri, Mar 5, 2021 at 12:49 AM Mark Thomas  wrote:

> The proposed Apache Tomcat 8.5.64 release is now available for voting.
>
> The notable changes compared to the 8.5.63 release are:
>
> - Improvements to Async and non-blocking IO error handling
>
> - Improvements to handling of OpenSSL errors
>
> - Align the behaviour when null is passed to the ServletResponse
>methods setCharacterEncoding(), setContentType() and setLocale()
>with the recent clarification from the Jakarta Servlet project of
>the expected behaviour in these cases.
>
> Along with lots of other bug fixes and improvements.
>
> For full details, see the changelog:
> https://ci.apache.org/projects/tomcat/tomcat85/docs/changelog.html
>
> It can be obtained from:
> https://dist.apache.org/repos/dist/dev/tomcat/tomcat-8/v8.5.64/
>
> The Maven staging repo is:
> https://repository.apache.org/content/repositories/orgapachetomcat-1302/
>
> The tag is:
> https://github.com/apache/tomcat/tree/8.5.64
> c47f86adea090175669df8b2ca04c93050bcaf8c
>
> The proposed 8.5.64 release is:
> [ ] Broken - do not release
> [X] Stable - go ahead and release as 8.5.64
>
> Rémy


Re: [tomcat] branch master updated: Add the option for distributions to disable dependency downloads

2021-03-05 Thread Rémy Maucherat
On Fri, Mar 5, 2021 at 10:41 AM Mark Thomas  wrote:

> On 05/03/2021 09:29, r...@apache.org wrote:
> > This is an automated email from the ASF dual-hosted git repository.
> >
> > remm pushed a commit to branch master
> > in repository https://gitbox.apache.org/repos/asf/tomcat.git
> >
> >
> > The following commit(s) were added to refs/heads/master by this push:
> >   new de3a101  Add the option for distributions to disable
> dependency downloads
> > de3a101 is described below
> >
> > commit de3a101b24bbd5e79283a822ea7b64cc11286267
> > Author: remm 
> > AuthorDate: Fri Mar 5 10:29:18 2021 +0100
> >
> >  Add the option for distributions to disable dependency downloads
> >
> >  The dependencies will still be checked but no download will actually
> >  occur. This allows a distribution to provide its own built binaries
> >  while making sure no other ones are included.
>
> Could you explain the use for this some more. I don't understand what
> problem it is trying to solve.
>

I wasn't convinced initially either.

It's the usual Linux distribution business that rebuilds everything. When
Tomcat adds a new dependency, it could be included via automatic non
rebuilt download and instead they only want to use the jars from their own
packages so they prefer an error.

Rémy


Re: [VOTE] Release Apache Tomcat 9.0.44

2021-03-05 Thread Rémy Maucherat
On Thu, Mar 4, 2021 at 11:22 PM Mark Thomas  wrote:

> The proposed Apache Tomcat 9.0.44 release is now available for voting.
>
> The notable changes compared to the 9.0.43 release are:
>
> - Improvements to Async and non-blocking IO error handling
>
> - Improvements to handling of OpenSSL errors
>
> - Align the behaviour when null is passed to the ServletResponse
>methods setCharacterEncoding(), setContentType() and setLocale()
>with the recent clarification from the Jakarta Servlet project of
>the expected behaviour in these cases.
>
>
> Along with lots of other bug fixes and improvements.
>
> For full details, see the changelog:
> https://ci.apache.org/projects/tomcat/tomcat9/docs/changelog.html
>
> It can be obtained from:
> https://dist.apache.org/repos/dist/dev/tomcat/tomcat-9/v9.0.44/
> The Maven staging repo is:
> https://repository.apache.org/content/repositories/orgapachetomcat-1301/
> The tag is:
> https://github.com/apache/tomcat/tree/9.0.44
> 7b4007a6a77300056f4681b064d7332c2284cbdd
>
> The proposed 9.0.44 release is:
> [ ] Broken - do not release
> [X] Stable - go ahead and release as 9.0.44
>
> Rémy


Re: [VOTE] Release Apache Tomcat 10.0.3

2021-03-05 Thread Rémy Maucherat
On Thu, Mar 4, 2021 at 10:20 PM Mark Thomas  wrote:

> The proposed Apache Tomcat 10.0.3 release is now available for
> voting.
>
> Apache Tomcat 10.x implements Jakarta EE 9 and, as such, the primary
> package for all the specification APIs has changed from javax.* to
> jakarta.*
>
> Applications that run on Tomcat 9 will not run on Tomcat 10 without
> changes. Java EE applications designed for Tomcat 9 and earlier may be
> placed in the $CATALINA_BASE/webapps-javaee directory and Tomcat will
> automatically convert them to Jakarta EE and copy them to the webapps
> directory
>
> The notable changes compared to 10.0.2 are:
>
> - Integration of the Apache Tomcat Migration Tool for Jakarta EE via
>the webapps-javaee directory
>
> - Improvements to Async and non-blocking IO error handling
>
> - Add support for Unix Domain Sockets to the APR/Native connector
>
> Along with lots of other bug fixes and improvements.
>
>
> For full details, see the changelog:
> https://ci.apache.org/projects/tomcat/tomcat10/docs/changelog.html
>
> It can be obtained from:
> https://dist.apache.org/repos/dist/dev/tomcat/tomcat-10/v10.0.3/
> The Maven staging repo is:
> https://repository.apache.org/content/repositories/orgapachetomcat-1300
> The tag is:
> https://github.com/apache/tomcat/tree/10.0.3
> 50c2c705bb74e35ada28147aa2649cb43398b3f9
>
> The proposed 10.0.3 release is:
> [X] Broken - do not release
> [ ] Beta   - go ahead and release as 10.0.3 (beta)
> [ ] Stable - go ahead and release as 10.0.3 (stable)
>
> The auto deployment from the migration folder fails with:
05-Mar-2021 10:11:35.541 WARNING [main]
org.apache.catalina.startup.HostConfig.migrateLegacyApp Migration failure
java.nio.file.NoSuchFileException:
/home/remm/Work/tomcat/apache-tomcat-trunk/output/build/webapps/examples-tc85
at
java.base/sun.nio.fs.UnixException.translateToIOException(UnixException.java:92)
at
java.base/sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:111)
at
java.base/sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:116)
at java.base/sun.nio.fs.UnixCopyFile.move(UnixCopyFile.java:430)
at
java.base/sun.nio.fs.UnixFileSystemProvider.move(UnixFileSystemProvider.java:267)
at java.base/java.nio.file.Files.move(Files.java:1422)
at
org.apache.catalina.startup.HostConfig.migrateLegacyApp(HostConfig.java:1301)
at
org.apache.catalina.startup.HostConfig$MigrateApp.run(HostConfig.java:2040)

Fixed with
https://github.com/apache/tomcat/commit/1e99eaf16aee080bcb8b8c363373d4f4d2676bbc

Rémy


Re: [tomcat] branch master updated: Fix SpotBugs warnings

2021-03-04 Thread Rémy Maucherat
On Thu, Feb 25, 2021 at 4:58 PM  wrote:

> This is an automated email from the ASF dual-hosted git repository.
>
> markt pushed a commit to branch master
> in repository https://gitbox.apache.org/repos/asf/tomcat.git
>
>
> The following commit(s) were added to refs/heads/master by this push:
>  new 35a2e04  Fix SpotBugs warnings
> 35a2e04 is described below
>
> commit 35a2e044d5988466b21b664610ce57b502833cd2
> Author: Mark Thomas 
> AuthorDate: Thu Feb 25 15:57:30 2021 +
>
> Fix SpotBugs warnings
> ---
>  java/org/apache/catalina/startup/HostConfig.java   |  8 +---
>  res/findbugs/filter-false-positives.xml|  6 ++
>  .../catalina/nonblocking/TestNonBlockingAPI.java   | 18
> +-
>  3 files changed, 20 insertions(+), 12 deletions(-)
>
> diff --git a/java/org/apache/catalina/startup/HostConfig.java
> b/java/org/apache/catalina/startup/HostConfig.java
> index 6f332b2..b34ea58 100644
> --- a/java/org/apache/catalina/startup/HostConfig.java
> +++ b/java/org/apache/catalina/startup/HostConfig.java
> @@ -1237,6 +1237,7 @@ public class HostConfig implements LifecycleListener
> {
>  ExecutorService es = host.getStartStopExecutor();
>  List> results = new ArrayList<>();
>
> +// Should not be null as we test above if this is a directory
>  String[] migrationCandidates = legacyAppBase.list();
>  for (String migrationCandidate : migrationCandidates) {
>  File source = new File(legacyAppBase, migrationCandidate);
> @@ -1282,7 +1283,8 @@ public class HostConfig implements LifecycleListener
> {
>  tempNew = File.createTempFile("new", null,
> host.getLegacyAppBaseFile());
>  tempOld = File.createTempFile("old", null,
> host.getLegacyAppBaseFile());
>  // createTempFile is not directly compatible with
> directories, so cleanup
> -tempNew.delete();
> +Files.delete(tempNew.toPath());
> +Files.delete(tempOld.toPath());
>
>  // The use of defaults is deliberate here to avoid having to
>  // recreate every configuration option on the host. Better to
> change
> @@ -1295,8 +1297,8 @@ public class HostConfig implements LifecycleListener
> {
>  migration.execute();
>
>  // Use rename
> -destination.renameTo(tempOld);
> -tempNew.renameTo(destination);
> +Files.move(destination.toPath(), tempOld.toPath());
>

Ok, I'm super sorry I didn't retest after this seemingly cosmetic change
until the release vote ... This breaks the migration feature as it now
needs a if (destination.exists()) { } wrapping Files.move, otherwise it
will throw an IOE if the webapp was not previously migrated.

So 10.0.3 is broken. This won't affect 9 and 8.5.

Rémy


> +Files.move(tempNew.toPath(), destination.toPath());
>  ExpandWar.delete(tempOld);
>
>  } catch (Throwable t) {
> diff --git a/res/findbugs/filter-false-positives.xml
> b/res/findbugs/filter-false-positives.xml
> index e47e128..69cfd78 100644
> --- a/res/findbugs/filter-false-positives.xml
> +++ b/res/findbugs/filter-false-positives.xml
> @@ -532,6 +532,12 @@
>  
>
>
> +
> +
> +
> +
> +  
> +  
>  
>  
>  
> diff --git a/test/org/apache/catalina/nonblocking/TestNonBlockingAPI.java
> b/test/org/apache/catalina/nonblocking/TestNonBlockingAPI.java
> index 0ca028b..97c6a22 100644
> --- a/test/org/apache/catalina/nonblocking/TestNonBlockingAPI.java
> +++ b/test/org/apache/catalina/nonblocking/TestNonBlockingAPI.java
> @@ -1376,9 +1376,9 @@ public class TestNonBlockingAPI extends
> TomcatBaseTest {
>
>  private static final long serialVersionUID = 1L;
>
> -private final CountDownLatch responseCommitLatch;
> -private final CountDownLatch clientCloseLatch;
> -private final CountDownLatch asyncCompleteLatch;
> +private final transient CountDownLatch responseCommitLatch;
> +private final transient CountDownLatch clientCloseLatch;
> +private final transient CountDownLatch asyncCompleteLatch;
>  private final boolean swallowIoException;
>
>  public NBWriteServlet02(CountDownLatch responseCommitLatch,
> CountDownLatch clientCloseLatch,
> @@ -1441,7 +1441,7 @@ public class TestNonBlockingAPI extends
> TomcatBaseTest {
>  private final CountDownLatch responseCommitLatch;
>  private final CountDownLatch clientCloseLatch;
>  private final boolean swallowIoException;
> -private volatile int stage = 0;
> +private volatile AtomicInteger stage = new AtomicInteger(0);
>
>  public TestWriteListener02(AsyncContext ac, CountDownLatch
> responseCommitLatch,
>  CountDownLatch clientCloseLatch, boolean
> swallowIoException) {
> @@ -1456,12 +1456,12 @@ public class TestNonBlockingAPI extends
> TomcatBaseTest {
>  try {
>  

Re: Tagging 10.0.x (and 9.0.x, 8.5.x)

2021-03-03 Thread Rémy Maucherat
On Wed, Mar 3, 2021 at 12:47 PM Mark Thomas  wrote:

> Hi all,
>
> It is the beginning of the month so it is release time again.
>
> I am currently working on a fix for some async error handling bugs
> reported via Spring WebFlux:
> https://github.com/spring-projects/spring-framework/issues/26434
>
> I think I have a fix but I want to try and create a test case first.
>
> I have a couple of other small things to do so I expect I'll be tagging
> some time tomorrow unless the new tests uncover additional issues.
>

+1

Rémy


>
> Mark
>
> -
> To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org
> For additional commands, e-mail: dev-h...@tomcat.apache.org
>
>


Re: tomcat with epollWait always return keyCount=0, but the read buffer still have data and cause many close_wait

2021-02-22 Thread Rémy Maucherat
On Tue, Feb 23, 2021 at 4:07 AM Dan Zheng  wrote:

> 1. Environment
> OS: CentOS Linux release 7.8.2003 (Core)
>
> JDK: java version "1.8.0_181"
> Java(TM) SE Runtime Environment (build 1.8.0_181-b13)
> Java HotSpot(TM) 64-Bit Server VM (build 25.181-b13, mixed mode)
>
> Tomcat: tomcat-embed-core-8.5.32 with spring-boot-2.0.4-RELEASE
>
> 2. Reproduce Steps
>
> 1. request large data download api, then request 2000+ request to
> another lightweight api
> 2. the system now is Full GC, and the 2000+ request will blocked, then
> close all these request
> 3. after system throw OufOfMemoryError, the memory will be released, the
> cpu and memory  occupation is normal, the system is ok to visit mysql
> database and execute the schedule job
> 4. request any api, the response is always slow, and too may close_wait
> [image: image.png]
>
> 3. Problem Shooting
> a) I check the thread with jstack, ClientPoller in NioEndpoint,
> BlockPoller in NioBlockingSelector are both in
>
> "http-nio-8043-ClientPoller-0" #83 daemon prio=5 os_prio=0
> tid=0x7faf58b5 nid=0x4a82 runnable [0x7faefc8d7000]
>java.lang.Thread.State: RUNNABLE
> at sun.nio.ch.EPollArrayWrapper.epollWait(Native Method)
> at sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269)
> at sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93)
> at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86)
> - locked <0xe3c8b478> (a sun.nio.ch.Util$3)
> - locked <0xe3c8b468> (a
> java.util.Collections$UnmodifiableSet)
> - locked <0xe3c8b330> (a sun.nio.ch.EPollSelectorImpl)
> at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97)
> at
> org.apache.tomcat.util.net.NioEndpoint$Poller.run(NioEndpoint.java:798)
> at java.lang.Thread.run(Thread.java:748)
>
>Locked ownable synchronizers:
> - None
>
> 
> "NioBlockingSelector.BlockPoller-1" #72 daemon prio=5 os_prio=0
> tid=0x7faf591fa800 nid=0x4a77 runnable [0x7faefd3e2000]
>java.lang.Thread.State: RUNNABLE
> at sun.nio.ch.EPollArrayWrapper.epollWait(Native Method)
> at sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269)
> at sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93)
> at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86)
> - locked <0xe3c8bb78> (a sun.nio.ch.Util$3)
> - locked <0xe3c8bb68> (a
> java.util.Collections$UnmodifiableSet)
> - locked <0xe3c8ba40> (a sun.nio.ch.EPollSelectorImpl)
> at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97)
> at
> org.apache.tomcat.util.net.NioBlockingSelector$BlockPoller.run(NioBlockingSelector.java:298)
>
>Locked ownable synchronizers:
>
> 
>
> b) arthas check https://github.com/alibaba/arthas
> i hook the process to see what are the two poller select(selectorTimeout)
> return,
>
> the keyCount = 0, but the read buffer have 191 byte to read, why epollWait
> always return keyCount = 0?
>
> the expected behavior is, tomcat can read the data from buffer and the
> close the socket successfully
>
> c) test with another method
> I change the protocol to "Http11Nio2Protocol", and the close_wait will be
> recycled
>
> I set java.nio.channels.spi.SelectorProvider
> with PollSelectorProvider, the close_wait will be recycled too
>

As you found out, this is a JVM bug and there are workarounds if you
experience it:
https://bz.apache.org/bugzilla/show_bug.cgi?id=63802

Rémy


Re: [tomcat] branch master updated: Allow casual runtime use of the migration tool

2021-02-17 Thread Rémy Maucherat
On Wed, Feb 17, 2021 at 10:52 AM Mark Thomas  wrote:

> On 17/02/2021 09:49, Rémy Maucherat wrote:
> > On Wed, Feb 17, 2021 at 10:23 AM Mark Thomas  wrote:
> >
> >> On 16/02/2021 16:07, r...@apache.org wrote:
> >>> This is an automated email from the ASF dual-hosted git repository.
> >>>
> >>> remm pushed a commit to branch master
> >>> in repository https://gitbox.apache.org/repos/asf/tomcat.git
> >>>
> >>>
> >>> The following commit(s) were added to refs/heads/master by this push:
> >>>  new abd1dea  Allow casual runtime use of the migration tool
> >>> abd1dea is described below
> >>>
> >>> commit abd1dead7804e99e3215e8e01a6cf7448a6b9f36
> >>> Author: remm 
> >>> AuthorDate: Tue Feb 16 17:06:27 2021 +0100
> >>>
> >>> Allow casual runtime use of the migration tool
> >>>
> >>> This allows specifying a profile which will be used for a
> >>> ClassFileTransformer. Nothing much to report, it does basic things
> >> but
> >>> does not do classloader resources or static files.
> >>
> >> I was about to query why you are using reflection when I remembered that
> >>
> >
> > Will remove it obviously, just testing ...
>
> Cool.
>
> >> the library isn't availaable by default. I was planning on changing that
> >> as soon as 0.2.0 was released as part of my implementation of migration
> >> at deployment time.
> >>
> >> Once the library is available by default we can simplify this some.
> >>
> >
> > If you want it available by default "soon", then a new tag is needed as
> the
> > 0.2.0 shaded JAR causes an error (the classpath attribute of the manifest
> > causes a failure as it tries to lookup for the dependent JARs).
>
> We should be able to avoid that by adding it to the list of JARs not to
> scan (which we would want to do anyway as there is no need to scan it).
>

Oops, I had forgotten that, "problem solved" since it has to be done
anyway, I agree :)
The runtime hack works well enough to play but is obviously unreliable.
Example: with the Tomcat 9 examples webapp, the included JSTL impl won't
work since the classes there are not statically converted to jakarta. A
deploy time tool is much more appropriate to resolve that kind of situation
and move any EE impl portions to the new namespace.

Rémy


>
> I agree the classpath needed fixing for other use cases.
>
> >> I was also planning on adding migrate.[sh|bat] scripts to the bin
> >> directory so the migration tool was available on the command line.
> >>
> >
> >  Good idea !
>
> Thanks.
>
> Mark
>
> -
> To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org
> For additional commands, e-mail: dev-h...@tomcat.apache.org
>
>


Re: [tomcat] branch master updated: Allow casual runtime use of the migration tool

2021-02-17 Thread Rémy Maucherat
On Wed, Feb 17, 2021 at 10:23 AM Mark Thomas  wrote:

> On 16/02/2021 16:07, r...@apache.org wrote:
> > This is an automated email from the ASF dual-hosted git repository.
> >
> > remm pushed a commit to branch master
> > in repository https://gitbox.apache.org/repos/asf/tomcat.git
> >
> >
> > The following commit(s) were added to refs/heads/master by this push:
> >  new abd1dea  Allow casual runtime use of the migration tool
> > abd1dea is described below
> >
> > commit abd1dead7804e99e3215e8e01a6cf7448a6b9f36
> > Author: remm 
> > AuthorDate: Tue Feb 16 17:06:27 2021 +0100
> >
> > Allow casual runtime use of the migration tool
> >
> > This allows specifying a profile which will be used for a
> > ClassFileTransformer. Nothing much to report, it does basic things
> but
> > does not do classloader resources or static files.
>
> I was about to query why you are using reflection when I remembered that
>

Will remove it obviously, just testing ...


> the library isn't availaable by default. I was planning on changing that
> as soon as 0.2.0 was released as part of my implementation of migration
> at deployment time.
>
> Once the library is available by default we can simplify this some.
>

If you want it available by default "soon", then a new tag is needed as the
0.2.0 shaded JAR causes an error (the classpath attribute of the manifest
causes a failure as it tries to lookup for the dependent JARs).


>
> I was also planning on adding migrate.[sh|bat] scripts to the bin
> directory so the migration tool was available on the command line.
>

 Good idea !

Rémy


Re: buildbot failure in on tomcat-trunk

2021-02-16 Thread Rémy Maucherat
On Tue, Feb 16, 2021 at 3:54 PM  wrote:

> The Buildbot has detected a new failure on builder tomcat-trunk while
> building tomcat. Full details are available at:
> https://ci.apache.org/builders/tomcat-trunk/builds/5682
>
> Buildbot URL: https://ci.apache.org/
>
> Buildslave for this Build: asf946_ubuntu
>
> Build Reason: The AnyBranchScheduler scheduler named 'on-tomcat-commit'
> triggered this build
> Build Source Stamp: [branch master]
> 01ceebb8e2aeaad7eea97982900928a3dbaf3056
> Blamelist: remm 
>
> BUILD FAILED: failed compile_1
>

A certificate has expired.
https://ci.apache.org/projects/tomcat/tomcat10/logs/5682/TEST-org.apache.tomcat.util.net.TestSSLHostConfigCompat.NIO.txt

Rémy


>
> Sincerely,
>  -The Buildbot
>
>
>
>
> -
> To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org
> For additional commands, e-mail: dev-h...@tomcat.apache.org
>
>


Re: [tomcat-jakartaee-migration] branch master updated: Fix bug. Constant pool size is defined as u2

2021-02-15 Thread Rémy Maucherat
On Thu, Feb 11, 2021 at 3:47 PM Mark Thomas  wrote:

> On 11/02/2021 14:39, Rémy Maucherat wrote:
> > On Thu, Feb 11, 2021 at 12:08 PM Mark Thomas  wrote:
> >
> >> On 10/02/2021 17:27, Raymond Auge wrote:
> >>> missing requirement
> >>> [org.apache.servicemix.bundles.spring-beans [6](R 6.0)]
> >>> osgi.wiring.package; (osgi.wiring.package=jakarta.inject)]
> >>>
> >>> Is the jakarta.inject package exported by someone (in the framework)?
> >>>
> >>> I'm not sure how JIRA sets up it's OSGi runtime.
> >>
> >> Thanks. That was one of my working theories. It is helpful to have it
> >> confirmed.
> >>
> >> I have spent some more time on this today without success. As far as I
> >> can tell, I have converted every reference to javax.inject to jakarta
> >> .inject but I am obviously missing something but I can't figure out
> >> what. I think I am going to leave this for now.
> >>
> >> I'll start on the 0.2.0 release shortly and then plan to think some more
> >> about integration of this tool with Tomcat 10.
> >>
> >
> > +1 for the release.
> > If we're talking about its integration, I can help and/or do it.
>
> Thanks for the offer. I have the deployment approach about 95% complete
> ;). Maybe look at the runtime approach so we can compare?
>

Using the tool as a ClassFileTransformer works well (so this does not cover
classloader resources which need WebResource filtering, or static files),
except actually using the shaded JAR with Tomcat did not work and needed a
manifest tweak to drop its class-path.

Rémy


>
> Mark
>
> -
> To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org
> For additional commands, e-mail: dev-h...@tomcat.apache.org
>
>


Re: [VOTE] Apache Tomcat migration tool for Jakarta EE 0.2.0

2021-02-12 Thread Rémy Maucherat
On Thu, Feb 11, 2021 at 6:47 PM Mark Thomas  wrote:

> The proposed Apache Tomcat migration tool for Jakarta EE 0.2.0 is now
> available for voting.
>
> The significant changes since 0.1.0 are:
>
> - Various fixes to the packages that are and are not converted
>
> - A new option to process zip archives in memory to support zip files
>   that use options that are incompatible with a streaming approach
>
> - A new option to exclude files from transformation
>
> It can be obtained from:
> https://dist.apache.org/repos/dist/dev/tomcat/jakartaee-migration/v0.2.0/
>
> The Maven staging repo is:
> https://repository.apache.org/content/repositories/orgapachetomcat-1299/
>
> The tag is:
> https://github.com/apache/tomcat-jakartaee-migration/tree/0.2.0
> 041e256e92078cd3b16a9adcd77faf257e3a5c88
>
> The proposed 0.2.0 release is:
>
> [ ] -1: Broken. Do not release because...
> [X] +1: Acceptable. Go ahead and release.
>

Rémy

>
> Thanks,
>
> Mark
>
> -
> To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org
> For additional commands, e-mail: dev-h...@tomcat.apache.org
>
>


Re: [tomcat-jakartaee-migration] branch master updated: Fix bug. Constant pool size is defined as u2

2021-02-11 Thread Rémy Maucherat
On Thu, Feb 11, 2021 at 3:47 PM Mark Thomas  wrote:

> On 11/02/2021 14:39, Rémy Maucherat wrote:
> > On Thu, Feb 11, 2021 at 12:08 PM Mark Thomas  wrote:
> >
> >> On 10/02/2021 17:27, Raymond Auge wrote:
> >>> missing requirement
> >>> [org.apache.servicemix.bundles.spring-beans [6](R 6.0)]
> >>> osgi.wiring.package; (osgi.wiring.package=jakarta.inject)]
> >>>
> >>> Is the jakarta.inject package exported by someone (in the framework)?
> >>>
> >>> I'm not sure how JIRA sets up it's OSGi runtime.
> >>
> >> Thanks. That was one of my working theories. It is helpful to have it
> >> confirmed.
> >>
> >> I have spent some more time on this today without success. As far as I
> >> can tell, I have converted every reference to javax.inject to jakarta
> >> .inject but I am obviously missing something but I can't figure out
> >> what. I think I am going to leave this for now.
> >>
> >> I'll start on the 0.2.0 release shortly and then plan to think some more
> >> about integration of this tool with Tomcat 10.
> >>
> >
> > +1 for the release.
> > If we're talking about its integration, I can help and/or do it.
>
> Thanks for the offer. I have the deployment approach about 95% complete
> ;). Maybe look at the runtime approach so we can compare?
>

I don't think the runtime option is a good plan for static resources. It
would be "ok" for classes which would avoid the JAR repackaging troubles,
however.

I'll add the ClassFileTransformer hook to ClassConverter before your
release, for experimentation with our WebappClassLoader. This won't convert
classloader resources though (URLClassLoader.getResourceAsStream).

Rémy


>
> Mark
>
> -
> To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org
> For additional commands, e-mail: dev-h...@tomcat.apache.org
>
>


Re: [tomcat-jakartaee-migration] branch master updated: Fix bug. Constant pool size is defined as u2

2021-02-11 Thread Rémy Maucherat
On Thu, Feb 11, 2021 at 12:08 PM Mark Thomas  wrote:

> On 10/02/2021 17:27, Raymond Auge wrote:
> > missing requirement
> > [org.apache.servicemix.bundles.spring-beans [6](R 6.0)]
> > osgi.wiring.package; (osgi.wiring.package=jakarta.inject)]
> >
> > Is the jakarta.inject package exported by someone (in the framework)?
> >
> > I'm not sure how JIRA sets up it's OSGi runtime.
>
> Thanks. That was one of my working theories. It is helpful to have it
> confirmed.
>
> I have spent some more time on this today without success. As far as I
> can tell, I have converted every reference to javax.inject to jakarta
> .inject but I am obviously missing something but I can't figure out
> what. I think I am going to leave this for now.
>
> I'll start on the 0.2.0 release shortly and then plan to think some more
> about integration of this tool with Tomcat 10.
>

+1 for the release.
If we're talking about its integration, I can help and/or do it.

Rémy


>
> Mark
>
> -
> To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org
> For additional commands, e-mail: dev-h...@tomcat.apache.org
>
>


Re: Integrating migration tool into Tomcat 10

2021-02-10 Thread Rémy Maucherat
On Tue, Feb 9, 2021 at 10:12 PM Mark Thomas  wrote:

> Hi all,
>
> I've been looking at the options to integrate the Java EE to Jakarta EE
> migration functionality into Tomcat 10.
>
> There are essentially two ways to do this: deployment time and runtime.
>
> The simplest solution to implement is probably a separate webapps-legacy
> directory (or some other name) where WARs and DIRs added to that
> directory get converted to Jakarta EE with the new version being placed
> in the webapps directory. There are complexities (such as handling an
> updates of the legacy WAR) but they should be manageable.
>
> A more complex version of the deploy time solution has the legacy web
> application placed in webapps, identified as a legacy webapp and then
> replaced with the new version (several ways to do this). The hard part
> here is the identification of the webapp as a Java EE app. The only
> reliable way to do this is class scanning and that is slow.
>
> The runtime approach converts classes and static resources as they are
> loaded. The classes are relatively easy to handle. A
> ClassFileTransformer can be added to the WebappClassLoader to do this.
> The static files are more of a problem. The good news is that the
> WebResources refactoring means static file access all goes through the
> same code but the filtering required essentially means we'd need to load
> static files into memory - regardless of size, convert them, and then
> update the cache with the converted version. That is likely to have a
> performance impact.
>
> Because of the performance impacts of handling static resources, the
> runtime approach also needs a way to identify applications that need
> conversion. I don't see a reliable, performant way to do that. Which, I
> think, leaves us with the simple deployment time approach and something
> (filename, special directory, something else) to mark an app as needing
> conversion.
>
> A final point, which probably should have been first, is is there a
> demand for this feature? Early indications from the users list and $work
> is that there is (going to be) a demand for this feature.
>
> Thoughts?
>

Overall it seems more people would like it now, for various reasons.
However, there's zero consensus on how since everyone came up with a
different idea.

I'd like to note the tool is the appropriate solution for most cases, so we
cover that already. The main problem with the tool is the perceived effort
from users.

After reading everything, I'd say it's worth adding a second integrated
option, and think I'm now swaying towards the runtime option. The main
problem would be the detection of legacy webapps. We could simply mandate
using an explicit new attribute on the Context element (and done !) so
nobody pays any needless costs.

Rémy


>
> Mark
>
> -
> To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org
> For additional commands, e-mail: dev-h...@tomcat.apache.org
>
>


Re: Integrating migration tool into Tomcat 10

2021-02-09 Thread Rémy Maucherat
On Tue, Feb 9, 2021 at 10:12 PM Mark Thomas  wrote:

> Hi all,
>
> I've been looking at the options to integrate the Java EE to Jakarta EE
> migration functionality into Tomcat 10.
>
> There are essentially two ways to do this: deployment time and runtime.
>
> The simplest solution to implement is probably a separate webapps-legacy
> directory (or some other name) where WARs and DIRs added to that
> directory get converted to Jakarta EE with the new version being placed
> in the webapps directory. There are complexities (such as handling an
> updates of the legacy WAR) but they should be manageable.
>
> A more complex version of the deploy time solution has the legacy web
> application placed in webapps, identified as a legacy webapp and then
> replaced with the new version (several ways to do this). The hard part
> here is the identification of the webapp as a Java EE app. The only
> reliable way to do this is class scanning and that is slow.
>
> The runtime approach converts classes and static resources as they are
> loaded. The classes are relatively easy to handle. A
> ClassFileTransformer can be added to the WebappClassLoader to do this.
> The static files are more of a problem. The good news is that the
> WebResources refactoring means static file access all goes through the
> same code but the filtering required essentially means we'd need to load
> static files into memory - regardless of size, convert them, and then
> update the cache with the converted version. That is likely to have a
> performance impact.
>
> Because of the performance impacts of handling static resources, the
> runtime approach also needs a way to identify applications that need
> conversion. I don't see a reliable, performant way to do that. Which, I
> think, leaves us with the simple deployment time approach and something
> (filename, special directory, something else) to mark an app as needing
> conversion.
>
> A final point, which probably should have been first, is is there a
> demand for this feature? Early indications from the users list and $work
> is that there is (going to be) a demand for this feature.
>
> Thoughts?
>

I kind of proposed the "simplest" option [the one using a separate appBase
for the EE 8 and earlier webapps] a couple times. A slightly more complex
deploy time option could look more polished and maybe preferable, like that
"filename" idea maybe ? Or a marker file ? I'm not sure.

There's a demand for the feature for sure, but probably only up to the
point people realize it's not actually that helpful. This adds a step to
deployment which may fail. The offline tool however is more efficient and
safer. Also the tool now has quite a few "advanced" options [the ones you
just added] for migrating trickier webapps, so how is an automatic
migration with defaults going to work out ? This looks like asking people
to fill up the BZ with stuff.

So assuming the feature goes in, maybe a hybrid solution could work better
than a pure runtime or deploy time solution. The classloader classes and
resources could rely on a runtime conversion [a bit costly but probably
safer], while the static resources could be converted at deployment time
[it's hard to give decent guarantees on resource use if done at runtime].

Rémy

>
> Mark
>
> -
> To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org
> For additional commands, e-mail: dev-h...@tomcat.apache.org
>
>


Re: JDK 16 is now in the Release Candidate Phase

2021-02-08 Thread Rémy Maucherat
On Mon, Feb 8, 2021 at 11:18 AM Martin Grigorov 
wrote:

> Hi Rémy,
>
> On Mon, Feb 8, 2021 at 11:23 AM Rémy Maucherat  wrote:
>
> > On Mon, Feb 8, 2021 at 10:10 AM Martin Grigorov 
> > wrote:
> >
> > > Hi devs,
> > >
> > > TestXxxEndpoint fails concitently on both JDK 16 b35 and JDK 17 b8
> with:
> > >
> > > Testsuite: org.apache.tomcat.util.net.TestXxxEndpoint
> > > Tests run: 3, Failures: 1, Errors: 0, Skipped: 0, Time elapsed: 8.983
> sec
> > > - Standard Error -
> > > 08-Feb-2021 08:38:08.129 INFO [main]
> > > org.apache.catalina.startup.LoggingBaseTest.setUp Starting test case
> > > [testUnixDomainSocket]
> > > 08-Feb-2021 08:38:08.867 INFO [main]
> > > org.apache.coyote.AbstractProtocol.destroy Destroying ProtocolHandler
> > > ["http-nio2-127.0.0.1-auto-1"]
> > > 08-Feb-2021 08:38:08.926 INFO [main]
> > > org.apache.catalina.startup.LoggingBaseTest.setUp Starting test case
> > > [testStartStopBindOnStart]
> > > 08-Feb-2021 08:38:11.842 INFO [main]
> > > org.apache.coyote.AbstractProtocol.init Initializing ProtocolHandler
> > > ["http-nio2-127.0.0.1-auto-2"]
> > > 08-Feb-2021 08:38:11.976 INFO [main]
> > > org.apache.catalina.core.StandardService.startInternal Starting service
> > > [Tomcat]
> > > 08-Feb-2021 08:38:11.979 INFO [main]
> > > org.apache.catalina.core.StandardEngine.startInternal Starting Servlet
> > > engine: [Apache Tomcat/10.0.3-dev]
> > > 08-Feb-2021 08:38:12.813 INFO [main]
> > > org.apache.catalina.startup.ContextConfig.getDefaultWebXmlFragment No
> > > global web.xml found
> > > 08-Feb-2021 08:38:14.322 INFO [main]
> > > org.apache.jasper.servlet.TldScanner.scanJars At least one JAR was
> > scanned
> > > for TLDs yet contained no TLDs. Enable debug logging for this logger
> for
> > a
> > > complete list of JAR
> > > s that were scanned but no TLDs were found in them. Skipping unneeded
> > JARs
> > > during scanning can improve startup time and JSP compilation time.
> > > 08-Feb-2021 08:38:14.462 INFO [main]
> > > org.apache.catalina.core.ApplicationContext.log ContextListener:
> > > contextInitialized()
> > > 08-Feb-2021 08:38:14.463 INFO [main]
> > > org.apache.catalina.core.ApplicationContext.log SessionListener:
> > > contextInitialized()
> > > 08-Feb-2021 08:38:14.483 INFO [main]
> > > org.apache.catalina.core.ApplicationContext.log ContextListener:
> > > attributeAdded('StockTicker', 'async.Stockticker@3a12c404')
> > > 08-Feb-2021 08:38:14.667 INFO [main]
> > > org.apache.coyote.AbstractProtocol.start Starting ProtocolHandler
> > > ["http-nio2-127.0.0.1-auto-2"]
> > > 08-Feb-2021 08:38:14.812 INFO [main]
> > > org.apache.coyote.AbstractProtocol.stop Stopping ProtocolHandler
> > > ["http-nio2-127.0.0.1-auto-2-34601"]
> > > 08-Feb-2021 08:38:14.824 INFO [main]
> > > org.apache.coyote.AbstractProtocol.start Starting ProtocolHandler
> > > ["http-nio2-127.0.0.1-auto-2"]
> > > 08-Feb-2021 08:38:14.966 INFO [main]
> > > org.apache.coyote.AbstractProtocol.pause Pausing ProtocolHandler
> > > ["http-nio2-127.0.0.1-auto-2"]
> > > 08-Feb-2021 08:38:14.967 INFO [main]
> > > org.apache.catalina.core.StandardService.stopInternal Stopping service
> > > [Tomcat]
> > > 08-Feb-2021 08:38:14.971 INFO [main]
> > > org.apache.catalina.core.ApplicationContext.log SessionListener:
> > > contextDestroyed()
> > > 08-Feb-2021 08:38:14.972 INFO [main]
> > > org.apache.catalina.core.ApplicationContext.log ContextListener:
> > > contextDestroyed()
> > > 08-Feb-2021 08:38:15.024 INFO [main]
> > > org.apache.coyote.AbstractProtocol.stop Stopping ProtocolHandler
> > > ["http-nio2-127.0.0.1-auto-2"]
> > > 08-Feb-2021 08:38:15.128 INFO [main]
> > > org.apache.coyote.AbstractProtocol.destroy Destroying ProtocolHandler
> > > ["http-nio2-127.0.0.1-auto-2"]
> > > 08-Feb-2021 08:38:15.134 INFO [main]
> > > org.apache.catalina.startup.LoggingBaseTest.setUp Starting test case
> > > [testStartStopBindOnInit]
> > > 08-Feb-2021 08:38:15.135 INFO [main]
> > > org.apache.coyote.AbstractProtocol.init Initializing ProtocolHandler
> > > ["http-nio2-127.0.0.1-auto-3"]
> > > 08-Feb-2021 08:38:15.240 INFO [main]
> > > org.apache.catalina.core.StandardService.

Re: JDK 16 is now in the Release Candidate Phase

2021-02-08 Thread Rémy Maucherat
On Mon, Feb 8, 2021 at 10:10 AM Martin Grigorov 
wrote:

> Hi devs,
>
> TestXxxEndpoint fails concitently on both JDK 16 b35 and JDK 17 b8 with:
>
> Testsuite: org.apache.tomcat.util.net.TestXxxEndpoint
> Tests run: 3, Failures: 1, Errors: 0, Skipped: 0, Time elapsed: 8.983 sec
> - Standard Error -
> 08-Feb-2021 08:38:08.129 INFO [main]
> org.apache.catalina.startup.LoggingBaseTest.setUp Starting test case
> [testUnixDomainSocket]
> 08-Feb-2021 08:38:08.867 INFO [main]
> org.apache.coyote.AbstractProtocol.destroy Destroying ProtocolHandler
> ["http-nio2-127.0.0.1-auto-1"]
> 08-Feb-2021 08:38:08.926 INFO [main]
> org.apache.catalina.startup.LoggingBaseTest.setUp Starting test case
> [testStartStopBindOnStart]
> 08-Feb-2021 08:38:11.842 INFO [main]
> org.apache.coyote.AbstractProtocol.init Initializing ProtocolHandler
> ["http-nio2-127.0.0.1-auto-2"]
> 08-Feb-2021 08:38:11.976 INFO [main]
> org.apache.catalina.core.StandardService.startInternal Starting service
> [Tomcat]
> 08-Feb-2021 08:38:11.979 INFO [main]
> org.apache.catalina.core.StandardEngine.startInternal Starting Servlet
> engine: [Apache Tomcat/10.0.3-dev]
> 08-Feb-2021 08:38:12.813 INFO [main]
> org.apache.catalina.startup.ContextConfig.getDefaultWebXmlFragment No
> global web.xml found
> 08-Feb-2021 08:38:14.322 INFO [main]
> org.apache.jasper.servlet.TldScanner.scanJars At least one JAR was scanned
> for TLDs yet contained no TLDs. Enable debug logging for this logger for a
> complete list of JAR
> s that were scanned but no TLDs were found in them. Skipping unneeded JARs
> during scanning can improve startup time and JSP compilation time.
> 08-Feb-2021 08:38:14.462 INFO [main]
> org.apache.catalina.core.ApplicationContext.log ContextListener:
> contextInitialized()
> 08-Feb-2021 08:38:14.463 INFO [main]
> org.apache.catalina.core.ApplicationContext.log SessionListener:
> contextInitialized()
> 08-Feb-2021 08:38:14.483 INFO [main]
> org.apache.catalina.core.ApplicationContext.log ContextListener:
> attributeAdded('StockTicker', 'async.Stockticker@3a12c404')
> 08-Feb-2021 08:38:14.667 INFO [main]
> org.apache.coyote.AbstractProtocol.start Starting ProtocolHandler
> ["http-nio2-127.0.0.1-auto-2"]
> 08-Feb-2021 08:38:14.812 INFO [main]
> org.apache.coyote.AbstractProtocol.stop Stopping ProtocolHandler
> ["http-nio2-127.0.0.1-auto-2-34601"]
> 08-Feb-2021 08:38:14.824 INFO [main]
> org.apache.coyote.AbstractProtocol.start Starting ProtocolHandler
> ["http-nio2-127.0.0.1-auto-2"]
> 08-Feb-2021 08:38:14.966 INFO [main]
> org.apache.coyote.AbstractProtocol.pause Pausing ProtocolHandler
> ["http-nio2-127.0.0.1-auto-2"]
> 08-Feb-2021 08:38:14.967 INFO [main]
> org.apache.catalina.core.StandardService.stopInternal Stopping service
> [Tomcat]
> 08-Feb-2021 08:38:14.971 INFO [main]
> org.apache.catalina.core.ApplicationContext.log SessionListener:
> contextDestroyed()
> 08-Feb-2021 08:38:14.972 INFO [main]
> org.apache.catalina.core.ApplicationContext.log ContextListener:
> contextDestroyed()
> 08-Feb-2021 08:38:15.024 INFO [main]
> org.apache.coyote.AbstractProtocol.stop Stopping ProtocolHandler
> ["http-nio2-127.0.0.1-auto-2"]
> 08-Feb-2021 08:38:15.128 INFO [main]
> org.apache.coyote.AbstractProtocol.destroy Destroying ProtocolHandler
> ["http-nio2-127.0.0.1-auto-2"]
> 08-Feb-2021 08:38:15.134 INFO [main]
> org.apache.catalina.startup.LoggingBaseTest.setUp Starting test case
> [testStartStopBindOnInit]
> 08-Feb-2021 08:38:15.135 INFO [main]
> org.apache.coyote.AbstractProtocol.init Initializing ProtocolHandler
> ["http-nio2-127.0.0.1-auto-3"]
> 08-Feb-2021 08:38:15.240 INFO [main]
> org.apache.catalina.core.StandardService.startInternal Starting service
> [Tomcat]
> 08-Feb-2021 08:38:15.240 INFO [main]
> org.apache.catalina.core.StandardEngine.startInternal Starting Servlet
> engine: [Apache Tomcat/10.0.3-dev]
> 08-Feb-2021 08:38:15.252 INFO [main]
> org.apache.catalina.startup.ContextConfig.getDefaultWebXmlFragment No
> global web.xml found
> 08-Feb-2021 08:38:15.747 INFO [main]
> org.apache.jasper.servlet.TldScanner.scanJars At least one JAR was scanned
> for TLDs yet contained no TLDs. Enable debug logging for this logger for a
> complete list of JAR
> s that were scanned but no TLDs were found in them. Skipping unneeded JARs
> during scanning can improve startup time and JSP compilation time.
> 08-Feb-2021 08:38:15.767 INFO [main]
> org.apache.catalina.core.ApplicationContext.log ContextListener:
> contextInitialized()
> 08-Feb-2021 08:38:15.768 INFO [main]
> org.apache.catalina.core.ApplicationContext.log SessionListener:
> contextInitialized()
> 08-Feb-2021 08:38:15.769 INFO [main]
> org.apache.catalina.core.ApplicationContext.log ContextListener:
> attributeAdded('StockTicker', 'async.Stockticker@2f48b3d2')
> 08-Feb-2021 08:38:15.774 INFO [main]
> org.apache.coyote.AbstractProtocol.start Starting ProtocolHandler
> ["http-nio2-127.0.0.1-auto-3-39117"]
> 08-Feb-2021 08:38:15.817 INFO [main]
> 

Re: Applications setting connection specific HTTP headers

2021-02-04 Thread Rémy Maucherat
On Thu, Feb 4, 2021 at 5:31 PM Michael Osipov  wrote:

> Am 2021-02-03 um 13:03 schrieb Mark Thomas:
> > Hi all,
> >
> > We have an open PR related to this for HTTP/2 (#277) and I am seeing
> > related issues at $work with HTTP.
> >
> > In short, applications are doing things like:
> >
> > response.setHeader("Transfer-Encoding", "chunked");
> >
> > which, as you'd expect is causing problems if:
> > - Tomcat doesn't chunk the response
> > - Tomcat does chunk the response, adds its own "chunked" value and the
> >user agent rightly objects to "chunked" appearing twice
> >
> > And so on.
> >
> > I'd like to put something into Tomcat to address this.
> >
> > I think it should be disabled by default so correctly written
> > applications pay a very small penalty. Along the lines of
> >
> > if (someSetting != null) {
> >  // Do header checks
> > }
> >
> > In terms of options I think we need:
> > - something representing the current, allow anything, behaviour
> > - an option to log (with a stack trace so the offending code can be
> >identified) attempts to set such headers
> > - an option to ignore attempts to set such headers
> >
> > Do we need an option that throws an exception if there is an attempt to
> > set such headers?
> >
> > Do we need an option to control which headers and which values will
> > trigger this behaviour? This would make the configuration rather more
> > complex. You'd need to be able to set multiple combinations of header,
> > value and action.
> >
> > Is adding debug (no stacktrace) and trace (with stacktrace) logging to
> > addHeader() sufficient? For identifying faulty code this helps but it
> > doesn't provide a way to work-around the problem. For that you need
> > something that blocks the adding of the header.
> >
> > I'm still considering what might be the best way to fix this. Hence the
> > brain dump above. Thoughts?
>
> Re-reading the PR and your description, I do not really understand why
> the container should handle and automagically fix bad application code?
> Doing non-sense in appication code shall lead to undefined behavior.
> Autofixing means that devs will never fix the real cause and Tomcat will
> handle the symptom.
>
> Can you explain why the problems at work cannot be fixed in the webapp
> itself?
>

Probably it's: Customer X using library Y, says there's no possible way he
could ever update library Y with a fixed version. Thankfully, getting
Tomcat devs to work around the library issue instead is easy to do.
Solution !

Rémy


>
> M
>
> -
> To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org
> For additional commands, e-mail: dev-h...@tomcat.apache.org
>
>


Re: [VOTE] Release Apache Tomcat 7.0.108

2021-02-04 Thread Rémy Maucherat
On Thu, Jan 28, 2021 at 10:49 AM Violeta Georgieva 
wrote:

> The proposed Apache Tomcat 7.0.108 release is now available for voting.
>
> It can be obtained from:
> https://dist.apache.org/repos/dist/dev/tomcat/tomcat-7/v7.0.108/
> The Maven staging repo is:
> https://repository.apache.org/content/repositories/orgapachetomcat-1295/
> The git tag is:
> https://github.com/apache/tomcat/tree/7.0.108
> b57a2ea4466a2d4ea03a0f90e3f0d6c485b3cfea
>
> The proposed 7.0.108 release is:
> [ ] Broken - do not release
> [X] Stable - go ahead and release as 7.0.108 Stable
>
> Regards,
> Violeta
>

So at most one more release to go and 7 is done.

Rémy


Re: Applications setting connection specific HTTP headers

2021-02-03 Thread Rémy Maucherat
On Wed, Feb 3, 2021 at 1:03 PM Mark Thomas  wrote:

> Hi all,
>
> We have an open PR related to this for HTTP/2 (#277) and I am seeing
> related issues at $work with HTTP.
>
> In short, applications are doing things like:
>
> response.setHeader("Transfer-Encoding", "chunked");
>
> which, as you'd expect is causing problems if:
> - Tomcat doesn't chunk the response
> - Tomcat does chunk the response, adds its own "chunked" value and the
>   user agent rightly objects to "chunked" appearing twice
>
> And so on.
>
> I'd like to put something into Tomcat to address this.
>
> I think it should be disabled by default so correctly written
> applications pay a very small penalty. Along the lines of
>
> if (someSetting != null) {
> // Do header checks
> }
>
> In terms of options I think we need:
> - something representing the current, allow anything, behaviour
> - an option to log (with a stack trace so the offending code can be
>   identified) attempts to set such headers
> - an option to ignore attempts to set such headers
>
> Do we need an option that throws an exception if there is an attempt to
> set such headers?
>
> Do we need an option to control which headers and which values will
> trigger this behaviour? This would make the configuration rather more
> complex. You'd need to be able to set multiple combinations of header,
> value and action.
>
> Is adding debug (no stacktrace) and trace (with stacktrace) logging to
> addHeader() sufficient? For identifying faulty code this helps but it
> doesn't provide a way to work-around the problem. For that you need
> something that blocks the adding of the header.
>
> I'm still considering what might be the best way to fix this. Hence the
> brain dump above. Thoughts?
>

There has been some debate about this before, and you did add quite a bit
of code to catch things that would break the protocol. So it seems this
would go above and beyond, and attempt to catch *anything* that could make
a response non compliant with the underlying protocol ?

Rémy


>
> Mark
>
>
> -
> To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org
> For additional commands, e-mail: dev-h...@tomcat.apache.org
>
>


Re: [VOTE][RESULT] Release Apache Tomcat 10.0.2

2021-02-02 Thread Rémy Maucherat
On Tue, Feb 2, 2021 at 8:09 PM Mark Thomas  wrote:

> The following votes were cast:
>
> Binding:
> +1 (stable): markt, remm, mgrigorov
>
> No other votes were cast.
>
> The vote therefore passes.
>
> This will be the first stable 10.0.x release. Woot!
>

Congratulations ! :)

Rémy


>
> Thanks to everyone who contributed to this release.
>
> Mark
>
> -
> To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org
> For additional commands, e-mail: dev-h...@tomcat.apache.org
>
>


Re: [VOTE] Release Apache Tomcat 8.5.63

2021-01-29 Thread Rémy Maucherat
On Fri, Jan 29, 2021 at 12:44 PM Mark Thomas  wrote:

> The proposed Apache Tomcat 8.5.63 release is now available for voting.
>
> The notable changes compared to the 8.5.61 release are:
>
> - Add a new StringInterpreter interface that allows applications to
>   provide customised string attribute value to type conversion within
>   JSPs. This allows applications to provide a conversion
>   implementation that is optimised for the application.
>
> - Add peerAddress to coyote request, which contains the IP address of
>   the direct connection peer. If a reverse proxy sits in front of
>   Tomcat and the protocol used is AJP or HTTP in combination with the
>   RemoteIp(Valve|Filter), the peer address might differ from the
>   remoteAddress. The latter then contains the address of the client in
>   front of the reverse proxy, not the address of the proxy itself.
>
> - Escape elements in the access log that need to be escaped for the
>   access log to be parsed unambiguously.
>
> Along with lots of other bug fixes and improvements.
>
> For full details, see the changelog:
> https://ci.apache.org/projects/tomcat/tomcat85/docs/changelog.html
>
> It can be obtained from:
> https://dist.apache.org/repos/dist/dev/tomcat/tomcat-8/v8.5.63/
>
> The Maven staging repo is:
> https://repository.apache.org/content/repositories/orgapachetomcat-1298/
>
> The tag is:
> https://github.com/apache/tomcat/tree/8.5.63
> eb8d36c30857866536e8c931731c9f86980b00a6
>
> The proposed 8.5.63 release is:
> [ ] Broken - do not release
> [X] Stable - go ahead and release as 8.5.63
>

Rémy

>
> -
> To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org
> For additional commands, e-mail: dev-h...@tomcat.apache.org
>
>


Re: [VOTE] Release Apache Tomcat 9.0.43

2021-01-29 Thread Rémy Maucherat
On Thu, Jan 28, 2021 at 9:56 PM Mark Thomas  wrote:

> The proposed Apache Tomcat 9.0.43 release is now available for voting.
>
> The notable changes compared to the 9.0.41 release are:
>
> - Add support for using Unix domain sockets for NIO when running on
>   Java 16 or later.
>
> - Add a new StringInterpreter interface that allows applications to
>   provide customised string attribute value to type conversion within
>   JSPs. This allows applications to provide a conversion
>   implementation that is optimised for the application.
>
> - Add peerAddress to coyote request, which contains the IP address of
>   the direct connection peer. If a reverse proxy sits in front of
>   Tomcat and the protocol used is AJP or HTTP in combination with the
>   RemoteIp(Valve|Filter), the peer address might differ from the
>   remoteAddress. The latter then contains the address of the client in
>   front of the reverse proxy, not the address of the proxy itself.
>
> Along with lots of other bug fixes and improvements.
>
> For full details, see the changelog:
> https://ci.apache.org/projects/tomcat/tomcat9/docs/changelog.html
>
> It can be obtained from:
> https://dist.apache.org/repos/dist/dev/tomcat/tomcat-9/v9.0.43/
> The Maven staging repo is:
> https://repository.apache.org/content/repositories/orgapachetomcat-1297/
> The tag is:
> https://github.com/apache/tomcat/tree/9.0.43
> dc8bcd9c0704235319d322ca3d4c32263a054766
>
> The proposed 9.0.43 release is:
> [ ] Broken - do not release
> [X] Stable - go ahead and release as 9.0.43
>

Rémy


>
> -
> To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org
> For additional commands, e-mail: dev-h...@tomcat.apache.org
>
>


Re: [VOTE] Release Apache Tomcat 10.0.2

2021-01-29 Thread Rémy Maucherat
On Thu, Jan 28, 2021 at 8:13 PM Mark Thomas  wrote:

> The proposed Apache Tomcat 10.0.2 release is now available for
> voting.
>
> Apache Tomcat 10.x implements Jakarta EE 9 and, as such, the primary
> package for all the specification APIs has changed from javax.* to
> jakarta.*
> Applications that run on Tomcat 9 will not run on Tomcat 10 without
> changes.
>
> The notable changes compared to 10.0.0 are:
>
> - Add support for using Unix domain sockets for NIO when running on
>   Java 16 or later.
>
> - Add a new StringInterpreter interface that allows applications to
>   provide customised string attribute value to type conversion within
>   JSPs. This allows applications to provide a conversion
>   implementation that is optimised for the application.
>
> - Add peerAddress to coyote request, which contains the IP address of
>   the direct connection peer. If a reverse proxy sits in front of
>   Tomcat and the protocol used is AJP or HTTP in combination with the
>   RemoteIp(Valve|Filter), the peer address might differ from the
>   remoteAddress. The latter then contains the address of the client in
>   front of the reverse proxy, not the address of the proxy itself.
>
> Along with lots of other bug fixes and improvements.
>
>
> For full details, see the changelog:
> https://ci.apache.org/projects/tomcat/tomcat10/docs/changelog.html
>
> It can be obtained from:
> https://dist.apache.org/repos/dist/dev/tomcat/tomcat-10/v10.0.2/
> The Maven staging repo is:
> https://repository.apache.org/content/repositories/orgapachetomcat-1296
> The tag is:
> https://github.com/apache/tomcat/tree/10.0.2
> 228209117457e9b30d96f235c45efac9d4b8d9cb
>
> The proposed 10.0.2 release is:
> [ ] Broken - do not release
> [ ] Beta   - go ahead and release as 10.0.2 (beta)
> [X] Stable - go ahead and release as 10.0.2 (stable)
>

Rémy


>
> -
> To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org
> For additional commands, e-mail: dev-h...@tomcat.apache.org
>
>


Re: 8.5.x unit tests crashing

2021-01-29 Thread Rémy Maucherat
On Fri, Jan 29, 2021 at 10:08 AM Mark Thomas  wrote:

> On 29/01/2021 09:01, Rémy Maucherat wrote:
> > On Fri, Jan 29, 2021 at 9:12 AM Mark Thomas  wrote:
> >
> >> On 29/01/2021 08:06, Mark Thomas wrote:
> >>> Heads up. When I ran my pre-tag tests for 8.5.x I started to see a
> bunch
> >>> of JVM crashes on Windows with both Java 7 and Java 8. The crashes were
> >>> always in a compilation thread which was unusual.
> >>>
> >>> I am currently doing the binary search to figure out which commit
> >>> triggered the issue.
> >>
> >> That was quicker than I expected. Running the HTTP/2 tests on their own
> >> is enough to trigger the issue.
> >>
> >> It was the fix for bug 65111 - freeing the direct buffers - that
> >> triggered this.
> >>
> >
> > Almost certainly the swap with SocketBufferHandler.EMPTY saves 9 and 10
> > from crashing. It's not certain this is 100% safe actually, but possibly
> > good enough. I did reproduce a (rare) crash with 8.5, so no good option
> > other than take it out for now.
>
> I was thinking about moving the cleaning of the direct buffers to
> destroySocket()
>
> I'll do some tests.
>

Yes, ideally, waiting until after the socket is removed from the poller is
a good plan.

Rémy


Re: 8.5.x unit tests crashing

2021-01-29 Thread Rémy Maucherat
On Fri, Jan 29, 2021 at 9:12 AM Mark Thomas  wrote:

> On 29/01/2021 08:06, Mark Thomas wrote:
> > Heads up. When I ran my pre-tag tests for 8.5.x I started to see a bunch
> > of JVM crashes on Windows with both Java 7 and Java 8. The crashes were
> > always in a compilation thread which was unusual.
> >
> > I am currently doing the binary search to figure out which commit
> > triggered the issue.
>
> That was quicker than I expected. Running the HTTP/2 tests on their own
> is enough to trigger the issue.
>
> It was the fix for bug 65111 - freeing the direct buffers - that
> triggered this.
>

Almost certainly the swap with SocketBufferHandler.EMPTY saves 9 and 10
from crashing. It's not certain this is 100% safe actually, but possibly
good enough. I did reproduce a (rare) crash with 8.5, so no good option
other than take it out for now.

Rémy


>
> Mark
>
> -
> To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org
> For additional commands, e-mail: dev-h...@tomcat.apache.org
>
>


Re: 8.5.x unit tests crashing

2021-01-29 Thread Rémy Maucherat
On Fri, Jan 29, 2021 at 9:12 AM Mark Thomas  wrote:

> On 29/01/2021 08:06, Mark Thomas wrote:
> > Heads up. When I ran my pre-tag tests for 8.5.x I started to see a bunch
> > of JVM crashes on Windows with both Java 7 and Java 8. The crashes were
> > always in a compilation thread which was unusual.
> >
> > I am currently doing the binary search to figure out which commit
> > triggered the issue.
>
> That was quicker than I expected. Running the HTTP/2 tests on their own
> is enough to trigger the issue.
>
> It was the fix for bug 65111 - freeing the direct buffers - that
> triggered this.
>

I think it's a good idea to take this out then, a little possible leak is
better than a crash ... I haven't seen any crash on CI or on my test run
with 10, so I did backport.

Rémy


Re: [VOTE] Release Apache Tomcat 8.5.62

2021-01-28 Thread Rémy Maucherat
On Wed, Jan 27, 2021 at 8:25 PM Mark Thomas  wrote:

> The proposed Apache Tomcat 8.5.62 release is now available for voting.
>
> The notable changes compared to the 8.5.61 release are:
>
> - Add a new StringInterpreter interface that allows applications to
>   provide customised string attribute value to type conversion within
>   JSPs. This allows applications to provide a conversion
>   implementation that is optimised for the application.
>
> - Add peerAddress to coyote request, which contains the IP address of
>   the direct connection peer. If a reverse proxy sits in front of
>   Tomcat and the protocol used is AJP or HTTP in combination with the
>   RemoteIp(Valve|Filter), the peer address might differ from the
>   remoteAddress. The latter then contains the address of the client in
>   front of the reverse proxy, not the address of the proxy itself.
>
> - Escape elements in the access log that need to be escaped for the
>   access log to be parsed unambiguously.
>
> Along with lots of other bug fixes and improvements.
>
> For full details, see the changelog:
> https://ci.apache.org/projects/tomcat/tomcat85/docs/changelog.html
>
> It can be obtained from:
> https://dist.apache.org/repos/dist/dev/tomcat/tomcat-8/v8.5.62/
>
> The Maven staging repo is:
> https://repository.apache.org/content/repositories/orgapachetomcat-1294/
>
> The tag is:
> https://github.com/apache/tomcat/tree/8.5.62
> 0c41d44e32bc4479f0de02e6eb29bb703549a05c
>
> The proposed 8.5.62 release is:
> [ ] Broken - do not release
> [X] Stable - go ahead and release as 8.5.62
>

Rémy

>
> -
> To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org
> For additional commands, e-mail: dev-h...@tomcat.apache.org
>
>


Re: [VOTE] Release Apache Tomcat 9.0.42

2021-01-28 Thread Rémy Maucherat
On Wed, Jan 27, 2021 at 6:40 PM Mark Thomas  wrote:

> The proposed Apache Tomcat 9.0.42 release is now available for voting.
>
> The notable changes compared to the 9.0.41 release are:
>
> - Add support for using Unix domain sockets for NIO when running on
>   Java 16 or later.
>
> - Add a new StringInterpreter interface that allows applications to
>   provide customised string attribute value to type conversion within
>   JSPs. This allows applications to provide a conversion
>   implementation that is optimised for the application.
>
> - Add peerAddress to coyote request, which contains the IP address of
>   the direct connection peer. If a reverse proxy sits in front of
>   Tomcat and the protocol used is AJP or HTTP in combination with the
>   RemoteIp(Valve|Filter), the peer address might differ from the
>   remoteAddress. The latter then contains the address of the client in
>   front of the reverse proxy, not the address of the proxy itself.
>
> Along with lots of other bug fixes and improvements.
>
> For full details, see the changelog:
> https://ci.apache.org/projects/tomcat/tomcat9/docs/changelog.html
>
> It can be obtained from:
> https://dist.apache.org/repos/dist/dev/tomcat/tomcat-9/v9.0.42/
> The Maven staging repo is:
> https://repository.apache.org/content/repositories/orgapachetomcat-1293/
> The tag is:
> https://github.com/apache/tomcat/tree/9.0.42
> 868b50e7af1dd6c3489ba0fda86dfc1ff1b8c8cb
>
> The proposed 9.0.42 release is:
> [ ] Broken - do not release
> [X] Stable - go ahead and release as 9.0.42
>
> Rémy


Re: [VOTE] Release Apache Tomcat 10.0.1

2021-01-27 Thread Rémy Maucherat
On Wed, Jan 27, 2021 at 4:08 PM Mark Thomas  wrote:

> The proposed Apache Tomcat 10.0.1 release is now available for
> voting.
>
> Apache Tomcat 10.x implements Jakarta EE 9 and, as such, the primary
> package for all the specification APIs has changed from javax.* to
> jakarta.*
> Applications that run on Tomcat 9 will not run on Tomcat 10 without
> changes.
>
> The notable changes compared to 10.0.1 are:
>
> - Add support for using Unix domain sockets for NIO when running on
>   Java 16 or later.
>
> - Add a new StringInterpreter interface that allows applications to
>   provide customised string attribute value to type conversion within
>   JSPs. This allows applications to provide a conversion
>   implementation that is optimised for the application.
>
> - Add peerAddress to coyote request, which contains the IP address of
>   the direct connection peer. If a reverse proxy sits in front of
>   Tomcat and the protocol used is AJP or HTTP in combination with the
>   RemoteIp(Valve|Filter), the peer address might differ from the
>   remoteAddress. The latter then contains the address of the client in
>   front of the reverse proxy, not the address of the proxy itself.
>
> Along with lots of other bug fixes and improvements.
>
>
> For full details, see the changelog:
> https://ci.apache.org/projects/tomcat/tomcat10/docs/changelog.html
>
> It can be obtained from:
> https://dist.apache.org/repos/dist/dev/tomcat/tomcat-10/v10.0.1/
> The Maven staging repo is:
> https://repository.apache.org/content/repositories/orgapachetomcat-1292/
> The tag is:
> https://github.com/apache/tomcat/tree/10.0.1
> e4344b6bd67359e1690312674d83710a793f1d5b
>
> The proposed 10.0.1 release is:
> [ ] Broken - do not release
> [ ] Beta   - go ahead and release as 10.0.1 (beta)
> [X] Stable - go ahead and release as 10.0.1 (stable)
>
> No reason to not upgrade it to stable since it's looking good.

I am personally interested to know if users would be interested in an
integrated additional deployment step for legacy webapps using the
migration tool. The benefit is intuitive but this could be a great bad
idea: problems can occur, and it is often best to do this offline with a
simple tool instead of on a real server.

Rémy


Re: [tomcat] branch 9.0.x updated: Fix a SpotBugs warning - log an message when permission setting fails

2021-01-26 Thread Rémy Maucherat
On Tue, Jan 26, 2021 at 3:11 PM Mark Thomas  wrote:

> On 26/01/2021 14:06, Rémy Maucherat wrote:
> > On Tue, Jan 26, 2021 at 2:56 PM  wrote:
> >
> >> This is an automated email from the ASF dual-hosted git repository.
> >>
> >> markt pushed a commit to branch 9.0.x
> >> in repository https://gitbox.apache.org/repos/asf/tomcat.git
> >>
> >>
> >> The following commit(s) were added to refs/heads/9.0.x by this push:
> >>  new 9ebeda2  Fix a SpotBugs warning - log an message when
> permission
> >> setting fails
> >> 9ebeda2 is described below
> >>
> >> commit 9ebeda2df488803d61b371c0cc36db83fe04f197
> >> Author: Mark Thomas 
> >> AuthorDate: Tue Jan 26 13:54:52 2021 +
> >>
> >> Fix a SpotBugs warning - log an message when permission setting
> fails
> >>
> >
> > Actually, the "else" seems useless at the moment and should be dropped
> > altogether.
>
> I can do that as I am in the area. Worth logging if attrs is null?
>

The result cannot be null, it could only throw an exception. I didn't
translate the code for the PR very accurately, actually they wanted to set
the default permissions when getUnixDomainSocketPathPermissions() is null.
So we'll see later if that is really needed (I think the user could do it
since it could be sensitive).

Rémy


>
> Mark
>
> >
> > Rémy
> >
> >
> >> ---
> >>  java/org/apache/tomcat/util/net/LocalStrings.properties |  3 +++
> >>  java/org/apache/tomcat/util/net/NioEndpoint.java| 12
> +---
> >>  2 files changed, 12 insertions(+), 3 deletions(-)
> >>
> >> diff --git a/java/org/apache/tomcat/util/net/LocalStrings.properties
> >> b/java/org/apache/tomcat/util/net/LocalStrings.properties
> >> index bcf697c..257f4bf 100644
> >> --- a/java/org/apache/tomcat/util/net/LocalStrings.properties
> >> +++ b/java/org/apache/tomcat/util/net/LocalStrings.properties
> >> @@ -97,6 +97,9 @@ endpoint.nio.keyProcessingError=Error processing
> >> selection key
> >>  endpoint.nio.latchMustBeZero=Latch must be at count zero or null
> >>  endpoint.nio.nullLatch=Latch cannot be null
> >>  endpoint.nio.nullSocketChannel=Invalid null socket channel while
> >> processing poller event
> >> +endpoint.nio.perms.execFail=Failed to set execute permissions for Unix
> >> domain socket [{0}]
> >> +endpoint.nio.perms.readFail=Failed to set read permissions for Unix
> >> domain socket [{0}]
> >> +endpoint.nio.perms.writeFail=Failed to set write permissions for Unix
> >> domain socket [{0}]
> >>  endpoint.nio.pollerEventError=Error processing poller event
> >>  endpoint.nio.registerFail=Failed to register socket with selector from
> >> poller
> >>  endpoint.nio.selectorCloseFail=Failed to close selector when closing
> the
> >> poller
> >> diff --git a/java/org/apache/tomcat/util/net/NioEndpoint.java
> >> b/java/org/apache/tomcat/util/net/NioEndpoint.java
> >> index 4ccfd4f..86e377d 100644
> >> --- a/java/org/apache/tomcat/util/net/NioEndpoint.java
> >> +++ b/java/org/apache/tomcat/util/net/NioEndpoint.java
> >> @@ -272,9 +272,15 @@ public class NioEndpoint extends
> >> AbstractJsseEndpoint
> >>  Files.setAttribute(path, attrs.name(),
> >> attrs.value());
> >>  } else {
> >>  java.io.File file = path.toFile();
> >> -file.setReadable(true, false);
> >> -file.setWritable(true, false);
> >> -file.setExecutable(false, false);
> >> +if (!file.setReadable(true, false)) {
> >> +
> >> log.warn(sm.getString("endpoint.nio.perms.readFail", path));
> >> +}
> >> +if (!file.setWritable(true, false)) {
> >> +
> >> log.warn(sm.getString("endpoint.nio.perms.writeFail", path));
> >> +}
> >> +if (!file.setExecutable(false, false)) {
> >> +
> >> log.warn(sm.getString("endpoint.nio.perms.execFail", path));
> >> +}
> >>  }
> >>  }
> >>  } else {
> >>
> >>
> >> -
> >> To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org
> >> For additional commands, e-mail: dev-h...@tomcat.apache.org
> >>
> >>
> >
>
>
> -
> To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org
> For additional commands, e-mail: dev-h...@tomcat.apache.org
>
>


Re: [tomcat] branch 9.0.x updated: Fix a SpotBugs warning - log an message when permission setting fails

2021-01-26 Thread Rémy Maucherat
On Tue, Jan 26, 2021 at 2:56 PM  wrote:

> This is an automated email from the ASF dual-hosted git repository.
>
> markt pushed a commit to branch 9.0.x
> in repository https://gitbox.apache.org/repos/asf/tomcat.git
>
>
> The following commit(s) were added to refs/heads/9.0.x by this push:
>  new 9ebeda2  Fix a SpotBugs warning - log an message when permission
> setting fails
> 9ebeda2 is described below
>
> commit 9ebeda2df488803d61b371c0cc36db83fe04f197
> Author: Mark Thomas 
> AuthorDate: Tue Jan 26 13:54:52 2021 +
>
> Fix a SpotBugs warning - log an message when permission setting fails
>

Actually, the "else" seems useless at the moment and should be dropped
altogether.

Rémy


> ---
>  java/org/apache/tomcat/util/net/LocalStrings.properties |  3 +++
>  java/org/apache/tomcat/util/net/NioEndpoint.java| 12 +---
>  2 files changed, 12 insertions(+), 3 deletions(-)
>
> diff --git a/java/org/apache/tomcat/util/net/LocalStrings.properties
> b/java/org/apache/tomcat/util/net/LocalStrings.properties
> index bcf697c..257f4bf 100644
> --- a/java/org/apache/tomcat/util/net/LocalStrings.properties
> +++ b/java/org/apache/tomcat/util/net/LocalStrings.properties
> @@ -97,6 +97,9 @@ endpoint.nio.keyProcessingError=Error processing
> selection key
>  endpoint.nio.latchMustBeZero=Latch must be at count zero or null
>  endpoint.nio.nullLatch=Latch cannot be null
>  endpoint.nio.nullSocketChannel=Invalid null socket channel while
> processing poller event
> +endpoint.nio.perms.execFail=Failed to set execute permissions for Unix
> domain socket [{0}]
> +endpoint.nio.perms.readFail=Failed to set read permissions for Unix
> domain socket [{0}]
> +endpoint.nio.perms.writeFail=Failed to set write permissions for Unix
> domain socket [{0}]
>  endpoint.nio.pollerEventError=Error processing poller event
>  endpoint.nio.registerFail=Failed to register socket with selector from
> poller
>  endpoint.nio.selectorCloseFail=Failed to close selector when closing the
> poller
> diff --git a/java/org/apache/tomcat/util/net/NioEndpoint.java
> b/java/org/apache/tomcat/util/net/NioEndpoint.java
> index 4ccfd4f..86e377d 100644
> --- a/java/org/apache/tomcat/util/net/NioEndpoint.java
> +++ b/java/org/apache/tomcat/util/net/NioEndpoint.java
> @@ -272,9 +272,15 @@ public class NioEndpoint extends
> AbstractJsseEndpoint
>  Files.setAttribute(path, attrs.name(),
> attrs.value());
>  } else {
>  java.io.File file = path.toFile();
> -file.setReadable(true, false);
> -file.setWritable(true, false);
> -file.setExecutable(false, false);
> +if (!file.setReadable(true, false)) {
> +
> log.warn(sm.getString("endpoint.nio.perms.readFail", path));
> +}
> +if (!file.setWritable(true, false)) {
> +
> log.warn(sm.getString("endpoint.nio.perms.writeFail", path));
> +}
> +if (!file.setExecutable(false, false)) {
> +
> log.warn(sm.getString("endpoint.nio.perms.execFail", path));
> +}
>  }
>  }
>  } else {
>
>
> -
> To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org
> For additional commands, e-mail: dev-h...@tomcat.apache.org
>
>


Re: [tomcat] branch master updated: Add UDS test case

2021-01-22 Thread Rémy Maucherat
On Fri, Jan 22, 2021 at 9:31 AM Martin Grigorov 
wrote:

> On Mon, Jan 18, 2021 at 1:07 PM  wrote:
>
> > This is an automated email from the ASF dual-hosted git repository.
> >
> > remm pushed a commit to branch master
> > in repository https://gitbox.apache.org/repos/asf/tomcat.git
> >
> >
> > The following commit(s) were added to refs/heads/master by this push:
> >  new 16e3ef4  Add UDS test case
> >  new bda7cbb  Merge branch 'master' of g...@github.com:
> > apache/tomcat.git
> > 16e3ef4 is described below
> >
> > commit 16e3ef43f2fd3aa4ac9dcf14a8e985ef0754022c
> > Author: remm 
> > AuthorDate: Mon Jan 18 12:06:22 2021 +0100
> >
> > Add UDS test case
> >
> > And a Java 16 compat flag.
> > ---
> >  java/org/apache/tomcat/util/compat/JreCompat.java  |  9 +++
> >  .../apache/tomcat/util/net/TestXxxEndpoint.java| 29
> > ++
> >  2 files changed, 38 insertions(+)
> >
> > diff --git a/java/org/apache/tomcat/util/compat/JreCompat.java
> > b/java/org/apache/tomcat/util/compat/JreCompat.java
> > index b8b3b48..d58c1a0 100644
> > --- a/java/org/apache/tomcat/util/compat/JreCompat.java
> > +++ b/java/org/apache/tomcat/util/compat/JreCompat.java
> > @@ -45,6 +45,7 @@ public class JreCompat {
> >
> >  private static final JreCompat instance;
> >  private static final boolean graalAvailable;
> > +private static final boolean jre16Available;
> >  private static final boolean jre11Available;
> >  private static final boolean jre9Available;
> >  private static final StringManager sm =
> > StringManager.getManager(JreCompat.class);
> > @@ -69,12 +70,15 @@ public class JreCompat {
> >  if (Jre16Compat.isSupported()) {
> >  instance = new Jre16Compat();
> >  jre9Available = true;
> > +jre16Available = true;
> >  } else if (Jre9Compat.isSupported()) {
> >  instance = new Jre9Compat();
> >  jre9Available = true;
> > +jre16Available = false;
> >  } else {
> >  instance = new JreCompat();
> >  jre9Available = false;
> > +jre16Available = false;
> >  }
> >  jre11Available = instance.jarFileRuntimeMajorVersion() >= 11;
> >
> > @@ -116,6 +120,11 @@ public class JreCompat {
> >  }
> >
> >
> > +public static boolean isJre16Available() {
> > +return jre16Available;
> > +}
> > +
> > +
> >  // Java 8 implementation of Java 9 methods
> >
> >  /**
> > diff --git a/test/org/apache/tomcat/util/net/TestXxxEndpoint.java
> > b/test/org/apache/tomcat/util/net/TestXxxEndpoint.java
> > index 2db184f..12a4015 100644
> > --- a/test/org/apache/tomcat/util/net/TestXxxEndpoint.java
> > +++ b/test/org/apache/tomcat/util/net/TestXxxEndpoint.java
> > @@ -19,8 +19,12 @@ package org.apache.tomcat.util.net;
> >  import java.io.File;
> >  import java.net.InetAddress;
> >  import java.net.ServerSocket;
> > +import java.net.SocketAddress;
> > +import java.nio.ByteBuffer;
> > +import java.nio.channels.SocketChannel;
> >
> >  import org.junit.Assert;
> > +import org.junit.Assume;
> >  import org.junit.Test;
> >
> >  import org.apache.catalina.connector.Connector;
> > @@ -29,6 +33,7 @@ import org.apache.catalina.startup.TomcatBaseTest;
> >  import org.apache.tomcat.jni.Error;
> >  import org.apache.tomcat.jni.Library;
> >  import org.apache.tomcat.jni.Pool;
> > +import org.apache.tomcat.util.compat.JreCompat;
> >
> >  /**
> >   * Test case for the Endpoint implementations. The testing framework
> will
> > ensure
> > @@ -206,4 +211,28 @@ public class TestXxxEndpoint extends TomcatBaseTest
> {
> >  Assert.assertNull(e);
> >  tomcat.getConnector().start();
> >  }
> > +
> > +@Test
> > +public void testUnixDomainSocket() throws Exception {
> > +
> >
>
> maybe move Assume.assumeTrue("JDK 16 is
> required", JreCompat.isJre16Available()); here ? Before starting Tomcat
> instance
>
>
> > +Tomcat tomcat = getTomcatInstance();
> > +Connector c = tomcat.getConnector();
> > +Assume.assumeTrue("SSL renegotiation has to be supported for
> this
> > test",
> >
>
> Is this copy/paste error ?
> The SSL renegotiation message seems unrelated.
>
>
> > +c.getProtocolHandlerClassName().contains("Nio")
> > +&& JreCompat.isJre16Available());
> > +
> > +final String unixDomainSocketPath = "/tmp/testUnixDomainSocket";
> > +Assert.assertTrue(c.setProperty("unixDomainSocketPath",
> > unixDomainSocketPath));
> > +tomcat.start();
> > +
> > +SocketAddress sa =
> > JreCompat.getInstance().getUnixDomainSocketAddress(unixDomainSocketPath);
> > +ByteBuffer response = ByteBuffer.allocate(1024);
> > +try (SocketChannel socket =
> > JreCompat.getInstance().openUnixDomainSocketChannel()) {
> > +socket.connect(sa);
> > +socket.write(ByteBuffer.wrap("OPTIONS *
> > HTTP/1.0\r\n\r\n".getBytes()));
> >
>
> Is 

Re: "Missing" break in listenerStart?

2021-01-12 Thread Rémy Maucherat
On Tue, Jan 12, 2021 at 5:49 PM Romain Manni-Bucau 
wrote:

> Hi all,
>
> I suspect it is intended but if so I wonder if it needs some toggle to
> disable that behavior: is the fact to not break when a listener (start)
> fails intended? ([1])
>
> An ASF friend hit that with 2 listeners and second one was failling after
> first one failed because it was depending on it.
>
> Since this is not uncommon I wonder if it should get a break once ok=false
> (issue can be listenerStop which should probably be independent of start
> chain behavior since some listener only impl it) or if we should have a
> flag in StandardContext to stop at first start failure.
>
> Anything already thought on it?
>

For all other subcomponents of the context, the behavior is: set ok to
false, log the error and continue. It should stay that way. However, since
a ServletContextListener is a Servlet API component, then the Servlet
specification is supposed to resolve this one way or the other, but I don't
think it does. In a similar case the language is "log and fail to deploy".
As this is application related it could be reasonable to stop there.

Rémy


>
> [1]
>
> https://github.com/apache/tomcat/blob/master/java/org/apache/catalina/core/StandardContext.java#L4669
>
> Romain Manni-Bucau
> @rmannibucau  |  Blog
>  | Old Blog
>  | Github <
> https://github.com/rmannibucau> |
> LinkedIn  | Book
> <
> https://www.packtpub.com/application-development/java-ee-8-high-performance
> >
>


Re: channelSendOptions default may cause problems

2021-01-06 Thread Rémy Maucherat
On Wed, Jan 6, 2021 at 11:58 AM jean-frederic clere 
wrote:

> Hi,
>
> While testing the tomcat clustering I have noted that at the start from
> time to the attribute replication is failing.
> While debugging I have the messages:
> +++
> 05-Jan-2021 10:25:07.046 FINE [Tribes-Task-Receiver[Catalina-Channel]-6]
> org.apache.catalina.ha.session.DeltaManager.handleSESSION_DELTA Manager
> [localhost#/demo-1.0]: received session delta for unknown session
> [596AEB7A68DE2D8F6B9819D4F4F55CDA]
> 05-Jan-2021 10:25:07.045 FINE [Tribes-Task-Receiver[Catalina-Channel]-5]
> org.apache.catalina.ha.session.DeltaManager.messageReceived Manager
> [localhost#/demo-1.0]: Received SessionMessage of
> type=[SESSION-MODIFIED] from
> [org.apache.catalina.tribes.membership.MemberImpl[tcp://{10, 128, 2,
> 182}:4000,{10, 128, 2, 182},4000, alive=10037, securePort=-1, UDP
> Port=-1, id={-87 87 4 -12 89 -46 96 94 12 20 -103 -109 -56 120 -16 74 },
> payload={}, command={}, domain={}]]
> 05-Jan-2021 10:25:07.046 FINE [Tribes-Task-Receiver[Catalina-Channel]-5]
> org.apache.catalina.ha.session.DeltaManager.handleSESSION_CREATED
> Manager [localhost#/demo-1.0]: received session created message for
> session [596AEB7A68DE2D8F6B9819D4F4F55CDA]
>
> +++
> It looks like the delta is processed before the session creation and it
> is ignored.
>
> When using the channelSendOptions="6" I am NOT getting the "received
> session delta for unknown session" and the stuff is working perfectly.
>
> Should we change the default for channelSendOptions to 6? - the actual
> value is 8 -
>

The documentation for the values is here:
http://tomcat.apache.org/tomcat-10.0-doc/config/cluster.html#SimpleTcpCluster_Attributes

So the default is the fastest, but if it's not reliable for many reasonable
use cases it may not be a good idea.

Rémy


>
> --
> Cheers
>
> Jean-Frederic
>
> -
> To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org
> For additional commands, e-mail: dev-h...@tomcat.apache.org
>
>


Re: [tomcat-jakartaee-migration] branch master updated: Convert the javax.jms package with the EE profile (Fixes #6)

2020-12-29 Thread Rémy Maucherat
On Tue, Dec 29, 2020 at 11:03 PM  wrote:

> This is an automated email from the ASF dual-hosted git repository.
>
> ebourg pushed a commit to branch master
> in repository
> https://gitbox.apache.org/repos/asf/tomcat-jakartaee-migration.git
>
>
> The following commit(s) were added to refs/heads/master by this push:
>  new c094325  Convert the javax.jms package with the EE profile (Fixes
> #6)
> c094325 is described below
>
> commit c0943251b8fd3d7570d6c13cf461cd939aaf9ab0
> Author: Emmanuel Bourg 
> AuthorDate: Tue Dec 29 23:03:17 2020 +0100
>
> Convert the javax.jms package with the EE profile (Fixes #6)
> ---
>  src/main/java/org/apache/tomcat/jakartaee/EESpecProfile.java | 2 +-
>  src/test/java/org/apache/tomcat/jakartaee/EESpecProfileTest.java | 2 ++
>  2 files changed, 3 insertions(+), 1 deletion(-)
>
> diff --git a/src/main/java/org/apache/tomcat/jakartaee/EESpecProfile.java
> b/src/main/java/org/apache/tomcat/jakartaee/EESpecProfile.java
> index d4ae075..32781c1 100644
> --- a/src/main/java/org/apache/tomcat/jakartaee/EESpecProfile.java
> +++ b/src/main/java/org/apache/tomcat/jakartaee/EESpecProfile.java
> @@ -26,7 +26,7 @@ public enum EESpecProfile {
>
>
>  
> TOMCAT("javax([/\\.](annotation(?![/\\.]processing)|ejb|el|mail|persistence|security[/\\.]auth[/\\.]message|servlet|transaction(?![/\\.]xa)|websocket))"),
>
> -
> EE("javax([/\\.](activation|annotation(?![/\\.]processing)|decorator|ejb|el|enterprise|json|interceptor|inject|mail|persistence|"
> +
> EE("javax([/\\.](activation|annotation(?![/\\.]processing)|decorator|ejb|el|enterprise|jmx|json|interceptor|inject|mail|persistence|"
>  +
> "security[/\\.]auth[/\\.]message|servlet|transaction(?![/\\.]xa)|validation|websocket|ws[/\\.]rs|"
>  +
> "xml[/\\.](bind|namespace|rpc|soap|stream|ws|XMLConstants)))");
>
> diff --git
> a/src/test/java/org/apache/tomcat/jakartaee/EESpecProfileTest.java
> b/src/test/java/org/apache/tomcat/jakartaee/EESpecProfileTest.java
> index 0eba27f..bab8010 100644
> --- a/src/test/java/org/apache/tomcat/jakartaee/EESpecProfileTest.java
> +++ b/src/test/java/org/apache/tomcat/jakartaee/EESpecProfileTest.java
> @@ -41,6 +41,7 @@ public class EESpecProfileTest {
>  assertEquals("javax.activation",
> profile.convert("javax.activation"));
>  assertEquals("javax.decorator",
> profile.convert("javax.decorator"));
>  assertEquals("javax.enterprise",
> profile.convert("javax.enterprise"));
> +assertEquals("javax.jmx", profile.convert("javax.jmx"));
>

jmx -> jms typo ?

Rémy

 assertEquals("javax.json", profile.convert("javax.json"));
>  assertEquals("javax.interceptor",
> profile.convert("javax.interceptor"));
>  assertEquals("javax.inject", profile.convert("javax.inject"));
> @@ -72,6 +73,7 @@ public class EESpecProfileTest {
>  assertEquals("jakarta.ejb", profile.convert("javax.ejb"));
>  assertEquals("jakarta.el", profile.convert("javax.el"));
>  assertEquals("jakarta.enterprise",
> profile.convert("javax.enterprise"));
> +assertEquals("jakarta.jmx", profile.convert("javax.jmx"));
>  assertEquals("jakarta.json", profile.convert("javax.json"));
>  assertEquals("jakarta.interceptor",
> profile.convert("javax.interceptor"));
>  assertEquals("jakarta.inject", profile.convert("javax.inject"));
>
>
> -
> To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org
> For additional commands, e-mail: dev-h...@tomcat.apache.org
>
>


Re: Tomcat 10 startup time and JMX as an opt in fature?

2020-12-28 Thread Rémy Maucherat
On Sun, Dec 27, 2020 at 4:16 PM Romain Manni-Bucau 
wrote:

> Hi everyone,
>
> wonder if there is some work planned in tomcat 10 to 1. makes it start
> faster and 2. make jmx optional.
>
> I did some tests and locally a tomcat start is about 400ms.
> I was surprised to see that Registry.disableRegistry() was taking already
> 65ms whereas it should be almost nothing so refactored the code, added a
> RegistryFactory and really noop impl (not even a mock) and went down to
> 1ms, way better right? except the remaining 64ms went to the next line (new
> StandardHost()). After some investigation time is mainly classloading time
> (in all senses).
>
> So I wonder if it wouldnt make sense to optimize tomcat startup time +
> finally make JMX optional since it is no more used by users in most cases -
> gues we must keep it as an optional plugin since it is used "historically".
>
> The proposal would be to:
>
> 1. reduce classloading tree as much as possible for default case - here
> having jmx optional will help a lot
> 2. probably generate resource bundle as .java at build time to avoid all
> the classloading resource bundle implies (it is like 74 loadClass + as much
> getResource for a simple startup and all loadclass are misses)
> 2.bis. probably add a mode where StringManager uses the default bundle
> without passing through the resource bundle layer
> 3. cut dead code when we know it is inactive (typically the case for jmx)
> 4. open point: bypassing Context: in "main" mode (as the code shared
> after), you just want to bind some logic in a servlet/filter/valve, you
> don't always care about handling contexts (or a single one) so wonder if we
> shouldnt evaluate the fact to add the user logic in a valve for this kind
> of bench and maybe document this case (spring boot could inherit from it
> since it almost never use anything else that default spring servlet and
> binds everything in it).
>
> Here is the kind of case I'd like to see insanely fast (<100ms - and it is
> possible regarding what it does):
>
> public class Main {
> public static void main(String[] args) throws LifecycleException,
> InterruptedException {
> final long start = System.nanoTime();
> Registry.disableRegistry();
> final long startDisableRegistry = System.nanoTime();
>
> final Host host = new StandardHost();
> host.setName("localhost");
> final long startHost = System.nanoTime();
>
> final Engine engine = new StandardEngine();
> engine.setName("Tomcat");
> engine.setDefaultHost("localhost");
> engine.setRealm(new NullRealm());
> engine.addChild(host);
> final long startEngine = System.nanoTime();
>
> final var protocolHandler = new Http11Nio2Protocol();
> protocolHandler.setPort(8080);
> final long startHttp11Nio2Protocol = System.nanoTime();
>
> final Service service = new StandardService();
> service.setName("Tomcat");
> service.setContainer(engine);
> service.addConnector(new Connector(protocolHandler));
> final long startService = System.nanoTime();
>
> final StandardServer server = new StandardServer();
> server.setPort(-1);
> server.addService(service);
> final long startServer = System.nanoTime();
>
> final StandardContext context = new StandardContext();
> context.setUseNaming(false);
> context.setClearReferencesStopThreads(false);
> context.setClearReferencesStopTimerThreads(false);
> context.setClearReferencesHttpClientKeepAliveThread(false);
> context.setClearReferencesRmiTargets(false);
> context.setClearReferencesThreadLocals(false);
> context.setClearReferencesObjectStreamClassCaches(false);
> context.setWorkDir(System.getProperty("java.io.tmpdir"));
> context.setName("");
> context.setPath("");
> context.setDocBase(null);
> context.addLifecycleListener(event -> {
> if (event.getType().equals(Lifecycle.CONFIGURE_START_EVENT)) {
> context.setConfigured(true);
> if (context.getLoginConfig() == null) {
> context.setLoginConfig(new LoginConfig("NONE", null,
> null, null));
> context.getPipeline().addValve(new
> NonLoginAuthenticator());
> }
> }
> });
> context.addServletContainerInitializer((classes, ctx) -> {
> ctx.addServlet("default", new DefaultServlet())
> .addMapping("/");
> }, Set.of());
> final long startContext = System.nanoTime();
>
> host.addChild(context);
>
> server.init();
> final long inited = System.nanoTime();
> server.start();
> final long started = System.nanoTime();
> // new CountDownLatch(1).await();
>
> final var contextClassLoader =
> Thread.currentThread().getContextClassLoader();
>

Re: Compat versions

2020-12-18 Thread Rémy Maucherat
On Fri, Dec 18, 2020 at 2:56 PM Romain Manni-Bucau 
wrote:

> Hmm, a few thoughts on this topic:
>
> 1. there is no reason to use reflection in jrecompat (all the java > build
> time version) impl can be generated in ant build using asm or bcel
> 2. all JreCompat are kind of hardcoded SPI, if this API gets a "ordinal" -
> priority - and a "matches(int javaMajor)" then it becomes simple to pick
> only the highest impl matching current java version (ie on java 8 it will
> use the default, on java 12 it will use java12 or 11 or  8 impl, on
> java 17 it will use java 17 impl etc).
>
> So overall it is mainly about making the maintenance of the code easier but
> not about dropping any feature or reducing support list IMHO.
> Build time generation can help a lot with that (@Implement(spi =
> JreCompat.class, since = 12) which would lead to jrecompat12, jrecompat13,
> , jrecompatLast override of an impl for a method - being said playing
> with inheritence between versions enables to drop duplication).
> This kind of tool is not harder than the jakarta migration tool and can
> solve that "fast pace" issue IMHO.
> Requires some build/tool investment but it likely interesting to keep a
> high support level for end user and reduce maintenance cost for tomcat
> itself.
>
> Hope it makes sense.
>

This is more than what I expected to do.

But ok, I got the main feedback from you and Chris, the compatibility
should be kept per branch. I have a hard time seeing us give any support to
users running on Java 13 (just an example) though, but ok it doesn't need
to be made more difficult.

Rémy


>
> Romain Manni-Bucau
> @rmannibucau <https://twitter.com/rmannibucau> |  Blog
> <https://rmannibucau.metawerx.net/> | Old Blog
> <http://rmannibucau.wordpress.com> | Github <
> https://github.com/rmannibucau> |
> LinkedIn <https://www.linkedin.com/in/rmannibucau> | Book
> <
> https://www.packtpub.com/application-development/java-ee-8-high-performance
> >
>
>
> Le ven. 18 déc. 2020 à 14:33, Christopher Schultz <
> ch...@christopherschultz.net> a écrit :
>
> > Rémy,
> >
> > On 12/18/20 08:20, Rémy Maucherat wrote:
> > > On Fri, Dec 18, 2020 at 12:19 PM Martin Grigorov  >
> > > wrote:
> > >
> > >> On Fri, Dec 18, 2020 at 11:12 AM Rémy Maucherat 
> > wrote:
> > >>
> > >>> Hi,
> > >>>
> > >>> I'd like to refactor the compat classes to align with the LTS
> versions:
> > >>> - Move Jre9Compat to Jre11Compat
> > >>> - I'll probably refactor out GraalCompat
> > >>> - For the upcoming Java 12+ features, they will all go to a new
> > >> Jre17Compat
> > >>>
> > >>
> > >> s/Java 12+/Java 17+/, right ?
> > >>
> > >
> > > I mean: for any features that are only present in Java 12 to 17 that we
> > > would want to use in Tomcat, then they will all be implemented through
> > > reflection in a Jre17Compat class. Example, if UDS support is added, it
> > > will go into that class even though previously it would have been in
> > > Jre16Compat.
> > >
> > > Effectively, this drops guaranteed support for all Java non LTS
> releases
> > > except the most recent one, so you either have to use one of the LTS or
> > the
> > > most recent Java.
> >
> > I don't see a reason to restrict users in this way.
> >
> > If a feature is added in Java 9, why not put it in a Java9Compat class
> > and allow it to work on Java 9, 10, etc.? I realize that may add more
> > JavaXCompat classes but I really don't see a particular reason to
> > restrict or misrepresent Java versions.
> >
> > -chris
> >
> > -
> > To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org
> > For additional commands, e-mail: dev-h...@tomcat.apache.org
> >
> >
>


Re: Compat versions

2020-12-18 Thread Rémy Maucherat
On Fri, Dec 18, 2020 at 12:19 PM Martin Grigorov 
wrote:

> On Fri, Dec 18, 2020 at 11:12 AM Rémy Maucherat  wrote:
>
> > Hi,
> >
> > I'd like to refactor the compat classes to align with the LTS versions:
> > - Move Jre9Compat to Jre11Compat
> > - I'll probably refactor out GraalCompat
> > - For the upcoming Java 12+ features, they will all go to a new
> Jre17Compat
> >
>
> s/Java 12+/Java 17+/, right ?
>

I mean: for any features that are only present in Java 12 to 17 that we
would want to use in Tomcat, then they will all be implemented through
reflection in a Jre17Compat class. Example, if UDS support is added, it
will go into that class even though previously it would have been in
Jre16Compat.

Effectively, this drops guaranteed support for all Java non LTS releases
except the most recent one, so you either have to use one of the LTS or the
most recent Java.


>
>
> > class, which will actually be useable in the meantime by the latest Java
> > milestone release from which the interesting features come from
> >
> > Although not fully accurate, this is more maintainable as the interim non
> > LTS Java releases quickly come and go and are not good platforms for
> > Tomcat.
> >
>
> +1
>

Rémy

>
>
>
> >
> > Rémy
> >
>


Compat versions

2020-12-18 Thread Rémy Maucherat
Hi,

I'd like to refactor the compat classes to align with the LTS versions:
- Move Jre9Compat to Jre11Compat
- I'll probably refactor out GraalCompat
- For the upcoming Java 12+ features, they will all go to a new Jre17Compat
class, which will actually be useable in the meantime by the latest Java
milestone release from which the interesting features come from

Although not fully accurate, this is more maintainable as the interim non
LTS Java releases quickly come and go and are not good platforms for Tomcat.

Rémy


Re: Objection to the deprecation of the tomcat-native/APR connector

2020-12-17 Thread Rémy Maucherat
On Thu, Dec 17, 2020 at 12:12 PM Mladen Adamović 
wrote:

> But what I have discovered from migrating from APR to Nio2 that our
> processor usage went down. We never tried Nio2, I have setup APR back in
> 2014 as I've read it has a better performance.
>

I would still say the APR connector is faster, just like the java.io
connector was the fastest overall. But it can depend on what you are doing,
maybe if your use case uses the poller more, then it could be significantly
less efficient. The main problems are that it is crash prone (and it's
expensive and complex to make it safe), and it doesn't have feature parity
with NIO.

About NIO2, Oracle has started updating and fixing NIO again. NIO2 is now
lagging a bit in features (no inherited channel, no UDP - NIO just got
fully rewritten support -, and now no domain socket support). The IO API
war is not over yet though.

Rémy


>
> So I don't see a reason why APR should stay as users can easily migrate to
> NIO2...
>
>
>
> On Thu, Dec 17, 2020 at 11:41 AM Michael Osipov 
> wrote:
>
> > Am 2020-12-16 um 16:38 schrieb Emmanuel Bourg:
> > > Le 11/12/2020 à 17:56, Michael Osipov a écrit :
> > >
> > >> Well, isn't that really a OS vendor problem not ours? We can provide
> > >> grace periods, but that's pretty much it. They need to solve that, not
> > >> us on a volunteer basis.
> > >
> > > Note that (most) Debian Developers are volunteers too.
> >
> > This makes siauation even worse to sit on old version and continue back
> > porting for downstream.
> >
> > >> FreeBSD's port of libtcnative is uptodate. I provide patches on
> regular
> > >> basis. Vendors like Debian or Red Hat lag years behind.
> > >
> > > I don't know the state in Red Hat, but in Debian tomcat-native is
> > > up-to-date [1]. For the stable release there are backports with more
> > > recent versions available.
> >
> > Thanks for the info, wasn't aware of that. Guess it takes the maintainer
> > do that otherwise it will stick to very old versions.
> >
> > I am horribly surprised for RHEL 7:
> > > $ yum info tomcat-native 2>&- | grep Version
> > > Version: 1.2.23
> >
> > in contrast:
> > > $ yum info curl 2>&- | grep Version
> > > Version: 7.29.0
> >
> > This is unusable.
> >
> >
> > -
> > To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org
> > For additional commands, e-mail: dev-h...@tomcat.apache.org
> >
> >
>


Re: JDK 16 is in Rampdown Phase One

2020-12-15 Thread Rémy Maucherat
On Tue, Dec 15, 2020 at 1:52 PM Martin Grigorov 
wrote:

> Hi Rémy,
>
> On Mon, Dec 14, 2020 at 10:51 PM Mark Thomas  wrote:
>
> > On 14/12/2020 11:06, Martin Grigorov wrote:
> > > On Mon, Dec 14, 2020 at 12:30 PM Rémy Maucherat 
> wrote:
> > >
> > >> On Mon, Dec 14, 2020 at 8:53 AM Martin Grigorov  >
> > >> wrote:
> > >>
> > >>> Hi Tomcat team,
> > >>>
> > >>> The following tests fail on JDK 16 b28:
> > >>>
> > >>
> > >> Ok, so I installed JDK 16 and tested. No issues overall, nice !
> > >>
> > >> About this particular one however, the proper exceptions are now
> caught
> > and
> > >> everything (so it's "ok") but it's not possible to make the
> > functionality
> > >> work anymore by default. So the executor and its threads are still
> > hanging,
> > >> causing the assertions to fail [as well as the traces when stopping
> the
> > >> blocked threads]. Should we relax module security somehow to allow the
> > >> fields setAccessible(true) to work again ? [that doesn't sound like a
> > great
> > >> plan to me ...]
> > >>
> > >
> > > One can work it around by adding --add-opens=java.base/java.
> > > <http://java.io/>util.concurrent=ALL-UNNAMED to the JVM arguments
> > > I am not sure whether some module-info.java hackery can make it better.
> >
> > This is what we currently do:
> >
> > https://github.com/apache/tomcat/blob/master/bin/catalina.sh#L313
> >
> > (and the same for .bat). Looks like we need to add another entry there.
> >
>
> Your fix
> <
> https://github.com/apache/tomcat/commit/f42f1899eda28244218bf4d29602bc99574d4486
> >
> does not solve the issue for me. Do the tests pass for you with JDK 16 b28
> ?
>
> diff --git build.xml build.xml
> index 2c2b532..2c1a54f 100644
> --- build.xml
> +++ build.xml
> @@ -220,7 +220,7 @@
>
>   property="java9.test.option.4"
> - value="--add-opens=java.base/java.util=ALL-UNNAMED"/>
> +
> value="--add-opens=java.base/java.util.concurrent=ALL-UNNAMED"/>
>
>
> ^^ this fixes them!
>

Bad luck, one needs java.uti, the other one  java.util.concurrent.
OTOH, I have no idea why anyone cares ...

Rémy


  1   2   3   4   5   6   7   8   9   10   >