Re: Preparing to migrate to Tomcat 10.1 / Jakarta EE

2022-12-07 Thread Rémy Maucherat
On Tue, Dec 6, 2022 at 11:50 PM Christopher Schultz
 wrote:
>
> Rémy,
>
> On 12/6/22 16:42, Rémy Maucherat wrote:
> > On Tue, Dec 6, 2022 at 6:48 PM Konstantin Kolinko
> >  wrote:
> >>
> >> пн, 5 дек. 2022 г. в 23:32, Christopher Schultz 
> >> :
> >>>
> >>> All,
> >>>
> >>> I've started looking at the migration path at $work for Jakarta EE and
> >>> while Tomcat 10 has some fantastic features to help with that (hot
> >>> migration is killer IMO), there are still some potential barriers I'm
> >>> seeing to "slowly" migrating $work applications from Java EE -> Jakarta 
> >>> EE.
> >>>
> >>> When I say "slowly", I mean "not having to write one or more large
> >>> revision-control changes to lock-step-upgrade from e.g. Tomcat 8/9 to
> >>> Tomcat 10.1."
> >>>
> >>> One specific example is that we use Apache Velocity as our
> >>> page-generation language and AIUI, these templates will not have their
> >>> text scanned and replaced with updated strings. For example:
> >>>
> >>> #if($request.getAttribute('javax.servlet.error.request_uri'))
> >>> ...
> >>> #end
> >>>
> >>> [... skipped]
> >>>
> >>> I can think of a few ways to facilitate a dual-deployment (Tomcat 8/9 vs
> >>> 10 aka Java EE vs Jakarta EE) application, at least for the specific
> >>> example I present above:
> >>>
> >>> [... skipped]
> >>
> >> 6. I think that Migration Tool can be updated to change string
> >> constants such as the above one.
>
> Does this happen at the "source" level? For example, if I have .jsp
> files does it search-and-replace .jsp source, or only the resulting
> .class files.

Yes, you can see here the extensions that are processed as text:
https://github.com/apache/tomcat-jakartaee-migration/blob/main/src/main/java/org/apache/tomcat/jakartaee/TextConverter.java#L40
Once converted they should be processed properly. For example a JSP
source that is rewritten will use jakarta everywhere and it should be
fine. The examples webapp works (JSP, taglibs, websocket, etc) ;)

> Does this work with the on-the-fly migration, or only with the "offline"
> tool?

The deployment time conversion is equivalent to running the tool with
the default configuration, so there should be no big surprises.
It's also possible to use classloading time conversion but it's more
limited (in particular: annotations are not supported, as well as spec
API classes relocation - like if your webapp includes some javax. APIs
that should be converted). The only real benefit is that it wouldn't
break proprietary software licenses.

> I'm happy to simply add *.vm to the list of things we process. That
> would actually be a good and simple fix for both me and potentially
> others looking to upgrade.

.vm is clearly not part of the list at the moment. I think it should be added.

> > It should do that here:
> > https://github.com/apache/tomcat-jakartaee-migration/blob/main/src/main/java/org/apache/tomcat/jakartaee/TextConverter.java
> > You can see the extensions that are processed in the source.
> >
> > However, there are some limitations, like the string has to appear as
> > a whole in the file, and so on. Although not strictly compliant, it
> > could be worthwhile to consider attributes names starting with
> > "javax." as if they started with "jakarta.".
>
> This limitation isn't a problem for me: we always use "full strings" for
> those kinds of things.

Ok. I'm sure there is someone out there doing PREFIX + something though.

Rémy

>
> Thanks,
> -chris
>
> -
> To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org
> For additional commands, e-mail: dev-h...@tomcat.apache.org
>

-
To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org
For additional commands, e-mail: dev-h...@tomcat.apache.org



Re: Preparing to migrate to Tomcat 10.1 / Jakarta EE

2022-12-06 Thread Christopher Schultz

Rémy,

On 12/6/22 16:42, Rémy Maucherat wrote:

On Tue, Dec 6, 2022 at 6:48 PM Konstantin Kolinko
 wrote:


пн, 5 дек. 2022 г. в 23:32, Christopher Schultz :


All,

I've started looking at the migration path at $work for Jakarta EE and
while Tomcat 10 has some fantastic features to help with that (hot
migration is killer IMO), there are still some potential barriers I'm
seeing to "slowly" migrating $work applications from Java EE -> Jakarta EE.

When I say "slowly", I mean "not having to write one or more large
revision-control changes to lock-step-upgrade from e.g. Tomcat 8/9 to
Tomcat 10.1."

One specific example is that we use Apache Velocity as our
page-generation language and AIUI, these templates will not have their
text scanned and replaced with updated strings. For example:

#if($request.getAttribute('javax.servlet.error.request_uri'))
...
#end

[... skipped]

I can think of a few ways to facilitate a dual-deployment (Tomcat 8/9 vs
10 aka Java EE vs Jakarta EE) application, at least for the specific
example I present above:

[... skipped]


6. I think that Migration Tool can be updated to change string
constants such as the above one.


Does this happen at the "source" level? For example, if I have .jsp 
files does it search-and-replace .jsp source, or only the resulting 
.class files.


Does this work with the on-the-fly migration, or only with the "offline" 
tool?


I'm happy to simply add *.vm to the list of things we process. That 
would actually be a good and simple fix for both me and potentially 
others looking to upgrade.



It should do that here:
https://github.com/apache/tomcat-jakartaee-migration/blob/main/src/main/java/org/apache/tomcat/jakartaee/TextConverter.java
You can see the extensions that are processed in the source.

However, there are some limitations, like the string has to appear as
a whole in the file, and so on. Although not strictly compliant, it
could be worthwhile to consider attributes names starting with
"javax." as if they started with "jakarta.".


This limitation isn't a problem for me: we always use "full strings" for 
those kinds of things.


Thanks,
-chris

-
To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org
For additional commands, e-mail: dev-h...@tomcat.apache.org



Re: Preparing to migrate to Tomcat 10.1 / Jakarta EE

2022-12-06 Thread Rémy Maucherat
On Tue, Dec 6, 2022 at 6:48 PM Konstantin Kolinko
 wrote:
>
> пн, 5 дек. 2022 г. в 23:32, Christopher Schultz 
> :
> >
> > All,
> >
> > I've started looking at the migration path at $work for Jakarta EE and
> > while Tomcat 10 has some fantastic features to help with that (hot
> > migration is killer IMO), there are still some potential barriers I'm
> > seeing to "slowly" migrating $work applications from Java EE -> Jakarta EE.
> >
> > When I say "slowly", I mean "not having to write one or more large
> > revision-control changes to lock-step-upgrade from e.g. Tomcat 8/9 to
> > Tomcat 10.1."
> >
> > One specific example is that we use Apache Velocity as our
> > page-generation language and AIUI, these templates will not have their
> > text scanned and replaced with updated strings. For example:
> >
> > #if($request.getAttribute('javax.servlet.error.request_uri'))
> > ...
> > #end
> >
> > [... skipped]
> >
> > I can think of a few ways to facilitate a dual-deployment (Tomcat 8/9 vs
> > 10 aka Java EE vs Jakarta EE) application, at least for the specific
> > example I present above:
> >
> > [... skipped]
>
> 6. I think that Migration Tool can be updated to change string
> constants such as the above one.

It should do that here:
https://github.com/apache/tomcat-jakartaee-migration/blob/main/src/main/java/org/apache/tomcat/jakartaee/TextConverter.java
You can see the extensions that are processed in the source.

However, there are some limitations, like the string has to appear as
a whole in the file, and so on. Although not strictly compliant, it
could be worthwhile to consider attributes names starting with
"javax." as if they started with "jakarta.".

Rémy


> There is a limited set of those names, and I expect them to always
> appear as a whole string among string constants.
>
> Going into java/javax/servlet of Tomcat 9 and running a search for `"javax.`,
> except '.LocalStrings"' l10n resources names:
>
> grep --line-number --recursive '"javax.' | grep  --invert-match
> '.LocalStrings"' - >../found.txt
>
> I get the following 36 lines:
>
> [[[
> AsyncContext.java:30:"javax.servlet.async.request_uri";
> AsyncContext.java:36:"javax.servlet.async.context_path";
> AsyncContext.java:42:"javax.servlet.async.mapping";
> AsyncContext.java:48:"javax.servlet.async.path_info";
> AsyncContext.java:54:"javax.servlet.async.servlet_path";
> AsyncContext.java:60:"javax.servlet.async.query_string";
> jsp/PageContext.java:139:public static final String PAGE =
> "javax.servlet.jsp.jspPage";
> jsp/PageContext.java:145:public static final String PAGECONTEXT =
> "javax.servlet.jsp.jspPageContext";
> jsp/PageContext.java:151:public static final String REQUEST =
> "javax.servlet.jsp.jspRequest";
> jsp/PageContext.java:157:public static final String RESPONSE =
> "javax.servlet.jsp.jspResponse";
> jsp/PageContext.java:163:public static final String CONFIG =
> "javax.servlet.jsp.jspConfig";
> jsp/PageContext.java:169:public static final String SESSION =
> "javax.servlet.jsp.jspSession";
> jsp/PageContext.java:174:public static final String OUT =
> "javax.servlet.jsp.jspOut";
> jsp/PageContext.java:180:public static final String APPLICATION =
> "javax.servlet.jsp.jspApplication";
> jsp/PageContext.java:187:public static final String EXCEPTION =
> "javax.servlet.jsp.jspException";
> RequestDispatcher.java:47:static final String FORWARD_REQUEST_URI
> = "javax.servlet.forward.request_uri";
> RequestDispatcher.java:58:static final String FORWARD_CONTEXT_PATH
> = "javax.servlet.forward.context_path";
> RequestDispatcher.java:69:static final String FORWARD_MAPPING =
> "javax.servlet.forward.mapping";
> RequestDispatcher.java:80:static final String FORWARD_PATH_INFO =
> "javax.servlet.forward.path_info";
> RequestDispatcher.java:91:static final String FORWARD_SERVLET_PATH
> = "javax.servlet.forward.servlet_path";
> RequestDispatcher.java:102:static final String
> FORWARD_QUERY_STRING = "javax.servlet.forward.query_string";
> RequestDispatcher.java:114:static final String INCLUDE_REQUEST_URI
> = "javax.servlet.include.request_uri";
> RequestDispatcher.java:126:static final String
> INCLUDE_CONTEXT_PATH = "javax.servlet.include.context_path";
> RequestDispatcher.java:138:static final String INCLUDE_PATH_INFO =
> "javax.servlet.include.path_info";
> RequestDispatcher.java:150:static final String INCLUDE_MAPPING =
> "javax.servlet.include.mapping";
> RequestDispatcher.java:162:static final String
> INCLUDE_SERVLET_PATH = "javax.servlet.include.servlet_path";
> RequestDispatcher.java:174:static final String
> INCLUDE_QUERY_STRING = "javax.servlet.include.query_string";
> RequestDispatcher.java:184:public static final String
> ERROR_EXCEPTION = "javax.servlet.error.exception";
> RequestDispatcher.java:194:public static final String
> ERROR_EXCEPTION_TYPE = 

Re: Preparing to migrate to Tomcat 10.1 / Jakarta EE

2022-12-06 Thread Konstantin Kolinko
пн, 5 дек. 2022 г. в 23:32, Christopher Schultz :
>
> All,
>
> I've started looking at the migration path at $work for Jakarta EE and
> while Tomcat 10 has some fantastic features to help with that (hot
> migration is killer IMO), there are still some potential barriers I'm
> seeing to "slowly" migrating $work applications from Java EE -> Jakarta EE.
>
> When I say "slowly", I mean "not having to write one or more large
> revision-control changes to lock-step-upgrade from e.g. Tomcat 8/9 to
> Tomcat 10.1."
>
> One specific example is that we use Apache Velocity as our
> page-generation language and AIUI, these templates will not have their
> text scanned and replaced with updated strings. For example:
>
> #if($request.getAttribute('javax.servlet.error.request_uri'))
> ...
> #end
>
> [... skipped]
>
> I can think of a few ways to facilitate a dual-deployment (Tomcat 8/9 vs
> 10 aka Java EE vs Jakarta EE) application, at least for the specific
> example I present above:
>
> [... skipped]

6. I think that Migration Tool can be updated to change string
constants such as the above one.

There is a limited set of those names, and I expect them to always
appear as a whole string among string constants.

Going into java/javax/servlet of Tomcat 9 and running a search for `"javax.`,
except '.LocalStrings"' l10n resources names:

grep --line-number --recursive '"javax.' | grep  --invert-match
'.LocalStrings"' - >../found.txt

I get the following 36 lines:

[[[
AsyncContext.java:30:"javax.servlet.async.request_uri";
AsyncContext.java:36:"javax.servlet.async.context_path";
AsyncContext.java:42:"javax.servlet.async.mapping";
AsyncContext.java:48:"javax.servlet.async.path_info";
AsyncContext.java:54:"javax.servlet.async.servlet_path";
AsyncContext.java:60:"javax.servlet.async.query_string";
jsp/PageContext.java:139:public static final String PAGE =
"javax.servlet.jsp.jspPage";
jsp/PageContext.java:145:public static final String PAGECONTEXT =
"javax.servlet.jsp.jspPageContext";
jsp/PageContext.java:151:public static final String REQUEST =
"javax.servlet.jsp.jspRequest";
jsp/PageContext.java:157:public static final String RESPONSE =
"javax.servlet.jsp.jspResponse";
jsp/PageContext.java:163:public static final String CONFIG =
"javax.servlet.jsp.jspConfig";
jsp/PageContext.java:169:public static final String SESSION =
"javax.servlet.jsp.jspSession";
jsp/PageContext.java:174:public static final String OUT =
"javax.servlet.jsp.jspOut";
jsp/PageContext.java:180:public static final String APPLICATION =
"javax.servlet.jsp.jspApplication";
jsp/PageContext.java:187:public static final String EXCEPTION =
"javax.servlet.jsp.jspException";
RequestDispatcher.java:47:static final String FORWARD_REQUEST_URI
= "javax.servlet.forward.request_uri";
RequestDispatcher.java:58:static final String FORWARD_CONTEXT_PATH
= "javax.servlet.forward.context_path";
RequestDispatcher.java:69:static final String FORWARD_MAPPING =
"javax.servlet.forward.mapping";
RequestDispatcher.java:80:static final String FORWARD_PATH_INFO =
"javax.servlet.forward.path_info";
RequestDispatcher.java:91:static final String FORWARD_SERVLET_PATH
= "javax.servlet.forward.servlet_path";
RequestDispatcher.java:102:static final String
FORWARD_QUERY_STRING = "javax.servlet.forward.query_string";
RequestDispatcher.java:114:static final String INCLUDE_REQUEST_URI
= "javax.servlet.include.request_uri";
RequestDispatcher.java:126:static final String
INCLUDE_CONTEXT_PATH = "javax.servlet.include.context_path";
RequestDispatcher.java:138:static final String INCLUDE_PATH_INFO =
"javax.servlet.include.path_info";
RequestDispatcher.java:150:static final String INCLUDE_MAPPING =
"javax.servlet.include.mapping";
RequestDispatcher.java:162:static final String
INCLUDE_SERVLET_PATH = "javax.servlet.include.servlet_path";
RequestDispatcher.java:174:static final String
INCLUDE_QUERY_STRING = "javax.servlet.include.query_string";
RequestDispatcher.java:184:public static final String
ERROR_EXCEPTION = "javax.servlet.error.exception";
RequestDispatcher.java:194:public static final String
ERROR_EXCEPTION_TYPE = "javax.servlet.error.exception_type";
RequestDispatcher.java:204:public static final String
ERROR_MESSAGE = "javax.servlet.error.message";
RequestDispatcher.java:214:public static final String
ERROR_REQUEST_URI = "javax.servlet.error.request_uri";
RequestDispatcher.java:224:public static final String
ERROR_SERVLET_NAME = "javax.servlet.error.servlet_name";
RequestDispatcher.java:234:public static final String
ERROR_STATUS_CODE = "javax.servlet.error.status_code";
ServletContext.java:58:public static final String TEMPDIR =
"javax.servlet.context.tempdir";
ServletContext.java:66:public static final String ORDERED_LIBS =
"javax.servlet.context.orderedLibs";
]]]

7. If you look into implementation of

Re: Preparing to migrate to Tomcat 10.1 / Jakarta EE

2022-12-06 Thread Christopher Schultz

Mark,

On 12/6/22 03:28, Mark Thomas wrote:

On 05/12/2022 20:32, Christopher Schultz wrote:



4. Patch the server to provide #3 without having to include a Filter 
in the application deployment.


5. Extend the migration tool so you can add file extension / file type 
mappings in addition to the built-in ones. Or maybe even override the 
built-in ones. This is a lot more work than #4.



I'm kind of interested in idea #4, and wanted to ask for feedback.


I think #4 could be a nest solution applicable to multiple users.

If placed at the end of the current getAttribute() code it would have a 
minimal performance impact.


We should log (debug level?, separate logger like 
NioEndpoint.handshake?) when this happens.


I can't be the only one in this situation where the Tomcat Migration 
Tool is great but doesn't cover some use-cases and therefore upgrades 
are risky and/or problematic.


I'm sure. I'm expecting a bunch of issues to start showing up as folks 
start migrating.


I think we as a community should be encouraging users to start 
upgrading to get over the "Jakarta Hurdle" and I think that this could 
be another small tool to help motivate that migration.


WDYT?


+1.


Cool. Based upon that feedback, I'll start working on a Filter as a PoC. 
It can easily be converted to a Valve or just added to Request.getAttribute.


Time for a good re-reading of the servlet spec documentation ;)

-chris

-
To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org
For additional commands, e-mail: dev-h...@tomcat.apache.org



Re: Preparing to migrate to Tomcat 10.1 / Jakarta EE

2022-12-06 Thread Mark Thomas

On 05/12/2022 20:32, Christopher Schultz wrote:



4. Patch the server to provide #3 without having to include a Filter in 
the application deployment.


5. Extend the migration tool so you can add file extension / file type 
mappings in addition to the built-in ones. Or maybe even override the 
built-in ones. This is a lot more work than #4.



I'm kind of interested in idea #4, and wanted to ask for feedback.


I think #4 could be a nest solution applicable to multiple users.

If placed at the end of the current getAttribute() code it would have a 
minimal performance impact.


We should log (debug level?, separate logger like 
NioEndpoint.handshake?) when this happens.


I can't be the only one in this situation where the Tomcat Migration 
Tool is great but doesn't cover some use-cases and therefore upgrades 
are risky and/or problematic.


I'm sure. I'm expecting a bunch of issues to start showing up as folks 
start migrating.


I think we as a community should be encouraging users to start upgrading 
to get over the "Jakarta Hurdle" and I think that this could be another 
small tool to help motivate that migration.


WDYT?


+1.

Mark

-
To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org
For additional commands, e-mail: dev-h...@tomcat.apache.org



Preparing to migrate to Tomcat 10.1 / Jakarta EE

2022-12-05 Thread Christopher Schultz

All,

I've started looking at the migration path at $work for Jakarta EE and 
while Tomcat 10 has some fantastic features to help with that (hot 
migration is killer IMO), there are still some potential barriers I'm 
seeing to "slowly" migrating $work applications from Java EE -> Jakarta EE.


When I say "slowly", I mean "not having to write one or more large 
revision-control changes to lock-step-upgrade from e.g. Tomcat 8/9 to 
Tomcat 10.1."


One specific example is that we use Apache Velocity as our 
page-generation language and AIUI, these templates will not have their 
text scanned and replaced with updated strings. For example:


#if($request.getAttribute('javax.servlet.error.request_uri'))
...
#end

This code looks to see if the request has the 
"javax.serlet.error.request_uri" attribute set and then takes some 
appropriate action.


In order to use this on Tomcat 10.1, we would need to replace that with 
the appropriate string. In order to make it work on *either* Tomcat 8/9 
or 10, we'd need to change the code to:


#if($request.getAttribute('jakarta.servlet.error.request_uri') || 
$request.getAttribute('javax.servlet.error.request_uri'))


... and then we'd have semi-useless code laying around after the upgrade 
which would need to be cleaned-up afterward. Doable, but not super ideal.


This is the first such example I came across when looking at doing an 
upgrade in a development environment at $work, and I'm sure I will come 
up with some others.


I can think of a few ways to facilitate a dual-deployment (Tomcat 8/9 vs 
10 aka Java EE vs Jakarta EE) application, at least for the specific 
example I present above:


1. Change everything all at once, force-upgrade to Tomcat 10.1 all at 
once. I find this infeasible, so I'm going to ignore it as an option for 
now.


2. Write dual-checking code (as shown above) where we simply have either 
2 different code-paths for different environments, or double-up on our 
checking and usage of various Java/Jakarta request attributes. Yuck.


3. Write a Filter which copies from javax.servlet.* -> jakarta.servlet.* 
(or vice versa). This would allow us to immediately use the application 
under either environment.


4. Patch the server to provide #3 without having to include a Filter in 
the application deployment.


I'm kind of interested in idea #4, and wanted to ask for feedback.

I can't be the only one in this situation where the Tomcat Migration 
Tool is great but doesn't cover some use-cases and therefore upgrades 
are risky and/or problematic.


I think we as a community should be encouraging users to start upgrading 
to get over the "Jakarta Hurdle" and I think that this could be another 
small tool to help motivate that migration.


WDYT?

Thanks,
-chris

-
To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org
For additional commands, e-mail: dev-h...@tomcat.apache.org