Re: Issues when building native image with GraalVM

[Rémy Maucherat]

On Tue, Dec 28, 2021 at 7:18 PM Mar R  wrote:
>
> Tomcat 10.0.14
> Windows 10 x64 21H1 OS Build 19043.1415
>
> openjdk version "17.0.1" 2021-10-19
> OpenJDK Runtime Environment GraalVM CE 21.3.0 (build
> 17.0.1+12-jvmci-21.3-b05)
> OpenJDK 64-Bit Server VM GraalVM CE 21.3.0 (build 17.0.1+12-jvmci-21.3-b05,
> mixed mode, sharing)
>
> NetBeans 12.6, maven webapp project
> Maven 3.8.4
> Ant 1.10.12
>
> Followed guide at https://tomcat.apache.org/tomcat-10.0-doc/graal.html
>
> Sorry in advance if not all issues should be reported here but instead are
> graalvm native image related, I'm reporting everything here and on their
> github fro completeness.
>
> https://drive.google.com/file/d/17flFW5nlNCdojlvJxCOy23NJBj03p46u/view?usp=sharing
> In this link you can find a folder with everything I used from tomcat
> stuffed folder downloaded from the github link present in the tomcat AOT
> guide, to all commands I put in a script for easier testing purpose, source
> webapp, built webapp, screenshots, everything :D
>
> I'll start by saying I'm well aware I could definitely have overlooked
> something, anyway I managed to get to the final native image and it works,
> BUT:
>
> If you remove .class files from webapps/webappname folder, when you run the
> binary file, it won't find those files, only jsps. This is strange because
> those files are actually present in the fat jar, and this actually beats
> the purpose of builidng a native image if you have to ship it with java
> files.
>
> If there is a class extending ServletContextListener (InitClass in this
> webapp), when running command with "--catalina -generateCode" parameter,
> where you have to access all jsps and servlets, the class extending
> ServletContextListener will be initialized/used(?) like 2 times, but the
> second time not completly, resetting everything done in the first
> initialization.
> In this webapp it happens when you access the only servlet present.
> Anyway all code is in the google drive link and in InitClass i put some
> prints so you can see what I mean.
> In another bigger webapp my InitClass initializes a lot of things used by
> the webapp basically screwing it.
> This issues happens only in this phase becasue in the second phase where
> the parameter is "useGeneratedCode" it works normally both with this simple
> test webapp and in my other one
>
> When building with native-image command, if
> "-J--add-exports=java.management/sun.management=ALL-UNNAMED" isn't used as
> additional parameter, errors will pop on run crashing the app. (screenshot
> in the google drive link)

Before looking at it more deeply (later ...), let's take this step by step:
- Even though you are generating a fat JAR and building a native
image, the webapps should stay unchanged. All classes and JARs in
WEB-INF will be needed for annotation scanning and whatever
"classloading". So this is not a bug. Don't worry, regular Java
classes that have not been compiled in a native image are never going
to be magically dynamically loaded and run, since this isn't possible
at all.
- Java 17 with Graal is completely untested (Java 11 is) since it's
very new. Since you seem very adventurous, you can also try to compile
in the Panama support for OpenSSL.
- I haven't tested the JMX support, which is fairly new in Graal.
Adding the module declaration on the command line if needed depending
on the Java version doesn't seem that surprising. If you were running
Java 8 you wouldn't need it. For the longest time, JMX was disabled
with Graal (with no way to enable it). Since it supposedly "works",
the regular Tomcat standalone defaults are used (so you can use the
command line to disable it again; to be honest I would probably do
that ...).
- You can keep your server.xml (more flexible), but once everything
works (if you get there then it's good !) you can indeed generate the
equivalent code for server.xml/context.xml with -generateCode and
avoid some reflection. This is really completely optional. So it seems
there's a problem with ServletContextListener then (this is strange
since it's a web.xml component, for which I never added the actual
code generation - it was tempting but way too complex if the goal was
to support everything). So I'll try to understand what is going on.

Rémy

-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org

Issues when building native image with GraalVM

[Mar R]

Tomcat 10.0.14
Windows 10 x64 21H1 OS Build 19043.1415

openjdk version "17.0.1" 2021-10-19
OpenJDK Runtime Environment GraalVM CE 21.3.0 (build
17.0.1+12-jvmci-21.3-b05)
OpenJDK 64-Bit Server VM GraalVM CE 21.3.0 (build 17.0.1+12-jvmci-21.3-b05,
mixed mode, sharing)

NetBeans 12.6, maven webapp project
Maven 3.8.4
Ant 1.10.12

Followed guide at https://tomcat.apache.org/tomcat-10.0-doc/graal.html

Sorry in advance if not all issues should be reported here but instead are
graalvm native image related, I'm reporting everything here and on their
github fro completeness.

https://drive.google.com/file/d/17flFW5nlNCdojlvJxCOy23NJBj03p46u/view?usp=sharing
In this link you can find a folder with everything I used from tomcat
stuffed folder downloaded from the github link present in the tomcat AOT
guide, to all commands I put in a script for easier testing purpose, source
webapp, built webapp, screenshots, everything :D

I'll start by saying I'm well aware I could definitely have overlooked
something, anyway I managed to get to the final native image and it works,
BUT:

If you remove .class files from webapps/webappname folder, when you run the
binary file, it won't find those files, only jsps. This is strange because
those files are actually present in the fat jar, and this actually beats
the purpose of builidng a native image if you have to ship it with java
files.

If there is a class extending ServletContextListener (InitClass in this
webapp), when running command with "--catalina -generateCode" parameter,
where you have to access all jsps and servlets, the class extending
ServletContextListener will be initialized/used(?) like 2 times, but the
second time not completly, resetting everything done in the first
initialization.
In this webapp it happens when you access the only servlet present.
Anyway all code is in the google drive link and in InitClass i put some
prints so you can see what I mean.
In another bigger webapp my InitClass initializes a lot of things used by
the webapp basically screwing it.
This issues happens only in this phase becasue in the second phase where
the parameter is "useGeneratedCode" it works normally both with this simple
test webapp and in my other one

When building with native-image command, if
"-J--add-exports=java.management/sun.management=ALL-UNNAMED" isn't used as
additional parameter, errors will pop on run crashing the app. (screenshot
in the google drive link)

Re: javax.servlet vs jakarta.servlet?

[Mark Thomas]

On 28/12/2021 14:58, Michael B Allen wrote:

On Tue, Dec 28, 2021 at 3:29 AM Johan Compagner  wrote:


Will that really work?


No.

Clearly I'm not paying attention because after reading surprisingly
little information about this fundamental incompatibility and
downloading and trying Tomcat 10, there is no way to use a
javax.servlet library or application with a jakarta.servlet app server
like Tomcat 10.


Yes there is. Just place the Java EE web application in the 
webapps-javaee directory and Tomcat will convert automatically to 
Jakarta EE and then deploy it.



So now I REALLY want to know if people actually use
Tomcat 10 and what the release policy will be going forward for Tomcat
9. Is Tomcat 9 going to be maintained separately with security fixes
and such?


It is the intention of the Tomcat community to continue to support 
Tomcat 9 for as long as there are users that want a supported Java EE 
platform.


Normally, we support 3 versions in parallel with a major version 
reaching EOL every ~3 years. Typically a major version will be supported 
for ~10 years.


Tomcat 9 will be different. When Tomcat 9 reaches the normal EOL point 
(roughly 2027) 9.0.x releases will stop but Tomcat 9.10.x releases will 
start. These will essentially track Tomcat 10.x releases but with the 
Java EE API rather than the Jakarta EE API.


When Tomcat 10.x reaches end of life (~2030) 9.10.x releases will stop 
and 9.11.x releases will start. These will essentially track Tomcat 11.x 
releases but with the Java EE API rather than the Jakarta EE API.


And so on for as long as there is community demand for the Java EE 
support and there are 3 Tomcat PMC members willing to vote for the release.



There must be thousands of libraries and applications that cannot be
converted for various reasons.


The only application I haven't been able to convert so far was Jira - 
primarily because the error messages when Jira failed to start were 
extremely unhelpful. I'm fairly sure the conversion process was missing 
some OSGI config somewhere but I couldn't figure out what it was.


If anyone has an app that the conversion tool doesn't handle correctly, 
open a bug, tell us what it isn't getting right and we should be able to 
fix it.



Even active open source projects are
probably going to punt on this. Think about how many github projects
there are with one guy using it for some thing at work and so he
doesn't want to make up some screwy build process that creates two
jars. This basically guarantees that people will be using Tomcat 9 for
many years.


No need for any build process changes if you don't want to. Just use 
webapps-javaee and the automatic conversion.


If there are apps that genuinely can't migrate (and I have a hard time 
believing there are really any apps in this category) then Tomcat 9 will 
continue to be available.



Also, correct me if I'm wrong but it seems to me that the two
interfaces should easily coexist.


Consider yourself corrected.


The Servlet interface is just that -
an interface. So just create an implementation of javax.servlet that
uses the jakarta.servlet code as it's implementation. In many cases
the code could literally be like:

   package javax.servlet.http;

   public class HttpServletRequestWrapper extends
jakarta.servlet.http.ServletRequestWrapper implements
HttpServletRequest
   {
 // nothing to see here
   }


This illustrates the my point. You would not be able to cast an instance 
of javax.servlet.http.HttpServletRequestWrapper to 
javax.servlet.http.ServletRequestWrapper


Alternatively, just try and write a class (that compiles) that 
implements javax.servlet.http.HttpServletRequest and 
jakarta.servlet.http.HttpServletRequest


You can't. Well, you might be able to do get it to work with dynamic 
proxies but the complexity would not be pretty - and I don't think you 
could hide it from the web application which defeats the point of trying.



The only code that might be different is details about sessions or
something that jakarta.servlet wants to do a little differently. But
in that case, the ability of OOP to override methods to selectively
change some behavior can be utilized well here. Unless something in
jakarta.servlet changes that fundamentally breaks compatibility with
javax.servlet, code using javax.servlet shouldn't know the difference.


It is much, much more complicated than that.


And if something does change that breaks backward compatibility, that
would be a slightly frightening change in policy regarding backward
compatibility of Java code in general since just about any piece of
Java code written in literally the last 20 years will run on the
latest platform with only minor tweaks.


Yes and no. The Java team is making some more invasive changes. The 
removal of the security manager is the one that jumps immediately to mind.


Ignoring the package change, the biggest difference in Servlet 6 and the 
other Jakarta EE 10 APIs (Tomcat 10.1.x) is that a lot of t

Re: javax.servlet vs jakarta.servlet?

[Johan Compagner]

i think the whole point was that they had to break with the "javax" naming
because oracle doesn't allow that outside it stuff.

not sure if Tomcat 10 would be allowed to just have both classes/interfaces
for compatibility

Also this would be a big hurdle moving forwards (and not sure if everything
is really an interface and if there are no concrete classes in the spec)

But for tomcat 10 and maybe for tomcat 11 this would be a good migration
path that it would just run both of them just as easy.

But anyway we kind of have this

Apache Tomcat® - Apache Tomcat Migration Tool for Jakarta EE Software
Downloads 

or:

"Applications that run on Tomcat 9 and earlier will not run on Tomcat 10
without changes. Java EE based applications designed for Tomcat 9 and
earlier may be placed in the $CATALINA_BASE/webapps-javaee directory and
Tomcat will automatically convert them to Jakarta EE and copy them to the
webapps directory. This conversion is performed using the Apache Tomcat
migration tool for Jakarta EE tool
 which is also
available as a separate download
 for off-line use."

but i agree a bit with your sentiment, this is a major break, its kind of a
new spec (they versioned it 4.0 to 5.0 but it kind of just a 1.0 again,
starting over)

So i also do think that the maintenance window of Tomcat 9 will be quite
long, but tomcat versions are supported for quite some time, 8.5 is still
supported and the first release was beginning of 2014.
Tomcat 9 was 3 years later or so, so i guess it will be supported maybe
even close to 2030...




On Tue, 28 Dec 2021 at 16:00, Michael B Allen  wrote:

> On Tue, Dec 28, 2021 at 3:29 AM Johan Compagner 
> wrote:
> >
> > Will that really work?
>
> No.
>
> Clearly I'm not paying attention because after reading surprisingly
> little information about this fundamental incompatibility and
> downloading and trying Tomcat 10, there is no way to use a
> javax.servlet library or application with a jakarta.servlet app server
> like Tomcat 10. So now I REALLY want to know if people actually use
> Tomcat 10 and what the release policy will be going forward for Tomcat
> 9. Is Tomcat 9 going to be maintained separately with security fixes
> and such?
>
> There must be thousands of libraries and applications that cannot be
> converted for various reasons. Even active open source projects are
> probably going to punt on this. Think about how many github projects
> there are with one guy using it for some thing at work and so he
> doesn't want to make up some screwy build process that creates two
> jars. This basically guarantees that people will be using Tomcat 9 for
> many years.
>
> Also, correct me if I'm wrong but it seems to me that the two
> interfaces should easily coexist. The Servlet interface is just that -
> an interface. So just create an implementation of javax.servlet that
> uses the jakarta.servlet code as it's implementation. In many cases
> the code could literally be like:
>
>   package javax.servlet.http;
>
>   public class HttpServletRequestWrapper extends
> jakarta.servlet.http.ServletRequestWrapper implements
> HttpServletRequest
>   {
> // nothing to see here
>   }
>
> The only code that might be different is details about sessions or
> something that jakarta.servlet wants to do a little differently. But
> in that case, the ability of OOP to override methods to selectively
> change some behavior can be utilized well here. Unless something in
> jakarta.servlet changes that fundamentally breaks compatibility with
> javax.servlet, code using javax.servlet shouldn't know the difference.
> And if something does change that breaks backward compatibility, that
> would be a slightly frightening change in policy regarding backward
> compatibility of Java code in general since just about any piece of
> Java code written in literally the last 20 years will run on the
> latest platform with only minor tweaks.
>
> The effort required for every product and project in the universe to
> create a build process that generates two jars is massive compared to
> the effort that would be required for Tomcat to support both
> javax.servlet and jakarta.servlet even if it were difficult to do for
> some reason I'm not seeing.
>
> Mike
>
> --
> Michael B Allen
> Java Active Directory Integration
> http://www.ioplex.com/
>
> -
> To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
> For additional commands, e-mail: users-h...@tomcat.apache.org
>
>

-- 
Johan Compagner
Servoy

Re: javax.servlet vs jakarta.servlet?

[Michael B Allen]

On Tue, Dec 28, 2021 at 3:29 AM Johan Compagner  wrote:
>
> Will that really work?

No.

Clearly I'm not paying attention because after reading surprisingly
little information about this fundamental incompatibility and
downloading and trying Tomcat 10, there is no way to use a
javax.servlet library or application with a jakarta.servlet app server
like Tomcat 10. So now I REALLY want to know if people actually use
Tomcat 10 and what the release policy will be going forward for Tomcat
9. Is Tomcat 9 going to be maintained separately with security fixes
and such?

There must be thousands of libraries and applications that cannot be
converted for various reasons. Even active open source projects are
probably going to punt on this. Think about how many github projects
there are with one guy using it for some thing at work and so he
doesn't want to make up some screwy build process that creates two
jars. This basically guarantees that people will be using Tomcat 9 for
many years.

Also, correct me if I'm wrong but it seems to me that the two
interfaces should easily coexist. The Servlet interface is just that -
an interface. So just create an implementation of javax.servlet that
uses the jakarta.servlet code as it's implementation. In many cases
the code could literally be like:

  package javax.servlet.http;

  public class HttpServletRequestWrapper extends
jakarta.servlet.http.ServletRequestWrapper implements
HttpServletRequest
  {
// nothing to see here
  }

The only code that might be different is details about sessions or
something that jakarta.servlet wants to do a little differently. But
in that case, the ability of OOP to override methods to selectively
change some behavior can be utilized well here. Unless something in
jakarta.servlet changes that fundamentally breaks compatibility with
javax.servlet, code using javax.servlet shouldn't know the difference.
And if something does change that breaks backward compatibility, that
would be a slightly frightening change in policy regarding backward
compatibility of Java code in general since just about any piece of
Java code written in literally the last 20 years will run on the
latest platform with only minor tweaks.

The effort required for every product and project in the universe to
create a build process that generates two jars is massive compared to
the effort that would be required for Tomcat to support both
javax.servlet and jakarta.servlet even if it were difficult to do for
some reason I'm not seeing.

Mike

--
Michael B Allen
Java Active Directory Integration
http://www.ioplex.com/

-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org

Re: ClassFileTransformer in Tomcat 10 common classloader

[Olaf Kock]

On 28.12.21 00:36, Chew Kok Hoor wrote:
> We're using the old javax.servlet namespace for compatibility reasons.
>
> Some of our jar files are re-used by different web-apps, therefore we
> placed them in the common classloader.
>
> Is it possible to convert them dynamically, just like how we do it for
> servlets in the per app WEB-INF folder, by using the following in the
> context file:
>
> 

On top of what's already discussed:

You're not giving any indication that you need jakarta-ee at all, so if
you use javax.servlet: Just do it, and deploy on Tomcat 9 - that's a
very good choice that won't go away any time soon. If there's no need
for Tomcat 10, save yourself the hassle - at least for now.

Later on, you might want to divide your webapps into two groups: Those
that don't require any compatibility handling, because they're fully
migrated (or automatically migratable) to jakarta-ee, and those that
don't. Easy setup with a reverse proxy sending the traffic to either of
the two (or multiple) backends.

Olaf




-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org

Re: javax.servlet vs jakarta.servlet?

[Johan Compagner]

Will that really work?

tomcat 10 internally really expects jakarta.xxx now right?
so if you have a class MyServlet extends javax.Servlet
i don't think tomcat 10 will do anything with that (if that is not
converted first through the migration tool)

I have a feeling that you really can't mix, but i can be wrong

johan



On Tue, 28 Dec 2021 at 01:35, Michael B Allen  wrote:

> Hello Tomcat Users,
>
> Does anyone have a sense of how many people are using the jakarta.servlet
> API?
>
> I have a product that includes some classes that use javax.servlet and
> I'm trying to figure out how to support people who want to use
> jakarta.servlet. I plan to install Tomcat 10 momentarily but I assume
> one just has to add whatever javax.servlet jar to make Tomcat 10 work
> with a javax.servlet lib or app?
>
> Mike
>
> --
> Michael B Allen
> Java Active Directory Integration
> http://www.ioplex.com/
>
> -
> To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
> For additional commands, e-mail: users-h...@tomcat.apache.org
>
>

-- 
Johan Compagner
Servoy