Re: [ant] branch master updated: add java.security.manager=allow while launching against Java 19
On Tue, Feb 8, 2022 at 5:53 AM Jaikiran Pai wrote: > Hello Earl, > > On 08/02/22 12:59 am, Earl Hood wrote: > > How exactly does setting the sysprop for only 18 and 19 allow folks to > test > > their stuff? If Ant currently depends on the security manager to > operate, > > why not set the sysprop regardless, then remove in future when a > > replacement API exists? > > Java 18 and 19 now throw a runtime exception by default when > System.setSecurityManager is called at runtime. This behaviour can be > changed to prevent the exception being thrown and let it behave like > older versions, by setting the Java system property > java.security.manager=allow. We (Ant) cannot set it by default to all > versions of Java because this "allow" value was only introduced in Java > 12 > > https://www.oracle.com/java/technologies/javase/12-relnote-issues.html#JDK-8191053. > > Ant 1.10.x supports using earlier versions than Java 12 (like Java 8), > so we (Ant) cannot blindly set that value without these Java version > checks. > FWIW, NetBeans added a SecurityManager called "allow", that uninstalls itself as soon as possible: https://github.com/apache/netbeans/blob/master/platform/o.n.bootstrap/src/allow.java Then -Djava.security.manager=allow works on the platforms supported by NetBeans - before JDK 12, "allow" is installed and quickly uninstalled, but setting another SecurityManager is allowed. Jan > > Since I work on a project that embeds Ant and uses it API, I am trying > > assess what I need to do on my end to mitigate the problem. I do not use > > the launcher scripts, but invoke Ant directly as an embedded service > (same > > JVM) or via an external JVM process (most common usage). > > The way the JDK implements the security manager removal, setting of > java.security.manager=allow is only allowed (and expected to work) when > launching Java. What that means is one cannot use System.setProperty() > API at runtime to set this property (it won't work). So users of Java > will have to set this value at launch time. Right now, users who use Ant > to build their project with Java 18 or 19, can workaround this issue by > setting the very broad ANT_OPTS environment variable to include > "-Djava.security.manager=allow". However, given the number of projects > out there that use Ant and various ways it gets used, I did not want > users to go fiddle with their build scripts to set up this value in > ANT_OPTS (that too conditionally based on Java versions). Instead, it's > much more useful if Ant itself did this in its own launch script, thus > allowing users to just download this newer version of Ant and continue > building their projects like they currently do. > > Now coming to your embedded case, whoever/whatever launches the original > JVM within which you launch Ant, will have to be responsible for setting > this system property while launching the JVM. There's no other way past > this if you want to use it against Java 18 or 19. > > > -Jaikiran > > > - > To unsubscribe, e-mail: dev-unsubscr...@ant.apache.org > For additional commands, e-mail: dev-h...@ant.apache.org > >
Re: [ant] branch master updated: add java.security.manager=allow while launching against Java 19
On 2022-02-08, Jaikiran Pai wrote: > Hello Earl, > On 08/02/22 12:59 am, Earl Hood wrote: >> How exactly does setting the sysprop for only 18 and 19 allow folks to test >> their stuff? If Ant currently depends on the security manager to operate, >> why not set the sysprop regardless, then remove in future when a >> replacement API exists? > Java 18 and 19 now throw a runtime exception by default when > System.setSecurityManager is called at runtime. This behaviour can be > changed to prevent the exception being thrown and let it behave like > older versions, by setting the Java system property > java.security.manager=allow. We (Ant) cannot set it by default to all > versions of Java because this "allow" value was only introduced in > Java 12 > https://www.oracle.com/java/technologies/javase/12-relnote-issues.html#JDK-8191053. > Ant > 1.10.x supports using earlier versions than Java 12 (like Java 8), so > we (Ant) cannot blindly set that value without these Java version > checks. I couldn't find what the property does prior to Java 12, so let me add this for completeness. This is what happens when you set it on Java 8: Error occurred during initialization of VM java.lang.InternalError: Could not create SecurityManager: allow at sun.misc.Launcher.(Launcher.java:106) at sun.misc.Launcher.(Launcher.java:54) at java.lang.ClassLoader.initSystemClassLoader(ClassLoader.java:1444) at java.lang.ClassLoader.getSystemClassLoader(ClassLoader.java:1429) Stefan - To unsubscribe, e-mail: dev-unsubscr...@ant.apache.org For additional commands, e-mail: dev-h...@ant.apache.org
Re: [ant] branch master updated: add java.security.manager=allow while launching against Java 19
Hello Earl, On 08/02/22 12:59 am, Earl Hood wrote: How exactly does setting the sysprop for only 18 and 19 allow folks to test their stuff? If Ant currently depends on the security manager to operate, why not set the sysprop regardless, then remove in future when a replacement API exists? Java 18 and 19 now throw a runtime exception by default when System.setSecurityManager is called at runtime. This behaviour can be changed to prevent the exception being thrown and let it behave like older versions, by setting the Java system property java.security.manager=allow. We (Ant) cannot set it by default to all versions of Java because this "allow" value was only introduced in Java 12 https://www.oracle.com/java/technologies/javase/12-relnote-issues.html#JDK-8191053. Ant 1.10.x supports using earlier versions than Java 12 (like Java 8), so we (Ant) cannot blindly set that value without these Java version checks. Since I work on a project that embeds Ant and uses it API, I am trying assess what I need to do on my end to mitigate the problem. I do not use the launcher scripts, but invoke Ant directly as an embedded service (same JVM) or via an external JVM process (most common usage). The way the JDK implements the security manager removal, setting of java.security.manager=allow is only allowed (and expected to work) when launching Java. What that means is one cannot use System.setProperty() API at runtime to set this property (it won't work). So users of Java will have to set this value at launch time. Right now, users who use Ant to build their project with Java 18 or 19, can workaround this issue by setting the very broad ANT_OPTS environment variable to include "-Djava.security.manager=allow". However, given the number of projects out there that use Ant and various ways it gets used, I did not want users to go fiddle with their build scripts to set up this value in ANT_OPTS (that too conditionally based on Java versions). Instead, it's much more useful if Ant itself did this in its own launch script, thus allowing users to just download this newer version of Ant and continue building their projects like they currently do. Now coming to your embedded case, whoever/whatever launches the original JVM within which you launch Ant, will have to be responsible for setting this system property while launching the JVM. There's no other way past this if you want to use it against Java 18 or 19. -Jaikiran - To unsubscribe, e-mail: dev-unsubscr...@ant.apache.org For additional commands, e-mail: dev-h...@ant.apache.org
Re: [ant] branch master updated: add java.security.manager=allow while launching against Java 19
On 2022-02-07, Jaikiran Pai wrote:
> So the launch scripts (the Linux one and the .bat for Windows one)
> have both been updated to set this system property only for Java 18
> and 19. I expect this to be a temporary thing till the new API is
> available. Once the new API is available, I think we should just get
> rid of this setting of system property even for Java 18 and 19 (since
> I don't expect many to be using these releases once the newer versions
> are available).
You are more optimistic than me WRT a replacement API. :-)
If you believe this is just going to be an issue for two or three
releases then I can live with a lenghty if. I want to avoid that we need
to cut a new release with every new Java EA just because the replacement
API still hasn't been added.
> Now coming to the actual implementation of this, it took me multiple
> weekends to get the .bat version to correctly work. Mainly due to lack
> of easy access to a Windows setup plus my general lack of knowledge of
> bat scripting and some gotchas when it comes to .bat parsing and the
> "errorlevel" values.
I'm sorry to hear that. And I must admit my .bat programming skills are
no more exhaustive than yours, most probably even less so.
> Ultimately the one I committed ant.bat now launches the Java command
> twice and expects it to dump certain property values, which are then
> used by "find" to see if the version is 18 or 19.
Ouch
Would
jrunscript -e
"print(java.lang.System.getProperty('java.specification.version'))"
work? TBH I'm not sure jrunscript is available in a JRE rather than a
JDK for versions where there actually is a difference.
Also findstr[1] looks as if you could use it to bring the number of
extra jvm executions down to one as it should allow regexes so
find "java.specification.version = 1[89]"
may work - unless Java 20 still comes without the replacement API as it
looks as if the regex subset supported by findstr doesn't include
alternatives.
> With these changes the CI builds which run Ant tests against Java 18,
> 19 and previous version like Java 17 now work fine. However, like I
> said my scripting skills are minimal, so if any of these changes in
> these scripts can be done in a better way, please feel free to do
> so. I would do it myself, but it's going to take me trial and error
> methods to get it right :)
It would be completely unfair to place the burden on you. I can live
with the current solution even though I'm not happy with it. I might
find a bit of time to experiment this week, but I can't promise
anything.
Stefan
[1]
https://docs.microsoft.com/en-us/windows-server/administration/windows-commands/findstr
-
To unsubscribe, e-mail: dev-unsubscr...@ant.apache.org
For additional commands, e-mail: dev-h...@ant.apache.org
Re: [ant] branch master updated: add java.security.manager=allow while launching against Java 19
How exactly does setting the sysprop for only 18 and 19 allow folks to test their stuff? If Ant currently depends on the security manager to operate, why not set the sysprop regardless, then remove in future when a replacement API exists? Since I work on a project that embeds Ant and uses it API, I am trying assess what I need to do on my end to mitigate the problem. I do not use the launcher scripts, but invoke Ant directly as an embedded service (same JVM) or via an external JVM process (most common usage). Regards, --ewh On Mon, Feb 7, 2022, 4:18 AM Jaikiran Pai wrote: > Hello Stefan, > > I was planning to send out a mail about this change later tonight. But > good you brought this up. To give some background, the security manager > changes starting Java 18 make it such that setting of the security > manager at runtime now throws an exception, which effectively fails all > builds since Ant by default sets up a security manager. After various > experimentations and checking over with the JDK team, it appears that we > (Ant) can't get rid of setting the security manager till the JDK team > provides an API to prevent System.exit(...) calls. So in the meantime to > allow users of Ant to build their projects and test for any issues > against Java 18 (and now Java 19 EA), I decided to specifically set this > system property only for these specific versions. Initially I had only > done it for Java 18, hoping that a new API for System.exit(...) > prevention would be availbale in 19, but it isn't ready so far. So the > launch scripts (the Linux one and the .bat for Windows one) have both > been updated to set this system property only for Java 18 and 19. I > expect this to be a temporary thing till the new API is available. Once > the new API is available, I think we should just get rid of this setting > of system property even for Java 18 and 19 (since I don't expect many to > be using these releases once the newer versions are available). > > Now coming to the actual implementation of this, it took me multiple > weekends to get the .bat version to correctly work. Mainly due to lack > of easy access to a Windows setup plus my general lack of knowledge of > bat scripting and some gotchas when it comes to .bat parsing and the > "errorlevel" values. Ultimately the one I committed ant.bat now launches > the Java command twice and expects it to dump certain property values, > which are then used by "find" to see if the version is 18 or 19. So > essentially, launching Ant (on Windows) now additionally triggers > lauching of two Java process (they do exit) just to check the version. > It's not the best way, but I couldn't find any other way to do this. > > As for the Linux version of the ant launch script it does a similar > thing but in its case it just launches the Java program once and figures > out if the version is 18 or 19. > > With these changes the CI builds which run Ant tests against Java 18, 19 > and previous version like Java 17 now work fine. However, like I said my > scripting skills are minimal, so if any of these changes in these > scripts can be done in a better way, please feel free to do so. I would > do it myself, but it's going to take me trial and error methods to get > it right :) > > -Jaikiran > > On 07/02/22 12:26 pm, Stefan Bodewig wrote: > > On 2022-02-07, wrote: > > > >> - if [ "$JAVA_SPEC_VERSION" = "java.specification.version=18" ]; then > >> + if [ "$JAVA_SPEC_VERSION" = "java.specification.version=18" ] || [ > "$JAVA_SPEC_VERSION" = "java.specification.version=19" ]; then > > Bourne shell's case may make this more > > readable (not sure whether there is a Windows batch file equivalent) > > > > case $JAVA_SPEC_VERSION in > > java.specification.version=18 | \ > > java.specification.version=19 ) > > ... > > ;; > > esac > > > > But I'm afraid this is not going to scale :-) > > > > In the long run we probably are better off by enumerating the JDKs where > > we don't want to set the property (ten from 8 to 17, but this is a fixed > > list that doesn't need to change with every release). > > > > Stefan > > > > - > > To unsubscribe, e-mail: dev-unsubscr...@ant.apache.org > > For additional commands, e-mail: dev-h...@ant.apache.org > > > > - > To unsubscribe, e-mail: dev-unsubscr...@ant.apache.org > For additional commands, e-mail: dev-h...@ant.apache.org > >
Re: [ant] branch master updated: add java.security.manager=allow while launching against Java 19
Hello Stefan, I was planning to send out a mail about this change later tonight. But good you brought this up. To give some background, the security manager changes starting Java 18 make it such that setting of the security manager at runtime now throws an exception, which effectively fails all builds since Ant by default sets up a security manager. After various experimentations and checking over with the JDK team, it appears that we (Ant) can't get rid of setting the security manager till the JDK team provides an API to prevent System.exit(...) calls. So in the meantime to allow users of Ant to build their projects and test for any issues against Java 18 (and now Java 19 EA), I decided to specifically set this system property only for these specific versions. Initially I had only done it for Java 18, hoping that a new API for System.exit(...) prevention would be availbale in 19, but it isn't ready so far. So the launch scripts (the Linux one and the .bat for Windows one) have both been updated to set this system property only for Java 18 and 19. I expect this to be a temporary thing till the new API is available. Once the new API is available, I think we should just get rid of this setting of system property even for Java 18 and 19 (since I don't expect many to be using these releases once the newer versions are available). Now coming to the actual implementation of this, it took me multiple weekends to get the .bat version to correctly work. Mainly due to lack of easy access to a Windows setup plus my general lack of knowledge of bat scripting and some gotchas when it comes to .bat parsing and the "errorlevel" values. Ultimately the one I committed ant.bat now launches the Java command twice and expects it to dump certain property values, which are then used by "find" to see if the version is 18 or 19. So essentially, launching Ant (on Windows) now additionally triggers lauching of two Java process (they do exit) just to check the version. It's not the best way, but I couldn't find any other way to do this. As for the Linux version of the ant launch script it does a similar thing but in its case it just launches the Java program once and figures out if the version is 18 or 19. With these changes the CI builds which run Ant tests against Java 18, 19 and previous version like Java 17 now work fine. However, like I said my scripting skills are minimal, so if any of these changes in these scripts can be done in a better way, please feel free to do so. I would do it myself, but it's going to take me trial and error methods to get it right :) -Jaikiran On 07/02/22 12:26 pm, Stefan Bodewig wrote: On 2022-02-07, wrote: - if [ "$JAVA_SPEC_VERSION" = "java.specification.version=18" ]; then + if [ "$JAVA_SPEC_VERSION" = "java.specification.version=18" ] || [ "$JAVA_SPEC_VERSION" = "java.specification.version=19" ]; then Bourne shell's case may make this more readable (not sure whether there is a Windows batch file equivalent) case $JAVA_SPEC_VERSION in java.specification.version=18 | \ java.specification.version=19 ) ... ;; esac But I'm afraid this is not going to scale :-) In the long run we probably are better off by enumerating the JDKs where we don't want to set the property (ten from 8 to 17, but this is a fixed list that doesn't need to change with every release). Stefan - To unsubscribe, e-mail: dev-unsubscr...@ant.apache.org For additional commands, e-mail: dev-h...@ant.apache.org - To unsubscribe, e-mail: dev-unsubscr...@ant.apache.org For additional commands, e-mail: dev-h...@ant.apache.org
