Re: Working around a JRE bug
Mark, On 11/5/20 14:59, Mark Thomas wrote: Woot! The great folks at bnd have fixed this. It means depending on a snapshot but compared to the disruption of the alternatives I think that is acceptable for the short term. The issue with depending on a snapshot is reproducibility of builds. The simplest option (and infra seem OK with it) is to put a copy in my home dir on home.a.o. Sounds good to me. Oh, and ship a beer or two to the kind folks at bnd who jumped to fix this within ... 4 hours? How does this affect Maven builds? -chris On 05/11/2020 12:57, Mark Thomas wrote: All, The summary: - The JVM spec states that the ModulePackages attribute in module-info.class DOES NOT have to list ALL packages in the module - bnd is consistent with the JVM spec and only lists the packages that are required to be listed - the JRE uses a broken class loader optimisation that assumes that ModulePackages DOES list ALL packages present in the module When applications try and use our JARs with bnd provided module-info CNFE occur because the JRE can't find the module for some classes. For a fuller description of the issue see: https://bugs.openjdk.java.net/browse/JDK-8255854 This is likely the cause of several currently open bugs reports of CNFE when using modules. Possible solutions: 1. OpenJDK accepts the class loader optimisation is flawed and reverts it. Given the reaction so far to the reported bug this looks unlikely. Even if this were to happen, class loading performance would be impacted and it is going to take a long time before all the broken JREs have been updated. 2. The bnd project updates bnd to implement what amounts to an undocumented requirement that the ModulePackages attribute lists all packages in the module. This is probably the cleanest solution but depends on the goodwill of the bnd project who would be well within their rights to reject it as invalid based on the JVM spec. I haven't yet approached the bnd project. A fix along these lines might be ready for the next release round but is unlikely to be ready for this one. 3. We drop all the JPMS meta-data until we have a solution. I'm not sure of the consequences for users wanting to use Tomcat JARs in a JPMS environment. 4. We "patch" module-info after bnd has generated it via: - custom code (BCEL probably helps) - jar (if using Java 9+ jar rebuilds the module-info.class file) 5. We add "unnecessary" @aQute.bnd.annotation.jpms.Open annotations to packages so bnd includes them in module-info. It might be hard to remove these at a later date if folks start to depend on them. I am currently thinking along these lines: - Add @aQute.bnd.annotation.jpms.Open where necessary to fix this. - Document clearly in the Javadoc, change log, the release announcement and the RELEASE NOTES that this is a temporary workaround that will be removed as soon as a better fix is available. - Ask the bnd project to make a change to list all packages in a module in the ModulePackages attribute. Thoughts? Mark - To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org For additional commands, e-mail: dev-h...@tomcat.apache.org - To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org For additional commands, e-mail: dev-h...@tomcat.apache.org - To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org For additional commands, e-mail: dev-h...@tomcat.apache.org
Re: Working around a JRE bug
On 06/11/2020 10:15, Rémy Maucherat wrote: > On Thu, Nov 5, 2020 at 9:00 PM Mark Thomas wrote: > >> Woot! >> >> The great folks at bnd have fixed this. It means depending on a snapshot >> but compared to the disruption of the alternatives I think that is >> acceptable for the short term. >> >> The issue with depending on a snapshot is reproducibility of builds. The >> simplest option (and infra seem OK with it) is to put a copy in my home >> dir on home.a.o. >> >> Any objections? Better ideas? >> > > It's a lot better than not working. But I expect it will be a problem for > independent build systems. After some reflection I went for a slightly different approach. The build script now gets the latest 5.3.0 SNAPSHOT from the bnd snapshot repo and I will take an archive copy of the version I use for release builds (likely only the one version). That should minimise the issues for other builds but in the unlikely event of someone needing to re-run a build with the exact bnd version we used for a release rather than building from source (which would be possible) we have a copy. Given the nature of the work-around and the way we use bnd I'd be very surprised if any of the snapshots between now and the 5.3.0 release (or the 5.3.0 release itself) exhibited any observable change in behaviour in our build system. Mark - To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org For additional commands, e-mail: dev-h...@tomcat.apache.org
Re: Working around a JRE bug
On Thu, Nov 5, 2020 at 9:00 PM Mark Thomas wrote: > Woot! > > The great folks at bnd have fixed this. It means depending on a snapshot > but compared to the disruption of the alternatives I think that is > acceptable for the short term. > > The issue with depending on a snapshot is reproducibility of builds. The > simplest option (and infra seem OK with it) is to put a copy in my home > dir on home.a.o. > > Any objections? Better ideas? > It's a lot better than not working. But I expect it will be a problem for independent build systems. Rémy > > Mark > > > > On 05/11/2020 12:57, Mark Thomas wrote: > > All, > > > > The summary: > > > > - The JVM spec states that the ModulePackages attribute in > > module-info.class DOES NOT have to list ALL packages in the module > > - bnd is consistent with the JVM spec and only lists the packages that > > are required to be listed > > - the JRE uses a broken class loader optimisation that assumes that > > ModulePackages DOES list ALL packages present in the module > > > > When applications try and use our JARs with bnd provided module-info > > CNFE occur because the JRE can't find the module for some classes. > > > > For a fuller description of the issue see: > > https://bugs.openjdk.java.net/browse/JDK-8255854 > > > > This is likely the cause of several currently open bugs reports of CNFE > > when using modules. > > > > > > Possible solutions: > > > > 1. OpenJDK accepts the class loader optimisation is flawed and reverts > >it. > >Given the reaction so far to the reported bug this looks unlikely. > >Even if this were to happen, class loading performance would be > >impacted and it is going to take a long time before all the broken > >JREs have been updated. > > > > 2. The bnd project updates bnd to implement what amounts to an > >undocumented requirement that the ModulePackages attribute lists all > >packages in the module. > >This is probably the cleanest solution but depends on the goodwill of > >the bnd project who would be well within their rights to reject it as > >invalid based on the JVM spec. I haven't yet approached the bnd > >project. A fix along these lines might be ready for the next release > >round but is unlikely to be ready for this one. > > > > 3. We drop all the JPMS meta-data until we have a solution. > >I'm not sure of the consequences for users wanting to use Tomcat JARs > >in a JPMS environment. > > > > 4. We "patch" module-info after bnd has generated it via: > >- custom code (BCEL probably helps) > >- jar (if using Java 9+ jar rebuilds the module-info.class file) > > > > 5. We add "unnecessary" @aQute.bnd.annotation.jpms.Open annotations to > >packages so bnd includes them in module-info. > >It might be hard to remove these at a later date if folks start to > >depend on them. > > > > > > I am currently thinking along these lines: > > > > - Add @aQute.bnd.annotation.jpms.Open where necessary to fix this. > > - Document clearly in the Javadoc, change log, the release announcement > > and the RELEASE NOTES that this is a temporary workaround that will be > > removed as soon as a better fix is available. > > - Ask the bnd project to make a change to list all packages in a module > > in the ModulePackages attribute. > > > > Thoughts? > > > > Mark > > > > - > > To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org > > For additional commands, e-mail: dev-h...@tomcat.apache.org > > > > > - > To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org > For additional commands, e-mail: dev-h...@tomcat.apache.org > >
Re: Working around a JRE bug
Woot! The great folks at bnd have fixed this. It means depending on a snapshot but compared to the disruption of the alternatives I think that is acceptable for the short term. The issue with depending on a snapshot is reproducibility of builds. The simplest option (and infra seem OK with it) is to put a copy in my home dir on home.a.o. Any objections? Better ideas? Mark On 05/11/2020 12:57, Mark Thomas wrote: > All, > > The summary: > > - The JVM spec states that the ModulePackages attribute in > module-info.class DOES NOT have to list ALL packages in the module > - bnd is consistent with the JVM spec and only lists the packages that > are required to be listed > - the JRE uses a broken class loader optimisation that assumes that > ModulePackages DOES list ALL packages present in the module > > When applications try and use our JARs with bnd provided module-info > CNFE occur because the JRE can't find the module for some classes. > > For a fuller description of the issue see: > https://bugs.openjdk.java.net/browse/JDK-8255854 > > This is likely the cause of several currently open bugs reports of CNFE > when using modules. > > > Possible solutions: > > 1. OpenJDK accepts the class loader optimisation is flawed and reverts >it. >Given the reaction so far to the reported bug this looks unlikely. >Even if this were to happen, class loading performance would be >impacted and it is going to take a long time before all the broken >JREs have been updated. > > 2. The bnd project updates bnd to implement what amounts to an >undocumented requirement that the ModulePackages attribute lists all >packages in the module. >This is probably the cleanest solution but depends on the goodwill of >the bnd project who would be well within their rights to reject it as >invalid based on the JVM spec. I haven't yet approached the bnd >project. A fix along these lines might be ready for the next release >round but is unlikely to be ready for this one. > > 3. We drop all the JPMS meta-data until we have a solution. >I'm not sure of the consequences for users wanting to use Tomcat JARs >in a JPMS environment. > > 4. We "patch" module-info after bnd has generated it via: >- custom code (BCEL probably helps) >- jar (if using Java 9+ jar rebuilds the module-info.class file) > > 5. We add "unnecessary" @aQute.bnd.annotation.jpms.Open annotations to >packages so bnd includes them in module-info. >It might be hard to remove these at a later date if folks start to >depend on them. > > > I am currently thinking along these lines: > > - Add @aQute.bnd.annotation.jpms.Open where necessary to fix this. > - Document clearly in the Javadoc, change log, the release announcement > and the RELEASE NOTES that this is a temporary workaround that will be > removed as soon as a better fix is available. > - Ask the bnd project to make a change to list all packages in a module > in the ModulePackages attribute. > > Thoughts? > > Mark > > - > To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org > For additional commands, e-mail: dev-h...@tomcat.apache.org > - To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org For additional commands, e-mail: dev-h...@tomcat.apache.org
Working around a JRE bug
All, The summary: - The JVM spec states that the ModulePackages attribute in module-info.class DOES NOT have to list ALL packages in the module - bnd is consistent with the JVM spec and only lists the packages that are required to be listed - the JRE uses a broken class loader optimisation that assumes that ModulePackages DOES list ALL packages present in the module When applications try and use our JARs with bnd provided module-info CNFE occur because the JRE can't find the module for some classes. For a fuller description of the issue see: https://bugs.openjdk.java.net/browse/JDK-8255854 This is likely the cause of several currently open bugs reports of CNFE when using modules. Possible solutions: 1. OpenJDK accepts the class loader optimisation is flawed and reverts it. Given the reaction so far to the reported bug this looks unlikely. Even if this were to happen, class loading performance would be impacted and it is going to take a long time before all the broken JREs have been updated. 2. The bnd project updates bnd to implement what amounts to an undocumented requirement that the ModulePackages attribute lists all packages in the module. This is probably the cleanest solution but depends on the goodwill of the bnd project who would be well within their rights to reject it as invalid based on the JVM spec. I haven't yet approached the bnd project. A fix along these lines might be ready for the next release round but is unlikely to be ready for this one. 3. We drop all the JPMS meta-data until we have a solution. I'm not sure of the consequences for users wanting to use Tomcat JARs in a JPMS environment. 4. We "patch" module-info after bnd has generated it via: - custom code (BCEL probably helps) - jar (if using Java 9+ jar rebuilds the module-info.class file) 5. We add "unnecessary" @aQute.bnd.annotation.jpms.Open annotations to packages so bnd includes them in module-info. It might be hard to remove these at a later date if folks start to depend on them. I am currently thinking along these lines: - Add @aQute.bnd.annotation.jpms.Open where necessary to fix this. - Document clearly in the Javadoc, change log, the release announcement and the RELEASE NOTES that this is a temporary workaround that will be removed as soon as a better fix is available. - Ask the bnd project to make a change to list all packages in a module in the ModulePackages attribute. Thoughts? Mark - To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org For additional commands, e-mail: dev-h...@tomcat.apache.org
