Public API for programmatic registration of Taglet?
Hello I'm trying to write an extension of the standard doclet together with custom taglets in the same JAR file. The custom taglets and doclet need to share information. But I faced the following difficulties: When the Taglet.init(DocletEnvironment,Doclet) method is invoked, the doclet argument given by the caller is the internal HtmlDoclet wrapped by StandardDoclet, not my custom StandardDoclet subtype. So I can not get my doclet and taglet in touch that way. The lazy workaround - shared static fields - does not work because the taglet is loaded in a different class loader than the doclet. So even if I change a static field value from the taglet, the doclet never see that change. This is not a multi-threading issue; I tested with volatile static field. I also verified that the class loaders for the same class were different depending on whether the class of my JAR file is accessed from the doclet or from the taglet. I tried to wrap DocletEnvironment in order to return a customised ForwardingJavaFileManager, in an attempt to control which ClassLoader is used for loading the taglet. But the standard doclet does not seem to accept customized DocletEnvironment, as suggested by the error that I got: java.lang.ClassCastException: org.opengis.tools.doclet.ForwardingDocletEnvironment cannot be cast to jdk.javadoc/jdk.javadoc.internal.tool.DocEnvImpl at jdk.javadoc/jdk.javadoc.internal.doclets.toolkit.WorkArounds.(WorkArounds.java:101) at jdk.javadoc/jdk.javadoc.internal.doclets.toolkit.AbstractDoclet.run(AbstractDoclet.java:108) at jdk.javadoc/jdk.javadoc.doclet.StandardDoclet.run(StandardDoclet.java:72) (...snip...) The jdk.javadoc.internal.doclets.toolkit.taglets.TagletManager class has public void addCustomTag(Taglet) method which seems perfect for my need, but I found no public API for accessing this functionality. Is there a public way for a doclet to know its registered taglets, or any other workaround that I may have missed? If not, would the following evolution of Javadoc tools be possible? * When extending the StandardDoclet, the doclet given in Taglet.init(...) method should be the user StandardDoclet instance instead than the JDK internal HtmlDoclet. * Alternatively, a public API for addCustomTag(…) functionality would also work. Regards, Martin
Re: Public API for programmatic registration of Taglet?
On 4/18/18 9:08 AM, Martin Desruisseaux wrote: Hello I'm trying to write an extension of the standard doclet together with custom taglets in the same JAR file. The custom taglets and doclet need to share information. But I faced the following difficulties: When the Taglet.init(DocletEnvironment,Doclet) method is invoked, the doclet argument given by the caller is the internal HtmlDoclet wrapped by StandardDoclet, not my custom StandardDoclet subtype. So I can not get my doclet and taglet in touch that way. The lazy workaround - shared static fields - does not work because the taglet is loaded in a different class loader than the doclet. So even if I change a static field value from the taglet, the doclet never see that change. This is not a multi-threading issue; I tested with volatile static field. I also verified that the class loaders for the same class were different depending on whether the class of my JAR file is accessed from the doclet or from the taglet. I tried to wrap DocletEnvironment in order to return a customised ForwardingJavaFileManager, in an attempt to control which ClassLoader is used for loading the taglet. But the standard doclet does not seem to accept customized DocletEnvironment, as suggested by the error that I got: java.lang.ClassCastException: org.opengis.tools.doclet.ForwardingDocletEnvironment cannot be cast to jdk.javadoc/jdk.javadoc.internal.tool.DocEnvImpl at jdk.javadoc/jdk.javadoc.internal.doclets.toolkit.WorkArounds.(WorkArounds.java:101) at jdk.javadoc/jdk.javadoc.internal.doclets.toolkit.AbstractDoclet.run(AbstractDoclet.java:108) at jdk.javadoc/jdk.javadoc.doclet.StandardDoclet.run(StandardDoclet.java:72) (...snip...) The jdk.javadoc.internal.doclets.toolkit.taglets.TagletManager class has public void addCustomTag(Taglet) method which seems perfect for my need, but I found no public API for accessing this functionality. Is there a public way for a doclet to know its registered taglets, or any other workaround that I may have missed? If not, would the following evolution of Javadoc tools be possible? * When extending the StandardDoclet, the doclet given in Taglet.init(...) method should be the user StandardDoclet instance instead than the JDK internal HtmlDoclet. * Alternatively, a public API for addCustomTag(…) functionality would also work. Regards, Martin Martin, You are (regrettably) on the bleeding edge here with this use of the recent new Doclet API. I would somewhat rephrase your first bullet, to reflect the intent of the API * Whatever the doclet, the doclet given in Taglet.init(...) method should be the user doclet, whether specified explicitly or by default Given that redefinition, the behavior you describe can reasonably be described as a bug. This would require a minor spec change of the doclet parameter: "doclet - the doclet that instantiated this taglet" It is true that there is no public API (except for option support) on StandardDoclet to access some of the features of the standard doclet. And, I don't think we should go all the way with methods to access all the individual features. But, there is precedent in other similar APIs for adding methods to provide instances of objects that might otherwise be created in problematic classloaders. The most notable case is the ability to provide instances of annotation processors to javac [1] The problem in this case would be the specification of such a method. While it would be easy to write "Adds a custom tag." the order of the calls has a visible impact on the generated documentation that would be harder to specify and/or provide full control. -- Jon [1] https://docs.oracle.com/javase/9/docs/api/javax/tools/JavaCompiler.CompilationTask.html#setProcessors-java.lang.Iterable-
Re: Public API for programmatic registration of Taglet?
Hello Jonathan Thanks for your reply. Indeed, I understand that adding new API can be tricky. If it is desired to avoid that path, modifying the Taglet.init(…) specification the way you rephrased would help a lot. Alternatively, another way to work without any specification change may be to allow StandardDoclet.run(...) to accept arbitrary DocletEnvironment without throwing ClassCastException. Regards, Martin Le 18/04/2018 à 18:37, Jonathan Gibbons a écrit : > Martin, > > You are (regrettably) on the bleeding edge here with this use of the > recent new Doclet API. > > I would somewhat rephrase your first bullet, to reflect the intent of > the API > > * Whatever the doclet, the doclet given in Taglet.init(...) method > should be the user doclet, whether specified explicitly or by default > > Given that redefinition, the behavior you describe can reasonably be > described as a bug. This would require a minor spec change of the > doclet parameter: "doclet - the doclet that instantiated this taglet" > > It is true that there is no public API (except for option support) on > StandardDoclet to access some of the features of the standard doclet. > And, I don't think we should go all the way with methods to access all > the individual features. But, there is precedent in other similar APIs > for adding methods to provide instances of objects that might > otherwise be created in problematic classloaders. The most notable > case is the ability to provide instances of annotation processors to > javac [1] > The problem in this case would be the specification of such a method. > While it would be easy to write "Adds a custom tag." the order of the > calls has a visible impact on the generated documentation that would > be harder to specify and/or provide full control. > > -- Jon > > [1] > https://docs.oracle.com/javase/9/docs/api/javax/tools/JavaCompiler.CompilationTask.html#setProcessors-java.lang.Iterable- > > >
Re: Public API for programmatic registration of Taglet?
Yes, it is on my list to dig into the details of why the ClassCastException is being thrown. -- Jon On 4/18/18 9:53 AM, Martin Desruisseaux wrote: Hello Jonathan Thanks for your reply. Indeed, I understand that adding new API can be tricky. If it is desired to avoid that path, modifying the Taglet.init(…) specification the way you rephrased would help a lot. Alternatively, another way to work without any specification change may be to allow StandardDoclet.run(...) to accept arbitrary DocletEnvironment without throwing ClassCastException. Regards, Martin Le 18/04/2018 à 18:37, Jonathan Gibbons a écrit : Martin, You are (regrettably) on the bleeding edge here with this use of the recent new Doclet API. I would somewhat rephrase your first bullet, to reflect the intent of the API * Whatever the doclet, the doclet given in Taglet.init(...) method should be the user doclet, whether specified explicitly or by default Given that redefinition, the behavior you describe can reasonably be described as a bug. This would require a minor spec change of the doclet parameter: "doclet - the doclet that instantiated this taglet" It is true that there is no public API (except for option support) on StandardDoclet to access some of the features of the standard doclet. And, I don't think we should go all the way with methods to access all the individual features. But, there is precedent in other similar APIs for adding methods to provide instances of objects that might otherwise be created in problematic classloaders. The most notable case is the ability to provide instances of annotation processors to javac [1] The problem in this case would be the specification of such a method. While it would be easy to write "Adds a custom tag." the order of the calls has a visible impact on the generated documentation that would be harder to specify and/or provide full control. -- Jon [1] https://docs.oracle.com/javase/9/docs/api/javax/tools/JavaCompiler.CompilationTask.html#setProcessors-java.lang.Iterable-
Re: Public API for programmatic registration of Taglet?
OK, we can't easily accept arbitrary subtypes of DocletEnvironment, without embarking on a bunch of changes to other public API. This means the correct solution here is that Taglet.init should be called with the "user doclet" (which we expect people to provide subtypes of) and the system-provided DocletEnvironment (which we do not expect people to subtype.) The spec change here (for the @param doclet) is not a material change; it is a clarification of the original intent. We just need to think of the best words to describe the usage without getting into the internal details of StandardDoclet delegating to HtmlDoclet. -- Jon On 4/18/18 10:50 AM, Jonathan Gibbons wrote: Yes, it is on my list to dig into the details of why the ClassCastException is being thrown. -- Jon On 4/18/18 9:53 AM, Martin Desruisseaux wrote: Hello Jonathan Thanks for your reply. Indeed, I understand that adding new API can be tricky. If it is desired to avoid that path, modifying the Taglet.init(…) specification the way you rephrased would help a lot. Alternatively, another way to work without any specification change may be to allow StandardDoclet.run(...) to accept arbitrary DocletEnvironment without throwing ClassCastException. Regards, Martin Le 18/04/2018 à 18:37, Jonathan Gibbons a écrit : Martin, You are (regrettably) on the bleeding edge here with this use of the recent new Doclet API. I would somewhat rephrase your first bullet, to reflect the intent of the API * Whatever the doclet, the doclet given in Taglet.init(...) method should be the user doclet, whether specified explicitly or by default Given that redefinition, the behavior you describe can reasonably be described as a bug. This would require a minor spec change of the doclet parameter: "doclet - the doclet that instantiated this taglet" It is true that there is no public API (except for option support) on StandardDoclet to access some of the features of the standard doclet. And, I don't think we should go all the way with methods to access all the individual features. But, there is precedent in other similar APIs for adding methods to provide instances of objects that might otherwise be created in problematic classloaders. The most notable case is the ability to provide instances of annotation processors to javac [1] The problem in this case would be the specification of such a method. While it would be easy to write "Adds a custom tag." the order of the calls has a visible impact on the generated documentation that would be harder to specify and/or provide full control. -- Jon [1] https://docs.oracle.com/javase/9/docs/api/javax/tools/JavaCompiler.CompilationTask.html#setProcessors-java.lang.Iterable-
Re: Public API for programmatic registration of Taglet?
Tracked as JDK-8201817 https://bugs.openjdk.java.net/browse/JDK-8201817 -- Jon On 4/18/18 11:39 AM, Jonathan Gibbons wrote: OK, we can't easily accept arbitrary subtypes of DocletEnvironment, without embarking on a bunch of changes to other public API. This means the correct solution here is that Taglet.init should be called with the "user doclet" (which we expect people to provide subtypes of) and the system-provided DocletEnvironment (which we do not expect people to subtype.) The spec change here (for the @param doclet) is not a material change; it is a clarification of the original intent. We just need to think of the best words to describe the usage without getting into the internal details of StandardDoclet delegating to HtmlDoclet. -- Jon On 4/18/18 10:50 AM, Jonathan Gibbons wrote: Yes, it is on my list to dig into the details of why the ClassCastException is being thrown. -- Jon On 4/18/18 9:53 AM, Martin Desruisseaux wrote: Hello Jonathan Thanks for your reply. Indeed, I understand that adding new API can be tricky. If it is desired to avoid that path, modifying the Taglet.init(…) specification the way you rephrased would help a lot. Alternatively, another way to work without any specification change may be to allow StandardDoclet.run(...) to accept arbitrary DocletEnvironment without throwing ClassCastException. Regards, Martin Le 18/04/2018 à 18:37, Jonathan Gibbons a écrit : Martin, You are (regrettably) on the bleeding edge here with this use of the recent new Doclet API. I would somewhat rephrase your first bullet, to reflect the intent of the API * Whatever the doclet, the doclet given in Taglet.init(...) method should be the user doclet, whether specified explicitly or by default Given that redefinition, the behavior you describe can reasonably be described as a bug. This would require a minor spec change of the doclet parameter: "doclet - the doclet that instantiated this taglet" It is true that there is no public API (except for option support) on StandardDoclet to access some of the features of the standard doclet. And, I don't think we should go all the way with methods to access all the individual features. But, there is precedent in other similar APIs for adding methods to provide instances of objects that might otherwise be created in problematic classloaders. The most notable case is the ability to provide instances of annotation processors to javac [1] The problem in this case would be the specification of such a method. While it would be easy to write "Adds a custom tag." the order of the calls has a visible impact on the generated documentation that would be harder to specify and/or provide full control. -- Jon [1] https://docs.oracle.com/javase/9/docs/api/javax/tools/JavaCompiler.CompilationTask.html#setProcessors-java.lang.Iterable-
Re: Public API for programmatic registration of Taglet?
This is fine, thanks for investigating. Would the following implementation adjustment be also acceptable? * If the Doclet and the Taglet are defined in the same JAR file, then they share the same URLClassLoader. I'm not 100% sure that this is the right thing to do. But it would be helpful (while not indispensable) in the case that I described (doclet and taglet sharing common custom data structures). Martin Le 18/04/2018 à 20:39, Jonathan Gibbons a écrit : > OK, we can't easily accept arbitrary subtypes of DocletEnvironment, > without embarking on a bunch of changes to other public API. > > This means the correct solution here is that Taglet.init should be > called with the "user doclet" (which we expect people to provide > subtypes of) and the system-provided DocletEnvironment (which we do > not expect people to subtype.) > > The spec change here (for the @param doclet) is not a material change; > it is a clarification of the original intent. We just need to think of > the best words to describe the usage without getting into the internal > details of StandardDoclet delegating to HtmlDoclet. > > -- Jon
Re: Public API for programmatic registration of Taglet?
In the general case, they cannot share classloaders ... 1. If you're loading user-defined taglets into the system-defined Standard Doclet, they probably should not be in the same class loader. 2. The user may use the -tagletpath option to specify a different searchpath for taglets. You did qualify your question about whether they are defined in the same JAR file, but that raises a chicken-and-egg question, and still doesn't address the corner-case of differently ordered taglet paths and doclet paths. Ideally, I'd like to say that the taglet classloader should be one that delegates to the doclet classloader, but that does not appear to be practical without fixing some shortcomings elsewhere in other API (JavaFileManager.getClassLoader). I suspect the best answer will be to add the method you suggested earlier, to register a Taglet directly with the custom doclet, to side-step the classloader issues. That will need a bit more investigation. -- Jon On 4/18/18 3:24 PM, Martin Desruisseaux wrote: This is fine, thanks for investigating. Would the following implementation adjustment be also acceptable? * If the Doclet and the Taglet are defined in the same JAR file, then they share the same URLClassLoader. I'm not 100% sure that this is the right thing to do. But it would be helpful (while not indispensable) in the case that I described (doclet and taglet sharing common custom data structures). Martin Le 18/04/2018 à 20:39, Jonathan Gibbons a écrit : OK, we can't easily accept arbitrary subtypes of DocletEnvironment, without embarking on a bunch of changes to other public API. This means the correct solution here is that Taglet.init should be called with the "user doclet" (which we expect people to provide subtypes of) and the system-provided DocletEnvironment (which we do not expect people to subtype.) The spec change here (for the @param doclet) is not a material change; it is a clarification of the original intent. We just need to think of the best words to describe the usage without getting into the internal details of StandardDoclet delegating to HtmlDoclet. -- Jon
