I help maintain a few service-loader-based modular annotation processor libraries with the avaje framework, so I have also run into this and had to work around it.
*Ad 1:* So this is indeed an inconvenience but in my experience, it's not a crazy one. When processing is over we check the module-info's directives to throw compilation warnings like: *[ERROR] /M:/Dev/avaje-helidon-nima-api-example/src/main/java/module-info.java:[1,1] Missing `provides io.ava* *je.http.client.HttpClient.GeneratedComponent with com.jojo.helidon.api.client.httpclient.GeneratedHttpCompon* *ent;`* *[ERROR] /M:/Dev/avaje-helidon-nima-api-example/src/main/java/module-info.java:[1,1] Missing `provides io.ava* *je.jsonb.Jsonb.GeneratedComponent with com.jojo.helidon.api.jsonb.GeneratedJsonComponent;`* *[ERROR] /M:/Dev/avaje-helidon-nima-api-example/src/main/java/module-info.java:[1,1] Missing `provides io.ava* *je.validation.Validator.GeneratedComponent with com.jojo.helidon.api.controller.valid.GeneratedValidatorComp* *onent;`* *[ERROR] /M:/Dev/avaje-helidon-nima-api-example/src/main/java/module-info.java:[1,1] Missing "provides io.ava* *je.inject.spi.Module with com.jojo.helidon.api.ApiModule;"* The *real problem *with checking the module info in a processor is that ModuleElement is bugged such that if you call *ModuleElement.getDirectives* on the project module It breaks compilation <https://bugs.openjdk.org/browse/JDK-8315125>. (they called this one a duplicate but I still have the problem even in JDK 22-ea and JDK 23-ea) To get around this I have to use Filer to retrieve the module-info's sources file and parse it as a string to avoid calling *getDirectives. * I've built libraries to do this to make this easier, but the problem remains that if any other processor uses *getDirectives*, compilation will still break. *Ad 2:* I asked this question here a while back, and what I got out of it is that we needed to do some form of circular dependency if we truly wanted optional services, the tech for compiling multi-module jars isn't here yet. It's a pain to deploy when we change the plugins, but we rarely do so this has worked fine for us. Example: avaje-validator <https://github.com/avaje/avaje-validator> provides a plugin for avaje inject that itself depends on avaje-validator. *Ad 3: * Yeah I got nothing, we never had to do this. *Ad 4: * This one was the simplest, we can define an annotation to go on the user's service class and process them to generate META-INF files and validate the module-info. In this way, one cannot forget to add the proper module-info information. In some of the libs we do this with their processors, but we also have a dedicated library for handling this sort of thing. On Fri, Jan 19, 2024 at 8:04 AM Tomas Langer <tomas.lan...@oracle.com> wrote: > Helidon currently has around 300 modules with module-info.java. In > general, this has improved our module structure and design. > Yet, we are now encountering some major issues related to extensibility. > I will put down a few points that are problematic, and explain each in > detail further in the e-mail (it is quite long, sorry about that). > > 1. provider implementations cannot be code generated without major problems > 2. the provider interface module MUST be on module path, even if it could > have `requires static` > 3. the provider implementation must be public with public constructor > 4. duality of definition between module path and class path > > I am trying to propose a solution within the bounds of the current service > loader design. Of course there may be other solutions (both with current > design, or even creating a brand new extensibility solution in Java, this > is just to illustrate it). > > The first two issues are quite major, as they force us to recommend not to > use JPMS to our users... > > We could design our own extensibility approach, though that would require > the use of reflection to instantiate the services, and we would have to > live with the limits of the module system (where Java ServiceLoader works > around a few of them). > I feel the right way is to use what Java provides, so I would welcome any > help (and possibly changes in the language) to support our use cases. > > Thanks, > Tomas Langer > Architect, project Helidon > > > *Ad 1 - Code generation* > ------------------------------------- > Problem: We cannot code generate service implementations (well we can, but > the user must handcraft them in module-info.java, so we end up running the > APT, generating a service, failing the compiler to tell the user to add the > service, compiling again every time a new service is added). > Possible solution: Provide extensibility to module-info.java that can be > code generated > Without JPMS: it just works, as `META-INF/services` files can be code > generated without issues > > Details: > What I do not see is how we are supposed to do extensibility through > annotation processing. > There are a lot of usecases for this, such as: > - generating code for serializers/deserializers for objects that persist > to JSON, XML, YAML > - generating code for database entities > - generating descriptor for services in a service registry > > We actually want to implement these three use cases, and it is a major > pain for the user - I would not mind much if this hurt us, as framework > developers, but we must force the user to take action by breaking the > compilation, or come up with some really weird solution (such as source > code modification using some preprocessor before compilation, or > postprocessor running on bytecode to re-generate module-info.class) > > > *Ad 2 - Provider module cannot be optional dependency* > ------------------------------------- > Problem: We cannot declare `requires static` on a module that has the > ServiceLoader provider interface (or abstract class) > Possible solution: Change the rules for JPMS to allow this > Without JPMS: it just works, as `META-INF/services` to not impose any > classpath structure > > Details: > If a module (my.json) defines a provider interface (let's say > `JsonSerializer`), and I create a module with `MyJsonSerializer` (provides > JsonSerializer witih MyJsonSerializer), currently I MUST do the following > "requires my.json". If I do a "requires static my.json" I fail to start the > JVM if that module is not on module path. > > The service CANNOT be used unless the module is on module path (as anybody > attempting to load it must declare `uses` in their module info with a > proper `requires` on the `my.json` module. > > So what are my options right now? > - have a "requires my.json" and just dump the module on all my users (not > so good - people may want to use my library without JSON altogether, I may > also provide support for XML, YAML - all of these would need to be on > module path) > - create a module for each (one for JSON, XML, YAML + my library) - > resulting in 4 modules (and this may grow if I decide to support other > format); this looks kind of OK on the first look, but with the number of > modules we have, and the number of features we support, this gets out of > hand really really quickly; this approach is also very user unfriendly, as > now the user needs to understand 4 modules instead of just 1, and use the > right ones at the right time). > - considering the number of modules we already have, this would make our > project unmaintainable (and unusable for users) > > As JPMS already allows "static" dependencies, there should be no reason > not to allow it in this case as well. We can break the module system even > now - just use a class from static dependency in a public class - this will > fail at runtime only. The service loader is not different (this is a > reaction to text in https://bugs.openjdk.org/browse/JDK-8299504). > > > *Ad 3 - Provider implementation must be public with public constructor* > ------------------------------------- > Problem: This creates a new public API that we may not want to expose, or > document > Possible solution: Change the rules to allow package local service > provider implementations with package local constructors > Without JPMS: Same issue, even more problematic as there is no > restrictions on package visibility > > Details: > Provider implementations are not supposed to be visible to users - they > are not public API of my module (the fact that I provide a service is part > of my public API). > Right now there is only one option to work around this, and it only works > in JPMS, and in my opinion it brings in even more problems - put the > provider implementation in an un-exported package. > The problem with this approach is that now the provider implementation > MUST use only public methods of my module, thus creating even more public > APIs, where if I just put it in my exported package, I can use package > private methods of my other classes to implement the service (so I pay the > price of having one public class with one public constructor agains > multiple public classes and public methods). Also the "hiding" in > unexported package is lost when on classpath anyway... > > > *Ad 4 - Duality of definition between classpath and module path* > ------------------------------------- > Problem: To support services, we MUST declare them twice - once in > `provides` in module-info.java, once through META-INF/services > Possible solution: Java could read module descriptors even when running in > classpath mode to add service implementations and merge it with > META-INF/services information > Without JPMS: it just works, as `META-INF/services` is always honored on > classpath and for non-JPMS modules on module path > > Details: > This is again quite a pain for us as framework develoepers, and a pitfall > for users. When we started with JPMS, we had both created manually, which > obviously ended in a huge inconsistent mess. > So now we have a custom Maven plugin, that creates META-INF/services files > based on the content in module-info.java and fails on inconsistencies. > I do not consider this a nice solution for us, and definitely not for end > users. Also there is no way to find out that you forgot to add one (or the > other), as JVM just does not care. So basically you end up with a runtime > issue that is really hard to troubleshoot. >
