P.S. To be more constructive, one idea that was floated in this discussion is to use a small number of layers. Perhaps one for the container and one for each user application.
You claimed that doing so would be inefficient (and that that is the reason, or one reason, why you opted for the layer-per-module design). Thing is that we do need to support many modules per layer efficiently, and since there’s currently much work focused on improving startup time, knowing if there’s a problem there would be very useful. So if there’s a problem there, we’ll need to address it anyway, and if we do, then wouldn’t it also address yours? This way there is no addition of spec complexity, and many more usecases can benefit. > On 16 Dec 2024, at 22:39, Ron Pressler <ron.press...@oracle.com> wrote: > > > >> On 16 Dec 2024, at 19:38, David Lloyd <david.ll...@redhat.com> wrote: >> >> >> If you look back at the old jigsaw-dev archives, you'll see my name, and >> those of my colleagues, quite frequently. I am well aware of the intent of >> this system and how it was implemented. We do not (and you should not) >> consider these specifications and their implementations to be holy writ. >> They are in fact no different from those of any other project, specifically >> that they can and should be changed as needed. I am expressing such a need. > > Yes, and now I think I understand what it is :) Now that we understand the > problem, we could contemplate solutions. > > >> >> I'm asking for the OpenJDK team to instead consider a different design for >> service loading in the Java platform itself, in the absence of an >> alternative solution which can use what exists today. Unfortunately, the >> constraints around the JPMS are such that I do not believe there is another >> design that will generally work for us. > > I understand what solution you’re proposing, but as with any feature design, > the severity of the problem and its root causes need to be understood before > settling on a specific solution. > > The issue you’re facing arises when making a particular design choice: a > container that loads every module in a separate layer and with multiple > parents. We need to ask ourselves, how common is this design, why do people > pick it (yes, you explained why you picked it, but we’ll need to study > alternatives), and are we expecting the number of such use cases to grow or > shrink over time? > > >> I do believe that there are possible workarounds which belie the conceit >> that the encapsulation of the module system is as strong as you present it; >> the existence of `ModuleLayer.Controller` itself shows that it is nowhere >> near absolute. This level of encapsulation is much more superficial than, >> say, member access control, which is to say that it can be worked around in >> various ways without "breaking the rules" (for example, by having a class >> loader define generated helper classes in every package of a module, I can >> gain access to their original Lookup objects, breaking *any* form modular of >> encapsulation without breaking any of the rules of the platform >> specification). If you would recoil at such a thing, consider again the >> difference between "intent" and "specification". > > I would be interested to hear more about this, because our assumption is that > the encapsulation — once integrity by default is done, i.e. there is no > hidden use of Unsafe or JNI etc. — is, in fact, absolute (modulo bugs), to > the point that finally the runtime will be able to actually trust Java’s > invariants. Both the security of the platform and the correctness of the > compilers depend on it, so if you think we’re wrong — this is important. You > mention ModuleLayer.Controller, but I don’t see how it undermines anything > (in particular notice how the method that enables a capability with > potentially global impact — native access — is caller-sensitive). As an > example, how would library use it to mutate a string (something that was > trivial before strong encapsulation and the JNI restriction)? > >> >> I can tell you that several design choices of the JPMS had nothing to do >> with encapsulation principles and everything to do with the philosophy of >> those who created the constraints. There's nothing inherent in the system >> that makes it necessary to prevent circularity, *or* to eagerly load all >> modules in a layer, *or* to eagerly resolve services. > > True, but strong encapsulation is not the only benefit this feature provides > and not every aspect of it serves strong encapsulation. > >> there are real implications to these choices that mean that many kinds of >> Java application containers cannot be satisfactorily migrated to use JPMS >> modules. I hope you recognize that I am trying to change that, but that >> doing so may require some form of compromise from the platform itself. > > I presume you’re trying to use the feature because you want to enjoy the > benefits it offers now and in the future. Obviously, we want all our features > to be as easy to use to maximise the value users get from them, but we also > need to balance the cost of additional complexity and the utility it adds to > the ecosystem as a whole. So we can consider that, and you can consider if > there really is no acceptable design for a container other than loading every > module in its own layer and constructing a complex layer graph. Maybe it > turns out that *both* of these things are true, i.e. it would be a good idea > to change ServiceLoader and also your design. > >> >> Yes. Quarkus currently does not use JPMS modules for the user application at >> build time or in any deployment mode, both of which instead rely on an >> arrangement of specialized class loaders, with all classes ending up in flat >> unnamed modules. My current experiments revolve around trying to create more >> modular-oriented and encapsulated packaging options. > > So you want Quarkus to also load one module per layer? > > — Ron
