Re: Relationship to JSR 291 [was: Re: Bryan's comments]
Stanley M. Ho wrote:
Hi,
Glyn Normington wrote:
*Bryan Atsatt <[EMAIL PROTECTED]>* wrote on 30/05/2007 07:57:59:
...
> So the open issue is the richness of the import "language": must we
> support only lowest-common-denominator, or can we do better without
> over-complicating the design?
>
> I for one would like to be able to have a single module express
> dependencies on modules from both the same and different
ModuleSystems,
> *using the standard semantics of each*. This may be reaching too far,
> but we should at least explore it seriously while we figure out what
> interop means here...
At this point, I feel that is likely to be reaching too far, but I'm
happy to play along and see what we can learn along the way.
I agreed with Glyn that this might be reaching too far. Before we dive
too much into how to implement interoperability, I think one of the
outstanding questions we should answer first is what degree of import
interoperability we want to offer. There are four possibilities:
1. JSR 277 module imports OSGi module by module name
2. OSGi module imports JSR 277 module by module name
3. OSGi module imports JSR 277 module by package name
4. JSR 277 module imports OSGi module by package name
Let's ignore the 294 issues and the module initialization issues for now
to simplify this discussion.
I think we all agreed #1 and #2 are important to support and the
reflective APIs already enable these (of course, there are minor issues
we still have to address as we evolve the APIs.)
That said, it is unclear to me how important it is to support #3 and #4,
so I think the first question for this EG is whether we want to support
#3 and #4 at all. However, let's pretend #3 and #4 are important for
this specific discussion.
I think #3 is already possible since the OSGi framework could look up
the appropriate module which exported the package in the repository,
using the reflective APIs. Afterwards, the OSGi framework can then do
the necessary wiring. Yes, the reflective APIs can be further refined to
make it easier for the OSGi framework to query modules by exported
package name from the repository, but this is a minor issue that we can
address easily.
#4 is a bit complicated, because there is no import-by-package semantic
in the current module layer's APIs and I think we all agreed that we
don't want to support this semantic directly in the module system
defined by JSR 277.
Not me.
I do understand your point that this is more difficult, but I don't
really care--it is our job to do the Right Thing here. Sure, if it is
not feasible in the time frame we have, then so be it. But we haven't
even tried!
If we're forced to make a choice between the two models, I'd pick
import-by-package with zero hesitation.
OSGi has a lot of experience here, and we shouldn't ignore it.
Import-by-name was added late in the game, and is now seriously
downplayed, for good reason.
And Oracle has a lot of relevant experience as well, since the shared
loader mechanism I created for our AS stack ("shared libraries")
supports only the import-by-name model. We have *lots* of
shared-libraries (I've seen 100+ running simultaneously), used by many
different components in the stack, in addition to customer applications.
It is a tremendous pain when refactoring or even simple renaming has to
be done (I hope to shift this implementation onto 277 to solve this
problem, rather than inventing an interim approach). And this kind of
change happens far more frequently than you probably expect.
I don't want to repeat the same mistake here. Nor do I think that
leaving a long trail of extraneous "view" modules behind for
compatibility is a good solution.
Worse still, if the 277 APIs don't support import-by-package, then the
compiler won't support it either. If the compiler doesn't support it,
then we've lost a golden opportunity to move to a better world, one in
which packaging is irrelevant.
We're supposed to be eliminating "jar-hell" here. Let's not simply
replace it with "module-hell".
In this context, the question is really about what
it would take to make #4 possible if we want to support it, and so far
there are two different approaches we have discussed:
a. Treat import-by-package separately from import-by-module in the
reflective APIs and make the module layer fully aware of the
import-by-package concept.
b. Expose exported-package using the ModuleDefinition abstraction, and
provide necessary hooks (e.g. consistency checking) for the module
system to resolve this kind of dependency appropriately. The module
layer is not aware of the import-by-package concept at all. New import
dependency granularity could also be introduced in a similar manner in
the future without requiring significant changes in the module layer.
As Richard and I discussed in previous emails, it seems to make the most
sense for JSR 277 to define the minimal set of features possible in the
module layer to accomplish what needs to be accomplished to
Re: Relationship to JSR 291 [was: Re: Bryan's comments]
On May 30, 2007, at 5:55 PM, Stanley M. Ho wrote: Glyn Normington wrote: *Bryan Atsatt <[EMAIL PROTECTED]>* wrote on 30/05/2007 07:57:59: ... > So the open issue is the richness of the import "language": must we > support only lowest-common-denominator, or can we do better without > over-complicating the design? > > I for one would like to be able to have a single module express > dependencies on modules from both the same and different ModuleSystems, > *using the standard semantics of each*. This may be reaching too far, > but we should at least explore it seriously while we figure out what > interop means here... At this point, I feel that is likely to be reaching too far, but I'm happy to play along and see what we can learn along the way. I agreed with Glyn that this might be reaching too far. Before we dive too much into how to implement interoperability, I think one of the outstanding questions we should answer first is what degree of import interoperability we want to offer. There are four possibilities: 1. JSR 277 module imports OSGi module by module name 2. OSGi module imports JSR 277 module by module name 3. OSGi module imports JSR 277 module by package name 4. JSR 277 module imports OSGi module by package name Let's ignore the 294 issues and the module initialization issues for now to simplify this discussion. I think we all agreed #1 and #2 are important to support and the reflective APIs already enable these (of course, there are minor issues we still have to address as we evolve the APIs.) That said, it is unclear to me how important it is to support #3 and #4, so I think the first question for this EG is whether we want to support #3 and #4 at all. However, let's pretend #3 and #4 are important for this specific discussion. I think #3 is already possible since the OSGi framework could look up the appropriate module which exported the package in the repository, using the reflective APIs. Afterwards, the OSGi framework can then do the necessary wiring. Yes, the reflective APIs can be further refined to make it easier for the OSGi framework to query modules by exported package name from the repository, but this is a minor issue that we can address easily. I am not so sure supporting #3 is as straight forward as you suggest here. Specially, the OSGi modularity layer [for the most part] assumes that packages are atomic units; there is no way to specify that your export or import something smaller than a package. Since 277 modules declare exports in terms of classes, it is not clear how we would know if a 277 module was exporting an entire package. This would, at a minimum, require that we assume we can inspect the contents of the module and see that the contents of a contained package matches the module's export signature for that package. -> richard #4 is a bit complicated, because there is no import-by-package semantic in the current module layer's APIs and I think we all agreed that we don't want to support this semantic directly in the module system defined by JSR 277. In this context, the question is really about what it would take to make #4 possible if we want to support it, and so far there are two different approaches we have discussed: a. Treat import-by-package separately from import-by-module in the reflective APIs and make the module layer fully aware of the import-by-package concept. b. Expose exported-package using the ModuleDefinition abstraction, and provide necessary hooks (e.g. consistency checking) for the module system to resolve this kind of dependency appropriately. The module layer is not aware of the import-by-package concept at all. New import dependency granularity could also be introduced in a similar manner in the future without requiring significant changes in the module layer. As Richard and I discussed in previous emails, it seems to make the most sense for JSR 277 to define the minimal set of features possible in the module layer to accomplish what needs to be accomplished to provide core modularity support in the Java platform. I think (b) follows this principle nicely while (a) does not. If we decide to support #3 and #4, then how we want to address #4 would impact our overall design significantly. Is there anyone in the EG provide good reasons why #3 and #4 are important to be supported? If so, is there anyone in the EG think (b) is not feasible at all and we should do (a) instead to support #4? Can you explain your rationale? > RESOLUTION MODELS > > The current design requires that each ModuleSystem provide its own > resolution logic, and that each definition will be resolved by its > owning ModuleSystem. This model appears to provide flexibility for > significant differences in implementation, but we really don't know > enough at this point. Perhaps only an actual second implementation will > tell us if this provides useful flexibility. In the existing module systems we are aware, e.g. OSGi and NetBeans, we alread
Re: Exported resources
Hey Stanley, Sorry to be generating so much traffic while you're traveling! I'm not in any rush here, so feel free to take your time responding... Stanley M. Ho wrote: Hi Bryan, A module can use the ResourceBundle API to retrieve resources from other resource modules, but it can also use the ResourceBundle API to retrieve resources from the (target) module itself. In the latter case, the resources are currently not required to be exported from the target module. I don't think we want to force a module to export its own private resources simply because it wants to use the ResourceBundle API in this case, would you agree? No, I don't agree. That's what I said before :^). I *really* don't like the idea that I'm going to have to explicitly grant permission to some module to access my resource. This *significantly* complicates deployment. How will the grant occur, especially given that each environment can use an entirely different permission storage mechanism? Or are you thinking that the loaders in the module system will construct ProtectionDomains pre-wired with permissions? If so, exactly which entities will be blessed? Just the JRE code? What about all the existing non-JRE frameworks in use? Regarding there are differences in the access models between classes and resources, I also prefer symmetry if possible. However, the access model for classes and resources have always been different, and there are built-in JVM support for classes while there is none for resources, so it is unclear if we can completely eliminate this asymmetry after all. Sure, resources have never had JVM access control applied to them. But I don't think we should go to the other extreme (permissions) just so that I can give "friend" access to private resources. I'd much prefer that such resources simply be exported. And I don't see this as a particular problem, either. We've gotten along just fine without private resources so far (or maybe there is some big demand for this that I'm missing?). And what about the mismatch issue I brought up? Lots of existing frameworks use the "services" pattern, in which: 1. A well known resource name is used in a getResource() call. 2. The resource contains a string naming a provider class. 3. The provider class is loaded using Class.forName(). This is a simple but very powerful pattern, but it requires that *both* the resource and the named class be accessible to the framework. If we require a permission grant for the resource, but not for the class, it will be very easy to get access to one and not the other. Is it really worth opening up this potential headache to enable a friend model for resources? // Bryan - Stanley Bryan Atsatt wrote: I've been assuming that Module private resources should not be visible to *any* class outside of the module. Including ResourceBundle, or any other existing framework classes that do resource lookups (e.g. ServiceLoader, JSF, etc). If resources need to be visible to these existing classes, they must be exported. The very simple check I proposed (immediate caller) is sufficient to make this assertion. I believe your point is that if we used the permission model instead, it would become possible for a module to invoke an external class (e.g. ResourceBundle.getBundle()) and enable *it* to successfully load a private resource from the module. Aside from the permission *grant* mechanism this model would rely on, it is an entirely different model than that used for classes! (Though we haven't explicitly defined this in 294, it seems extremely unlikely that we will rely on permissions--none of the other access modes do so.) Such asymmetry is very disconcerting to me, and, I believe, just plain wrong... Consider that you could grant the ServiceLoader, for example, access to a resource that names a class that it could not instantiate. That class would have to be exported. I believe the resource should be as well. // Bryan
Re: Service providers
Hi Richard, Richard S. Hall wrote: Hey Stanley, ... To me, this sounds like what we call the "extender model" for the OSGi framework. The way it works is that bundles that want to participate in certain scenarios simply include some metadata inside of themselves, then some infrastructure can probe for this metadata and automatically do work on their behalf. This is how Declarative Services works in the R4 spec and also how Spring-OSGi works...it is becoming a very common model. The benefit of this approach is that it doesn't push upper layer concepts down into the modularity layer and keeps it simple. Off the top of my head, I don't see a reason why the Service Loader couldn't use a similar approach. -> richard Thanks. I will certainly check it out to see if this approach is feasible. - Stanley
Re: Exported resources
Hi Bryan, A module can use the ResourceBundle API to retrieve resources from other resource modules, but it can also use the ResourceBundle API to retrieve resources from the (target) module itself. In the latter case, the resources are currently not required to be exported from the target module. I don't think we want to force a module to export its own private resources simply because it wants to use the ResourceBundle API in this case, would you agree? Regarding there are differences in the access models between classes and resources, I also prefer symmetry if possible. However, the access model for classes and resources have always been different, and there are built-in JVM support for classes while there is none for resources, so it is unclear if we can completely eliminate this asymmetry after all. - Stanley Bryan Atsatt wrote: I've been assuming that Module private resources should not be visible to *any* class outside of the module. Including ResourceBundle, or any other existing framework classes that do resource lookups (e.g. ServiceLoader, JSF, etc). If resources need to be visible to these existing classes, they must be exported. The very simple check I proposed (immediate caller) is sufficient to make this assertion. I believe your point is that if we used the permission model instead, it would become possible for a module to invoke an external class (e.g. ResourceBundle.getBundle()) and enable *it* to successfully load a private resource from the module. Aside from the permission *grant* mechanism this model would rely on, it is an entirely different model than that used for classes! (Though we haven't explicitly defined this in 294, it seems extremely unlikely that we will rely on permissions--none of the other access modes do so.) Such asymmetry is very disconcerting to me, and, I believe, just plain wrong... Consider that you could grant the ServiceLoader, for example, access to a resource that names a class that it could not instantiate. That class would have to be exported. I believe the resource should be as well. // Bryan
Re: Relationship to JSR 291 [was: Re: Bryan's comments]
Hi, Glyn Normington wrote: *Bryan Atsatt <[EMAIL PROTECTED]>* wrote on 30/05/2007 07:57:59: ... > So the open issue is the richness of the import "language": must we > support only lowest-common-denominator, or can we do better without > over-complicating the design? > > I for one would like to be able to have a single module express > dependencies on modules from both the same and different ModuleSystems, > *using the standard semantics of each*. This may be reaching too far, > but we should at least explore it seriously while we figure out what > interop means here... At this point, I feel that is likely to be reaching too far, but I'm happy to play along and see what we can learn along the way. I agreed with Glyn that this might be reaching too far. Before we dive too much into how to implement interoperability, I think one of the outstanding questions we should answer first is what degree of import interoperability we want to offer. There are four possibilities: 1. JSR 277 module imports OSGi module by module name 2. OSGi module imports JSR 277 module by module name 3. OSGi module imports JSR 277 module by package name 4. JSR 277 module imports OSGi module by package name Let's ignore the 294 issues and the module initialization issues for now to simplify this discussion. I think we all agreed #1 and #2 are important to support and the reflective APIs already enable these (of course, there are minor issues we still have to address as we evolve the APIs.) That said, it is unclear to me how important it is to support #3 and #4, so I think the first question for this EG is whether we want to support #3 and #4 at all. However, let's pretend #3 and #4 are important for this specific discussion. I think #3 is already possible since the OSGi framework could look up the appropriate module which exported the package in the repository, using the reflective APIs. Afterwards, the OSGi framework can then do the necessary wiring. Yes, the reflective APIs can be further refined to make it easier for the OSGi framework to query modules by exported package name from the repository, but this is a minor issue that we can address easily. #4 is a bit complicated, because there is no import-by-package semantic in the current module layer's APIs and I think we all agreed that we don't want to support this semantic directly in the module system defined by JSR 277. In this context, the question is really about what it would take to make #4 possible if we want to support it, and so far there are two different approaches we have discussed: a. Treat import-by-package separately from import-by-module in the reflective APIs and make the module layer fully aware of the import-by-package concept. b. Expose exported-package using the ModuleDefinition abstraction, and provide necessary hooks (e.g. consistency checking) for the module system to resolve this kind of dependency appropriately. The module layer is not aware of the import-by-package concept at all. New import dependency granularity could also be introduced in a similar manner in the future without requiring significant changes in the module layer. As Richard and I discussed in previous emails, it seems to make the most sense for JSR 277 to define the minimal set of features possible in the module layer to accomplish what needs to be accomplished to provide core modularity support in the Java platform. I think (b) follows this principle nicely while (a) does not. If we decide to support #3 and #4, then how we want to address #4 would impact our overall design significantly. Is there anyone in the EG provide good reasons why #3 and #4 are important to be supported? If so, is there anyone in the EG think (b) is not feasible at all and we should do (a) instead to support #4? Can you explain your rationale? > RESOLUTION MODELS > > The current design requires that each ModuleSystem provide its own > resolution logic, and that each definition will be resolved by its > owning ModuleSystem. This model appears to provide flexibility for > significant differences in implementation, but we really don't know > enough at this point. Perhaps only an actual second implementation will > tell us if this provides useful flexibility. In the existing module systems we are aware, e.g. OSGi and NetBeans, we already know their resolution logics are different from each other. Even if they are migrated/re-based to JSR 277 in the future, I think we should expect their resolution logic would remain the same to maintain backward compatibility with their existing modules/bundles. In other words, their resolution logic would still be different from each other. A phased approach would be particularly beneficial if the initial phase could be delivered as part of Java 7 and subsequent phases implemented strictly on top of Java 7. But getting the API right up front might be tricky unless we can spot some really good abstractions or prototype the later phases sufficien
Re: Exported resources
I've been assuming that Module private resources should not be visible
to *any* class outside of the module. Including ResourceBundle, or any
other existing framework classes that do resource lookups (e.g.
ServiceLoader, JSF, etc). If resources need to be visible to these
existing classes, they must be exported. The very simple check I
proposed (immediate caller) is sufficient to make this assertion.
I believe your point is that if we used the permission model instead, it
would become possible for a module to invoke an external class (e.g.
ResourceBundle.getBundle()) and enable *it* to successfully load a
private resource from the module.
Aside from the permission *grant* mechanism this model would rely on, it
is an entirely different model than that used for classes! (Though we
haven't explicitly defined this in 294, it seems extremely unlikely that
we will rely on permissions--none of the other access modes do so.) Such
asymmetry is very disconcerting to me, and, I believe, just plain wrong...
Consider that you could grant the ServiceLoader, for example, access to
a resource that names a class that it could not instantiate. That class
would have to be exported. I believe the resource should be as well.
// Bryan
Stanley M. Ho wrote:
Hi Bryan,
Those resource-related methods in ClassLoader can be called by anyone,
including code that is part of the module, code that is from other
modules, or code that is part of the platform libraries (e.g.
ResourceBundle). The approach you described would require walking the
stack to get the caller's Module, but the real issue is that it is
difficult to determine who the actual caller is from the stack.
Treating the immediate caller on the stack as the actual caller wouldn't
be sufficient because the immediate caller could be called by someone
else who is the one actually making the call. On the other hand,
treating the originated caller on the stack as the actual caller would
be the right semantic, but this is basically the same as the security
permission approach.
- Stanley
Bryan Atsatt wrote:
Both solutions require stack walking (unless there is some new
implementation of the java security model I've not yet seen!).
The permission check does much more work than is necessary here. Take a
look at AccessController.checkPermission() to see what I mean.
And actually there is a very simple API to get the stack, which I've
used for years:
private static class StackAccessor extends SecurityManager {
public Class[] getStack() {
return getClassContext();
}
}
private static final STACK_ACCESSOR = new StackAccessor();
Now the enclosing class can simply call STACK_ACCESSOR.getStack().
// Bryan
Stanley M. Ho wrote:
Hi Bryan,
Bryan Atsatt wrote:
1. Definitely agree that resource search order should be identical to
class search order.
Glad to hear!
2. Using permissions to limit access to private resources seems like
overkill to me. The prototype implemented this in a very simple
fashion:
a. If resource is exported, return it, else
a. Get the caller's Module (get class from stack, get module from it)
b. If callerModule == this, return resource, else return null.
The issue is that this approach still requires stack walking and there
is no public API in the SE platform that let you implement this.
If stack walking is required for the check anyway, I think the security
permission approach is better that it is implementable with the existing
API in the SE platform.
- Stanley
Re: Service providers
Hey Stanley, Stanley M. Ho wrote: Richard S. Hall wrote: Hello Stanley, ... I guess the addition of service-related concepts into the module layer would be one example of feature creep for me. While I think it makes perfect sense to figure out how the Service Loader stuff will work on top of the module layer, pushing Service Loader concepts down into the module layer doesn't make any sense to me at all. It should be sufficient for the Service Loader to probe the installed modules and perhaps examine module metadata to determine if a module provides a service or not. From there, the Service Loader can create module instances and provider instances as necessary. If this is not possible in our current constructs, then we should address these shortcomings rather than adding higher layer concepts into the module layer. -> richard The current service provider strawman suggests a few changes in ModuleDefinition/Query classes, and you basically suggest that these changes could be eliminated if we have a more generic way to express services and service-providers in the module metadata and a generic way to examine the information from the module metadata, all without service-related APIs in the module layer. I think what you suggested makes sense, and I will look into this. To me, this sounds like what we call the "extender model" for the OSGi framework. The way it works is that bundles that want to participate in certain scenarios simply include some metadata inside of themselves, then some infrastructure can probe for this metadata and automatically do work on their behalf. This is how Declarative Services works in the R4 spec and also how Spring-OSGi works...it is becoming a very common model. The benefit of this approach is that it doesn't push upper layer concepts down into the modularity layer and keeps it simple. Off the top of my head, I don't see a reason why the Service Loader couldn't use a similar approach. -> richard
Service providers (was: Richard's comments)
Hi Richard, I have renamed the subject so we can discuss issues around the service-provider strawman in its own thread. Richard S. Hall wrote: Hello Stanley, ... I guess the addition of service-related concepts into the module layer would be one example of feature creep for me. While I think it makes perfect sense to figure out how the Service Loader stuff will work on top of the module layer, pushing Service Loader concepts down into the module layer doesn't make any sense to me at all. It should be sufficient for the Service Loader to probe the installed modules and perhaps examine module metadata to determine if a module provides a service or not. From there, the Service Loader can create module instances and provider instances as necessary. If this is not possible in our current constructs, then we should address these shortcomings rather than adding higher layer concepts into the module layer. -> richard The current service provider strawman suggests a few changes in ModuleDefinition/Query classes, and you basically suggest that these changes could be eliminated if we have a more generic way to express services and service-providers in the module metadata and a generic way to examine the information from the module metadata, all without service-related APIs in the module layer. I think what you suggested makes sense, and I will look into this. - Stanley
Re: Exported resources
Hi Bryan,
Those resource-related methods in ClassLoader can be called by anyone,
including code that is part of the module, code that is from other
modules, or code that is part of the platform libraries (e.g.
ResourceBundle). The approach you described would require walking the
stack to get the caller's Module, but the real issue is that it is
difficult to determine who the actual caller is from the stack.
Treating the immediate caller on the stack as the actual caller wouldn't
be sufficient because the immediate caller could be called by someone
else who is the one actually making the call. On the other hand,
treating the originated caller on the stack as the actual caller would
be the right semantic, but this is basically the same as the security
permission approach.
- Stanley
Bryan Atsatt wrote:
Both solutions require stack walking (unless there is some new
implementation of the java security model I've not yet seen!).
The permission check does much more work than is necessary here. Take a
look at AccessController.checkPermission() to see what I mean.
And actually there is a very simple API to get the stack, which I've
used for years:
private static class StackAccessor extends SecurityManager {
public Class[] getStack() {
return getClassContext();
}
}
private static final STACK_ACCESSOR = new StackAccessor();
Now the enclosing class can simply call STACK_ACCESSOR.getStack().
// Bryan
Stanley M. Ho wrote:
Hi Bryan,
Bryan Atsatt wrote:
1. Definitely agree that resource search order should be identical to
class search order.
Glad to hear!
2. Using permissions to limit access to private resources seems like
overkill to me. The prototype implemented this in a very simple fashion:
a. If resource is exported, return it, else
a. Get the caller's Module (get class from stack, get module from it)
b. If callerModule == this, return resource, else return null.
The issue is that this approach still requires stack walking and there
is no public API in the SE platform that let you implement this.
If stack walking is required for the check anyway, I think the security
permission approach is better that it is implementable with the existing
API in the SE platform.
- Stanley
Re: ClassLoader Deadlock fix
Hi JSR 277 experts, Below are some comments from Jeff Nisewanger and Karen Kinnear who are working on the ClassLoader API improvements for Java SE 7. If you are interested in contributing to the ClassLoader API improvements as a reviewer, you may contact Jeff and Karen at [EMAIL PROTECTED] and [EMAIL PROTECTED] - Stanley Original Message Hi, Sun is working on the ClassLoader Deadlock fix. Our goal is to work closely with the community, of which the 277 EG contains key members, in coming up with small modifications to the current ClassLoader APIs to resolve that issue. As we've mentioned before, and Glyn even points out in his referenced blog, we did add a temporary risky unstable workaround which only really handles a small special case for WebSphere 6.1 which should be supplanted by the new mechanism as that becomes available. One of the key issues here is that a prerequisite for avoiding the ClassLoader deadlocks is that the JVM must not hold the ClassLoader Object lock when it calls out to LoadClass[Internal]. And presumably the lock must not be held when the custom class loader's loadClass() is called. As anyone who has written a custom class loader would immediately recognize, changing that underlying assumption is inherently risky. It is all too easy to break existing custom class loaders that were perhaps unknowingly relying on that lock to protect critical data and operations. Those are the kind of breakages that are extremely timing dependent and can pass many test cases and then unexpectedly fail in the field. I'm sure you all also remember when Unix user level libraries were converted to being MT-safe. We didn't get them all right the first time around. We don't want to put our customers in that position. So a part of the ClassLoader API modifications would be for class loaders to explicitly declare that they are safe for parallel class loading, both for multiple classes in the same class loader, and for multiple instances of the same class in the same class loader. I have been assuming that this would be the default for JSR 277, but we can work that issue in detail when we have a strawman ClassLoader API proposal. In the meantime, thank you very much to Adrian Brock for his classcircularity example. I will be studying it in detail. If any other EG member has a test scenario they would like to describe to us or even better they have a test case for, we would very much appreciate it. And if you are interested in contributing to the ClassLoader API improvements as a reviewer, please let us know. Thank you, Jeff Nisewanger & Karen Kinnear
Re: Relationship to JSR 291 [was: Re: Bryan's comments]
Responses inline, and a few clarifications here (I was a bit tired when
I finished this last night :^)...
The main point I was trying to make is that resolution must occur within
a specific context, but I don't think my example APIs showed that well.
I was assuming that ImportResolver had a ModuleContext as a field, but
we can make this much cleaner and easier to understand by passing it as
an argument:
public abstract class ImportResolver {
public abstract Module resolve(ModuleDefinition def,
ModuleContext ctx);
}
And we can really tie this together by adding a convenience method to
ModuleContext:
public abstract class ModuleContext {
...
public Module resolve(ModuleDefinition def) {
return getImportResolver().resolve(def, this);
}
}
Now resolution becomes simply:
context.resolve(definition);
(I also left out any use of the ImportPolicy, as it isn't yet clear to
me that it should remain a separate type in this model.)
// Bryan
Glyn Normington wrote:
*Bryan Atsatt <[EMAIL PROTECTED]>* wrote on 30/05/2007 07:57:59:
> Andy Piper wrote:
> > At 23:19 25/05/2007, Stanley M. Ho wrote:
> >> Anyway, it seems the EG consensus so far is to not add import package
> >> support. If any EG member disagrees, please speak up.
> >
> > Well, it depends on what the solution for enabling interoperation
> > with JSR 291 is.
> > Our requirement is that there must be a solution, if that requires
> > import package, so be it. If not then not.
>
> Exactly.
>
> I think we can all agree that, at minimum, interoperation means that
> classes and resources are sharable *across* ModuleSystems at runtime.
>
> Which implies that *import dependencies must be resolvable across
> multiple ModuleSystem instances*. (BTW, I think we should change the
> state name "PREPARING" to "RESOLVING" in 7.2.1.)
Agreed. We must avoid the trap of thinking that module system interop.
can be achieved by exposing class loaders (as loadClass will happily
load unexported classes).
>
> So the open issue is the richness of the import "language": must we
> support only lowest-common-denominator, or can we do better without
> over-complicating the design?
>
> I for one would like to be able to have a single module express
> dependencies on modules from both the same and different ModuleSystems,
> *using the standard semantics of each*. This may be reaching too far,
> but we should at least explore it seriously while we figure out what
> interop means here...
At this point, I feel that is likely to be reaching too far, but I'm
happy to play along and see what we can learn along the way.
>
>
> BASICS
>
> So far, we only know of two different import semantics: module-name, and
> package-name. For discussion, let's call these:
>
> a. import-module
> b. import-package
>
> So, to start, we could:
>
> 1. Support declaration of both import types. If 294 supports imports at
> all, it should be relatively easy to support both, since a superpackage
> name is a module name, and it contains member package names. (Compiler
> support is clearly the critical issue here, but it will obviously
> require use of the 277 runtime, so the import *type* should be
> transparent to it.) At worst, we'd need two new annotation types.
A superpackage name is a deployment module name in the JSR 277 model of
one superpackage per deployment module, but I don't see any reason why a
JSR 291 deployment module should not contain more than one superpackage.
So if 294 were to support import, then its import-module would really be
a superpackage import rather than a development module import.
If we end up with nested superpackages, might it make sense to model
multiple superpackages by enclosing them in a single top-level one?
>
> 2. Provide API for both import types (e.g. ImportDependency has
> getModuleName() and getPackageName() methods, one of which will return
> null on a given instance).
>
> However, we know these are necessary but not sufficient. Leaving aside
> the resolution issue for a moment, support for import-package also
> suggests that we:
>
> 3. Enable a single module to declare different versions for each of its
> member packages.
>
> 4. Enable efficient Repository lookup by package name.
>
> I think these are relatively easy (but *could* be considered optional).
>
> We also need:
>
> 5. Standard Query types for lookup by module and package name.
>
>
> EXISTING DEPENDENCY RESOLUTION MODEL
>
> The more interesting issue is dependency resolution. But this hasn't
> been discussed in any real detail, so lets do so before talking further
> about import-package. To simplify this discussion, I'm ignoring
> bundled/custom import policies for now...
>
> Resolution in the current spec is delegated to the associated
> ModuleSystem instance (7.2.2 #8). While the details are not spelled out,
> the expectation appears to be that
>
Re: Relationship to JSR 291 [was: Re: Bryan's comments]
Bryan Atsatt <[EMAIL PROTECTED]> wrote on 30/05/2007 07:57:59: > Andy Piper wrote: > > At 23:19 25/05/2007, Stanley M. Ho wrote: > >> Anyway, it seems the EG consensus so far is to not add import package > >> support. If any EG member disagrees, please speak up. > > > > Well, it depends on what the solution for enabling interoperation > > with JSR 291 is. > > Our requirement is that there must be a solution, if that requires > > import package, so be it. If not then not. > > Exactly. > > I think we can all agree that, at minimum, interoperation means that > classes and resources are sharable *across* ModuleSystems at runtime. > > Which implies that *import dependencies must be resolvable across > multiple ModuleSystem instances*. (BTW, I think we should change the > state name "PREPARING" to "RESOLVING" in 7.2.1.) Agreed. We must avoid the trap of thinking that module system interop. can be achieved by exposing class loaders (as loadClass will happily load unexported classes). > > So the open issue is the richness of the import "language": must we > support only lowest-common-denominator, or can we do better without > over-complicating the design? > > I for one would like to be able to have a single module express > dependencies on modules from both the same and different ModuleSystems, > *using the standard semantics of each*. This may be reaching too far, > but we should at least explore it seriously while we figure out what > interop means here... At this point, I feel that is likely to be reaching too far, but I'm happy to play along and see what we can learn along the way. > > > BASICS > > So far, we only know of two different import semantics: module-name, and > package-name. For discussion, let's call these: > > a. import-module > b. import-package > > So, to start, we could: > > 1. Support declaration of both import types. If 294 supports imports at > all, it should be relatively easy to support both, since a superpackage > name is a module name, and it contains member package names. (Compiler > support is clearly the critical issue here, but it will obviously > require use of the 277 runtime, so the import *type* should be > transparent to it.) At worst, we'd need two new annotation types. A superpackage name is a deployment module name in the JSR 277 model of one superpackage per deployment module, but I don't see any reason why a JSR 291 deployment module should not contain more than one superpackage. So if 294 were to support import, then its import-module would really be a superpackage import rather than a development module import. > > 2. Provide API for both import types (e.g. ImportDependency has > getModuleName() and getPackageName() methods, one of which will return > null on a given instance). > > However, we know these are necessary but not sufficient. Leaving aside > the resolution issue for a moment, support for import-package also > suggests that we: > > 3. Enable a single module to declare different versions for each of its > member packages. > > 4. Enable efficient Repository lookup by package name. > > I think these are relatively easy (but *could* be considered optional). > > We also need: > > 5. Standard Query types for lookup by module and package name. > > > EXISTING DEPENDENCY RESOLUTION MODEL > > The more interesting issue is dependency resolution. But this hasn't > been discussed in any real detail, so lets do so before talking further > about import-package. To simplify this discussion, I'm ignoring > bundled/custom import policies for now... > > Resolution in the current spec is delegated to the associated > ModuleSystem instance (7.2.2 #8). While the details are not spelled out, > the expectation appears to be that > ModuleSystem.getModule(ModuleDefinition) must: > > - Select an initial repository. Call getRepository() on the input parameter. > > Then, for each ImportDependency of the definition: > > - Select a matching definition. Construct a Query from the > ImportDependency and use Repository.find() to lookup a matching > ModuleDefinition. > > - Get an instance. Use def.getModuleSystem().getModule(def). The > ModuleSystem is expected to return a cached instance if available, or > create/cache/return one if not. I think there also needs to be some 'resolution context' object which explicitly denotes a particular resolution so that each module system can keep track of the state of a resolution. This is required when two or more imports of a given module from another module system need to resolve to the same module instance. A resolution context may also be needed for back-tracking when a set of module instances created earlier in resolution turn out not to satisfy all the necessary constraints. > > > (TBD: The PlatformBinding must be taken into account somehow during > selection. ModuleDefinition must include an accessor for it, and either > Repository.find() should implicitly filter them, or the caller must > construct a Query which will do so. I thi
Re: ClassLoader Deadlock fix was: Re: Relationship to JSR 291 [was: Re: Bryan's comments]
Adrian <[EMAIL PROTECTED]> wrote on 25/05/2007 12:44:54: > On Thu, 2007-05-24 at 11:00 +0100, Glyn Normington wrote: > > A better approach would be for the Java 7 platform to provide first > > class support for JSR 291. This boils down to standardising the > > experimental class loader deadlock fix ([1]) > > Fixing the deadlock just moves the problem. > > You'll still get ClassCircularityErrors when competing threads > try to load classes using locks other than the classloader > synchronization or they don't synchronize on the loadClass() > or they release the lock during the classloading request > to let others have a go (again to avoid the deadlock). > > This is because of the way the dictionary class determines > whether a circular load is occuring. > > Although I haven't tried it with OpenJDK so maybe the > dictionary class contains some other fixes to workaround the > problem? > > The simple form of the problem: > http://bugs.sun.com/bugdatabase/view_bug.do?bug_id=4699981 > has been fixed in recent JDKs, but spurious CCEs > still exist in other cases. > e.g. the testAbstractFactoryConcurrent() here: > http://viewvc.jboss.org/cgi-bin/viewvc. > cgi/jbossas/projects/microcontainer/trunk/classloader/src/tests/org/jboss/test/classloader/delegate/test/DelegateUnitTestCase. > java?revision=62792&view=markup > will show CCEs in the log if you enable TRACE logging. > > 1445 TRACE [ClassLoaderManager] Run failed with exception > java.lang.ClassCircularityError: > org/jboss/test/classloader/delegate/support/b/TestFactoryImplementation > at java.lang.ClassLoader.defineClass1(Native Method) > at java.lang.ClassLoader.defineClass(ClassLoader.java:620) Some spurious CCEs were indeed fixed before the experimental deadlock fix was introduced, but there may be more to do to make this fix robust and complete. If you raise a sunbug for the above testcase, best to report it here so that class loader rearchitecture folks can take it into consideration for Java 7. Glyn Unless stated otherwise above: IBM United Kingdom Limited - Registered in England and Wales with number 741598. Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU
Re: Relationship to JSR 291 [was: Re: Bryan's comments]
Andy Piper wrote:
> At 23:19 25/05/2007, Stanley M. Ho wrote:
>> Anyway, it seems the EG consensus so far is to not add import package
>> support. If any EG member disagrees, please speak up.
>
> Well, it depends on what the solution for enabling interoperation
> with JSR 291 is.
> Our requirement is that there must be a solution, if that requires
> import package, so be it. If not then not.
Exactly.
I think we can all agree that, at minimum, interoperation means that
classes and resources are sharable *across* ModuleSystems at runtime.
Which implies that *import dependencies must be resolvable across
multiple ModuleSystem instances*. (BTW, I think we should change the
state name "PREPARING" to "RESOLVING" in 7.2.1.)
So the open issue is the richness of the import "language": must we
support only lowest-common-denominator, or can we do better without
over-complicating the design?
I for one would like to be able to have a single module express
dependencies on modules from both the same and different ModuleSystems,
*using the standard semantics of each*. This may be reaching too far,
but we should at least explore it seriously while we figure out what
interop means here...
BASICS
So far, we only know of two different import semantics: module-name, and
package-name. For discussion, let's call these:
a. import-module
b. import-package
So, to start, we could:
1. Support declaration of both import types. If 294 supports imports at
all, it should be relatively easy to support both, since a superpackage
name is a module name, and it contains member package names. (Compiler
support is clearly the critical issue here, but it will obviously
require use of the 277 runtime, so the import *type* should be
transparent to it.) At worst, we'd need two new annotation types.
2. Provide API for both import types (e.g. ImportDependency has
getModuleName() and getPackageName() methods, one of which will return
null on a given instance).
However, we know these are necessary but not sufficient. Leaving aside
the resolution issue for a moment, support for import-package also
suggests that we:
3. Enable a single module to declare different versions for each of its
member packages.
4. Enable efficient Repository lookup by package name.
I think these are relatively easy (but *could* be considered optional).
We also need:
5. Standard Query types for lookup by module and package name.
EXISTING DEPENDENCY RESOLUTION MODEL
The more interesting issue is dependency resolution. But this hasn't
been discussed in any real detail, so lets do so before talking further
about import-package. To simplify this discussion, I'm ignoring
bundled/custom import policies for now...
Resolution in the current spec is delegated to the associated
ModuleSystem instance (7.2.2 #8). While the details are not spelled out,
the expectation appears to be that
ModuleSystem.getModule(ModuleDefinition) must:
- Select an initial repository. Call getRepository() on the input parameter.
Then, for each ImportDependency of the definition:
- Select a matching definition. Construct a Query from the
ImportDependency and use Repository.find() to lookup a matching
ModuleDefinition.
- Get an instance. Use def.getModuleSystem().getModule(def). The
ModuleSystem is expected to return a cached instance if available, or
create/cache/return one if not.
(TBD: The PlatformBinding must be taken into account somehow during
selection. ModuleDefinition must include an accessor for it, and either
Repository.find() should implicitly filter them, or the caller must
construct a Query which will do so. I think we should add a
CURRENT_PLATFORM constant to Query, which will evaluate to true if no
binding is present in a definition.)
(The spec also talks about Repository as the mechanism of isolation
(6.4). This was the case in the prototype, where the repository itself
provided caching. It doesn't appear to work with the current design.
There is no need that I can see to isolate ModuleDefinition
instances--it is Module instances with their associated loaders that may
require isolation.)
(Also note that if ImportDependency was itself a Query subclass, there
would be no need to do any mapping. And since the ModuleDefinition
subclass must produce ImportDependency instances, it can even produce
more specialized Query instances if desired.)
REFINEMENT
I think we can improve on the existing model in several ways:
A. Provide a model for Module isolation (e.g. for EE, Applets, etc).
B. Encapsulate all selection logic in a single mechanism.
C. Eliminate the overhead of the repository lookup when a cached
instance exists.
Let me propose a new class that encapsulates the caching logic, enables
lookup using Query, and supports multiple instances for isolation:
public abstract class ModuleContext {
// Get the context used to define JRE modules.
public static ModuleContext getBootstrapContext(){...};
// Get the context used to define the main module.
pub
