Re: Relationship to JSR 291 [was: Re: Bryan's comments]
Bryan Atsatt [EMAIL PROTECTED] wrote on 30/05/2007 19:11:02:
Responses inline, and a few clarifications here (I was a bit tired when
I finished this last night :^)...
The main point I was trying to make is that resolution must occur within
a specific context, but I don't think my example APIs showed that well.
I was assuming that ImportResolver had a ModuleContext as a field, but
we can make this much cleaner and easier to understand by passing it as
an argument:
public abstract class ImportResolver {
public abstract Module resolve(ModuleDefinition def,
ModuleContext ctx);
}
And we can really tie this together by adding a convenience method to
ModuleContext:
public abstract class ModuleContext {
...
public Module resolve(ModuleDefinition def) {
return getImportResolver().resolve(def, this);
}
}
Now resolution becomes simply:
context.resolve(definition);
(I also left out any use of the ImportPolicy, as it isn't yet clear to
me that it should remain a separate type in this model.)
// Bryan
Glyn Normington wrote:
*Bryan Atsatt [EMAIL PROTECTED]* wrote on 30/05/2007 07:57:59:
Andy Piper wrote:
At 23:19 25/05/2007, Stanley M. Ho wrote:
Anyway, it seems the EG consensus so far is to not add import
package
support. If any EG member disagrees, please speak up.
Well, it depends on what the solution for enabling
interoperation
with JSR 291 is.
Our requirement is that there must be a solution, if that
requires
import package, so be it. If not then not.
Exactly.
I think we can all agree that, at minimum, interoperation means
that
classes and resources are sharable *across* ModuleSystems at
runtime.
Which implies that *import dependencies must be resolvable across
multiple ModuleSystem instances*. (BTW, I think we should change
the
state name PREPARING to RESOLVING in 7.2.1.)
Agreed. We must avoid the trap of thinking that module system interop.
can be achieved by exposing class loaders (as loadClass will happily
load unexported classes).
So the open issue is the richness of the import language: must we
support only lowest-common-denominator, or can we do better without
over-complicating the design?
I for one would like to be able to have a single module express
dependencies on modules from both the same and different
ModuleSystems,
*using the standard semantics of each*. This may be reaching too
far,
but we should at least explore it seriously while we figure out
what
interop means here...
At this point, I feel that is likely to be reaching too far, but I'm
happy to play along and see what we can learn along the way.
BASICS
So far, we only know of two different import semantics:
module-name, and
package-name. For discussion, let's call these:
a. import-module
b. import-package
So, to start, we could:
1. Support declaration of both import types. If 294 supports
imports at
all, it should be relatively easy to support both, since a
superpackage
name is a module name, and it contains member package names.
(Compiler
support is clearly the critical issue here, but it will obviously
require use of the 277 runtime, so the import *type* should be
transparent to it.) At worst, we'd need two new annotation types.
A superpackage name is a deployment module name in the JSR 277 model
of
one superpackage per deployment module, but I don't see any reason why
a
JSR 291 deployment module should not contain more than one
superpackage.
So if 294 were to support import, then its import-module would really
be
a superpackage import rather than a development module import.
If we end up with nested superpackages, might it make sense to model
multiple superpackages by enclosing them in a single top-level one?
That is an option, but of course each nested superpackage has to name its
parent, so it wouldn't be possible to combine superpackages from
independent groups or sources without either modifying their superpackage
declarations or getting them to agree on the name of the parent
superpackage and code it themselves.
2. Provide API for both import types (e.g. ImportDependency has
getModuleName() and getPackageName() methods, one of which will
return
null on a given instance).
However, we know these are necessary but not sufficient. Leaving
aside
the resolution issue for a moment, support for import-package also
suggests that we:
3. Enable a single module to declare different versions for each of
its
member packages.
4. Enable efficient Repository lookup by package name.
I think these are relatively easy (but *could* be considered
optional).
We also need:
5. Standard Query types for lookup by module and package name.
Re: Exported resources
Some feedback from an observer which may help:
Bryan misses the need for extender model to load internal impl classes
ala
Bundle.loadClass in OSGi.
You do not want to have to export the service impl class from a module.
You want to hide the class from others' casual loading. However an
extender bundle (e.g. ServiceLoader) will need to be able to load that
class to make it instances of it available to others under the service
interface class.
Glyn
Bryan Atsatt [EMAIL PROTECTED] wrote on 30/05/2007 21:21:36:
I've been assuming that Module private resources should not be visible
to *any* class outside of the module. Including ResourceBundle, or any
other existing framework classes that do resource lookups (e.g.
ServiceLoader, JSF, etc). If resources need to be visible to these
existing classes, they must be exported. The very simple check I
proposed (immediate caller) is sufficient to make this assertion.
I believe your point is that if we used the permission model instead, it
would become possible for a module to invoke an external class (e.g.
ResourceBundle.getBundle()) and enable *it* to successfully load a
private resource from the module.
Aside from the permission *grant* mechanism this model would rely on, it
is an entirely different model than that used for classes! (Though we
haven't explicitly defined this in 294, it seems extremely unlikely that
we will rely on permissions--none of the other access modes do so.) Such
asymmetry is very disconcerting to me, and, I believe, just plain
wrong...
Consider that you could grant the ServiceLoader, for example, access to
a resource that names a class that it could not instantiate. That class
would have to be exported. I believe the resource should be as well.
// Bryan
Stanley M. Ho wrote:
Hi Bryan,
Those resource-related methods in ClassLoader can be called by anyone,
including code that is part of the module, code that is from other
modules, or code that is part of the platform libraries (e.g.
ResourceBundle). The approach you described would require walking the
stack to get the caller's Module, but the real issue is that it is
difficult to determine who the actual caller is from the stack.
Treating the immediate caller on the stack as the actual caller
wouldn't
be sufficient because the immediate caller could be called by someone
else who is the one actually making the call. On the other hand,
treating the originated caller on the stack as the actual caller would
be the right semantic, but this is basically the same as the security
permission approach.
- Stanley
Bryan Atsatt wrote:
Both solutions require stack walking (unless there is some new
implementation of the java security model I've not yet seen!).
The permission check does much more work than is necessary here. Take
a
look at AccessController.checkPermission() to see what I mean.
And actually there is a very simple API to get the stack, which I've
used for years:
private static class StackAccessor extends SecurityManager {
public Class[] getStack() {
return getClassContext();
}
}
private static final STACK_ACCESSOR = new StackAccessor();
Now the enclosing class can simply call STACK_ACCESSOR.getStack().
// Bryan
Stanley M. Ho wrote:
Hi Bryan,
Bryan Atsatt wrote:
1. Definitely agree that resource search order should be identical
to
class search order.
Glad to hear!
2. Using permissions to limit access to private resources seems
like
overkill to me. The prototype implemented this in a very simple
fashion:
a. If resource is exported, return it, else
a. Get the caller's Module (get class from stack, get module from
it)
b. If callerModule == this, return resource, else return null.
The issue is that this approach still requires stack walking and
there
is no public API in the SE platform that let you implement this.
If stack walking is required for the check anyway, I think the
security
permission approach is better that it is implementable with the
existing
API in the SE platform.
- Stanley
Unless stated otherwise above:
IBM United Kingdom Limited - Registered in England and Wales with number
741598.
Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU
Re: Exported resources
I don't think I'm missing anything, just looking at it from perhaps a
different perspective :^)
First, 294 will determine accessibility for non-exported classes. The
assumption at the moment is that they will not be accessible to *any*
class outside of the module.
I do understand the value of a friend semantic; it would certainly be
nice to grant certain frameworks special access. But, so far, that is
not on the table for classes.
And if we don't have it for classes, I can't see why we should have it
for resources.
// Bryan
Glyn Normington wrote:
Some feedback from an observer which may help:
Bryan misses the need for extender model to load internal impl
classes ala
Bundle.loadClass in OSGi.
You do not want to have to export the service impl class from a module.
You want to hide the class from others' casual loading. However an
extender bundle (e.g. ServiceLoader) will need to be able to load that
class to make it instances of it available to others under the service
interface class.
Glyn
*Bryan Atsatt [EMAIL PROTECTED]* wrote on 30/05/2007 21:21:36:
I've been assuming that Module private resources should not be visible
to *any* class outside of the module. Including ResourceBundle, or any
other existing framework classes that do resource lookups (e.g.
ServiceLoader, JSF, etc). If resources need to be visible to these
existing classes, they must be exported. The very simple check I
proposed (immediate caller) is sufficient to make this assertion.
I believe your point is that if we used the permission model instead, it
would become possible for a module to invoke an external class (e.g.
ResourceBundle.getBundle()) and enable *it* to successfully load a
private resource from the module.
Aside from the permission *grant* mechanism this model would rely on, it
is an entirely different model than that used for classes! (Though we
haven't explicitly defined this in 294, it seems extremely unlikely that
we will rely on permissions--none of the other access modes do so.) Such
asymmetry is very disconcerting to me, and, I believe, just plain
wrong...
Consider that you could grant the ServiceLoader, for example, access to
a resource that names a class that it could not instantiate. That class
would have to be exported. I believe the resource should be as well.
// Bryan
Stanley M. Ho wrote:
Hi Bryan,
Those resource-related methods in ClassLoader can be called by anyone,
including code that is part of the module, code that is from other
modules, or code that is part of the platform libraries (e.g.
ResourceBundle). The approach you described would require walking the
stack to get the caller's Module, but the real issue is that it is
difficult to determine who the actual caller is from the stack.
Treating the immediate caller on the stack as the actual caller
wouldn't
be sufficient because the immediate caller could be called by someone
else who is the one actually making the call. On the other hand,
treating the originated caller on the stack as the actual caller would
be the right semantic, but this is basically the same as the security
permission approach.
- Stanley
Bryan Atsatt wrote:
Both solutions require stack walking (unless there is some new
implementation of the java security model I've not yet seen!).
The permission check does much more work than is necessary here.
Take a
look at AccessController.checkPermission() to see what I mean.
And actually there is a very simple API to get the stack, which I've
used for years:
private static class StackAccessor extends SecurityManager {
public Class[] getStack() {
return getClassContext();
}
}
private static final STACK_ACCESSOR = new StackAccessor();
Now the enclosing class can simply call STACK_ACCESSOR.getStack().
// Bryan
Stanley M. Ho wrote:
Hi Bryan,
Bryan Atsatt wrote:
1. Definitely agree that resource search order should be
identical to
class search order.
Glad to hear!
2. Using permissions to limit access to private resources seems like
overkill to me. The prototype implemented this in a very simple
fashion:
a. If resource is exported, return it, else
a. Get the caller's Module (get class from stack, get module
from it)
b. If callerModule == this, return resource, else return null.
The issue is that this approach still requires stack walking and
there
is no public API in the SE platform that let you implement this.
If stack walking is required for the check anyway, I think the
security
permission approach is better that it is implementable with the
existing
API in the SE platform.
- Stanley
/
/
/Unless stated otherwise above:
IBM United Kingdom Limited - Registered
Re: Relationship to JSR 291 [was: Re: Bryan's comments]
Bryan Atsatt wrote: Stanley M. Ho wrote: Glyn Normington wrote: *Bryan Atsatt [EMAIL PROTECTED]* wrote on 30/05/2007 07:57:59: ... So the open issue is the richness of the import language: must we support only lowest-common-denominator, or can we do better without over-complicating the design? I for one would like to be able to have a single module express dependencies on modules from both the same and different ModuleSystems, *using the standard semantics of each*. This may be reaching too far, but we should at least explore it seriously while we figure out what interop means here... At this point, I feel that is likely to be reaching too far, but I'm happy to play along and see what we can learn along the way. I agreed with Glyn that this might be reaching too far. Before we dive too much into how to implement interoperability, I think one of the outstanding questions we should answer first is what degree of import interoperability we want to offer. There are four possibilities: 1. JSR 277 module imports OSGi module by module name 2. OSGi module imports JSR 277 module by module name 3. OSGi module imports JSR 277 module by package name 4. JSR 277 module imports OSGi module by package name Let's ignore the 294 issues and the module initialization issues for now to simplify this discussion. I think we all agreed #1 and #2 are important to support and the reflective APIs already enable these (of course, there are minor issues we still have to address as we evolve the APIs.) That said, it is unclear to me how important it is to support #3 and #4, so I think the first question for this EG is whether we want to support #3 and #4 at all. However, let's pretend #3 and #4 are important for this specific discussion. I think #3 is already possible since the OSGi framework could look up the appropriate module which exported the package in the repository, using the reflective APIs. Afterwards, the OSGi framework can then do the necessary wiring. Yes, the reflective APIs can be further refined to make it easier for the OSGi framework to query modules by exported package name from the repository, but this is a minor issue that we can address easily. #4 is a bit complicated, because there is no import-by-package semantic in the current module layer's APIs and I think we all agreed that we don't want to support this semantic directly in the module system defined by JSR 277. Not me. I do understand your point that this is more difficult, but I don't really care--it is our job to do the Right Thing here. Sure, if it is not feasible in the time frame we have, then so be it. But we haven't even tried! If we're forced to make a choice between the two models, I'd pick import-by-package with zero hesitation. OSGi has a lot of experience here, and we shouldn't ignore it. Import-by-name was added late in the game, and is now seriously downplayed, for good reason. Just for information, import-by-name (or module dependencies) was not added to OSGi R4 because they were thought to be widely needed or useful, adding them was always somewhat contentious which is why the R4 spec contained a section saying why you shouldn't use them (this section has been further expanded in R4.1). The main motivation for adding them was certainly legacy situation and for tightly coupled sub-systems. All in all, I agree with everything that Bryan is saying and I argued these same points early on; however, I was under the impression that we had long since lost that battle. Bryan, it is unclear to me whether your view is that we need to make sure that 277 supports other modules systems the use import-by-package or if you think that 277 should somehow directly support import-by-package (i.e., explicitly expose such concepts). - richard And Oracle has a lot of relevant experience as well, since the shared loader mechanism I created for our AS stack (shared libraries) supports only the import-by-name model. We have *lots* of shared-libraries (I've seen 100+ running simultaneously), used by many different components in the stack, in addition to customer applications. It is a tremendous pain when refactoring or even simple renaming has to be done (I hope to shift this implementation onto 277 to solve this problem, rather than inventing an interim approach). And this kind of change happens far more frequently than you probably expect. I don't want to repeat the same mistake here. Nor do I think that leaving a long trail of extraneous view modules behind for compatibility is a good solution. Worse still, if the 277 APIs don't support import-by-package, then the compiler won't support it either. If the compiler doesn't support it, then we've lost a golden opportunity to move to a better world, one in which packaging is irrelevant. We're supposed to be eliminating jar-hell here. Let's not simply replace it with module-hell. In this context, the question is really about what it would take to make #4 possible if we
Re: Relationship to JSR 291 [was: Re: Bryan's comments]
On May 31, 2007, at 8:06 PM, Stanley M. Ho wrote: Hi Richard, Richard S.Hall wrote: I don't think I fully understand your concerns around exporting an entire package. As you mentioned, what's get exported in 277 module is in terms of classes and the information is in the module metadata, and I think we can probably determine the packages associated with these exported classes and treat these packages as exported packages from the OSGi perspective. There is no split package allowed in 277 module, so I think each of these exported packages would be an entire package. The point is that an OSGi exported package FOO specifically means that all public classes in the FOO package are accessible to importers. Such an export is not the same as a 277 module that just happens to export a class in its export signature from the FOO package. Without cracking open the module, there is no way to tell if its export signature contains all public classes from the FOO package (i.e., the entire package). - richard I think there is a misunderstanding here. Both JSR 277 and OSGi export all accessible classes in a package. JSR 294 changes what accessible means from public to exported public, and this affects 277 and OSGi in the same way. I don't think there is a conflict here. Perhaps so. I am operating off my [possibly dated] recollection that 277 modules explicitly list the classes that they export, which meant that they may expose arbitrary public classes from a package. If this is no longer the case, then this is fine. The point remains, all legacy OSGi dependencies on a package assume that all public classes are exported from that package. An OSGi dependency on a given package cannot be resolved to a 277 module whose exported public classes are not the same as all public classes in the specific package. - richard
