Re: ModuleDefinition construction WAS Re: Goals was: Re: Supporting OSGi Bundles in the Java Module System
Adrian Brock wrote: I see a number of possible use cases here (I'll continue with appserver example): 1) Fully JSR 277 The appserver and JavaEE specs are updated to fully understand JSR277/294. In this case, the JavaEE deployment process would probably make use of a JSR 277 repository. 2) Partial JSR 277 The appserver knows about JSR277/294 but the JavaEE specs haven't been updated (or they are legacy deployments). In this case the appserver could try to adapt the deployments to conform to the ModuleDefinition/Content. In this case, there probably wouldn't be a real repository implementation, instead the repository would be dummy. It would just be serving as an adapter from the old deployment packaging and metadata to the new one at runtime. 3) Mixed Similar to 2, but the opposite. The appserver knows about JSR277/294 and wants to support .jam files as well as .jar files in JavaEE deployments. i.e. .jam files would be unpacked from a .jar EAR deployment and then added to the module system runtime. Conclusion: I expect there are other possible uses. The question is whether Repository is the best api for cases (2) and (3). i.e. we implement a dummy repository with a dummy URL for the source or whether there should be a more abstract parent class (I called it ModuleSource) that doesn't assume a physical repository behind the scenes. At least for cases (2) and (3) all we really want to do is to PUSH a ModuleDefinition/Content to the jsr 277 module system such that we can get back a Module/Classloader with the imports resolved. In your example, if you want to create ModuleDefinitions from the .jar files, there are two ways to achieve it: 1. a. Create a ModuleContent based on the content of the .jar file. b. Generate the JAM's metadata based on information in the .jar file. c. Construct a JAM's ModuleDefinition with the ModuleContent using Modules.newJamModuleDefinition(). The ModuleDefinition would be associated with the JAM module system. 2. a. Create a ModuleContent based on the content of the .jar file. b. Create your own ModuleDefinition with the ModuleContent and use the JAM module system as the associated module system. As long as your ModuleDefinition exposes the appropriate metadata information reflectively, the JAM module system would treat it like other JAM's ModuleDefinitions. In either case, the JAM module system could be used to instantiate and initialize Module/ClassLoader from the ModuleDefinition. Creating a repository implementation is a different story. In the current draft API, you do need to override a number of methods in Repository in order to have a functional repository implementation. I still think Repository is the api for module systems to find ModuleDefinitions. The question is how we could change the API to make it much easier for developers to create custom repository implementation. Perhaps we should provide more useful default implementation in the Repository's methods so developers only need to override very small number of methods. Or perhaps what we really need is a repository builder which allows developers to provide a set of ModuleDefinitions and get back a basic repository implementation to use. I'll look into the draft API to see if further simplifications can be made. - Stanley
Re: Exported Classes and OSGi Re: Supporting OSGi Bundles in the Java Module System
Adrian Brock wrote: So a 291 module that hasn't been updated to reflect changes from 294 would just expose package exports and the importing 277 module would know how to deal with that? A 291 module which exports packages is fundamentally exporting public types, which will continue to work across module boundaries. If a JAM module imports a 291 module, it'll know how to deal with this if getExportedClasses() is not supported in the imported ModuleDefinition. The 291 module classes would be like legacy classes that don't belong to a module in the 294 sense. i.e. Class.getModuleInfo() would return null Types which are not in a named module are in the unnamed module; the changes in the Java Language Specification (JLS) will say this. null is probably as good a value as any for representing this. - Stanley
Re: Goals was: Re: Supporting OSGi Bundles in the Java Module System
Hi Adrian, Adrian Brock wrote: My critisms of using the JSR277 repositories to do the integration included: I want to better understand your concerns around developing repositories, since I believe some of your concerns have already been addressed in the draft API. * There are already lots of orthogonal reasons for swapping repository implementations On Mon, 2007-02-26 at 12:26 +0100, Adrian wrote: So besides the peer/hierarchy argument there is also the problem that the repository is dealing with too many concerns. 1) ModuleDefinition construction The draft API provides a Modules.newJamModuleDefinition() method to construct a ModuleDefinition from JAM's metadata and ModuleContent. The ModuleContent abstraction enables a module archive to be stored in the repository in any form under the cover. This should make it easier for a repository implementation to construct ModuleDefinitions from JAM files, or from other archive formats (as long as the metadata information of a module can be expressed as JAM's metadata, and the content of the module can be exposed through the ModuleContent abstraction). In your original use case in JavaEE, the appserver wants to use the module system for plain old module wiring but also wants to define its own modules for wars, ears, etc., and some of which looks nothing like jars/jams in structure. Do you think the draft API and the ModuleContent abstraction are sufficient to address the ModuleDefinition construction issue you were concerned with? 2) Model - parent/child or peer I think you meant the module system inteorp model, not the repository delegation model. Right? 3) QoS - what tools are available for the repository The Java Module System implementation will come with a standard tool to manage module archives in the repository, e.g. install/uninstall, etc. Is there any particular aspect of the tools support that you are concerned with but currently not possible in the draft API? 4) Location - file system/url based, etc. This is handled by the ModuleContent abstraction described above. * Developing a full repository just to define a different archive format is too much (e.g. legacy javaee deployments) I would like to know what you think about the draft API to see if it is still too much to develop a repository. - Stanley
Re: Exported Classes and OSGi Re: Supporting OSGi Bundles in the Java Module System
Hi Adrian,
Adrian Brock wrote:
On Mon, 2008-04-28 at 20:16 -0700, Stanley M. Ho wrote:
3.2 ModuleDefinition class
Two new methods are added in the ModuleDefinition class to return the
exported packages and member packages respectively. The export and
member definitions contained in the OSGi metadata are in package
granularity. In addition, a new PackageDefinition class is added to
allow an OSGi bundle to expose the metadata for an exported package.
This is required to meet the requirements (1) and (2).
ModuleDefinition class:
public abstract SetPackageDefinition getExportedPackageDefinitions();
public abstract SetPackageDefinition getMemberPackageDefinitions();
PackageDefinition class:
public abstract class PackageDefinition {
public String getName();
public Version getVersion();
public SetString getAttributeNames();
public String getAttribute(String name);
}
ModuleDefinition still refers to exported and member classes.
Although this is probably more of an issue for 294/291 interoperation,
I don't think 291 has such notions?
With modules in the language, only public types are accessible outside
their development modules. A deployment module system could leverage
this by considering public types (or a subset thereof) as 'exported',
and the granularity of exports can therefore be types or packages. The
JAM module system integrates tightly with development modules and it
will expose type-level exports, but other module systems can continue to
expose package-level exports. Thus, ModuleDefinition should support both
granularity levels. If a module system does not support type-level
exports (e.g. 291), the implementation of
ModuleDefinition.getExportedClasses()/getMemberClasses() could simply
throw UnsupportedOperationException.
- Stanley
Supporting OSGi Bundles in the Java Module System
Dear 277 observers, The original message is too big for this observer list, so I forward the original message with first attachment only. If any of you are interested in getting the latest API javadoc, please send email to [EMAIL PROTECTED] - Stanley Dear 277 experts, The first attachment is a draft spec for supporting OSGi bundles in the Java Module System. This is based on the past EG discussion in particular the proposals and inputs from Glyn, Richard, and Bryan. This is a work in progress and is expected to evolve based on further inputs from this EG. The second attachment is the latest API javadoc for your reference. We're currently updating the JSR 277 EDR2 specification and APIs to make the distinction between the framework/abstractions for the Java Module System more clear, and we expect they will be available for the EG to review in a few weeks after JavaOne. In the meantime, the draft is intended to provide enough information for the EG to review and provide inputs without the EDR2. Numerous EG members are preparing to attend and/or speak at JavaOne next week, so some of them may well not respond until afterwards. Finally, thanks to Bryan and his blog that serves really well to set up the context for this discussion. - Stanley and Alex Title: Supporting OSGi Bundles in the Java Module System Supporting OSGi Bundles in the Java Module System Mandy Chung and Stanley Ho Version 0.1 April 28, 2008 Copyright 2008 Sun Microsystems, Inc. JSR 277 Early Draft defines the framework for the Java Module System and provides the API to access Java modules such as searching for modules, loading and initializing a module, and also managing its life time. JSR 277 enables one or more module systems run on a single Java virtual machine. This document1 defines how JSR 277 supports OSGi bundles as defined in JSR 291. This document is a work in progress and is expected to evolve based on more input from the Expert Group. Section 1 describes various definitions in the Java Module System. Section 2 describes the requirements for supporting OSGi bundles in the Java Module System and section 3 describes the proposed API changes. Section 4 specifies how to map an OSGi bundle to a ModuleDefinition so they can be consumed by other modules in the Java Module System. Section 5 describes the repository mechanism that enables the OSGi module system to be plugged into the Java Module System framework to make OSGi bundles accessible by other module systems at runtime. Appendix A shows the code example illustrating how a JAM module system implementation uses the JSR 277 API to search and import OSGi bundles. 1This draft is built upon from the basis of the past EG discussions, proposals and suggestions for the interoperability support especially the ideas and invaluable inputs from Glyn Normington, Richard Hall and Bryan Atsatt. 0. Current Status The JSR 277 EDR2 is being updated to make the distinction between the framework/abstractions for the Java Module System and the JAM module system (See Section 1) clearer. This section highlights the main items that remain to be sorted out. Versioning scheme (see Section 4.2) Security Java Module Events 1. Definitions Java module A development abstraction that exists at compile-time in the Java programming language and is reified in the Java Virtual Machine. Module Definition A deployment abstraction that contains metadata, classes and resources, and is reified as ModuleDefinition as part of a Java Module System implementation. JAM module A concrete, portable deployment format which implements the Module Definition. Amongst other details, it has metadata, classes, resources, and can contain embedded JAR files and native libraries. The distribution format is called JAM (JAva Module) that is based on the JAR file format. Java Module System A set of runtime abstractions that includes ModuleSystem, Repository, ModuleDefinition (see "Module Definition" above), Module, ImportDependency and others. JAM Module System A concrete implementation of the Java Module System that supports JAM modules. It is the default implementation of the Java Module System for the Java SE platform. OSGi Module System A concrete implementation of the Java Module System by an OSGi container. 2. Requirements It shall be possible for an OSGi container to implement the Java Module System. It shall be possible for a JAM module to express an import dependency on any Module Definition in any Java Module System. Below provides an example to illustrate how a JAM module imports OSGi bundles. A Wombat application is a JAM module that processes shopping orders for a website. It depends on the Apache Xerces XML parser and the Apache Derby and both
Re: Updates on Interoperability
Hi Brett, We are in the process of finalizing the document, and hopefully we could provide more details to the EG next week. - Stanley Brett Porter wrote: On 03/04/2008, at 1:27 PM, Stanley M. Ho wrote: The good news is that we now have a prototype that allows JSR 277 modules to interoperate with OSGi bundles in certain degrees, and another prototype that enables interoperation with Maven. I'm in the process of sorting out the details with the development team so we could have a strawman for this EG to discuss soon. With my Maven hat on, this sounds quite interesting. Is there any update on the availability of this for discussion? Thanks, Brett -- Brett Porter [EMAIL PROTECTED] http://blogs.exist.com/bporter/
Re: Updates on Interoperability
The current APis do not prohibit module definition to have split packages. It is only when a Java module is instantiated from the module definition, the implementation of the default module system would perform shallow validation and consider split packages as an error. Other module system implementations can continue to have their own validation policies to resolve their modules. I hope this will become more clear when the details are available next week. - Stanley Bryan Atsatt wrote: Richard S. Hall wrote: Gordon Hirsch wrote: 1. Map its dependency declarations into a standard runtime representation. 2. Support a standard search model over its stored modules, using the runtime dependency expressions. 3. Map its stored module data into a standard runtime representation that can be returned by the search. One challenge lies in defining the standard runtime representations and standard search model in a universal enough way to encompass OSGi and other module systems. This implies embracing concepts (in these standard representations and search model) that were not universally liked by the EG early on. (Split packages and package-level import/export come to mind.) Package-level import/export seem like a must, but I still don't see split packages as a necessity. Strongly agreed on import-by-package, and that ModuleDefinition requires a method to enumerate exported packages (and probably an exportsPackage(String packageName) method to enable an efficient Query). On the split package front, however, it seems a little fuzzy to me. If an OSGi implementation of 277 is also going to remain OSGi Rx compliant, it will still need to support split packages, right? If so, don't we need to surface this fact at the 277 level? Perhaps that is as simple as acknowledging that Module.getImportedModules() may return more than one instance that exports a given package. // Bryan
Re: Fw: JSR 291 interoperation status
Hi Glyn, The work is in progress, and there are ongoing prototypings to figure out how it should work and to also validate the overall approach. I will send out the strawman proposal for the EG to review and discuss when it is ready. Interoperability is definitely something that I would like to address before this JSR goes public review. - Stanley Glyn Normington wrote: Hi Stanley Please could you provide an update on your work on interoperation between JSR 277 and JSR 291? If the work is still in progress, how's it coming along and roughly when might there be a strawman proposal to review? Thanks, Glyn
Re: Module system notification mechanism
Hi Bryan,
Bryan Atsatt wrote:
Um, I'm not arguing about the restriction in custom import policies, I
certainly understand why that is present.
I'm saying that an initializer *should* be able to rely on imported
modules, so we need a separate mechanism. And that separate mechanism
would kick in *after* resolution/validation had completed.
And yes, there is potentially a pathological case even here, if two
initializers have cyclic dependencies (not the Class kind ;^), but...
Seems to me that this has to be discovered and eliminated in
development; this is a variation on the order of static initialization
problem, and one that we can't solve here.
I think we agreed that in the presence of circular dependencies a module
cannot rely on its initializer being called before other code in the
module is called. The problem is that a module may not know if it is
part of a cyclic dependency chain because that depends on its imports,
their imports, etc. I don't think we can assume that this is a
pathological case that never occurs in the real world.
If a module cannot rely on its initializer to access other modules
safely or in the right order, then it is unclear to me if such support
would be very useful in practice. Instead, a module could rely on the
existing custom import policy or alternatively on other measures, for
example explicit checks in the code or infrastructure provided by the
container.
Are you just suggesting that we add start()/refresh() type methods to
ImportPolicy, rather than invent a new class/annotation? And that we
might then want to rename this class? For example, rename ImportPolicy
to ModuleInitializer, and add to it:
public abstract class ModuleInitializer {
public abstract ListModuleDefinition getImports(...);
public abstract void initialize(Module) throws Exception;
public abstract void release(Module);
}
If so, I'm OK with that approach as long as the class loading
restrictions during getImports() are made clear and the module state at
initialize() is clearly specified.
I'm fine with changing the name of the ImportPolicy class into
ModuleInitializer if it makes things more obvious to developers.
I think what you suggested is not unreasonable, but I want to make sure
this is something we really need to support, especially outside the
container environments. Specifically, I'm not yet convinced we should
provide an initialize() method that allows access to the imported
modules (excluding the standard SE modules) because of the potential
problems. Could you provide use cases for this outside a container
environment?
It is also unclear to me if providing the release() method would be
useful outside a container environment as well. As I hinted before, you
probably want to do the clean up in the finalizer or something similar
when nobody uses the module instance anymore, rather than when the
module instance is released from the module system while other existing
users could still access it.
- Stanley
Re: Module system notification mechanism
Hi Bryan, Bryan Atsatt wrote: Custom import policies cannot leverage any other modules; this would be a severe limitation for an initialization mechanism. There are good reasons why there is such restriction. When the module is being initialized, its imported modules might also be in the process of initialization. If one module accesses another module while both are in the middle of initialization, the result could be problematic. Also, suppose two modules have cyclic dependencies and each has its own initializer, what happens when an initializer is called in one module and that initializer wants to access the other module, while the other module's initializer wants to do the same? We could either prevent this from happening by saying the initializer could only access code in the standard SE modules and the code within its own module, or we could say that the initializer could access the module itself and all its imported modules - but developers need to be aware that there is a possibility that the imported module is only half-initialized when it is called by the importer's initializer. The latter could be very error-prone, and I'm not sure how useful it is to provide it to the developers. On the other hand, if we restrict which modules the initializer could access, then the initializer is not much different from the custom import policy - it's still a piece of code that is executed during module initialization, and exception would cause the module initialization to fail. In fact, the custom import policy was called ModuleInitializer in the original JSR 277 strawman, and the intention is to allow a module to do everything it needs to initialize itself properly. - Stanley
Re: Module system notification mechanism
Bryan Atsatt wrote: Stanley, I assume you are proposing a general mechanism in which any code can register listeners? If so, then I have to agree with Glyn that this is not sufficient to implement a module activator. The mechanism I have in mind can be registered only by code that have the appropriate security permissions, and untrusted code in the sandbox won't be able to listen to the notification. I am still a bit puzzled by why the module's state needs to be coupled directly to the result from the module's activator because this looks to me more like a solution to a problem than a requirement. For that model to work, I think there are couple underlying assumptions, but they might not be valid in our case: 1. If a module's activator is executed before the module is fully initialized, it can safely register something into some service lookup mechanism in the system. 2. The service lookup mechanism will make sure the registered thing from that module is not usable by other modules unless that module becomes fully initialized. 3. If the module's activator fails to execute when the module is being initialized, the module system knows how to work with the service lookup mechanism to undo all the unregisterations automatically and correctly. In this case, the module system probably needs to be highly integrated with the service lookup mechanism, and that mechanism should be the only one used by all the modules' activators. However, in practice, multiple service lookup mechanisms exist and can be used in the Java platform, and I don't think we should expect the module system will know how to work with these arbitrary mechanisms to provide the suggested module's state semantic in conjunction with the module's activator. Also, from our previous discussions, I think our consensus is to avoid pushing the service mechanism down to the module layer if possible. If the use case is that the module's activator should be called by the container when the module is fully initialized or is released, the notification mechanism should be sufficient for this purpose. If the use case is that a module instance should be released if its module's activator fails to start, the notification mechanism should still be sufficient, assuming the container will call ModuleSystem.releaseModule() after the container has received the module initialized notification but fails to start the activator. (Yes, there are side effects with releaseModule() as we discussed before, but it's a problem the container can probably deal with.) Is there any other use case that you think is important to consider but not allowed by this notification mechanism? - Stanley
Re: Module system notification mechanism
Hi Glyn, Glyn Normington wrote: Yes, but my point was that separating lifecycle out in that way would make it harder to enforce constraints like if a module's state is initialised, the module's activator completed successfully. The mechanism I suggested is simply for informing the application that something has happened. This is a notification only and does not change the state of the module system in any way. If we want to enforce constraints like you mentioned, one approach is to execute the activator code in the custom import policy. If this is not sufficient, then we'll probably need a different mechanism to handle this use case. - Stanley
Re: Strawman: Services and service-providers support
My original email did not seem to go through ... resend. - Stanley Original Message Subject: Re: Strawman: Services and service-providers support Date: Tue, 12 Jun 2007 17:07:05 -0700 From: Stanley M. Ho [EMAIL PROTECTED] Organization: Sun Microsystems, Inc. To: Java Community Process JSR #277 Expert List [EMAIL PROTECTED] Hi JSR 277 experts, Since I have not heard any further input on the services and service-providers strawman, I suppose the EG is fine with the strawman overall except the issue raised by Richard. Unless I hear any objection, I will incorporate the appropriate portion of the strawman based on the feedback you have provided into the next revision of the specification. Thanks, - Stanley P.S. I am still working on getting the strawman out to the observers through the infrastructure on openjdk.org, and it will take some time.
Re: Module isolation
Resend. Bryan Atsatt wrote: ... Stanley's idea was that a Repository implementation could return a *copy* of a ModuleDefinition created by a different Repository. Such a Repository would in fact form an isolation context. And caching Module instances becomes trivial: each definition can just have one as a field. While there are definite lifecycle issues here between such repository instances, they are probably solvable. And then we really *do* have an exact analogue of the loader delegation model, in which: Repository ~= ClassLoader ModuleDefinition ~= Class Module ~= Object Isolation of classes requires different Class instances. So isolation of modules requires different ModuleDefinition instances. Yes, that's the basic idea for isolation. In addition, the ModuleSystem object is the one actually handling the module instances' instantiation, initialization, and release, so the repository implementors do not have to be aware of most of the module runtime's complexities. - Stanley
Re: JSR 277 EG observer mailing list
This is a resend. Glyn Normington wrote: Hi Stanley Any chance of updating the public part of the JSR 277 page? The content of the public page is for the original JSR proposal, and is administrated and maintained by the JCP PMO office. I have contacted the office and the answer is no. Anyway, I think mentioning the observer list in the openjdk page + blogs + google should be sufficient for most people. - Stanley
Re: JSR 277 EG observer mailing list
Hi Bryan, Bryan Atsatt wrote: How are people supposed to find this? Shouldn't there be an easily spotted link somewhere on the public 277 page? Yes, it is mentioned in the JSR 277 community update page: http://jcp.org/en/egc/view?id=277 but unfortunately it requires JCP member login. It is also mentioned in the OpenJDK project which will cover the reference implementation (RI) of JSR 277 and JSR 294, and hopefully people will be able to find it: http://openjdk.java.net/projects/modules - Stanley
Re: Exported resources
Hi Bryan, I would like to get closures on a few open issues first, so my responses will be limited to those threads rather than all the new threads that have been started recently. Bryan Atsatt wrote: Hi Stanley, Sorry if I'm not being clear. Let me try to summarize: - I *do* want 277 to support private (non-exported) resources, but Good. - I do not think we should use permissions as the enforcement mechanism. - I believe the 'caller-class-is-member' enforcement mechanism is sufficient, and - I do realize that this means ResourceBundle.getBundle() (or similar) will fail to find private resources; these must be made public. Let me give more detailed reasoning for this position... You are of course correct that a class loader is free to add permissions to any class by assigning a ProtectionDomain with those permissions during defineClass(). (Let's call these hard-wired permissions.) So permission to access private resources could easily be hard-wired for other classes *in the same module*. To this point, there really isn't any meaningful difference between using permissions or using my approach: both allow module classes to directly access their private resources. But if we want to grant that same permission to *any* class outside of the module, it gets far more interesting. And it is this problem that I'm concerned with. For ResourceBundle.getBundle(), for example, to access a private resource, we must grant permission to the *ResourceBundle* class. And how do we do that? We could hard-wire permission for ResourceBundle, but... what about ServiceLoader? Ok, so let's just hard-wire permission for any JRE module. The system classes are granted with all permissions, so they will have sufficient permissions to access the private resources in a module. I don't think this is a real issue. Problem solved? Not at all: what about all the non JRE frameworks out in the world that need the same kind of access? I agreed that using other frameworks might be a concern. Clearly we cannot hard-wire permissions for these. So now we need to use policy file entries to do so. We could of course take this approach, but it either leaves some poor admin running around to update policy files as things break, or some fancy mechanism to grant access during framework module install. Ick. If the framework is bundled as part of your module, it will have the same set of permissions as the other code in your module, so it will have sufficient permissions to allow your code to access the private resources. If the framework is deployed through the typical mechanisms (e.g. extension classpath, etc.), it will have all permissions, hence it will also have sufficient permissions to access the private resources. Further, if the framework is deployed as another module in one of the system's repositories, it will likely have all permissions to allow your code to access the private resources in your module as well. Moreover, if the framework JAR/module is signed and is trusted, the system will typically grant all permissions to the code in the framework when the code is loaded. In other words, there is no need to explicitly grant permissions in most scenarios. The scenario where this may be an issue is when the framework is deployed through some other means (e.g. other modules in a URLRepository, JARs in custom classloaders) *and* the framework is unsigned (or if the framework is signed but not trusted for some reasons), it may not have sufficient permissions to allow your code to access the private resources through the framework API. However, it is unclear to me if there are frameworks deployed like this for this issue to become a real problem. Do you have any example that shows this is a real problem? And, hard-wired or not, permissions *are* a friend model. But 294 is not likely to provide such a model for classes. So, even if we hard-wire permission for ResourceBundle to access a private resource, how is it going to access a private ListResourceBundle class? Or how will the ServiceLoader gain access to the service provider class? How will third-party frameworks solve this problem? In a typical classloader, all public and private classes are visible (return from loadClass()) externally but only the public ones are accessible (invoke without access control failure). When a module uses the ResourceBundle API to load the ListResourceBundle from the module itself, the ResourceBundle API only loads the class from the module classloader and the class is returned to the code in the module for actual access. ServiceLoader works in similar way that the ServiceLoader only loads the service-provider class but it's the actual caller who accesses the class. Therefore, I don't think there is a real problem here. It is hard for me to see how all this complexity is justified, simply to hide resources that have never been hidden before. This doesn't mean there is no place for private resources! It just means that they can
Re: Query...
Bryan Atsatt wrote: And we haven't done so in the spec, either. I think we should. Query was not declared as Serializable in the spec because it was unclear if we wanted to allow custom Query implementations and how it might impact these custom Query implementations. Now that we have decided to allow custom Query implementations, if we make the Query serializable, the custom implementations will have to implement serialization properly, but I think this is still okay. However, if the query is serialized for it to be sent over the wire to a remote repository, then its usefulness is pretty much limited to the default query, because it is very unlikely that the remote repository will happen to have the custom implementation classes in place for the remote repository to deserialize the custom query properly. If the EG still thinks this is a useful thing to add, I can incorporate it into the next revision of the specification. - Stanley
Re: Relationship to JSR 291 [was: Re: Bryan's comments]
Hi Bryan, Bryan Atsatt wrote: ... Rather than surfacing the import-by-package at the API level, I think we can solve #3 by generalizing the import dependency concept as import-by-name, and this name can be mapped to superpackage-name in JSR 277 and OSGi-bundle-name or OSGi-exported-package-name in OSGi (i.e. exposing OSGi bundles and exported packages through implementing OSGiBundleModuleDefinition and OSGiPackageModuleDefinition). I think this is also what you intended to say at one point (correct me if I misunderstand you) by suggesting to change ImportDependency.getModuleName() into something like ImportDependency.getName(). I believe we are now thinking along the same lines. Good to know. I just sent an email on the topic of generalizing the import dependency concept, with the subject Import constraints. With the approach I suggest there, the distinctions between module-name and package-name are mostly transparent to the runtime (declarative support for package-name is not addressed). I will reply to that thread later. In case if the 277 module would like to import these OSGi exported packages using an OSGi-like resolution algorithm, it should be possible for it to use a custom import policy (or import override policy or maybe some new override mechanisms) to get the behavior it wants. Do you think this is a workable approach to address #3? I don't think custom code should be required; we need to make it easier than that or it doesn't qualify as first-class citizenship, at least not to me :^). The OSGi resolution algorithm has evolved significantly between R3 and R4, and it will likely continue to evolve in certain degree in future releases (Glyn/Richard, please correct me if I'm wrong). From this perspective, I don't think it makes sense to have build-in OSGi-like resolution algorithm in 277, because it will stick forever once 277 is part of the SE platform and cannot be evolve easily. I think it would makes more sense to allow OSGi-like resolution logic to be plugged into the 277 module which imports OSGi bundles/packages. We could argue whether it should be plugged in per-module in the form of custom import policy, or per-system(or per-repository?) in the form of import override policy, or maybe even have a new mechanism to allow certain class of modules (e.g. 277 modules importing OSGi modules, or 277 modules importing NetBeans modules, etc.) to always use a specific import policy automatically. But my point is that OSGi-like resolution logic is something that will need to evolve in the future and should not be built-in. I think if there is a way to plug in OSGi-like resolution logic into 277 module which imports OSGi bundles, it would still qualify as first-class citizenship to me. ;) And we don't yet have *any* import declaration mechanism (yes, we're assuming that this will be handled in 294, but it hasn't been brought up yet). I don't see this as a difficult issue either way. It would be natural to have both import-by-name and import-by-package in superpackages, if we have import at all. If we need to use annotations instead, it is easy to have both types. As we discussed (and I think you also agreed), we can make the import dependency to be generic with a name, and the name could be a module name (or a package name in the context of OSGi interoperability). We'll only need to have a single import-by-name either in the superpackage or in the form of annotation. - Stanley
Re: Relationship to JSR 291 [was: Re: Bryan's comments]
Hi, Glyn Normington wrote: *Bryan Atsatt [EMAIL PROTECTED]* wrote on 30/05/2007 07:57:59: ... So the open issue is the richness of the import language: must we support only lowest-common-denominator, or can we do better without over-complicating the design? I for one would like to be able to have a single module express dependencies on modules from both the same and different ModuleSystems, *using the standard semantics of each*. This may be reaching too far, but we should at least explore it seriously while we figure out what interop means here... At this point, I feel that is likely to be reaching too far, but I'm happy to play along and see what we can learn along the way. I agreed with Glyn that this might be reaching too far. Before we dive too much into how to implement interoperability, I think one of the outstanding questions we should answer first is what degree of import interoperability we want to offer. There are four possibilities: 1. JSR 277 module imports OSGi module by module name 2. OSGi module imports JSR 277 module by module name 3. OSGi module imports JSR 277 module by package name 4. JSR 277 module imports OSGi module by package name Let's ignore the 294 issues and the module initialization issues for now to simplify this discussion. I think we all agreed #1 and #2 are important to support and the reflective APIs already enable these (of course, there are minor issues we still have to address as we evolve the APIs.) That said, it is unclear to me how important it is to support #3 and #4, so I think the first question for this EG is whether we want to support #3 and #4 at all. However, let's pretend #3 and #4 are important for this specific discussion. I think #3 is already possible since the OSGi framework could look up the appropriate module which exported the package in the repository, using the reflective APIs. Afterwards, the OSGi framework can then do the necessary wiring. Yes, the reflective APIs can be further refined to make it easier for the OSGi framework to query modules by exported package name from the repository, but this is a minor issue that we can address easily. #4 is a bit complicated, because there is no import-by-package semantic in the current module layer's APIs and I think we all agreed that we don't want to support this semantic directly in the module system defined by JSR 277. In this context, the question is really about what it would take to make #4 possible if we want to support it, and so far there are two different approaches we have discussed: a. Treat import-by-package separately from import-by-module in the reflective APIs and make the module layer fully aware of the import-by-package concept. b. Expose exported-package using the ModuleDefinition abstraction, and provide necessary hooks (e.g. consistency checking) for the module system to resolve this kind of dependency appropriately. The module layer is not aware of the import-by-package concept at all. New import dependency granularity could also be introduced in a similar manner in the future without requiring significant changes in the module layer. As Richard and I discussed in previous emails, it seems to make the most sense for JSR 277 to define the minimal set of features possible in the module layer to accomplish what needs to be accomplished to provide core modularity support in the Java platform. I think (b) follows this principle nicely while (a) does not. If we decide to support #3 and #4, then how we want to address #4 would impact our overall design significantly. Is there anyone in the EG provide good reasons why #3 and #4 are important to be supported? If so, is there anyone in the EG think (b) is not feasible at all and we should do (a) instead to support #4? Can you explain your rationale? RESOLUTION MODELS The current design requires that each ModuleSystem provide its own resolution logic, and that each definition will be resolved by its owning ModuleSystem. This model appears to provide flexibility for significant differences in implementation, but we really don't know enough at this point. Perhaps only an actual second implementation will tell us if this provides useful flexibility. In the existing module systems we are aware, e.g. OSGi and NetBeans, we already know their resolution logics are different from each other. Even if they are migrated/re-based to JSR 277 in the future, I think we should expect their resolution logic would remain the same to maintain backward compatibility with their existing modules/bundles. In other words, their resolution logic would still be different from each other. A phased approach would be particularly beneficial if the initial phase could be delivered as part of Java 7 and subsequent phases implemented strictly on top of Java 7. But getting the API right up front might be tricky unless we can spot some really good abstractions or prototype the later phases sufficiently well. Is that
Re: Service providers
Hi Richard, Richard S. Hall wrote: Hey Stanley, ... To me, this sounds like what we call the extender model for the OSGi framework. The way it works is that bundles that want to participate in certain scenarios simply include some metadata inside of themselves, then some infrastructure can probe for this metadata and automatically do work on their behalf. This is how Declarative Services works in the R4 spec and also how Spring-OSGi works...it is becoming a very common model. The benefit of this approach is that it doesn't push upper layer concepts down into the modularity layer and keeps it simple. Off the top of my head, I don't see a reason why the Service Loader couldn't use a similar approach. - richard Thanks. I will certainly check it out to see if this approach is feasible. - Stanley
