Another situation that can occur in an object stream, is an outer object, dependency injects an object during construction. So a Bundle further down the stack (the stack is built during deserialization, not to be confused with the call stack) can resovle a class, but the immediate encapsulating object's Bundle cannot. To avoid creating a new incompatible bundle all preceeding bundles in the stack for that serialized object graph should be tried before installing a bundle from a codebase annotation.
Clearly we need to know if we're deserializing into an OSGi framework and modify the class resolution strategy to suit. Cheers, Peter. Sent from my Samsung device. Include original message ---- Original message ---- From: Peter <j...@zeus.net.au> Sent: 01/02/2017 01:19:36 pm To: dev@river.apache.org <dev@river.apache.org> Subject: Re: OSGi Thanks Nic, So it follows that the caller has visibility to all imported packages from other Bundle's ClassLoaders. But not packages from those bundles that haven't been imported. Which is the reason I propose switching context with each object in a serialized graph, to gain the visibility of the current object's bundle ClassLoader during deserialization, in order to be able to deserialize implementation classes. We need the codebase annotation when a class isn't reachable from the current context. Each class in a serialization stream is only resolved once. The codebase annotation is exact, it has an identity defined by it's normalized URI, which may be a specific version in a specific location, it should not contain a version range, or it will cause annotation loss and prevent loading of new proxy versions as Gregg mentioned earlier. A proxy class will not be visible from the clients calling stack Hence the proxy's exact bundle will be loaded. It is then of critical importance that the service api bundle is resovled, visible to both the client and proxy. The way the proxy bundle is wired depends on it's manifest package imports and the service api version currently loaded in the client jvm. Hence the service needs to publish the api package ranges it's proxy imports in an Entry (we need a new Entry in the Jini platform to support OSGi). The client must be using the latest currently loaded service api packages within the proxy's import ranges, as they will be selected by the framework for the proxy. There is a special case with the uses package declaration we need to understand and document, as this may cause selection of an earlier api package version. Hence Package Admin Service will need to be utilised to update the client before unmarshalling the proxy under rare circumstances when more than one service api version is in use at the client.. The codebase annotation should only be needed during deserialization for proxy and dependency injected objects. Proxy's and dependency injected objects will never be resolved to alternate bundle versions or a range This also prevents codebase annotation loss I think we're getting closer to understanding a possible solution. Cheers, Peter. Sent from my Samsung device. Include original message ---- Original message ---- From: Niclas Hedhman <nic...@hedhman.org> Sent: 01/02/2017 09:57:28 am To: dev@river.apache.org Subject: Re: OSGi As I think you know, the whole purpose of OSGi is to NOT tie the resolution to Bundles, but to explicitly Exported (with versions) packages. If people screw up and don't declare the Import/Export package meta data correctly, then ordinary OSGi may fail just as spectacularly. The difference being that Java serialization is slightly more sensitive to class changes than the code itself, and that was an issue from the start. "Back then" it was quickly realized that "long-term" storage of serialized state was a bad idea, but "long-term" is relative to code releases and with highly distributed systems, this is now a reality in networked applications as well as storing serialized objects to disk. With that in mind, one should seriously reconsider the status of Java Serialization in one's application, realize that every serialized class, incl possible exceptions, is a published API and needs to be treated as such, just as a REST API, SOAP interface or HTTP itself. Sorry for the rant... Back to OSGi; even if you really need to tie a class to a particular bundle, you can do that with attributes and the 'mandatory' directive. Cheers Niclas On Wed, Feb 1, 2017 at 3:29 AM, Michał Kłeczek <michal.klec...@xpro.biz> wrote: > Unfortunately it is not going to work in the general case as I've shown in > another message. > > The issue is that since you do not have influence on how the bundles of > ServiceProxy and ImplementationDetailServiceProxy are resolved, you cannot > guarantee that ImplementationDetail interface is going to be resolved from > the same bundle for both. > > Michal > > On Tue, 31 Jan 2017 at 06:38, Bharath Kumar <bharathkuma...@gmail.com> > wrote: > > > With help of some information from service providers, we can implement > > this use case in osgi environment easily. > > > > Suppose if we split these classes as separate bundles, we will have 2 > sets > > of bundles. > > > > 1. ImplementationDetail.api - contains ImplementationDetail service > api > > class and its dependencies > > 2. ImplementationDetail.proxy - smart proxy for ImplementationDetail > > service > > 3. ImplementationDetail.service - ImplementationDetail service > > implementation > > 4. ImplementationDetail.bootstrap - service bootstrap bundle for > > ImplementationDetail > > service. > > > > ImplementationDetail.bootstrap bundle creates the service and annotate > the > > codebase location. > > > > > > 1. forClient.api - contains ForClient service and its dependencies > > 2. forClient.proxy - smart proxy(ServiceProxy) for ForClient > > service. It imports > > classes from ImplementationDetail.api bundle and forClient.api bundle. > > 3. forClientservice - service implementation(ServiceBackend) and > > depends on forClient.proxy and forClient.api bundles > > 4 forClient.bootstrap - service bootstrap bundle for ForClient > service > > > > > > Here fooClient.proxy dependens on ImplementationDetail service. So it > can't > > be created until ImplementationDetail service is available in the > network. > > So forClient.bootstrap bundle registers a ServiceTemplate instance into > > osgi registry. This will help to find ImplementationDetail service from > the > > network. > > > > Once ImplementationDetail service is available in the network, > > forClient.bootstrap > > bundle creates the ServiceBackend and creates the proxy using > > ImplementationDetail service's proxy. Here we can pass the proxy > locations > > of ImplementationDetail.proxy and forClient.proxy bundles. > > > > > > *How to find codebase location of a service proxy?* > > > > Before registering the service, we can create an Entry object with code > > location. This information can be used by any service which will wrap > > another service proxy. As we know this information, we can create this > > Entry object. > > > > > > *Codebase information* > > > > If we have the below information in the codebase annotation, then > > RMIClassLoaderSPI can use these information to install or find the > correct > > compatible bundle in the client environment. > > > > - Service API bundle symbolic name, its version and location > > - Proxy bundle symbolic name , its version and location > > - Third party osgi bundles (libraries), their version and locations > > (optional). > > > > > > Even If we annotate the JERI based proxy with service api bundle > > information, we don't even need to patch the below classes. > > > > 1. BasicInvocationDispatcher > > 2. ServiceDiscoveryManager > > > > Because, RMIClassLoaderSPI can find the correct compatible service api > > bundle in client environment using codebase annotation. > > > > I'll create a POC and will share my experiences. > > > > > > On Tue, Jan 31, 2017 at 10:01 AM, Gregg Wonderly <ge...@cox.net> wrote: > > > > > The annotation for the exported services/classes is what is at issue > > > here. Here’s the perspectives I’m trying to make sure everyone sees. > > > > > > 1) Somehow, exported classes from one JVM need to be resolved in > another > > > JVM (at a minimum). The source of those classes today, is the codebase > > > specified by the service. A directed graph of JVMs exchanging classes > > > demands that all service like JVMs provide a codebase for client like > > JVMs > > > to be able to resolve the classes for objects traveling to the client > > form > > > the service. This is nothing we all don’t already know I believe. > > > > > > 2) If there is a 3rd party user of a class from one JVM which is handed > > > objects resolved by a middle man JVM (as Michal is mentioning here), > > there > > > is now a generally required class which all 3 JVMs need to be able to > > > resolve. As we know, Jini’s current implementation and basic design is > > > that a services codebase has to provide a way for clients to resolve > the > > > classes it exports in its service implementation. In the case Michal > is > > > mentioning, the demand would be for the middle man service to have the > > > classes that it wants the 3rd service to resolve, in some part of its > > > codebase. This is why I mentioned Objectspace Voyage earlier. I > wanted > > to > > > use it as an example of a mechanism which always packages class > > definitions > > > into the byte stream that is used for sending objects between VMs. > > Voyager > > > would extract the class definitions from the jars, wrap them into the > > > stream, and the remote JVM would be able to then resolve the classes by > > > constructing instances of the class using the byte[] data for the class > > > definition. > > > > > > Ultimately, no matter what the source of the byte[] data for the class > > > definition is, it has to be present, at some point in all VMs using > that > > > definition/version of the class. That’s what I am trying to say. The > > > issue is simply where would the class resolve from? I think that class > > > definition conveyance, between JVMs is something that we have choices > on. > > > But, practically, you can’t change “annotations” to make this work. If > > the > > > middle man above is a “proxy” service which bridges two different > > networks, > > > neither JVM on each network would have routing to get to the one on the > > > other side of the proxy JVM. This is why a mechanism like Objectspace > > > Voyager would be one way to send class definitions defined on one > network > > > to another JVM on another network via this proxy service. > > > > > > Of course other mechanisms for class conveyance are possible and in > fact > > > already exist. Maven and even OSGi provide class, version oriented > > > conveyance from a distribution point, into a particular JVM instance. > > Once > > > the class definition exists inside of one of those JVMs then we have > all > > > the other details about TCCL and creation of proper versions and > > resolution > > > from proper class loaders. > > > > > > I don’t think we have to dictate that a particular class conveyance > > > mechanism is the only one. But, to solve the problem of how to allow > > > classes hop between multiple JVMs, we have to specify how that might > > work > > > at the level that service instances are resolved and some kind of class > > > loading context is attached to that service. > > > > > > The reason I am talking specifically about directed graphs of class > > > loading is because I am first focused on the fact that there is a lot > > less > > > flexibility in trying to resolve through a large collection of specific > > > classes rather than an open set of classes resolved through a directed > > > graph of the code execution path which exposes the places and moments > of > > > object use in a much more controlled and natural way to me. > > > > > > Gregg > > > > > > > On Jan 30, 2017, at 9:14 AM, Michał Kłeczek (XPro Sp. z o. o.) < > > > michal.klec...@xpro.biz> wrote: > > > > > > > > It looks to me like we are talking past each other. > > > > > > > > Thread local resolution context is needed - we both agree on this. > > > > What we do not agree on is that the context should be a single > > > ClassLoader. It has to be a set of ClassLoaders to support situations > > when > > > dependencies are not hierarchical. > > > > > > > > The use case is simple - I want to implement "decorator" services > that > > > provide smart proxies wrapping (smart) proxies of other services. > > > > I also want to have Exporters provided as dynamic services which > would > > > allow my services to adapt to changing network environment. > > > > > > > > And I would like to stress - I am actually quite negative about OSGI > > > being the right environment for this. > > > > > > > > Thanks, > > > > Michal > > > > > > > > Gregg Wonderly wrote: > > > >> Maybe you can help me out here by explaining how it is that > execution > > > context and class visibility are both handled by OSGi bundles. For > > > example, one of my client applications is a desktop environment. It > does > > > service look up for all services registrations providing a “serviceUI”. > > It > > > then integrates all of those services into a desktop view where the UIs > > are > > > running at the same time with each one imbedded in a JDesktopPane or a > > > JTabbedPane or a JFrame or JDialog. There are callbacks from parts of > > that > > > environment into my application which in turn is interacting with the > > > ServiceUI component. You have AWT event threads which are calling out, > > > into the ServiceUIs and lots of other threads of execution which all, > > > ultimately, must have different class loading environments so that the > > > ServiceUI components can know where to load code from. > > > >> > > > >> It’s exactly TCCL that allows them to know that based on all the > other > > > class loading standards. The ClassLoader is exactly the thing that all > > of > > > them have in common if you include OSGi bundles as well. The important > > > detail, is that if the TCCL is not used as new ClassLoaders are > created, > > > then there is no context for those new ClassLoaders to reference, > > > universally. > > > >> > > > >> The important details are: > > > >> > > > >> 1) The desktop application has to be able to prefer certain > Entry > > > classes which define details that are presented to the user. > > > >> 2) When the user double clicks on a services icon, or right > > clicks > > > and selects “Open in new Frame”, an async worker thread needs a TCCL > > > pointing at the correct parent class loader for the service’s > > > URLClassLoader to reference so that the preferred classes work. > > > >> 3) Anytime that the AWT Event thread might be active inside of > > the > > > services UI implementation, it also needs to indicate the correct > parent > > > class loader if that UI component causes other class loading to occur. > > > >> 4) I am speaking specifically in the context of deferred class > > > loading which is controlled outside of the service discovery moment. > > > >> > > > >> > > > >>> On Jan 30, 2017, at 4:04 AM, Michał Kłeczek (XPro Sp. z o. o.) < > > > michal.klec...@xpro.biz> <mailto:michal.klec...@xpro.biz> wrote: > > > >>> > > > >>> What I think Jini designers did not realize is that class loading > can > > > be treated exactly as any other capability provided by a (possibly > > remote) > > > service. > > > >>> Once you realize that - it is possible to provide a kind of a > > > "universal container infrastructure" where different class loading > > > implementations may co-exist in a single JVM. > > > >> > > > >> That’s precisely what ClassLoader is for. TCCL is precisely to > allow > > > “some class” to know what context to associate newly loaded classes > with, > > > so that in such an environment, any code can load classes on behalf of > > some > > > other code/context. It doesn’t matter if it is TCCL or some other > class > > > management scheme such as OSGi bundles. We are talking about the same > > > detail, just implemented in a different way. > > > >> > > > >>> What's more - these class loading implementations may be dynamic > > > themselves - ie. it is a service that provides the client with a way to > > > load its own (proxy) classes. > > > >>> > > > >>> In other words: "there not enough Jini in Jini itself”. > > > >> > > > >> I am not sure I understand where the short coming is at then. Maybe > > > you can illustrate with an example where TCCL fails to allow some piece > > of > > > code to load classes on behalf of another piece of code? > > > >> > > > >> In my desktop application environment, there is a abstract class > which > > > is used by each serviceUI to allow the desktop to know if it provides > the > > > ability to open into one of the above mentioned JComponent subclasses > > > That class is preferred and provided and resolved using the codebase of > > the > > > desktop client. That class loading environment is then the place where > > the > > > service is finally resolved and classes created so that the proxy can > be > > > handed to the serviceUI component which ultimately only partially > > resolves > > > from the services codebase. > > > >> > > > >> It’s this class compatibility which needs to be lightweight. > > > >> > > > >>> We have _all_ the required pieces in place: > > > >>> - dynamic code loading and execution (ClassLoaders), > > > >>> - security model and implementation that allows restricting rights > of > > > the downloaded code, > > > >>> - and a serialization/deserialization which allows sending > arbitrary > > > data (and yes - code too) over the wire. > > > >>> > > > >>> It is just the matter of glueing the pieces together. > > > >> > > > >> Correct, but it’s a matter of class compatibility where a client > > > environment has to interact with a service and the serviceUI components > > > where TCCL excels and providing the ability to create class loaders > with > > > the correct parent context, for Java based code. OSGi introduces the > > > opportunity for some extra bells and whistles. But I don’t see that it > > can > > > completely eliminate the nature of TCCL and how it was intended to be > > used. > > > >> > > > >>> Thanks, > > > >>> Michal > > > >>> > > > >>> > > > >>> Gregg Wonderly wrote: > > > >>>> <snip> > > > >>>> I am not an OSGi user. I am not trying to be an OSGi opponent. > > What > > > I am trying to say is that I consider all the commentary in those > > articles > > > about TCCL not working to be just inexperience and argument to try and > > > justify a different position or interpretation of what the real problem > > is. > > > >>>> > > > >>>> The real problem is that there is not one “module” concept in Java > > > (another one is almost here in JDK 9/Jigsaw). No one is working > together > > > on this, and OSGi is solving problems in a small part of the world of > > > software. It works well for embedded, static systems. I think OSGi > > > misses the mark on dynamic systems because of the piecemeal loading and > > > resolving of classes. I am not sure that OSGi developers really > > understand > > > everything that Jini can do because of the choices made (and not made) > in > > > the design. The people who put Jini together had a great deal of years > > of > > > experience piecing together systems which needed to work well with a > > faster > > > degree of variability and adaptation to the environment then what most > > > people seem to experience in their classes and work environments which > > are > > > locked down by extremely controlled distribution strategies which end > up > > > slowing development in an attempt to control everything that doesn’t > > > actually cause quality to suffer. > > > >>>> > > > >>>> Gregg > > > >>>> > > > >>>> > > > >> > > > > > > > > > > > > > -- Niclas Hedhman, Software Developer http://polygene.apache.org <http://zest.apacheorg> - New Energy for Java
