Re: OSGi NP Complete Was: OSGi - deserialization remote invocation strategy
Also see the OSGi Enterprise specification, v6, Chapter 136, page 691, there's some discussion about the NP-complete nature of dependency resolution there as well. https://www.osgi.org/developer/downloads/release-6/release-6-download/ On 13/02/2017 5:19 PM, Peter wrote: OSGi Dependency resolution is. http://underlap.blogspot.com.au/2010/02/osgi-resolution-is-np-complete-so-what.html Which means if we want to support an OSGi environment properly, we may need some time to resolve the dependencies for a smart proxy, before deserializing the proxy, rather than downloading the proxy dynamically during unmarshalling, it's better to delay unmarshalling until the dependencies are resolved, so the client isn't impacted by delays. Cheers, Peter. On 13/02/2017 4:50 PM, Michał Kłeczek wrote: Sorry, NP Completness of what? I have been the first to mention NP hardness of constraint satisfaction problem but I am not sure if this is what you are asking about. Thanks, Michal Patricia Shanahan wrote: Are you literally claiming NP Completeness, or just using that as an analogy for really, really difficult?
Re: OSGi NP Complete Was: OSGi - deserialization remote invocation strategy
OSGi Dependency resolution is. http://underlap.blogspot.com.au/2010/02/osgi-resolution-is-np-complete-so-what.html Which means if we want to support an OSGi environment properly, we may need some time to resolve the dependencies for a smart proxy, before deserializing the proxy, rather than downloading the proxy dynamically during unmarshalling, it's better to delay unmarshalling until the dependencies are resolved, so the client isn't impacted by delays. Cheers, Peter. On 13/02/2017 4:50 PM, Michał Kłeczek wrote: Sorry, NP Completness of what? I have been the first to mention NP hardness of constraint satisfaction problem but I am not sure if this is what you are asking about. Thanks, Michal Patricia Shanahan wrote: Are you literally claiming NP Completeness, or just using that as an analogy for really, really difficult?
Re: OSGi NP Complete Was: OSGi - deserialization remote invocation strategy
Sorry, NP Completness of what? I have been the first to mention NP hardness of constraint satisfaction problem but I am not sure if this is what you are asking about. Thanks, Michal Patricia Shanahan wrote: Are you literally claiming NP Completeness, or just using that as an analogy for really, really difficult?
Re: OSGi NP Complete Was: OSGi - deserialization remote invocation strategy
Are you literally claiming NP Completeness, or just using that as an analogy for really, really difficult? On 2/11/2017 1:23 PM, Michał Kłeczek wrote: I am sorry but I think that to solve various issues we need to make sure fundamentals are right: 1. There is NO such a thing as "reflective non-smart" proxy - EVERY proxy is "smart" (even if it is "reflective") - there is an InvocationHandler down there, isn't there? 2. Solving this on service discovery level is trying to do it on the WRONG level of abstraction. Services DEPEND on class loading - not the other way around. 3. What you propose is a partial "solution". Not being able to register "smart" event listeners means no custom endpoints for example (UDPEndpoint anyone?) 4. Trying to squeeze partial solutions into the framework is IMHO a BIG no no. This is simply creating more code, more maintenance burden and more headache for users trying to workaround "edge, unsupported cases". Please - lets try to come up with the RIGHT solution that is going to REALLY fix class loading issues. Thanks, Michal Peter wrote: In a word, ServiceDiscoveryManager ServiceDiscoveryManager is the solution. ServiceDiscoveryManager performs discovery and looks up services from registrars based on filters. ServiceDiscoveryManager then performs local filtering. This allows time for proxy bundles to be installed, resolve, started and confirmed type compatible, prior to them being made available (via OSGi service registry if you so desire) for client use. The new interfaces that are part of JGDMS that I'd like to see their way into River, found here: https://github.com/pfirmstone/JGDMS/blob/Maven_build/modularize/JGDMS/jgdms-lib-dl/src/main/java/net/jini/lookup/SafeServiceRegistrar.java https://github.com/pfirmstone/JGDMS/blob/Maven_build/modularize/JGDMS/jgdms-lib-dl/src/main/java/net/jini/lookup/ServiceAttributesAccessor.java https://github.com/pfirmstone/JGDMS/blob/Maven_build/modularize/JGDMS/jgdms-lib-dl/src/main/java/net/jini/lookup/ServiceCodebaseAccessor.java https://github.com/pfirmstone/JGDMS/blob/Maven_build/modularize/JGDMS/jgdms-lib-dl/src/main/java/net/jini/lookup/ServiceIDAccessor.java https://github.com/pfirmstone/JGDMS/blob/Maven_build/modularize/JGDMS/jgdms-lib-dl/src/main/java/net/jini/lookup/ServiceProxyAccessor.java ServiceCodebaseAccessor is also used as part of secure discovery, but the codebase string and certs are transferred as primitives over the network. In this case codebase annotations don't need to be included in the stream, the JERI endpoints don't need them at all. How so? We can use ServiceItemFilter and ProxyPreparer to install, resolve and start out proxy codebase, before downloading the proxy. The interfaces listed above allow an array bootstrap proxy's (java.lang.reflect.Proxy) to be obtained from SafeServiceRegistrar. Firstly the bootstrap proxy's JERI endpoint will be loaded in the ServiceDiscoveryManager's ClassLoader, so after we've retrieved the codebase annotation and signers, created a bundle for the proxy, resolved it's dependencies (via OSGi resolution and repository services), we need to remarshall the bootstrap proxy into a MarshalledInstance, then unmarshall it using the ClassLoader of the recently started proxy bundle. Then when we cast the bootstrap proxy to ServiceProxyAccessor and retrieve the smart proxy, it will be loaded into the same ClassLoader that the bootstrap proxy uses, our newly provisioned and loaded bundle (the correct ClassLoader) without need to serialize any annotations, then the smart proxy can have constraints applied etc and be registered as an OSGi service with the OSGi registrar, where client code can interact with the remote proxy. Now if public Serializable classes that are imported by the proxy's bundle (service api) or private classes in the proxy's bundle can be deserialized and the JERI endpoint has a reference to the ClassLoader of the proxy. This should be good enough so we don't require the "bundle stack" proposed earlier, which also saves the need to explain it and simplifies the solution (the intent of the bundle stack was to allow deserialization of private classes within other bundles whose packages have been imported by the proxy). The client won't be able to pass a smart proxy to the service (like a Listener), but it can still pass a non smart proxy and it will still function. So clients can still export their own remote service (albiet without a codebase, excluding smart proxy's), but it'll be good enough for a listener etc. Cheers, Peter. On 8/02/2017 1:09 PM, Niclas Hedhman wrote: Maybe there are some misunderstanding somewhere... see below; On Wed, Feb 8, 2017 at 3:35 AM, Peter wrote: I'm currently only considering OSGi server -> OSGi client. Mick's investigating all four options. Ok, makes it a lot easier for me to follow. Not expecting the client calling bundle to resolve everything, hence the stack, so we have the full visibility of the bundl
Re: OSGi NP Complete Was: OSGi - deserialization remote invocation strategy
Interestingly ServiceCodebaseAccessor.getClassAnnotation() could be modified to accept a string parameter. This would allow different smart proxy codebases to be provisioned for different environments. By provisioning and deserializing into the ClassLoader for the provisioned codebase, a service can support multiple client environments. Although this doesn't provide the full gammut of seamlessly passing smart proxys and private serializable objects that extend the service api, as remote method arguments, which is an admirable and ideal goal, my pragmatic side tells me it's an acceptable compromise to constrain functionality within the known limitations of the jvm and ClassLoaders. Jim Waldo lamented (sorry don't have a )reference the change in Java 1.2 that tied class identity to ClassLoaders, see following paper: http://www.bracha.org/classloaders.ps The proxy ClassLoader of the local environment controlls class visibility, annotations are not required in the serialized object stream, if it is deserialised into the proxy ClassLoader. As Nic and Mic have mentioned OSGi's NP Complete resolution means that codebase provisioning is something that needs to performed by a background thread, it's not something that's practical to perform during unmarshalling, it needs to occur prior to deserialization. Gregg has demonstrated that delayed unmarshalling has significant performance benefits in desktop applications. Delayed unmarshalling also has security benefits, allowing authentication to be performed prior to codebase downloads and deserialization. Terms: Dynamic proxy - proxy that's generated dynamically, it doesn’t use or require a codebase. Smart proxy - proxy that performs local processing / computation, it requires a codebase. Cheers, Peter. Sent from my Samsung device. Include original message Original message From: Peter Sent: 12/02/2017 08:42:23 am To: dev@river.apache.org Subject: Re: OSGi NP Complete Was: OSGi - deserialization remote invocation strategy Thanks Michal, See inline below. On 12/02/2017 7:23 AM, Michał Kłeczek wrote: > I am sorry but I think that to solve various issues we need to make > sure fundamentals are right: > > 1. There is NO such a thing as "reflective non-smart" proxy - EVERY > proxy is "smart" (even if it is "reflective") - there is an > InvocationHandler down there, isn't there? Easy fixed, we change the terminology to a proxy without a codebase download, "dynamic proxy" was a term that Sun used for it previously. Sun used the terminology "Smart Proxy" to describe a proxy that had a codebase. Dynamic proxy's still have an InvocationHandler, it's part of one of the existing jeri implementations and already installed at both ends. > 2. Solving this on service discovery level is trying to do it on the > WRONG level of abstraction. Services DEPEND on class loading - not the > other way around. Can you explain a little more please? > 3. What you propose is a partial "solution". Not being able to > register "smart" event listeners means no custom endpoints for example > (UDPEndpoint anyone?) Yes, you can still have custom endpoints, but the endpoints need to be installed at both ends, prior. Initially, smart proxy's cannot be used for remote objects, only exported services. > 4. Trying to squeeze partial solutions into the framework is IMHO a > BIG no no. > This is simply creating more code, more maintenance burden and more > headache for users trying to workaround "edge, unsupported cases". Not really, it provides working solutions for the majority of cases, with minimal changes, low hanging fruit so to speak and allows adoption, the other cases can be worked on later, when there are more users and more developers to assist. The proxy implementation (provider) is abstracted behind the service api completely, typical of OSGi practise, ServiceDiscoveryManager allows time for service proxy's to be wired up, prior to deserialization using delayed unmarshalling. The consumer must use the service api to interact with the service and must honour the service api documentation, which may state which classes may be overridden, that providers must interact using certain classes in the local jvm and not transfer them remotely, a good way to do that is ensure they're not serializable. A listener is intended to be implemented, however the consumer would only be able to export a remote object that doesn't have a codebase and still be able expect the provider to deserialize it. Unless of course we create a new service that allows a client to request a bundle be installed, resolved and started in a remote end, allowing consumers to request remote provisioning. Still I'd like to allow the solution to grow, we shouldn't expect to be able to do it all at once. We can still agree that at some point addressing the remaining functionality would be nice
