Re: Ugly things done to support multiple ContentHandlerFactory and URLStreamHandlerFactory
Alan, Tom, On 13/10/16 19:59, Alan Bateman wrote: .. Speaking of net-dev, then maybe this thread should move there as this topic is really more of a URL issue rather than module system issue. I have replied to this over on net-dev [1] ( I hope that is ok ). We can continue the discussion there. -Chris. []1 http://mail.openjdk.java.net/pipermail/net-dev/2016-October/010380.html
Re: Ugly things done to support multiple ContentHandlerFactory and URLStreamHandlerFactory
On 13/10/2016 19:40, Thomas Watson wrote: I don't think service loader lookup is going to help us here. Each framework instance will need its own provider but the set of providers will be constant to what is discovered on the system class loader. As additional frameworks come up their providers will have no chance to be discovered. Furthermore each framework implementation can be loaded by a non-system class loader. For the usecase I am concerned about the framework implementation will not be loaded by the system class loader but instead by some dynamic class loader that can be thrown away when the framework instance is torn down. For Eclipse a small launcher is the only jar on the system class loader. The framework itself is loaded with a custom class loader the launcher creates. I should have been clearer. I was trying to point out that you can locate URLStreamHandlerProvider implementations that aren't built-in or deployed on the class path yourself. That should be enough to create the URLStreamHandler and do the wrapping that I think you are doing. : I have not been able to determine why an SPI is needed for providing URLStreamHandlerFactory implementations to the service loader but not for ContentHandlerFactory implementations. The URLStreamHandlerProvider is just an empty abstract class that implements URLStreamHandlerFactory. Why couldn't the provider type just be URLStreamHandlerFactory directly similar to how ContentHandlerFactory is? This is a security sensitive area and the permission check needed here couldn't be reliably enforced if the interface URLStreamHandleFactory were the service type. There is discussion on net-dev where this point was discussed. Speaking of net-dev, then maybe this thread should move there as this topic is really more of a URL issue rather than module system issue. -Alan
Re: Ugly things done to support multiple ContentHandlerFactory and URLStreamHandlerFactory
> From: Alan Bateman > The javadoc for the 4-arg URL constructor has all the details on how it > interacts with the system-wide stream handler factory, also how the > system class loader is used to locate protocol handlers that are > intended to used system-wide (this includes the ability to override > non-core protocol handlers). I've no doubt that this won't be exactly > what you want but I hope you can see how URLStreamHandlerProvider is > used as the service type. I don't think service loader lookup is going to help us here. Each framework instance will need its own provider but the set of providers will be constant to what is discovered on the system class loader. As additional frameworks come up their providers will have no chance to be discovered. Furthermore each framework implementation can be loaded by a non-system class loader. For the usecase I am concerned about the framework implementation will not be loaded by the system class loader but instead by some dynamic class loader that can be thrown away when the framework instance is torn down. For Eclipse a small launcher is the only jar on the system class loader. The framework itself is loaded with a custom class loader the launcher creates. It appears we will have to continue to rely on the static methods to set our factory instances. > > There are also updates to allow ContentHandlerFactory implementations be > deployed as modules. The details on that are in the javadoc for > URLConnection::getContent. This one did not need introducing a new > service type. I have not been able to determine why an SPI is needed for providing URLStreamHandlerFactory implementations to the service loader but not for ContentHandlerFactory implementations. The URLStreamHandlerProvider is just an empty abstract class that implements URLStreamHandlerFactory. Why couldn't the provider type just be URLStreamHandlerFactory directly similar to how ContentHandlerFactory is? Thanks for your time. Tom
Re: Ugly things done to support multiple ContentHandlerFactory and URLStreamHandlerFactory
On 13/10/2016 17:23, Thomas Watson wrote: : Thanks for the pointer Alan. I'll have a look. Is there any additional documentation on this besides the javadoc? At first glance it is unclear to me what discovers the providers? Is the the URL class itself based on the thread context class loader? How do providers interact with the statically set factory. I also don't see a provider SPI type for java.net.ContentHandlerFactory. The javadoc for the 4-arg URL constructor has all the details on how it interacts with the system-wide stream handler factory, also how the system class loader is used to locate protocol handlers that are intended to used system-wide (this includes the ability to override non-core protocol handlers). I've no doubt that this won't be exactly what you want but I hope you can see how URLStreamHandlerProvider is used as the service type. There are also updates to allow ContentHandlerFactory implementations be deployed as modules. The details on that are in the javadoc for URLConnection::getContent. This one did not need introducing a new service type. -Alan.
Re: Ugly things done to support multiple ContentHandlerFactory and URLStreamHandlerFactory
> From: Alan Bateman > Have you looked at the changes in this area in JDK 9, specifically the > changes on how protocol handlers are located and the new URL stream > handler service-provider type URLStreamHandlerProvider? > > -Alan > Thanks for the pointer Alan. I'll have a look. Is there any additional documentation on this besides the javadoc? At first glance it is unclear to me what discovers the providers? Is the the URL class itself based on the thread context class loader? How do providers interact with the statically set factory. I also don't see a provider SPI type for java.net.ContentHandlerFactory. Tom
Re: Ugly things done to support multiple ContentHandlerFactory and URLStreamHandlerFactory
On 13/10/2016 16:46, Thomas Watson wrote: If not, are there going to be command line options that will allow us to open up deep reflection of specific VM types as a compatibility mode until we can get a proper solution? There is a command-line option, in this case `--add-exports-private java.base/java.net=ALL-UNNAMED` will allow code on the class path (or any custom class loader) to continue to break into non-public members of java.net.URL or other types in this package. -Alan
Re: Ugly things done to support multiple ContentHandlerFactory and URLStreamHandlerFactory
On 13/10/2016 16:46, Thomas Watson wrote: Now that jigsaw is prohibiting deep reflection on types provided by the boot modules we are running into an issue with the ugly things we do [1] in order to support multiple factories in Equinox (an OSGi implementation). Each Equinox instance has its own context aware ContentHandlerFactory and URLStreamHandlerFactory which serves up handlers that are available for the particular instance of Equinox. In order to do this we have to resort to some very ugly reflection. This allows us to gain access to the 'base' factory object set in the VM so that each instance of the framework can register itself with the 'base' factory. The 'base' factory handlers do the complicated work of figuring out which factory to call based on the context which is calling the base handler. We also resort to reflection on framework shutdown in order to flush our factories out from the VM so that the base factory is not pinning our framework in memory. I am also aware that the Apache Felix OSGi Framework implementation uses similar deep reflection in its implementation. Are there any plans in Java 9 that will make this scenario easier to achieve without resorting to the use of deep reflection on the URL types from the VM? If not, are there going to be command line options that will allow us to open up deep reflection of specific VM types as a compatibility mode until we can get a proper solution? At a minimum I think we need someway to unset the statically set handlers in the VM to avoid pinning our base factories. I can think of other non-reflective ways to gain access to the base handler instance in order to register each framework instance with the base handler. For example, the base factory could implement a specific protocol which allows a connection to register another framework. Have you looked at the changes in this area in JDK 9, specifically the changes on how protocol handlers are located and the new URL stream handler service-provider type URLStreamHandlerProvider? -Alan
Ugly things done to support multiple ContentHandlerFactory and URLStreamHandlerFactory
Now that jigsaw is prohibiting deep reflection on types provided by the boot modules we are running into an issue with the ugly things we do [1] in order to support multiple factories in Equinox (an OSGi implementation). Each Equinox instance has its own context aware ContentHandlerFactory and URLStreamHandlerFactory which serves up handlers that are available for the particular instance of Equinox. In order to do this we have to resort to some very ugly reflection. This allows us to gain access to the 'base' factory object set in the VM so that each instance of the framework can register itself with the 'base' factory. The 'base' factory handlers do the complicated work of figuring out which factory to call based on the context which is calling the base handler. We also resort to reflection on framework shutdown in order to flush our factories out from the VM so that the base factory is not pinning our framework in memory. I am also aware that the Apache Felix OSGi Framework implementation uses similar deep reflection in its implementation. Are there any plans in Java 9 that will make this scenario easier to achieve without resorting to the use of deep reflection on the URL types from the VM? If not, are there going to be command line options that will allow us to open up deep reflection of specific VM types as a compatibility mode until we can get a proper solution? At a minimum I think we need someway to unset the statically set handlers in the VM to avoid pinning our base factories. I can think of other non-reflective ways to gain access to the base handler instance in order to register each framework instance with the base handler. For example, the base factory could implement a specific protocol which allows a connection to register another framework. But we are still left with a delima of how to wrap the handlers returned by various framework factory instances from the base handlers. All the interesting methods on java.net.URLStreamHandler are protected. Even though the proxy base handler extends java.net.URLStreamHandler it is not allowed to call the protected methods on another instance of java.net.URLStreamHandler. We resorted to deep reflection on the java.net.URLStreamHandler class methods to do that. The java.net.URLStreamHandler class is not an interface so we cannot easily create proxy classes using java.lang.reflect.Proxy. Tom [1] https://bugs.eclipse.org/bugs/show_bug.cgi?id=502209
