Hi David Thanks for taking the time!
On 05.10.2012 14:04:06 David Bosschaert wrote: > Hi Jeremias, > > I took some time to look at your code, just echoing what I think that > it does to make sure that I understand things correctly. > I think it implements a dynamic services mechanism that works both > inside OSGi as well as outside of OSGi. Correct? Yes. > Services are declared using the META-INF/services resources, just like > they are done with ServiceLoader. The client side doesn't use the > java.util.ServiceLoader.load() API. Right. That's to enable Java 5 compatibility and to allow for more dynamics adding and removing plug-ins than java.util.ServiceLoader can offer. > In the case we're outside of OSGi there is a custom non-OSGi > ServiceTracker/ServiceListener that reacts to services arriving / > disappearing. Right now, it doesn't handle anything disappearing (the equivalent of ServiceLoader.reload()). It does one discovery run and sticks to the discovered set, practically never calling ServiceListener.removed(). > When you're inside OSGi you can look up these services from the > Service Registry as SPI-Fly has put them there. Exactly. I was really pleased with SPI Fly's approach even if it made some of my initial work obsolete (but it was flawed, too). > Am I correctly summarizing? Did I miss anything? Spot on. > Next question is would it make sense to add this functionality to Aries? > I think it does. To me many of the ideas in here match with the OSGi > Connect RFP 145 (http://www.osgi.org/bugzilla/show_bug.cgi?id=145) and > I think that, besides its practical use today, this code could be a > valuable input to the standardization process of OSGi Connect. Overall > the charter of OSGi Connect is to create a dynamic services > environment that works both inside OSGi and out. To me the overall > goal of your code seems similar. > If we all agree that it would be suitable for this component to reside > in Aries, I think we should strive to make it ultimately compliant > with the OSGi Connect spec, when that's available. > > Does this make sense to you? As I understand it OSGi Connect's goal is to use a subset of the OSGi framework (most importantly the service layer but not the module layer). So you can use the OSGi ServiceTracker to lookup services. In that case, my library isn't needed and probably not very useful, since it actually strives not to use OSGi APIs at all. So, I'm not quite getting your point here. I got about one too many hints that some people may have reservations when introducing OSGi to a plain Java project ("Do we all have to learn OSGi? Can I still use X in plain Java? etc."). OSGi, unfortunately, is still not as widely adopted as I would like. I've noticed how a low-level ServiceTracker can provoke reactions like: "Does it have to be that complicated?" At least, until they get the power of it. So, my main goal was to really just shield everyone from OSGi as much as possible. Basically, I just wanted to provide an easy migration path without the requirement to learn about OSGi beyond including manifest metadata. If my thingy helps OSGi Connect, that's great but I frankly don't see how. I'm probably still missing something. Jeremias Maerki > Best regards, > > David > > On 1 October 2012 14:24, David Bosschaert <[email protected]> wrote: > > Thanks for reminding me, Jeremias. I downloaded it but then I got > > distracted. Hope to have a look soon. > > > > David > > > > On 1 October 2012 03:27, Jeremias Maerki <[email protected]> wrote: > >> Hi David, > >> did you have any chance to look at it already? > >> > >> BTW, I've just found a possible memory leak in the OSGi case when the > >> client doesn't (or can't easily) remove the ServiceListener. Usually, > >> there's only one ServiceListener per client class (static context) so > >> it's not totally serious, but it prevents GC'ing the bundle. I'm working > >> on that now. > >> > >> Cheers, > >> Jeremias Maerki > >> > >> > >> On 30.08.2012 17:29:47 David Bosschaert wrote: > >>> Thanks Jeremias! > >>> > >>> I'll have a look at this soon. > >>> > >>> Cheers, > >>> > >>> David > >>> > >>> On 23 August 2012 15:39, Jeremias Maerki <[email protected]> wrote: > >>> > Hi David > >>> > > >>> > Sorry for the delay. As life plays, you get side-tracked all the time, > >>> > but since this is important to me I had to revisit eventually. > >>> > > >>> > I've simplified and cleaned up the code some more and added some tests. > >>> > It's now ready for review: > >>> > http://www.jeremias-maerki.ch/download/osgi/ch.jm.util.services-2.0.0.dev-src.zip > >>> > > >>> > So, if you still think this could be a good addition to Aries, I can > >>> > rename the packages, switch to a Maven build (sigh) and prepare a > >>> > package to vote upon for inclusion in Aries. > >>> > > >>> > What could we name it? > >>> > - org.apache.aries.spifly.consumer > >>> > - org.apache.aries.spifly.discovery > >>> > - org.apache.aries.discovery > >>> > - org.apache.aries.plugin.discovery > >>> > - org.apache.aries.spi.catch ;-) > >>> > > >>> > You mentioned earlier that you don't see it as part of SPI-Fly. In its > >>> > current state it actually relies on a working implementation of RFP 167 > >>> > (SPI Service Loader support) because I removed the extender that looks > >>> > for META-INF/services from 1.0 to 2.0. So, while it is not part of RFC > >>> > 167, it is still thighly connected to SPI-Fly, although not through > >>> > package imports. > >>> > > >>> > On RFP 143: like I already mentioned, I do find it interesting. It can > >>> > serve as a different approach to solving the same problem. However, my > >>> > approach gets away with a 20KB JAR, compared to the 220KB of PojoSR > >>> > (still small!). But my main argument is actually this: I need to find a > >>> > way to sell an easy way to the Apache XML Graphics project to achieve > >>> > OSGi compatibility including plug-in support without throwing everything > >>> > upside down. Practically none of the committers are familiar with OSGi. > >>> > Introducing an OSGi-compatible service registry is generally cool for > >>> > someone who knows OSGi already but the others would probably ask: why do > >>> > we need to learn about this? After all, the service registry can be a > >>> > bit verbose for a newbie with ServiceReference, BundleContext and such. > >>> > I believe that selling OSGi Connect would be more difficult in such a > >>> > case than my small proposal. > >>> > > >>> > In the end, I'm looking for the best home for it. I think Aries is a > >>> > logical place. > >>> > > >>> > When looking at PojoSR I was wondering about use cases. Personally, I'd > >>> > rather have a full OSGi environment any time even if it means that I > >>> > have to be able to maintain a bundle cache/storage folder on the local > >>> > file system (I actually wonder why these are not pluggable and > >>> > virtualizable, yet, but that's a different story). Use case 4.2 (WARs) > >>> > in the RFP draft makes a lot of sense, though. There you don't usually > >>> > have any service dynamics. Or a command-line application that uses some > >>> > code that usually runs in a OSGi-enabled server. I have such a case that > >>> > I might explore eventually. But frankly, OSGi Connect doesn't create too > >>> > much attraction for me since I will usually want to work in a full OSGi > >>> > environment. And when I can't do that I'm likely happy with plain > >>> > META-INF/services. I guess I can't correctly imagine the possibilities > >>> > that are mentioned in use case 4.3 (Application Frameworks). > >>> > > >>> > Cheers, > >>> > Jeremias Maerki > >>> > > >>> > > >>> > On 13.07.2012 16:07:58 David Bosschaert wrote: > >>> >> Hi Jeremias, > >>> >> > >>> >> Just FYI OSGi RFP 143 has just been published which relates precisely > >>> >> to the issue of using a Service Registry without going full-blown on > >>> >> the modularity. Ultimately the idea is that there will be an OSGi > >>> >> specification around this. > >>> >> Your work might fit with this idea as well and I would be very > >>> >> interested in your thoughts around this topic. > >>> >> > >>> >> See here: > >>> >> http://blog.osgi.org/2012/07/new-rfps-available-for-feedback.html > >>> >> and here: https://www.osgi.org/bugzilla/show_bug.cgi?id=145 > >>> >> > >>> >> Best regards, > >>> >> > >>> >> David > >>> >> > >>> >> On 13 June 2012 08:02, Jeremias Maerki <[email protected]> wrote: > >>> >> > Hi David > >>> >> > > >>> >> > Thanks a lot for your feedback! PojoSR is interesting. You could say > >>> >> > that there is a certain overlap. If I understand this right, PojoSR > >>> >> > is > >>> >> > suggesting to detect not-quite-OSGi bundles via > >>> >> > ClassLoader.getResources("META-INF/MANIFEST.MF") rather than to > >>> >> > detect > >>> >> > individual services via "META-INF/services/xy". It has a focus on > >>> >> > maintaining a service registry that is based on OSGi principles > >>> >> > (bundle > >>> >> > activator and all) rather than working off the old JAR service > >>> >> > providers. > >>> >> > It brings OSGi to plain Java rather than plain Java to OSGi. ;-) As > >>> >> > an > >>> >> > OSGi addict, that is very appealing to me. The problem is selling > >>> >> > it, I > >>> >> > guess. Even selling OSGi compatibility in a plain Java project > >>> >> > requires > >>> >> > addressing things like: "no, you don't have to change your > >>> >> > application > >>> >> > to OSGi." > >>> >> > > >>> >> > So, both approaches have their strong points. Let's try to put them > >>> >> > next > >>> >> > to each other: > >>> >> > > >>> >> > PojoSR: > >>> >> > - ability to work OSGi-like in plain Java, no changes running in > >>> >> > real OSGi > >>> >> > - runtime dependency on PojoSR, OSGi core and compendium in plain > >>> >> > Java > >>> >> > - existing applications using META-INF/services have to change their > >>> >> > plug-in approach to OSGi-style services > >>> >> > - existing plug-ins need to get OSGi metadata > >>> >> > > >>> >> > my approach: > >>> >> > - developers are shielded from OSGi except for the build-time > >>> >> > dependency > >>> >> > (OSGi core) and the build changes for OSGi metadata > >>> >> > - one runtime dependency in plain Java (the ServiceListener > >>> >> > infrastructure) > >>> >> > - existing applications have to switch from ServiceLoader (or > >>> >> > Services > >>> >> > in Apache XML Graphics) to a ServiceListener (ServiceTracker-like) > >>> >> > for obtaining plug-ins > >>> >> > - existing plug-ins need to add OSGi metadata as per RFC 167. > >>> >> > - probably has a considerably lower learning curve for people that > >>> >> > haven't had contact with OSGi, yet. > >>> >> > > >>> >> > I guess one question is whether PojoSR works with SPI-Fly in plain > >>> >> > Java. > >>> >> > That would make this very nice. But I suspect that might not be so > >>> >> > simple. If it doesn't work, all plug-ins have to be changed to > >>> >> > publish > >>> >> > services through a BundleContext. That would make it impossible to > >>> >> > retro-fit an older plug-in just by adding OSGi metadata to the JAR. > >>> >> > > >>> >> > Anyway, I'll put together a submission in the next few days (probably > >>> >> > next week due to other obligations). We can then see where to go from > >>> >> > there. I'd be happy to help with the spec however I can. I guess my > >>> >> > code > >>> >> > will at least be a useful additional discussion base. > >>> >> > > >>> >> > Thanks, > >>> >> > Jeremias Maerki > >>> >> > > >>> >> > > >>> >> > On 12.06.2012 14:47:01 David Bosschaert wrote: > >>> >> >> Hi Jeremias, > >>> >> >> > >>> >> >> Many thanks for the suggestion. You are highlighting a valid point: > >>> >> >> namely that people often have the need for services, like the ones > >>> >> >> supported in OSGi, but are unable to (or don't want to) modularize > >>> >> >> their project in order to be able to use OSGi. > >>> >> >> There is currently some work underway in the OSGi expert groups that > >>> >> >> relates to this. > >>> >> >> > >>> >> >> The PojoSR project (http://code.google.com/p/pojosr/) is a > >>> >> >> relatively > >>> >> >> mature project that implements most of the OSGi Service Registry and > >>> >> >> also supports things like the BundleActivator without doing the > >>> >> >> modularity part of OSGi. So in a way it is similar to what you have > >>> >> >> written, if I'm not mistaken. > >>> >> >> > >>> >> >> In the OSGi Core Platform Expert Group there is currently an effort > >>> >> >> underway to create a specification inspired by PojoSR. I will ensure > >>> >> >> that during the specification work your work is also considered as > >>> >> >> input, if you wish [1]. > >>> >> >> > >>> >> >> So in short - there will be a specification around this in the not > >>> >> >> too > >>> >> >> distant future, your work can be considered a stepping stone to this > >>> >> >> specification and it could even possibly become a compliant > >>> >> >> implementation in the future. I assume that PojoSR will also become > >>> >> >> compliant but I think that there is always room for multiple > >>> >> >> implementations of a spec. > >>> >> >> > >>> >> >> If you are interested in updating your implementation to comply with > >>> >> >> the spec in the future then I think that Apache Aries would be a > >>> >> >> good > >>> >> >> place to put your implementation right now from where it can > >>> >> >> mature. I > >>> >> >> think that it should not be a subcomponent of SPI-Fly but rather a > >>> >> >> top-level component of its own, within Aries. > >>> >> >> > >>> >> >> Best regards, > >>> >> >> > >>> >> >> David > >>> >> >> > >>> >> >> [1] or, if you'd rather do that yourself, let me know :) > >>> >> >> > >>> >> >> On 8 June 2012 10:42, Jeremias Maerki <[email protected]> > >>> >> >> wrote: > >>> >> >> > Hi there, > >>> >> >> > > >>> >> >> > due to my involvement with the Apache XML Graphics project > >>> >> >> > (FOP/Batik) > >>> >> >> > where we make heavy use of META-INF/services, I've got a big > >>> >> >> > interest in > >>> >> >> > how best to bring FOP and Batik into OSGi. My particular > >>> >> >> > requirement > >>> >> >> > here is that FOP and Batik have to continue working in plain Java > >>> >> >> > but > >>> >> >> > profit from the OSGi service registry for plug-in discovery when > >>> >> >> > possible. > >>> >> >> > > >>> >> >> > So, two years ago I came up with this: > >>> >> >> > http://www.jeremias-maerki.ch/development/osgi/jar-services.html > >>> >> >> > > >>> >> >> > That works half-way good enough although I've never felt > >>> >> >> > comfortable > >>> >> >> > enough to push it back to the XML Graphics project as I've had a > >>> >> >> > few > >>> >> >> > flaws in the above implementation. So, by now SPI Fly does a lot > >>> >> >> > of > >>> >> >> > things much better than my approach and it is following an RFC > >>> >> >> > that I > >>> >> >> > find very useful. > >>> >> >> > > >>> >> >> > However, on the client side, RFC 167 is a bit heavy on J2SE-1.6's > >>> >> >> > ServiceLoader. Apache XML Graphics is still on J2SE-1.5 (yes, I > >>> >> >> > know, I > >>> >> >> > know) so it contains its own Services.java to find plug-ins via > >>> >> >> > META-INF/services. Furthermore, with ServiceLoader you basically > >>> >> >> > have to > >>> >> >> > poll for changes. > >>> >> >> > > >>> >> >> > Finally getting to my point: I'd like to offer a little API that > >>> >> >> > brings > >>> >> >> > some of the service dynamics you get with OSGi services. This can > >>> >> >> > already be seen on the page indicated above although I've slightly > >>> >> >> > modified the API. Essentially, to get notified about new or > >>> >> >> > disappearing > >>> >> >> > plug-ins/services, you can register a service listener. And that > >>> >> >> > would > >>> >> >> > look like this (client code): > >>> >> >> > > >>> >> >> > ServiceTracker<ImageWriter> tracker = > >>> >> >> > Plugins.getServiceTracker(ImageWriter.class); > >>> >> >> > tracker.addServiceListener(new > >>> >> >> > ServiceListener<ImageWriter>() { > >>> >> >> > > >>> >> >> > public void added(ImageWriter writer) { > >>> >> >> > register(writer); > >>> >> >> > } > >>> >> >> > > >>> >> >> > public void removed(ImageWriter writer) { > >>> >> >> > unregister(writer); > >>> >> >> > } > >>> >> >> > > >>> >> >> > }); > >>> >> >> > > >>> >> >> > Where Plugins is: > >>> >> >> > > >>> >> >> > public class Plugins { > >>> >> >> > > >>> >> >> > /** > >>> >> >> > * This is the services singleton. > >>> >> >> > */ > >>> >> >> > private static final Services SERVICES = new Services(); > >>> >> >> > > >>> >> >> > public static void setServicesBackend(ServicesBackend backend) > >>> >> >> > { > >>> >> >> > SERVICES.setServicesBackend(backend); > >>> >> >> > } > >>> >> >> > > >>> >> >> > public static <T> ServiceTracker<T> getServiceTracker(Class<T> > >>> >> >> > providerIntf) { > >>> >> >> > return SERVICES.getServiceTracker(providerIntf); > >>> >> >> > } > >>> >> >> > > >>> >> >> > } > >>> >> >> > > >>> >> >> > And the BundleActivator for the client looks like this: > >>> >> >> > > >>> >> >> > public class Activator implements BundleActivator { > >>> >> >> > > >>> >> >> > private volatile ServicesOSGi services; > >>> >> >> > > >>> >> >> > /** {@inheritDoc} */ > >>> >> >> > public void start(BundleContext context) throws Exception { > >>> >> >> > this.services = new ServicesOSGi(context); > >>> >> >> > Plugins.setServicesBackend(this.services); > >>> >> >> > } > >>> >> >> > > >>> >> >> > /** {@inheritDoc} */ > >>> >> >> > public void stop(BundleContext context) throws Exception { > >>> >> >> > Plugins.setServicesBackend(null); > >>> >> >> > if (this.services != null) { > >>> >> >> > this.services.close(); > >>> >> >> > } > >>> >> >> > } > >>> >> >> > > >>> >> >> > } > >>> >> >> > > >>> >> >> > What happens is this: by default "Services" starts up with a > >>> >> >> > plain-Java > >>> >> >> > backend that simply looks up services through classic means. When > >>> >> >> > the > >>> >> >> > BundleActivator is called, the backend is replaced by > >>> >> >> > ServicesOSGi that, > >>> >> >> > instead, gets the plug-ins from the service registry. In the > >>> >> >> > background, > >>> >> >> > any previously discovered plug-ins are "removed()" and the new > >>> >> >> > ones from > >>> >> >> > the registry "added()". > >>> >> >> > > >>> >> >> > In the case of classic Java, the "removed()" method is never > >>> >> >> > called > >>> >> >> > (probably doesn't ever need to be), i.e. no dynamics there. > >>> >> >> > > >>> >> >> > To conclude: the goal is to profit from OSGi service dynamics when > >>> >> >> > discovering plug-ins while preserving the possibility to run > >>> >> >> > without > >>> >> >> > OSGi API runtime dependencies and still in J2SE-1.5. Furthermore, > >>> >> >> > the > >>> >> >> > OSGi-specific code shall be as minimal as possible to shield > >>> >> >> > those with > >>> >> >> > no OSGi knowledge in the team from the learning curve. > >>> >> >> > > >>> >> >> > And now, I'm wondering if you're interested to adopt this as a > >>> >> >> > new part > >>> >> >> > in SPI Fly as a useful alternative to using ServiceLoader. If you > >>> >> >> > don't > >>> >> >> > want it, I'm going to publish it under Apache Extras. > >>> >> >> > > >>> >> >> > Thanks, > >>> >> >> > Jeremias Maerki > >>> >> >> > > >>> >> > > >>> > > >>
