Hi, I think it could be useful to have a broader overview of use cases and have them categorised. What comes to mind besides the cache-invalidation case you mentioned (which I think is still a good candidate for external observation events) is:
* exactly once : eg when uploading an asset you want a certain job to be started exactly once. Currently this uses internal events only (so kind of not related to this external event topic) but it has the risk of failing when the instance crashes before the job was created. Perhaps we could add some framework support for this (auto-creation of a job?) * maybe there's also at-least-once, not sure, but it might also be solved with auto-spawning a job (perhaps less costly than exactly-once) If we're comfortable having covered all use cases we can probably disable external observation event support entirely - but until then, even switching the default to be internal-only would probably reduce external events a lot. Cheers, Stefan On 19/10/15 14:46, "Carsten Ziegeler" <[email protected]> wrote: >Hi, > >with https://issues.apache.org/jira/browse/SLING-4751 we have a new API >for dealing with change events on both sides, the reporting part and the >client using the events. The basic idea is to have a more scalable >approach. > >Now, if we think of large scalable systems, there is one thing that is >not very optimal: dealing with external observation events. Obviously >sending all observation events to all potentially interested parties is >not really cheap. >There are additional things, like e.g. Oak is not providing the commit >info for external events. Therefore the user of the change is unknown. >With other resource providers, like mongo, cassandra etc. there is not >even an infrastructure to send external change events across instances. > >I was wondering what is the best way to deal with this. It would be >great if we could just say: observation is only for local changes. This >would simplify a lot of things and makes scaling observation handling >much easier. >For a lot of things, this is usually sufficient. For example, if you >want to trigger an action based on some changes, you want to trigger >this action only once anyway. Doing it on the instance where the change >happened is therefore straightforward. >Unfortunately, there are use cases where you want to receive external >changes, and these usually boil down to caching use caches. For example >the jsp script cache needs to be aware of jsp changes, no matter where >the change happened. >All these use cases could be changed to different patterns, e.g. on >access check if the cached entry is still valid, period checks etc. On >the other hand, an on access cache reduces general request performance >dramatically, period checks have the downside that there is a delay of >detecting the change etc. An observation based cache is pretty optimal >as it only invalidates if there is a change and it only checks/recreates >a cached entry if necessary. > >With the move to the new observation api we now have the chance to come >up with the best solution. We just need to figure out, what this is. > >Thoughts? > >Regards >Carsten >-- >Carsten Ziegeler >Adobe Research Switzerland >[email protected]
