On Monday 19 October 2015 14:46:05 Carsten Ziegeler wrote: > Hi, hi,
> with https://issues.apache.org/jira/browse/SLING-4751 we have a new API > for dealing with change events on both sides, the reporting part and the > client using the events. The basic idea is to have a more scalable approach. > > Now, if we think of large scalable systems, there is one thing that is > not very optimal: dealing with external observation events. Obviously > sending all observation events to all potentially interested parties is > not really cheap. > There are additional things, like e.g. Oak is not providing the commit > info for external events. Therefore the user of the change is unknown. > With other resource providers, like mongo, cassandra etc. there is not > even an infrastructure to send external change events across instances. > > I was wondering what is the best way to deal with this. It would be > great if we could just say: observation is only for local changes. This > would simplify a lot of things and makes scaling observation handling > much easier. > For a lot of things, this is usually sufficient. For example, if you > want to trigger an action based on some changes, you want to trigger > this action only once anyway. Doing it on the instance where the change > happened is therefore straightforward. > Unfortunately, there are use cases where you want to receive external > changes, and these usually boil down to caching use caches. For example > the jsp script cache needs to be aware of jsp changes, no matter where > the change happened. > All these use cases could be changed to different patterns, e.g. on > access check if the cached entry is still valid, period checks etc. On > the other hand, an on access cache reduces general request performance > dramatically, period checks have the downside that there is a delay of > detecting the change etc. An observation based cache is pretty optimal > as it only invalidates if there is a change and it only checks/recreates > a cached entry if necessary. > > With the move to the new observation api we now have the chance to come > up with the best solution. We just need to figure out, what this is. > > Thoughts? have you considered something like Apache Ignite or Apache Kafka? Regards, O. > Regards > Carsten
