2013/4/17 Jukka Zitting <[email protected]>: > Hi, > > On Wed, Apr 17, 2013 at 3:35 AM, Ian Boston <[email protected]> wrote: >> When mentioning AMQ, I forgot to mention that one size doesn't fit all use >> cases. When I first connected AMQ to Jackrabbit observation via OSGi >> events, AMQ rapidly became swamped with the volume of events. OSGi events >> were just about able to cope but as soon as the events were propagated over >> the network the volume was too great. > > Indeed. Most message queue systems out there are designed for > application/document level messages (like "document added", "action > taken", etc.), i.e. the granularity of events is generally coarser > than the field-level events ("property changed", "node added", etc.) > you get with JCR observation. > > My thinking here is that the best approach for most use cases will be > to use a commit hook to consolidate the low level changes to less > granular application events that can then be processed in the > background using JCR observation, OSGi events, a generic message queue > system, or directly by a polling thread in the client application.
I agree with this line of thinking. Instead of doing any performance improvement or event aggregation at a low-level event producer site, I think that Oak itself should generate as many meaningful events as possible. With a sensible eventing SPI, it is easy for an Oak user to implement "event listener pipes" or "event transformers" which filter, aggregate, and package low-level events into fewer but bigger messages, depending on the concrete event distribution requirements. The originally suggested aggregation on a revision level would be an example for such a "custom" aggregation. If MQ or OSGi event listener implementations are flooded too quickly, Oak (or an Oak extension module) could provide a sample implementation for event accumulation (at the producer site) and dispersion (at the consumer site), to transparently package events into fewer but bigger physical messages for transport - while still being able to consume each low-level event at the consumer site if needed. Oak itself, however, shouldn't make too many architecture / topology assumptions about the components that build on it. I feel that concrete implementation facts (internal and external to Oak) are beginning to leak into Oak's eventing SPI. Cheers, Lukas
