Hi, As you're probably aware there are currently several different issues being worked upon related to the observation queue limit problem ([0], epic [1]). I wanted to discuss yet another improvement and first ask what the list thinks.
What about requiring observation listeners to either consume only internal or only external events, but never both together, we wouldn't support that anymore. (And if you're in a cluster you want to be very careful with consuming external events in the first place - but that's another topic) The root problem of the 'queue hitting the limit' as of today is that it throws away the CommitInfo, thus doesn't know anymore if it's an internal or an external event (besides actually loosing the CommitInfo details). If we separate listeners into purely internal vs external, then a queue as a whole is either purely internal or external and we no longer have this issue. We could continue to throw away the CommitInfo (or avoid that using a persisted obs queue ([2])), but we could then still say with certainty if it's an internal or an external event. A user that would want to receive both internal and external events could simply create two listeners. Those would both receive events as expected. The only difference would be that the two stream of events would not be in sync - but I doubt that this would be a big loss. We could thus introduce 'ExcludeInternal' and demand in ObservationManager.addEventListener that the listener is flagged with one of ExcludeInternal or ExcludeExternal. Wdyt? Cheers, Stefan -- [0] - https://issues.apache.org/jira/browse/OAK-2683 [1] - https://issues.apache.org/jira/browse/OAK-4614 [2] - https://issues.apache.org/jira/browse/OAK-4581
