Hi Chetan, (see below)
On 02/09/16 13:26, "Chetan Mehrotra" <[email protected]> wrote: >On Fri, Sep 2, 2016 at 4:00 PM, Stefan Egli <[email protected]> wrote: >> If we >> separate listeners into purely internal vs external, then a queue as a >>whole >> is either purely internal or external and we no longer have this issue. > >Not sure here on how this would work. The observation queue is made up >of ContentChange which is a tuple of [root NodeState , CommitInfo >(null for external)] > >--- NS1-L---NS2-L--NS3---NS4-L---NS5-L ---NS6-L > >--- a -------- /a/b ----- /a/c --- /a/c > /a/b /a/b > /a/d > >So if we dedicate a queue for local changes only what would happen. > >If we drop NS3 then while diffing [NS2-L, NS4-L] /a/c would be >reported as "added" and "local". Now we have a listener which listens >for locally added nt:file node such it can start some processing job >for it. Such a listener would then think its a locally added node and >would start a duplicate job Good point. We could probably fix this though by not only storing 1 root NodeState per ContentChange, but store 2: a 'from' and a 'to' (the 'from' is currently implicit, as that's taken from the previous entry, but if we skip entries, then it needs to be re-added). So with that, we could safely drop external changes as 'uninterested' and diffing would still report the correct thing. > >In general I believe > >Listener for external Change >-------------------------------------- >listener which are listening for external changes are maintaining some >state and purge/refresh it upon detecting change in interested paths. >They would work fine if multiple content change occurrences are merged > >[NS4-L, NS5-L] + [NS5-L,NS6-L] = [NS4, NS6] (external) as they would >still detect the change > >An example of this is LuceneIndexObserver which sets queue size to 5 >and does not care its local or not. It just interested in if index >node is updated > >Listener for local Change >---------------------------------- > >Such a listener is more particular about type of change and is doing >some persisted state change i.e. like registering a job, invoking some >third party service to update the value. This listener is only >interested in local as it know same listener is also active on other >cluster node (homogeneous cluster setup) so if a node gets added it >only need to react on the cluster node where it got added. One thing this reminds me of is a use-case where you have say 3 cluster nodes, each one handling mainly local events lets say. All fine. Then 1 node crashes while likely it's (local) observation queue wasn't entirely empty. Those events would then probably not get handled by anyone (and that node wouldn't necessarily be restarted as the cluster continues normally, it could be restarted as a new clusterNodeId..). So maybe there's an issue there. > >So for such it needs to be ensured that mixed content changes are not >compacted. So its fine to > >[NS4-L, NS5-L] + [NS5-L,NS6-L] = [NS4, NS6] (can be treated as >local with loss of user identity which caused the change) >[NS2-L, NS3] + [NS3, NS4-L] = [NS2-L, NS4-L] (cannot be treated as >local) I think keeping the 'from/to' tuple instead of just 1 root NodeState would make the above picture more simple. Cheers, Stefan > >Just thinking out loud here to understand the problem space better :) > >Chetan Mehrotra
