[
https://issues.apache.org/jira/browse/OAK-4581?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15526714#comment-15526714
]
Stefan Egli edited comment on OAK-4581 at 9/27/16 5:02 PM:
-----------------------------------------------------------
[~cziegeler], re
bq. Now, an additional reason at least for Sling was that the event bridge we
have was reading all added/changed nodes to get the resource type property.
-I can't find this dependency in the code, do you know where that's coded in
JcrResourceListener? IIUC then the reason for having these
addedEvents/changedEvents/removedEvents maps in onEvent is to be able to build
a JcrResourceChanged obj that contains all changed properties (unrelated to
resource type - but perhaps that's hidden somewhere down deep..)-
EDIT: Are you referring to
[OsgiObservationBridge.sendOsgiEvent:131|https://github.com/apache/sling/blob/9c424dfca6a802a6b66b4b7981a313c5856a0e1f/bundles/resourceresolver/src/main/java/org/apache/sling/resourceresolver/impl/observation/OsgiObservationBridge.java#L131]
? IIUC that reads the current resource explicitly to get the resourceType. So
sure, having the resource type as part of the event (as OAK-4853 would provide)
would be a handy thing, even if slighlty unexpected. However, I fail to see how
this justifies the addedEvents/changedEvents/removedEvents in
JcrResourceListener. IIUC the reason for building these maps is to be able to
generate _correct_ JcrResourceChange objs - as they must contain all properties
of a particular node - and those come in separate {{Events}}. So if the goal is
to prevent those maps to become huge (or to not have them at all), then this
has nothing to do with the resource type in my view. If we shouldn't rely on
the breadth-first traversal, then the only alternative would be to get all
properties that have been changed/added/removed on the corresponding {{Event}}
of the node (which is slightly different from OAK-4853). wdyt?
was (Author: egli):
[~cziegeler], re
bq. Now, an additional reason at least for Sling was that the event bridge we
have was reading all added/changed nodes to get the resource type property.
I can't find this dependency in the code, do you know where that's coded in
JcrResourceListener? IIUC then the reason for having these
addedEvents/changedEvents/removedEvents maps in onEvent is to be able to build
a JcrResourceChanged obj that contains all changed properties (unrelated to
resource type - but perhaps that's hidden somewhere down deep..)
> Persistent local journal for more reliable event generation
> -----------------------------------------------------------
>
> Key: OAK-4581
> URL: https://issues.apache.org/jira/browse/OAK-4581
> Project: Jackrabbit Oak
> Issue Type: New Feature
> Components: core
> Reporter: Chetan Mehrotra
> Assignee: Stefan Egli
> Labels: observation
> Fix For: 1.6
>
> Attachments: OAK-4581.v0.patch
>
>
> As discussed in OAK-2683 "hitting the observation queue limit" has multiple
> drawbacks. Quite a bit of work is done to make diff generation faster.
> However there are still chances of event queue getting filled up.
> This issue is meant to implement a persistent event journal. Idea here being
> # NodeStore would push the diff into a persistent store via a synchronous
> observer
> # Observors which are meant to handle such events in async way (by virtue of
> being wrapped in BackgroundObserver) would instead pull the events from this
> persisted journal
> h3. A - What is persisted
> h4. 1 - Serialized Root States and CommitInfo
> In this approach we just persist the root states in serialized form.
> * DocumentNodeStore - This means storing the root revision vector
> * SegmentNodeStore - {color:red}Q1 - What does serialized form of
> SegmentNodeStore root state looks like{color} - Possible the RecordId of
> "root" state
> Note that with OAK-4528 DocumentNodeStore can rely on persisted remote
> journal to determine the affected paths. Which reduces the need for
> persisting complete diff locally.
> Event generation logic would then "deserialize" the persisted root states and
> then generate the diff as currently done via NodeState comparison
> h4. 2 - Serialized commit diff and CommitInfo
> In this approach we can save the diff in JSOP form. The diff only contains
> information about affected path. Similar to what is current being stored in
> DocumentNodeStore journal
> h4. CommitInfo
> The commit info would also need to be serialized. So it needs to be ensure
> whatever is stored there can be serialized or re calculated
> h3. B - How it is persisted
> h4. 1 - Use a secondary segment NodeStore
> OAK-4180 makes use of SegmentNodeStore as a secondary store for caching.
> [~mreutegg] suggested that for persisted local journal we can also utilize a
> SegmentNodeStore instance. Care needs to be taken for compaction. Either via
> generation approach or relying on online compaction
> h4. 2- Make use of write ahead log implementations
> [~ianeboston] suggested that we can make use of some write ahead log
> implementation like [1], [2] or [3]
> h3. C - How changes get pulled
> Some points to consider for event generation logic
> # Would need a way to keep pointers to journal entry on per listener basis.
> This would allow each Listener to "pull" content changes and generate diff as
> per its speed and keeping in memory overhead low
> # The journal should survive restarts
> [1] http://www.mapdb.org/javadoc/latest/mapdb/org/mapdb/WriteAheadLog.html
> [2]
> https://github.com/apache/activemq/tree/master/activemq-kahadb-store/src/main/java/org/apache/activemq/store/kahadb/disk/journal
> [3]
> https://github.com/elastic/elasticsearch/tree/master/core/src/main/java/org/elasticsearch/index/translog
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
