[jira] [Commented] (SLING-4751) New Observation Support
[ https://issues.apache.org/jira/browse/SLING-4751?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14958937#comment-14958937 ] Carsten Ziegeler commented on SLING-4751: - Both :) > New Observation Support > --- > > Key: SLING-4751 > URL: https://issues.apache.org/jira/browse/SLING-4751 > Project: Sling > Issue Type: Improvement > Components: API, JCR, ResourceResolver >Reporter: Carsten Ziegeler >Assignee: Tomek Rękawek > Fix For: API 2.10.0, Resource Resolver 1.2.8 > > > Mail thread: > http://mail-archives.apache.org/mod_mbox/sling-dev/201505.mbox/%3C555983F2.20402%40apache.org%3E > Starting mail > Right now, resources changes are propagated through event admin - which > at the time sounded like a good idea. Over time, this has shown to be a > bottle neck. > Basically there are at least three problems: > - the sender of the resource events does not know whether there is a > receiver, therefore events for each and every change need to be sent > - event objects are immutable and therefore all relevant data needs to > be calculated upfront, even if it's not used. For example a resource > event contains the resource type which needs to be fetched from the > repository, even if no one is interested in it. > - receivers of the events can't easily act on behalf of the user who > initiated the change. > I created a new listener api at [1] which defines a ResourceListener > interface and some ways how to specify the events one is interested in. > The user aware resource listener allows to act on behalf of the user (if > that information is available). > On the other side, a new service, the ObservationReporter [2] is > defined. Resource providers report changes through this interface. The > payload of such an event is an interface which allows for lazy retrieval > of the information. > We can also use this mechanism for compatibility and an implementation > of the observation reporter might sent all events via the event admin. > [1] > https://svn.apache.org/repos/asf/sling/whiteboard/cziegeler/api-v3/src/main/java/org/apache/sling/api/resource/observation/ > [2] > https://svn.apache.org/repos/asf/sling/whiteboard/cziegeler -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (SLING-4751) New Observation Support
[ https://issues.apache.org/jira/browse/SLING-4751?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14958936#comment-14958936 ] Carsten Ziegeler commented on SLING-4751: - [~tomek.rekawek] Would it be possible to split up the patch into a patches per module? > New Observation Support > --- > > Key: SLING-4751 > URL: https://issues.apache.org/jira/browse/SLING-4751 > Project: Sling > Issue Type: Improvement > Components: API, JCR, ResourceResolver >Reporter: Carsten Ziegeler >Assignee: Tomek Rękawek > Fix For: API 2.10.0, Resource Resolver 1.2.8 > > > Mail thread: > http://mail-archives.apache.org/mod_mbox/sling-dev/201505.mbox/%3C555983F2.20402%40apache.org%3E > Starting mail > Right now, resources changes are propagated through event admin - which > at the time sounded like a good idea. Over time, this has shown to be a > bottle neck. > Basically there are at least three problems: > - the sender of the resource events does not know whether there is a > receiver, therefore events for each and every change need to be sent > - event objects are immutable and therefore all relevant data needs to > be calculated upfront, even if it's not used. For example a resource > event contains the resource type which needs to be fetched from the > repository, even if no one is interested in it. > - receivers of the events can't easily act on behalf of the user who > initiated the change. > I created a new listener api at [1] which defines a ResourceListener > interface and some ways how to specify the events one is interested in. > The user aware resource listener allows to act on behalf of the user (if > that information is available). > On the other side, a new service, the ObservationReporter [2] is > defined. Resource providers report changes through this interface. The > payload of such an event is an interface which allows for lazy retrieval > of the information. > We can also use this mechanism for compatibility and an implementation > of the observation reporter might sent all events via the event admin. > [1] > https://svn.apache.org/repos/asf/sling/whiteboard/cziegeler/api-v3/src/main/java/org/apache/sling/api/resource/observation/ > [2] > https://svn.apache.org/repos/asf/sling/whiteboard/cziegeler -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (SLING-4751) New Observation Support
[ https://issues.apache.org/jira/browse/SLING-4751?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14944791#comment-14944791 ] Tomek Rękawek commented on SLING-4751: -- Do you mean removing bridge completely or just disabling the external events support? > New Observation Support > --- > > Key: SLING-4751 > URL: https://issues.apache.org/jira/browse/SLING-4751 > Project: Sling > Issue Type: Improvement > Components: API, JCR, ResourceResolver >Reporter: Carsten Ziegeler >Assignee: Tomek Rękawek > Fix For: API 2.10.0, Resource Resolver 1.2.8 > > > Mail thread: > http://mail-archives.apache.org/mod_mbox/sling-dev/201505.mbox/%3C555983F2.20402%40apache.org%3E > Starting mail > Right now, resources changes are propagated through event admin - which > at the time sounded like a good idea. Over time, this has shown to be a > bottle neck. > Basically there are at least three problems: > - the sender of the resource events does not know whether there is a > receiver, therefore events for each and every change need to be sent > - event objects are immutable and therefore all relevant data needs to > be calculated upfront, even if it's not used. For example a resource > event contains the resource type which needs to be fetched from the > repository, even if no one is interested in it. > - receivers of the events can't easily act on behalf of the user who > initiated the change. > I created a new listener api at [1] which defines a ResourceListener > interface and some ways how to specify the events one is interested in. > The user aware resource listener allows to act on behalf of the user (if > that information is available). > On the other side, a new service, the ObservationReporter [2] is > defined. Resource providers report changes through this interface. The > payload of such an event is an interface which allows for lazy retrieval > of the information. > We can also use this mechanism for compatibility and an implementation > of the observation reporter might sent all events via the event admin. > [1] > https://svn.apache.org/repos/asf/sling/whiteboard/cziegeler/api-v3/src/main/java/org/apache/sling/api/resource/observation/ > [2] > https://svn.apache.org/repos/asf/sling/whiteboard/cziegeler -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (SLING-4751) New Observation Support
[
https://issues.apache.org/jira/browse/SLING-4751?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14943142#comment-14943142
]
Stefan Egli commented on SLING-4751:
A comment from a scalability point of view: thanks to
{{ExternalResourceListener}} we're trying hard to reduce the number of external
events that need to be processed, that's great. But IIUC then that's not the
case for the legacy OsgiObservationBridge case: that one still consumes all
external events for all paths. Is there any way this could be improved? If not,
is it an option to deprecate it, or is that already the case?
> New Observation Support
> ---
>
> Key: SLING-4751
> URL: https://issues.apache.org/jira/browse/SLING-4751
> Project: Sling
> Issue Type: Improvement
> Components: API, JCR, ResourceResolver
>Reporter: Carsten Ziegeler
>Assignee: Tomek Rękawek
> Fix For: API 2.10.0, Resource Resolver 1.2.8
>
>
> Mail thread:
> http://mail-archives.apache.org/mod_mbox/sling-dev/201505.mbox/%3C555983F2.20402%40apache.org%3E
> Starting mail
> Right now, resources changes are propagated through event admin - which
> at the time sounded like a good idea. Over time, this has shown to be a
> bottle neck.
> Basically there are at least three problems:
> - the sender of the resource events does not know whether there is a
> receiver, therefore events for each and every change need to be sent
> - event objects are immutable and therefore all relevant data needs to
> be calculated upfront, even if it's not used. For example a resource
> event contains the resource type which needs to be fetched from the
> repository, even if no one is interested in it.
> - receivers of the events can't easily act on behalf of the user who
> initiated the change.
> I created a new listener api at [1] which defines a ResourceListener
> interface and some ways how to specify the events one is interested in.
> The user aware resource listener allows to act on behalf of the user (if
> that information is available).
> On the other side, a new service, the ObservationReporter [2] is
> defined. Resource providers report changes through this interface. The
> payload of such an event is an interface which allows for lazy retrieval
> of the information.
> We can also use this mechanism for compatibility and an implementation
> of the observation reporter might sent all events via the event admin.
> [1]
> https://svn.apache.org/repos/asf/sling/whiteboard/cziegeler/api-v3/src/main/java/org/apache/sling/api/resource/observation/
> [2]
> https://svn.apache.org/repos/asf/sling/whiteboard/cziegeler
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (SLING-4751) New Observation Support
[ https://issues.apache.org/jira/browse/SLING-4751?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14943201#comment-14943201 ] Carsten Ziegeler commented on SLING-4751: - I think we have to get away from the bridge - this is the only way to make it scale at all Therefore deprecating the topics is a good thing We could also add a configuration to the bridge to ignore external events. However, it might be tricky to find out whether the whole app still works with disabling it. > New Observation Support > --- > > Key: SLING-4751 > URL: https://issues.apache.org/jira/browse/SLING-4751 > Project: Sling > Issue Type: Improvement > Components: API, JCR, ResourceResolver >Reporter: Carsten Ziegeler >Assignee: Tomek Rękawek > Fix For: API 2.10.0, Resource Resolver 1.2.8 > > > Mail thread: > http://mail-archives.apache.org/mod_mbox/sling-dev/201505.mbox/%3C555983F2.20402%40apache.org%3E > Starting mail > Right now, resources changes are propagated through event admin - which > at the time sounded like a good idea. Over time, this has shown to be a > bottle neck. > Basically there are at least three problems: > - the sender of the resource events does not know whether there is a > receiver, therefore events for each and every change need to be sent > - event objects are immutable and therefore all relevant data needs to > be calculated upfront, even if it's not used. For example a resource > event contains the resource type which needs to be fetched from the > repository, even if no one is interested in it. > - receivers of the events can't easily act on behalf of the user who > initiated the change. > I created a new listener api at [1] which defines a ResourceListener > interface and some ways how to specify the events one is interested in. > The user aware resource listener allows to act on behalf of the user (if > that information is available). > On the other side, a new service, the ObservationReporter [2] is > defined. Resource providers report changes through this interface. The > payload of such an event is an interface which allows for lazy retrieval > of the information. > We can also use this mechanism for compatibility and an implementation > of the observation reporter might sent all events via the event admin. > [1] > https://svn.apache.org/repos/asf/sling/whiteboard/cziegeler/api-v3/src/main/java/org/apache/sling/api/resource/observation/ > [2] > https://svn.apache.org/repos/asf/sling/whiteboard/cziegeler -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (SLING-4751) New Observation Support
[ https://issues.apache.org/jira/browse/SLING-4751?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14943211#comment-14943211 ] Carsten Ziegeler commented on SLING-4751: - EventHandler's can filter to just receive local (or external events), as this info is a property of the event (or it's missing). We could lookup all event handlers and do magic, but then we don't need this new observation support at all. I wouldn't go that way and simply move away from it completely > New Observation Support > --- > > Key: SLING-4751 > URL: https://issues.apache.org/jira/browse/SLING-4751 > Project: Sling > Issue Type: Improvement > Components: API, JCR, ResourceResolver >Reporter: Carsten Ziegeler >Assignee: Tomek Rękawek > Fix For: API 2.10.0, Resource Resolver 1.2.8 > > > Mail thread: > http://mail-archives.apache.org/mod_mbox/sling-dev/201505.mbox/%3C555983F2.20402%40apache.org%3E > Starting mail > Right now, resources changes are propagated through event admin - which > at the time sounded like a good idea. Over time, this has shown to be a > bottle neck. > Basically there are at least three problems: > - the sender of the resource events does not know whether there is a > receiver, therefore events for each and every change need to be sent > - event objects are immutable and therefore all relevant data needs to > be calculated upfront, even if it's not used. For example a resource > event contains the resource type which needs to be fetched from the > repository, even if no one is interested in it. > - receivers of the events can't easily act on behalf of the user who > initiated the change. > I created a new listener api at [1] which defines a ResourceListener > interface and some ways how to specify the events one is interested in. > The user aware resource listener allows to act on behalf of the user (if > that information is available). > On the other side, a new service, the ObservationReporter [2] is > defined. Resource providers report changes through this interface. The > payload of such an event is an interface which allows for lazy retrieval > of the information. > We can also use this mechanism for compatibility and an implementation > of the observation reporter might sent all events via the event admin. > [1] > https://svn.apache.org/repos/asf/sling/whiteboard/cziegeler/api-v3/src/main/java/org/apache/sling/api/resource/observation/ > [2] > https://svn.apache.org/repos/asf/sling/whiteboard/cziegeler -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (SLING-4751) New Observation Support
[
https://issues.apache.org/jira/browse/SLING-4751?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14943203#comment-14943203
]
Stefan Egli commented on SLING-4751:
bq. Do you have any suggestions what can be done to increase the performance?
Not very concrete ones yet. Perhaps some fiddling with EventHandler properties.
The internal vs external part could perhaps be modelled via some
{{EventConstants.EVENT_FILTER}}? Or by introducing
{{SlingConstants.TOPIC_RESOURCE_EXTERNAL_*}} ..
In any case, the goal should be to limit the number of paths for which external
events should be delivered by the repository (eg Oak). If the external events
are filtered further up the stack, ie after Oak has delivered them to the
{{OsgiObservationBridge}}, then I think it's too late: the work has already
been done. Best would be if {{OsgiObservationBridge}} could come up with a
limited lists of paths for it wants external events to be delivered. Perhaps
that is feasible with some peeking into EventAdmin?
> New Observation Support
> ---
>
> Key: SLING-4751
> URL: https://issues.apache.org/jira/browse/SLING-4751
> Project: Sling
> Issue Type: Improvement
> Components: API, JCR, ResourceResolver
>Reporter: Carsten Ziegeler
>Assignee: Tomek Rękawek
> Fix For: API 2.10.0, Resource Resolver 1.2.8
>
>
> Mail thread:
> http://mail-archives.apache.org/mod_mbox/sling-dev/201505.mbox/%3C555983F2.20402%40apache.org%3E
> Starting mail
> Right now, resources changes are propagated through event admin - which
> at the time sounded like a good idea. Over time, this has shown to be a
> bottle neck.
> Basically there are at least three problems:
> - the sender of the resource events does not know whether there is a
> receiver, therefore events for each and every change need to be sent
> - event objects are immutable and therefore all relevant data needs to
> be calculated upfront, even if it's not used. For example a resource
> event contains the resource type which needs to be fetched from the
> repository, even if no one is interested in it.
> - receivers of the events can't easily act on behalf of the user who
> initiated the change.
> I created a new listener api at [1] which defines a ResourceListener
> interface and some ways how to specify the events one is interested in.
> The user aware resource listener allows to act on behalf of the user (if
> that information is available).
> On the other side, a new service, the ObservationReporter [2] is
> defined. Resource providers report changes through this interface. The
> payload of such an event is an interface which allows for lazy retrieval
> of the information.
> We can also use this mechanism for compatibility and an implementation
> of the observation reporter might sent all events via the event admin.
> [1]
> https://svn.apache.org/repos/asf/sling/whiteboard/cziegeler/api-v3/src/main/java/org/apache/sling/api/resource/observation/
> [2]
> https://svn.apache.org/repos/asf/sling/whiteboard/cziegeler
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (SLING-4751) New Observation Support
[
https://issues.apache.org/jira/browse/SLING-4751?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14943168#comment-14943168
]
Tomek Rękawek commented on SLING-4751:
--
In the current trunk version, the OSGi events are created both for the internal
and external JCR/Oak events. New {{OsgiObservationBridge}} tries to be
compatible with this approach. Do you have any suggestions what can be done to
increase the performance? Dropping external events would make things faster,
but it may also break the compatibility.
Not sure what would be the best part to mark as deprecated. Maybe OSGi topics
at {{SlingConstants.TOPIC_RESOURCE_*}}?
> New Observation Support
> ---
>
> Key: SLING-4751
> URL: https://issues.apache.org/jira/browse/SLING-4751
> Project: Sling
> Issue Type: Improvement
> Components: API, JCR, ResourceResolver
>Reporter: Carsten Ziegeler
>Assignee: Tomek Rękawek
> Fix For: API 2.10.0, Resource Resolver 1.2.8
>
>
> Mail thread:
> http://mail-archives.apache.org/mod_mbox/sling-dev/201505.mbox/%3C555983F2.20402%40apache.org%3E
> Starting mail
> Right now, resources changes are propagated through event admin - which
> at the time sounded like a good idea. Over time, this has shown to be a
> bottle neck.
> Basically there are at least three problems:
> - the sender of the resource events does not know whether there is a
> receiver, therefore events for each and every change need to be sent
> - event objects are immutable and therefore all relevant data needs to
> be calculated upfront, even if it's not used. For example a resource
> event contains the resource type which needs to be fetched from the
> repository, even if no one is interested in it.
> - receivers of the events can't easily act on behalf of the user who
> initiated the change.
> I created a new listener api at [1] which defines a ResourceListener
> interface and some ways how to specify the events one is interested in.
> The user aware resource listener allows to act on behalf of the user (if
> that information is available).
> On the other side, a new service, the ObservationReporter [2] is
> defined. Resource providers report changes through this interface. The
> payload of such an event is an interface which allows for lazy retrieval
> of the information.
> We can also use this mechanism for compatibility and an implementation
> of the observation reporter might sent all events via the event admin.
> [1]
> https://svn.apache.org/repos/asf/sling/whiteboard/cziegeler/api-v3/src/main/java/org/apache/sling/api/resource/observation/
> [2]
> https://svn.apache.org/repos/asf/sling/whiteboard/cziegeler
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (SLING-4751) New Observation Support
[ https://issues.apache.org/jira/browse/SLING-4751?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14943247#comment-14943247 ] Stefan Egli commented on SLING-4751: bq. I think we have to get away from the bridge - this is the only way to make it scale at all +1 > New Observation Support > --- > > Key: SLING-4751 > URL: https://issues.apache.org/jira/browse/SLING-4751 > Project: Sling > Issue Type: Improvement > Components: API, JCR, ResourceResolver >Reporter: Carsten Ziegeler >Assignee: Tomek Rękawek > Fix For: API 2.10.0, Resource Resolver 1.2.8 > > > Mail thread: > http://mail-archives.apache.org/mod_mbox/sling-dev/201505.mbox/%3C555983F2.20402%40apache.org%3E > Starting mail > Right now, resources changes are propagated through event admin - which > at the time sounded like a good idea. Over time, this has shown to be a > bottle neck. > Basically there are at least three problems: > - the sender of the resource events does not know whether there is a > receiver, therefore events for each and every change need to be sent > - event objects are immutable and therefore all relevant data needs to > be calculated upfront, even if it's not used. For example a resource > event contains the resource type which needs to be fetched from the > repository, even if no one is interested in it. > - receivers of the events can't easily act on behalf of the user who > initiated the change. > I created a new listener api at [1] which defines a ResourceListener > interface and some ways how to specify the events one is interested in. > The user aware resource listener allows to act on behalf of the user (if > that information is available). > On the other side, a new service, the ObservationReporter [2] is > defined. Resource providers report changes through this interface. The > payload of such an event is an interface which allows for lazy retrieval > of the information. > We can also use this mechanism for compatibility and an implementation > of the observation reporter might sent all events via the event admin. > [1] > https://svn.apache.org/repos/asf/sling/whiteboard/cziegeler/api-v3/src/main/java/org/apache/sling/api/resource/observation/ > [2] > https://svn.apache.org/repos/asf/sling/whiteboard/cziegeler -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (SLING-4751) New Observation Support
[ https://issues.apache.org/jira/browse/SLING-4751?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14936577#comment-14936577 ] Tomek Rękawek commented on SLING-4751: -- Diff file (requires SLING-4750): https://github.com/trekawek/sling/pull/1.diff Browse changes files at: https://github.com/trekawek/sling/pull/1/files > New Observation Support > --- > > Key: SLING-4751 > URL: https://issues.apache.org/jira/browse/SLING-4751 > Project: Sling > Issue Type: Improvement > Components: API, JCR, ResourceResolver >Reporter: Carsten Ziegeler >Assignee: Tomek Rękawek > Fix For: API 2.10.0, Resource Resolver 1.2.8 > > > Mail thread: > http://mail-archives.apache.org/mod_mbox/sling-dev/201505.mbox/%3C555983F2.20402%40apache.org%3E > Starting mail > Right now, resources changes are propagated through event admin - which > at the time sounded like a good idea. Over time, this has shown to be a > bottle neck. > Basically there are at least three problems: > - the sender of the resource events does not know whether there is a > receiver, therefore events for each and every change need to be sent > - event objects are immutable and therefore all relevant data needs to > be calculated upfront, even if it's not used. For example a resource > event contains the resource type which needs to be fetched from the > repository, even if no one is interested in it. > - receivers of the events can't easily act on behalf of the user who > initiated the change. > I created a new listener api at [1] which defines a ResourceListener > interface and some ways how to specify the events one is interested in. > The user aware resource listener allows to act on behalf of the user (if > that information is available). > On the other side, a new service, the ObservationReporter [2] is > defined. Resource providers report changes through this interface. The > payload of such an event is an interface which allows for lazy retrieval > of the information. > We can also use this mechanism for compatibility and an implementation > of the observation reporter might sent all events via the event admin. > [1] > https://svn.apache.org/repos/asf/sling/whiteboard/cziegeler/api-v3/src/main/java/org/apache/sling/api/resource/observation/ > [2] > https://svn.apache.org/repos/asf/sling/whiteboard/cziegeler -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (SLING-4751) New Observation Support
[ https://issues.apache.org/jira/browse/SLING-4751?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14907880#comment-14907880 ] Tomek Rękawek commented on SLING-4751: -- Starterd working here: https://github.com/trekawek/sling/tree/SLING-4751 It already passes the integration tests. > New Observation Support > --- > > Key: SLING-4751 > URL: https://issues.apache.org/jira/browse/SLING-4751 > Project: Sling > Issue Type: Improvement > Components: API, JCR, ResourceResolver >Reporter: Carsten Ziegeler >Assignee: Tomek Rękawek > Fix For: API 2.10.0, Resource Resolver 1.2.8 > > > Mail thread: > http://mail-archives.apache.org/mod_mbox/sling-dev/201505.mbox/%3C555983F2.20402%40apache.org%3E > Starting mail > Right now, resources changes are propagated through event admin - which > at the time sounded like a good idea. Over time, this has shown to be a > bottle neck. > Basically there are at least three problems: > - the sender of the resource events does not know whether there is a > receiver, therefore events for each and every change need to be sent > - event objects are immutable and therefore all relevant data needs to > be calculated upfront, even if it's not used. For example a resource > event contains the resource type which needs to be fetched from the > repository, even if no one is interested in it. > - receivers of the events can't easily act on behalf of the user who > initiated the change. > I created a new listener api at [1] which defines a ResourceListener > interface and some ways how to specify the events one is interested in. > The user aware resource listener allows to act on behalf of the user (if > that information is available). > On the other side, a new service, the ObservationReporter [2] is > defined. Resource providers report changes through this interface. The > payload of such an event is an interface which allows for lazy retrieval > of the information. > We can also use this mechanism for compatibility and an implementation > of the observation reporter might sent all events via the event admin. > [1] > https://svn.apache.org/repos/asf/sling/whiteboard/cziegeler/api-v3/src/main/java/org/apache/sling/api/resource/observation/ > [2] > https://svn.apache.org/repos/asf/sling/whiteboard/cziegeler -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (SLING-4751) New Observation Support
[ https://issues.apache.org/jira/browse/SLING-4751?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14900264#comment-14900264 ] Michael Dürig commented on SLING-4751: -- See also OAK-3372. Depending on where this is going this might also help. > New Observation Support > --- > > Key: SLING-4751 > URL: https://issues.apache.org/jira/browse/SLING-4751 > Project: Sling > Issue Type: Improvement > Components: API, JCR, ResourceResolver >Reporter: Carsten Ziegeler >Assignee: Carsten Ziegeler > Fix For: API 2.10.0, Resource Resolver 1.2.8 > > > Mail thread: > http://mail-archives.apache.org/mod_mbox/sling-dev/201505.mbox/%3C555983F2.20402%40apache.org%3E > Starting mail > Right now, resources changes are propagated through event admin - which > at the time sounded like a good idea. Over time, this has shown to be a > bottle neck. > Basically there are at least three problems: > - the sender of the resource events does not know whether there is a > receiver, therefore events for each and every change need to be sent > - event objects are immutable and therefore all relevant data needs to > be calculated upfront, even if it's not used. For example a resource > event contains the resource type which needs to be fetched from the > repository, even if no one is interested in it. > - receivers of the events can't easily act on behalf of the user who > initiated the change. > I created a new listener api at [1] which defines a ResourceListener > interface and some ways how to specify the events one is interested in. > The user aware resource listener allows to act on behalf of the user (if > that information is available). > On the other side, a new service, the ObservationReporter [2] is > defined. Resource providers report changes through this interface. The > payload of such an event is an interface which allows for lazy retrieval > of the information. > We can also use this mechanism for compatibility and an implementation > of the observation reporter might sent all events via the event admin. > [1] > https://svn.apache.org/repos/asf/sling/whiteboard/cziegeler/api-v3/src/main/java/org/apache/sling/api/resource/observation/ > [2] > https://svn.apache.org/repos/asf/sling/whiteboard/cziegeler -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (SLING-4751) New Observation Support
[ https://issues.apache.org/jira/browse/SLING-4751?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=1484#comment-1484 ] Carsten Ziegeler commented on SLING-4751: - Just in case we want to have reliable user info, using the Observer interface (and not the BackgroundObserver base class in the JCR bridge) will give us reliably user id for all local events. This has been discussed a while back on the oak dev list > New Observation Support > --- > > Key: SLING-4751 > URL: https://issues.apache.org/jira/browse/SLING-4751 > Project: Sling > Issue Type: Improvement > Components: API, JCR, ResourceResolver >Reporter: Carsten Ziegeler >Assignee: Carsten Ziegeler > Fix For: API 2.10.0, Resource Resolver 1.2.8 > > > Mail thread: > http://mail-archives.apache.org/mod_mbox/sling-dev/201505.mbox/%3C555983F2.20402%40apache.org%3E > Starting mail > Right now, resources changes are propagated through event admin - which > at the time sounded like a good idea. Over time, this has shown to be a > bottle neck. > Basically there are at least three problems: > - the sender of the resource events does not know whether there is a > receiver, therefore events for each and every change need to be sent > - event objects are immutable and therefore all relevant data needs to > be calculated upfront, even if it's not used. For example a resource > event contains the resource type which needs to be fetched from the > repository, even if no one is interested in it. > - receivers of the events can't easily act on behalf of the user who > initiated the change. > I created a new listener api at [1] which defines a ResourceListener > interface and some ways how to specify the events one is interested in. > The user aware resource listener allows to act on behalf of the user (if > that information is available). > On the other side, a new service, the ObservationReporter [2] is > defined. Resource providers report changes through this interface. The > payload of such an event is an interface which allows for lazy retrieval > of the information. > We can also use this mechanism for compatibility and an implementation > of the observation reporter might sent all events via the event admin. > [1] > https://svn.apache.org/repos/asf/sling/whiteboard/cziegeler/api-v3/src/main/java/org/apache/sling/api/resource/observation/ > [2] > https://svn.apache.org/repos/asf/sling/whiteboard/cziegeler -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (SLING-4751) New Observation Support
[ https://issues.apache.org/jira/browse/SLING-4751?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14585805#comment-14585805 ] Carsten Ziegeler commented on SLING-4751: - That's a good point. Answering in reverse order - we'll probably drop UserAwareResourceListener for the moment; there is no point in having this if the underlying store can't reliably provide the user information - The resource change object will hopefully not reference an opened resource resolver but a factory object of some kind. When additional data is requested, a resource resolver is created, fetches all data, and released (I guess it will not be a resource resolver, but a resource provider internal object). This step should be synchronized. - We should add to the javadocs that the resource change object is only valid within the onChange method. If its passed around or processed async, there is no guarantee. For passing around, maybe we should add a clone() method New Observation Support --- Key: SLING-4751 URL: https://issues.apache.org/jira/browse/SLING-4751 Project: Sling Issue Type: Improvement Components: API, JCR, ResourceResolver Reporter: Carsten Ziegeler Assignee: Carsten Ziegeler Fix For: API 2.10.0, Resource Resolver 1.2.6 Mail thread: http://mail-archives.apache.org/mod_mbox/sling-dev/201505.mbox/%3C555983F2.20402%40apache.org%3E Starting mail Right now, resources changes are propagated through event admin - which at the time sounded like a good idea. Over time, this has shown to be a bottle neck. Basically there are at least three problems: - the sender of the resource events does not know whether there is a receiver, therefore events for each and every change need to be sent - event objects are immutable and therefore all relevant data needs to be calculated upfront, even if it's not used. For example a resource event contains the resource type which needs to be fetched from the repository, even if no one is interested in it. - receivers of the events can't easily act on behalf of the user who initiated the change. I created a new listener api at [1] which defines a ResourceListener interface and some ways how to specify the events one is interested in. The user aware resource listener allows to act on behalf of the user (if that information is available). On the other side, a new service, the ObservationReporter [2] is defined. Resource providers report changes through this interface. The payload of such an event is an interface which allows for lazy retrieval of the information. We can also use this mechanism for compatibility and an implementation of the observation reporter might sent all events via the event admin. [1] https://svn.apache.org/repos/asf/sling/whiteboard/cziegeler/api-v3/src/main/java/org/apache/sling/api/resource/observation/ [2] https://svn.apache.org/repos/asf/sling/whiteboard/cziegeler -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (SLING-4751) New Observation Support
[ https://issues.apache.org/jira/browse/SLING-4751?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14585780#comment-14585780 ] Marc Pfaff commented on SLING-4751: --- Hi I have question related to the lazy loading and inspection of ResourceChange objects. I assume this feature will turn ResourceChange events into resource wrapper kind of objects. In this context, I wonder what the life cycle will be of the underlaying resolver used by the change objects? I'm asking because I would expect a ResourceListener implementation to have the freedom to process the change events asynchronously. In this case I have concerns that it's very easy for ResourceListener implementations to leave the thread boundary of the change objects, leading to thread contention issues on the resolver and underlaying JCR session. The same concern applies to the prototype of UserAwareResourceListener.onChange() with regards the passed resource resolver. New Observation Support --- Key: SLING-4751 URL: https://issues.apache.org/jira/browse/SLING-4751 Project: Sling Issue Type: Improvement Components: API, JCR, ResourceResolver Reporter: Carsten Ziegeler Assignee: Carsten Ziegeler Fix For: API 2.10.0, Resource Resolver 1.2.6 Mail thread: http://mail-archives.apache.org/mod_mbox/sling-dev/201505.mbox/%3C555983F2.20402%40apache.org%3E Starting mail Right now, resources changes are propagated through event admin - which at the time sounded like a good idea. Over time, this has shown to be a bottle neck. Basically there are at least three problems: - the sender of the resource events does not know whether there is a receiver, therefore events for each and every change need to be sent - event objects are immutable and therefore all relevant data needs to be calculated upfront, even if it's not used. For example a resource event contains the resource type which needs to be fetched from the repository, even if no one is interested in it. - receivers of the events can't easily act on behalf of the user who initiated the change. I created a new listener api at [1] which defines a ResourceListener interface and some ways how to specify the events one is interested in. The user aware resource listener allows to act on behalf of the user (if that information is available). On the other side, a new service, the ObservationReporter [2] is defined. Resource providers report changes through this interface. The payload of such an event is an interface which allows for lazy retrieval of the information. We can also use this mechanism for compatibility and an implementation of the observation reporter might sent all events via the event admin. [1] https://svn.apache.org/repos/asf/sling/whiteboard/cziegeler/api-v3/src/main/java/org/apache/sling/api/resource/observation/ [2] https://svn.apache.org/repos/asf/sling/whiteboard/cziegeler -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (SLING-4751) New Observation Support
[ https://issues.apache.org/jira/browse/SLING-4751?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14584681#comment-14584681 ] Carsten Ziegeler commented on SLING-4751: - One advantage of this new api, even if it would allow the same type of listeners as currently, is that the information can be fetched lazily. Right now, for each and every change, a repository read is necessary to get the resource type New Observation Support --- Key: SLING-4751 URL: https://issues.apache.org/jira/browse/SLING-4751 Project: Sling Issue Type: Improvement Components: API, JCR, ResourceResolver Reporter: Carsten Ziegeler Assignee: Carsten Ziegeler Fix For: API 2.10.0, Resource Resolver 1.2.6 Mail thread: http://mail-archives.apache.org/mod_mbox/sling-dev/201505.mbox/%3C555983F2.20402%40apache.org%3E Starting mail Right now, resources changes are propagated through event admin - which at the time sounded like a good idea. Over time, this has shown to be a bottle neck. Basically there are at least three problems: - the sender of the resource events does not know whether there is a receiver, therefore events for each and every change need to be sent - event objects are immutable and therefore all relevant data needs to be calculated upfront, even if it's not used. For example a resource event contains the resource type which needs to be fetched from the repository, even if no one is interested in it. - receivers of the events can't easily act on behalf of the user who initiated the change. I created a new listener api at [1] which defines a ResourceListener interface and some ways how to specify the events one is interested in. The user aware resource listener allows to act on behalf of the user (if that information is available). On the other side, a new service, the ObservationReporter [2] is defined. Resource providers report changes through this interface. The payload of such an event is an interface which allows for lazy retrieval of the information. We can also use this mechanism for compatibility and an implementation of the observation reporter might sent all events via the event admin. [1] https://svn.apache.org/repos/asf/sling/whiteboard/cziegeler/api-v3/src/main/java/org/apache/sling/api/resource/observation/ [2] https://svn.apache.org/repos/asf/sling/whiteboard/cziegeler -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (SLING-4751) New Observation Support
[ https://issues.apache.org/jira/browse/SLING-4751?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14584675#comment-14584675 ] Carsten Ziegeler commented on SLING-4751: - I've moved the prototype api to trunk New Observation Support --- Key: SLING-4751 URL: https://issues.apache.org/jira/browse/SLING-4751 Project: Sling Issue Type: Improvement Components: API, JCR, ResourceResolver Reporter: Carsten Ziegeler Assignee: Carsten Ziegeler Fix For: API 2.10.0, Resource Resolver 1.2.6 Mail thread: http://mail-archives.apache.org/mod_mbox/sling-dev/201505.mbox/%3C555983F2.20402%40apache.org%3E Starting mail Right now, resources changes are propagated through event admin - which at the time sounded like a good idea. Over time, this has shown to be a bottle neck. Basically there are at least three problems: - the sender of the resource events does not know whether there is a receiver, therefore events for each and every change need to be sent - event objects are immutable and therefore all relevant data needs to be calculated upfront, even if it's not used. For example a resource event contains the resource type which needs to be fetched from the repository, even if no one is interested in it. - receivers of the events can't easily act on behalf of the user who initiated the change. I created a new listener api at [1] which defines a ResourceListener interface and some ways how to specify the events one is interested in. The user aware resource listener allows to act on behalf of the user (if that information is available). On the other side, a new service, the ObservationReporter [2] is defined. Resource providers report changes through this interface. The payload of such an event is an interface which allows for lazy retrieval of the information. We can also use this mechanism for compatibility and an implementation of the observation reporter might sent all events via the event admin. [1] https://svn.apache.org/repos/asf/sling/whiteboard/cziegeler/api-v3/src/main/java/org/apache/sling/api/resource/observation/ [2] https://svn.apache.org/repos/asf/sling/whiteboard/cziegeler -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (SLING-4751) New Observation Support
[ https://issues.apache.org/jira/browse/SLING-4751?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14572759#comment-14572759 ] Stefan Egli commented on SLING-4751: So for the former cache cases you could add an 'AggregateResourceListener' which is only interested in changes at-or-under a certain path, but not all the details of what exactly has changed under that path. That would allow to optimize further down the stack in oak even - if you propagate this information accordingly. I think the advantage of having specialized ResourceListeners instead of just a generic one is that it makes the use cases much more explicit, thus making users also more aware of what implications they can on scalability/performance have when using the wrong listener. The generic one could even not be supported at all in a cluster for example. Additionally it would allow us to look at each listener class and optimize each case individually for scalability. With the goal that as few external changes as possible would have to be processed in a cluster. It would also allow to have monitoring etc on the different types perhaps even showing their influence they have on scalability/performance. All of these optimizations would become difficult to introduce with just having one generic type IMO. With, if not optimized, will eventually become a scalability bottleneck. New Observation Support --- Key: SLING-4751 URL: https://issues.apache.org/jira/browse/SLING-4751 Project: Sling Issue Type: Improvement Components: API, JCR, ResourceResolver Reporter: Carsten Ziegeler Assignee: Carsten Ziegeler Fix For: API 2.10.0, Resource Resolver 1.2.6 Mail thread: http://mail-archives.apache.org/mod_mbox/sling-dev/201505.mbox/%3C555983F2.20402%40apache.org%3E Starting mail Right now, resources changes are propagated through event admin - which at the time sounded like a good idea. Over time, this has shown to be a bottle neck. Basically there are at least three problems: - the sender of the resource events does not know whether there is a receiver, therefore events for each and every change need to be sent - event objects are immutable and therefore all relevant data needs to be calculated upfront, even if it's not used. For example a resource event contains the resource type which needs to be fetched from the repository, even if no one is interested in it. - receivers of the events can't easily act on behalf of the user who initiated the change. I created a new listener api at [1] which defines a ResourceListener interface and some ways how to specify the events one is interested in. The user aware resource listener allows to act on behalf of the user (if that information is available). On the other side, a new service, the ObservationReporter [2] is defined. Resource providers report changes through this interface. The payload of such an event is an interface which allows for lazy retrieval of the information. We can also use this mechanism for compatibility and an implementation of the observation reporter might sent all events via the event admin. [1] https://svn.apache.org/repos/asf/sling/whiteboard/cziegeler/api-v3/src/main/java/org/apache/sling/api/resource/observation/ [2] https://svn.apache.org/repos/asf/sling/whiteboard/cziegeler -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (SLING-4751) New Observation Support
[ https://issues.apache.org/jira/browse/SLING-4751?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14567172#comment-14567172 ] Michael Dürig commented on SLING-4751: -- Nothing really. Just saying that we've seen the problem with listeners being swamped and couldn't keep up in the past. This will only get worse once we really start to scale out. So it would good to come up with ways to pro actively cope with this instead of just degrading in an uncontrollable manner. New Observation Support --- Key: SLING-4751 URL: https://issues.apache.org/jira/browse/SLING-4751 Project: Sling Issue Type: Improvement Components: API, JCR, ResourceResolver Reporter: Carsten Ziegeler Assignee: Carsten Ziegeler Fix For: API 2.10.0, Resource Resolver 1.2.6 Mail thread: http://mail-archives.apache.org/mod_mbox/sling-dev/201505.mbox/%3C555983F2.20402%40apache.org%3E Starting mail Right now, resources changes are propagated through event admin - which at the time sounded like a good idea. Over time, this has shown to be a bottle neck. Basically there are at least three problems: - the sender of the resource events does not know whether there is a receiver, therefore events for each and every change need to be sent - event objects are immutable and therefore all relevant data needs to be calculated upfront, even if it's not used. For example a resource event contains the resource type which needs to be fetched from the repository, even if no one is interested in it. - receivers of the events can't easily act on behalf of the user who initiated the change. I created a new listener api at [1] which defines a ResourceListener interface and some ways how to specify the events one is interested in. The user aware resource listener allows to act on behalf of the user (if that information is available). On the other side, a new service, the ObservationReporter [2] is defined. Resource providers report changes through this interface. The payload of such an event is an interface which allows for lazy retrieval of the information. We can also use this mechanism for compatibility and an implementation of the observation reporter might sent all events via the event admin. [1] https://svn.apache.org/repos/asf/sling/whiteboard/cziegeler/api-v3/src/main/java/org/apache/sling/api/resource/observation/ [2] https://svn.apache.org/repos/asf/sling/whiteboard/cziegeler -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (SLING-4751) New Observation Support
[
https://issues.apache.org/jira/browse/SLING-4751?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14567013#comment-14567013
]
Stefan Egli commented on SLING-4751:
I see two problems with the previous (current) implementation - not sure if you
covered them already:
# the 'bridge' listener has to listen to any event (ie {{/}}) as it has no
idea on what paths the listeners are interested.
# there is no distinction between events generated by the local instance and
by any other instance in the cluster (external)
Re 1. the new ResourceListener has {{PATHS}} which is good. But I guess the
'bridge' listener will still have to listen to {{/}} so not much gained there?
Re 2. this I think would be an important distinction to have.
New Observation Support
---
Key: SLING-4751
URL: https://issues.apache.org/jira/browse/SLING-4751
Project: Sling
Issue Type: Improvement
Components: API, JCR, ResourceResolver
Reporter: Carsten Ziegeler
Assignee: Carsten Ziegeler
Fix For: API 2.10.0, Resource Resolver 1.2.6
Mail thread:
http://mail-archives.apache.org/mod_mbox/sling-dev/201505.mbox/%3C555983F2.20402%40apache.org%3E
Starting mail
Right now, resources changes are propagated through event admin - which
at the time sounded like a good idea. Over time, this has shown to be a
bottle neck.
Basically there are at least three problems:
- the sender of the resource events does not know whether there is a
receiver, therefore events for each and every change need to be sent
- event objects are immutable and therefore all relevant data needs to
be calculated upfront, even if it's not used. For example a resource
event contains the resource type which needs to be fetched from the
repository, even if no one is interested in it.
- receivers of the events can't easily act on behalf of the user who
initiated the change.
I created a new listener api at [1] which defines a ResourceListener
interface and some ways how to specify the events one is interested in.
The user aware resource listener allows to act on behalf of the user (if
that information is available).
On the other side, a new service, the ObservationReporter [2] is
defined. Resource providers report changes through this interface. The
payload of such an event is an interface which allows for lazy retrieval
of the information.
We can also use this mechanism for compatibility and an implementation
of the observation reporter might sent all events via the event admin.
[1]
https://svn.apache.org/repos/asf/sling/whiteboard/cziegeler/api-v3/src/main/java/org/apache/sling/api/resource/observation/
[2]
https://svn.apache.org/repos/asf/sling/whiteboard/cziegeler
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (SLING-4751) New Observation Support
[ https://issues.apache.org/jira/browse/SLING-4751?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14567160#comment-14567160 ] Carsten Ziegeler commented on SLING-4751: - Yes, one of the benefits of this approach is that the implementation knows the listener and therefore can be optimized. As long as we support the OSGi events, we don't gain anything, but with this we can deprecate it and move away from it. The distinction between local and external makes sense for as long as you have this information. For example with Oak this information is not there in all cases and it can happen that local events are reported as external one - which in turn makes it dangerous to listen to local events New Observation Support --- Key: SLING-4751 URL: https://issues.apache.org/jira/browse/SLING-4751 Project: Sling Issue Type: Improvement Components: API, JCR, ResourceResolver Reporter: Carsten Ziegeler Assignee: Carsten Ziegeler Fix For: API 2.10.0, Resource Resolver 1.2.6 Mail thread: http://mail-archives.apache.org/mod_mbox/sling-dev/201505.mbox/%3C555983F2.20402%40apache.org%3E Starting mail Right now, resources changes are propagated through event admin - which at the time sounded like a good idea. Over time, this has shown to be a bottle neck. Basically there are at least three problems: - the sender of the resource events does not know whether there is a receiver, therefore events for each and every change need to be sent - event objects are immutable and therefore all relevant data needs to be calculated upfront, even if it's not used. For example a resource event contains the resource type which needs to be fetched from the repository, even if no one is interested in it. - receivers of the events can't easily act on behalf of the user who initiated the change. I created a new listener api at [1] which defines a ResourceListener interface and some ways how to specify the events one is interested in. The user aware resource listener allows to act on behalf of the user (if that information is available). On the other side, a new service, the ObservationReporter [2] is defined. Resource providers report changes through this interface. The payload of such an event is an interface which allows for lazy retrieval of the information. We can also use this mechanism for compatibility and an implementation of the observation reporter might sent all events via the event admin. [1] https://svn.apache.org/repos/asf/sling/whiteboard/cziegeler/api-v3/src/main/java/org/apache/sling/api/resource/observation/ [2] https://svn.apache.org/repos/asf/sling/whiteboard/cziegeler -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (SLING-4751) New Observation Support
[ https://issues.apache.org/jira/browse/SLING-4751?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14567162#comment-14567162 ] Carsten Ziegeler commented on SLING-4751: - What exactly do you have in mind here? New Observation Support --- Key: SLING-4751 URL: https://issues.apache.org/jira/browse/SLING-4751 Project: Sling Issue Type: Improvement Components: API, JCR, ResourceResolver Reporter: Carsten Ziegeler Assignee: Carsten Ziegeler Fix For: API 2.10.0, Resource Resolver 1.2.6 Mail thread: http://mail-archives.apache.org/mod_mbox/sling-dev/201505.mbox/%3C555983F2.20402%40apache.org%3E Starting mail Right now, resources changes are propagated through event admin - which at the time sounded like a good idea. Over time, this has shown to be a bottle neck. Basically there are at least three problems: - the sender of the resource events does not know whether there is a receiver, therefore events for each and every change need to be sent - event objects are immutable and therefore all relevant data needs to be calculated upfront, even if it's not used. For example a resource event contains the resource type which needs to be fetched from the repository, even if no one is interested in it. - receivers of the events can't easily act on behalf of the user who initiated the change. I created a new listener api at [1] which defines a ResourceListener interface and some ways how to specify the events one is interested in. The user aware resource listener allows to act on behalf of the user (if that information is available). On the other side, a new service, the ObservationReporter [2] is defined. Resource providers report changes through this interface. The payload of such an event is an interface which allows for lazy retrieval of the information. We can also use this mechanism for compatibility and an implementation of the observation reporter might sent all events via the event admin. [1] https://svn.apache.org/repos/asf/sling/whiteboard/cziegeler/api-v3/src/main/java/org/apache/sling/api/resource/observation/ [2] https://svn.apache.org/repos/asf/sling/whiteboard/cziegeler -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (SLING-4751) New Observation Support
[ https://issues.apache.org/jira/browse/SLING-4751?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14567165#comment-14567165 ] Stefan Egli commented on SLING-4751: With OAK-2829 the goal is to speed up external events a lot thus helping in the overall goal to avoid hitting the observation queue limit. But that alone still does not truly scale to infinity: if there are hundreds of servers then external events become a delicate issue and ideally only very limited number of external events should have to be processed at all. The problem of having 'mixed' events that occur in case of hitting the limit is - agreed - a nasty one. But for the rest, maybe it would help to propagate this concept of 'internal vs external' events also to the ResourceListener? Where is this 'internal vs external' event not available other than in case of hitting the queue limit? New Observation Support --- Key: SLING-4751 URL: https://issues.apache.org/jira/browse/SLING-4751 Project: Sling Issue Type: Improvement Components: API, JCR, ResourceResolver Reporter: Carsten Ziegeler Assignee: Carsten Ziegeler Fix For: API 2.10.0, Resource Resolver 1.2.6 Mail thread: http://mail-archives.apache.org/mod_mbox/sling-dev/201505.mbox/%3C555983F2.20402%40apache.org%3E Starting mail Right now, resources changes are propagated through event admin - which at the time sounded like a good idea. Over time, this has shown to be a bottle neck. Basically there are at least three problems: - the sender of the resource events does not know whether there is a receiver, therefore events for each and every change need to be sent - event objects are immutable and therefore all relevant data needs to be calculated upfront, even if it's not used. For example a resource event contains the resource type which needs to be fetched from the repository, even if no one is interested in it. - receivers of the events can't easily act on behalf of the user who initiated the change. I created a new listener api at [1] which defines a ResourceListener interface and some ways how to specify the events one is interested in. The user aware resource listener allows to act on behalf of the user (if that information is available). On the other side, a new service, the ObservationReporter [2] is defined. Resource providers report changes through this interface. The payload of such an event is an interface which allows for lazy retrieval of the information. We can also use this mechanism for compatibility and an implementation of the observation reporter might sent all events via the event admin. [1] https://svn.apache.org/repos/asf/sling/whiteboard/cziegeler/api-v3/src/main/java/org/apache/sling/api/resource/observation/ [2] https://svn.apache.org/repos/asf/sling/whiteboard/cziegeler -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (SLING-4751) New Observation Support
[ https://issues.apache.org/jira/browse/SLING-4751?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14567030#comment-14567030 ] Michael Dürig commented on SLING-4751: -- Does this new mechanism provide a way to apply back pressure? Even though we don't have good back pressure support yet on the repository side, this is something that keeps coming up. The more we scale the repository the more crucial will this become. So I suggest we spare some thoughts on this now. New Observation Support --- Key: SLING-4751 URL: https://issues.apache.org/jira/browse/SLING-4751 Project: Sling Issue Type: Improvement Components: API, JCR, ResourceResolver Reporter: Carsten Ziegeler Assignee: Carsten Ziegeler Fix For: API 2.10.0, Resource Resolver 1.2.6 Mail thread: http://mail-archives.apache.org/mod_mbox/sling-dev/201505.mbox/%3C555983F2.20402%40apache.org%3E Starting mail Right now, resources changes are propagated through event admin - which at the time sounded like a good idea. Over time, this has shown to be a bottle neck. Basically there are at least three problems: - the sender of the resource events does not know whether there is a receiver, therefore events for each and every change need to be sent - event objects are immutable and therefore all relevant data needs to be calculated upfront, even if it's not used. For example a resource event contains the resource type which needs to be fetched from the repository, even if no one is interested in it. - receivers of the events can't easily act on behalf of the user who initiated the change. I created a new listener api at [1] which defines a ResourceListener interface and some ways how to specify the events one is interested in. The user aware resource listener allows to act on behalf of the user (if that information is available). On the other side, a new service, the ObservationReporter [2] is defined. Resource providers report changes through this interface. The payload of such an event is an interface which allows for lazy retrieval of the information. We can also use this mechanism for compatibility and an implementation of the observation reporter might sent all events via the event admin. [1] https://svn.apache.org/repos/asf/sling/whiteboard/cziegeler/api-v3/src/main/java/org/apache/sling/api/resource/observation/ [2] https://svn.apache.org/repos/asf/sling/whiteboard/cziegeler -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (SLING-4751) New Observation Support
[ https://issues.apache.org/jira/browse/SLING-4751?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14567464#comment-14567464 ] Stefan Egli commented on SLING-4751: Maybe we can group all of this into different use cases to better see what kinds of listeners we expect and how we can support them? The 'exactly once' for example I think should be solved via a job. The 'caching' example could be done by listening on certain patterns (with a path prefix) instead of all child modifications too. The result could be that the listener would not be entirely free as to what it wants to listen on, but would have to be more specific (internal vs external, path vs child changes etc). Maybe this is going too far, but whatever can be optimized to be more scalable for the ResourceListener could in my view help. New Observation Support --- Key: SLING-4751 URL: https://issues.apache.org/jira/browse/SLING-4751 Project: Sling Issue Type: Improvement Components: API, JCR, ResourceResolver Reporter: Carsten Ziegeler Assignee: Carsten Ziegeler Fix For: API 2.10.0, Resource Resolver 1.2.6 Mail thread: http://mail-archives.apache.org/mod_mbox/sling-dev/201505.mbox/%3C555983F2.20402%40apache.org%3E Starting mail Right now, resources changes are propagated through event admin - which at the time sounded like a good idea. Over time, this has shown to be a bottle neck. Basically there are at least three problems: - the sender of the resource events does not know whether there is a receiver, therefore events for each and every change need to be sent - event objects are immutable and therefore all relevant data needs to be calculated upfront, even if it's not used. For example a resource event contains the resource type which needs to be fetched from the repository, even if no one is interested in it. - receivers of the events can't easily act on behalf of the user who initiated the change. I created a new listener api at [1] which defines a ResourceListener interface and some ways how to specify the events one is interested in. The user aware resource listener allows to act on behalf of the user (if that information is available). On the other side, a new service, the ObservationReporter [2] is defined. Resource providers report changes through this interface. The payload of such an event is an interface which allows for lazy retrieval of the information. We can also use this mechanism for compatibility and an implementation of the observation reporter might sent all events via the event admin. [1] https://svn.apache.org/repos/asf/sling/whiteboard/cziegeler/api-v3/src/main/java/org/apache/sling/api/resource/observation/ [2] https://svn.apache.org/repos/asf/sling/whiteboard/cziegeler -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (SLING-4751) New Observation Support
[ https://issues.apache.org/jira/browse/SLING-4751?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14567318#comment-14567318 ] Carsten Ziegeler commented on SLING-4751: - I think hitting the queue limit is the only case. Ok, let's see first why local vs external is interesting/important: it allows to distribute the processing of these events in an easy way. You want to make sure that at least one instance gets the events and that its clear which instance should act on the events. For this, local is a pretty good marker - however if there are case, no matter how rarely these might happen, where this is not guaranteed to work, it means no listener will pick up changes as they are marked as external on all instances. Now, the other typical use case for observation is to clear caches. In this case you need to report all changes on all instances in order to flush a cache. For example script compilation is a pretty good use case New Observation Support --- Key: SLING-4751 URL: https://issues.apache.org/jira/browse/SLING-4751 Project: Sling Issue Type: Improvement Components: API, JCR, ResourceResolver Reporter: Carsten Ziegeler Assignee: Carsten Ziegeler Fix For: API 2.10.0, Resource Resolver 1.2.6 Mail thread: http://mail-archives.apache.org/mod_mbox/sling-dev/201505.mbox/%3C555983F2.20402%40apache.org%3E Starting mail Right now, resources changes are propagated through event admin - which at the time sounded like a good idea. Over time, this has shown to be a bottle neck. Basically there are at least three problems: - the sender of the resource events does not know whether there is a receiver, therefore events for each and every change need to be sent - event objects are immutable and therefore all relevant data needs to be calculated upfront, even if it's not used. For example a resource event contains the resource type which needs to be fetched from the repository, even if no one is interested in it. - receivers of the events can't easily act on behalf of the user who initiated the change. I created a new listener api at [1] which defines a ResourceListener interface and some ways how to specify the events one is interested in. The user aware resource listener allows to act on behalf of the user (if that information is available). On the other side, a new service, the ObservationReporter [2] is defined. Resource providers report changes through this interface. The payload of such an event is an interface which allows for lazy retrieval of the information. We can also use this mechanism for compatibility and an implementation of the observation reporter might sent all events via the event admin. [1] https://svn.apache.org/repos/asf/sling/whiteboard/cziegeler/api-v3/src/main/java/org/apache/sling/api/resource/observation/ [2] https://svn.apache.org/repos/asf/sling/whiteboard/cziegeler -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (SLING-4751) New Observation Support
[ https://issues.apache.org/jira/browse/SLING-4751?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14567367#comment-14567367 ] Carsten Ziegeler commented on SLING-4751: - I also think, we should drop the event user aware listener for now as the info is not guaranteed by Oak either (OAK-2772). Once that's fixed we have this prototype interface and we can continue to work on it. New Observation Support --- Key: SLING-4751 URL: https://issues.apache.org/jira/browse/SLING-4751 Project: Sling Issue Type: Improvement Components: API, JCR, ResourceResolver Reporter: Carsten Ziegeler Assignee: Carsten Ziegeler Fix For: API 2.10.0, Resource Resolver 1.2.6 Mail thread: http://mail-archives.apache.org/mod_mbox/sling-dev/201505.mbox/%3C555983F2.20402%40apache.org%3E Starting mail Right now, resources changes are propagated through event admin - which at the time sounded like a good idea. Over time, this has shown to be a bottle neck. Basically there are at least three problems: - the sender of the resource events does not know whether there is a receiver, therefore events for each and every change need to be sent - event objects are immutable and therefore all relevant data needs to be calculated upfront, even if it's not used. For example a resource event contains the resource type which needs to be fetched from the repository, even if no one is interested in it. - receivers of the events can't easily act on behalf of the user who initiated the change. I created a new listener api at [1] which defines a ResourceListener interface and some ways how to specify the events one is interested in. The user aware resource listener allows to act on behalf of the user (if that information is available). On the other side, a new service, the ObservationReporter [2] is defined. Resource providers report changes through this interface. The payload of such an event is an interface which allows for lazy retrieval of the information. We can also use this mechanism for compatibility and an implementation of the observation reporter might sent all events via the event admin. [1] https://svn.apache.org/repos/asf/sling/whiteboard/cziegeler/api-v3/src/main/java/org/apache/sling/api/resource/observation/ [2] https://svn.apache.org/repos/asf/sling/whiteboard/cziegeler -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (SLING-4751) New Observation Support
[ https://issues.apache.org/jira/browse/SLING-4751?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14560923#comment-14560923 ] Carsten Ziegeler commented on SLING-4751: - Deal, thanks, I've updated the description. New Observation Support --- Key: SLING-4751 URL: https://issues.apache.org/jira/browse/SLING-4751 Project: Sling Issue Type: Improvement Components: API, JCR, ResourceResolver Reporter: Carsten Ziegeler Assignee: Carsten Ziegeler Fix For: API 2.10.0, Resource Resolver 1.2.6 Mail thread: http://mail-archives.apache.org/mod_mbox/sling-dev/201505.mbox/%3C555983F2.20402%40apache.org%3E Starting mail Right now, resources changes are propagated through event admin - which at the time sounded like a good idea. Over time, this has shown to be a bottle neck. Basically there are at least three problems: - the sender of the resource events does not know whether there is a receiver, therefore events for each and every change need to be sent - event objects are immutable and therefore all relevant data needs to be calculated upfront, even if it's not used. For example a resource event contains the resource type which needs to be fetched from the repository, even if no one is interested in it. - receivers of the events can't easily act on behalf of the user who initiated the change. I created a new listener api at [1] which defines a ResourceListener interface and some ways how to specify the events one is interested in. The user aware resource listener allows to act on behalf of the user (if that information is available). On the other side, a new service, the ObservationReporter [2] is defined. Resource providers report changes through this interface. The payload of such an event is an interface which allows for lazy retrieval of the information. We can also use this mechanism for compatibility and an implementation of the observation reporter might sent all events via the event admin. [1] https://svn.apache.org/repos/asf/sling/whiteboard/cziegeler/api-v3/src/main/java/org/apache/sling/api/resource/observation/ [2] https://svn.apache.org/repos/asf/sling/whiteboard/cziegeler -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (SLING-4751) New Observation Support
[ https://issues.apache.org/jira/browse/SLING-4751?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14560904#comment-14560904 ] Bertrand Delacretaz commented on SLING-4751: bq. I think the serviceuser property is currently better described... Still, I had to reread that description several times to understand it ;-) How about Required property containing the service user name to use for the ResourceResolver passed to onChange, if the change event does not provide the actual user ? New Observation Support --- Key: SLING-4751 URL: https://issues.apache.org/jira/browse/SLING-4751 Project: Sling Issue Type: Improvement Components: API, JCR, ResourceResolver Reporter: Carsten Ziegeler Assignee: Carsten Ziegeler Fix For: API 2.10.0, Resource Resolver 1.2.6 Mail thread: http://mail-archives.apache.org/mod_mbox/sling-dev/201505.mbox/%3C555983F2.20402%40apache.org%3E Starting mail Right now, resources changes are propagated through event admin - which at the time sounded like a good idea. Over time, this has shown to be a bottle neck. Basically there are at least three problems: - the sender of the resource events does not know whether there is a receiver, therefore events for each and every change need to be sent - event objects are immutable and therefore all relevant data needs to be calculated upfront, even if it's not used. For example a resource event contains the resource type which needs to be fetched from the repository, even if no one is interested in it. - receivers of the events can't easily act on behalf of the user who initiated the change. I created a new listener api at [1] which defines a ResourceListener interface and some ways how to specify the events one is interested in. The user aware resource listener allows to act on behalf of the user (if that information is available). On the other side, a new service, the ObservationReporter [2] is defined. Resource providers report changes through this interface. The payload of such an event is an interface which allows for lazy retrieval of the information. We can also use this mechanism for compatibility and an implementation of the observation reporter might sent all events via the event admin. [1] https://svn.apache.org/repos/asf/sling/whiteboard/cziegeler/api-v3/src/main/java/org/apache/sling/api/resource/observation/ [2] https://svn.apache.org/repos/asf/sling/whiteboard/cziegeler -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (SLING-4751) New Observation Support
[ https://issues.apache.org/jira/browse/SLING-4751?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14560882#comment-14560882 ] Carsten Ziegeler commented on SLING-4751: - For the reporting we can call the event admin through the reporter implementation. So providers just need to report it through the new api I've changed the CHANGES property as suggested I think the serviceuser property is currently better described, because the resolver will either have the privileges of the user who changed the content (if the info is available) or the privileges of the service user New Observation Support --- Key: SLING-4751 URL: https://issues.apache.org/jira/browse/SLING-4751 Project: Sling Issue Type: Improvement Components: API, JCR, ResourceResolver Reporter: Carsten Ziegeler Assignee: Carsten Ziegeler Fix For: API 2.10.0, Resource Resolver 1.2.6 Mail thread: http://mail-archives.apache.org/mod_mbox/sling-dev/201505.mbox/%3C555983F2.20402%40apache.org%3E Starting mail Right now, resources changes are propagated through event admin - which at the time sounded like a good idea. Over time, this has shown to be a bottle neck. Basically there are at least three problems: - the sender of the resource events does not know whether there is a receiver, therefore events for each and every change need to be sent - event objects are immutable and therefore all relevant data needs to be calculated upfront, even if it's not used. For example a resource event contains the resource type which needs to be fetched from the repository, even if no one is interested in it. - receivers of the events can't easily act on behalf of the user who initiated the change. I created a new listener api at [1] which defines a ResourceListener interface and some ways how to specify the events one is interested in. The user aware resource listener allows to act on behalf of the user (if that information is available). On the other side, a new service, the ObservationReporter [2] is defined. Resource providers report changes through this interface. The payload of such an event is an interface which allows for lazy retrieval of the information. We can also use this mechanism for compatibility and an implementation of the observation reporter might sent all events via the event admin. [1] https://svn.apache.org/repos/asf/sling/whiteboard/cziegeler/api-v3/src/main/java/org/apache/sling/api/resource/observation/ [2] https://svn.apache.org/repos/asf/sling/whiteboard/cziegeler -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (SLING-4751) New Observation Support
[ https://issues.apache.org/jira/browse/SLING-4751?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14560597#comment-14560597 ] Bertrand Delacretaz commented on SLING-4751: Looks good to me, thanks! How do you see the transition from the current mechanism based on OSGi events? I suppose resource providers will need to report to both, or a bridge needs to be implemented? A few minor suggestions: In ResourceListener and UserAwareResourceListener the changes property name should be resource.change.types In UserAwareResourceListener the resource.serviceuser comment can be improved as follows: The name of the service user to use for the ResourceResolver passed to onChange, if that resolver is not bound to the user who initiated the changes. New Observation Support --- Key: SLING-4751 URL: https://issues.apache.org/jira/browse/SLING-4751 Project: Sling Issue Type: Improvement Components: API, JCR, ResourceResolver Reporter: Carsten Ziegeler Assignee: Carsten Ziegeler Fix For: API 2.10.0, Resource Resolver 1.2.6 Mail thread: http://mail-archives.apache.org/mod_mbox/sling-dev/201505.mbox/%3C555983F2.20402%40apache.org%3E Starting mail Right now, resources changes are propagated through event admin - which at the time sounded like a good idea. Over time, this has shown to be a bottle neck. Basically there are at least three problems: - the sender of the resource events does not know whether there is a receiver, therefore events for each and every change need to be sent - event objects are immutable and therefore all relevant data needs to be calculated upfront, even if it's not used. For example a resource event contains the resource type which needs to be fetched from the repository, even if no one is interested in it. - receivers of the events can't easily act on behalf of the user who initiated the change. I created a new listener api at [1] which defines a ResourceListener interface and some ways how to specify the events one is interested in. The user aware resource listener allows to act on behalf of the user (if that information is available). On the other side, a new service, the ObservationReporter [2] is defined. Resource providers report changes through this interface. The payload of such an event is an interface which allows for lazy retrieval of the information. We can also use this mechanism for compatibility and an implementation of the observation reporter might sent all events via the event admin. [1] https://svn.apache.org/repos/asf/sling/whiteboard/cziegeler/api-v3/src/main/java/org/apache/sling/api/resource/observation/ [2] https://svn.apache.org/repos/asf/sling/whiteboard/cziegeler -- This message was sent by Atlassian JIRA (v6.3.4#6332)
