Re: What all changes does ResourceChangeListener are interested in

[Stefan Egli]

On 17/10/16 09:38, "Carsten Ziegeler"  wrote:

>Chetan Mehrotra wrote
>> On Mon, Oct 17, 2016 at 11:04 AM, Carsten Ziegeler
>> wrote:

> I would rather go with a "property names filter hint" - and if we
>detect
 other use cases we add another property for that.

 Makes sense. This would keep things simple and precise
>>>
>>> I plan to release API early this week :) So it would be great if we
>>> could get this in, or we move it to the next api release.
>> 
>> Would ObserverConfiguration be the place to expose this hint part?
>> 
>This needs enhancements in several areas:
>1. ResourceChangeListener we need to define a new service registration
>property
>2. And - as you note - ObserverConfiguration needs a getter method to
>expose the hint to the (jcr) implementation

I've added a patch with the above to SLING-6164 - would that then be
feasible to be included in the API?

Cheers,
Stefan
--
https://issues.apache.org/jira/browse/SLING-6164

Re: What all changes does ResourceChangeListener are interested in

[Carsten Ziegeler]

Chetan Mehrotra wrote
> On Mon, Oct 17, 2016 at 11:04 AM, Carsten Ziegeler  
> wrote:
>>>
 I would rather go with a "property names filter hint" - and if we detect
>>> other use cases we add another property for that.
>>>
>>> Makes sense. This would keep things simple and precise
>>
>> I plan to release API early this week :) So it would be great if we
>> could get this in, or we move it to the next api release.
> 
> Would ObserverConfiguration be the place to expose this hint part?
> 
This needs enhancements in several areas:
1. ResourceChangeListener we need to define a new service registration
property
2. And - as you note - ObserverConfiguration needs a getter method to
expose the hint to the (jcr) implementation

 Regards

Carsten

-- 
Carsten Ziegeler
Adobe Research Switzerland
cziege...@apache.org

Re: What all changes does ResourceChangeListener are interested in

[Chetan Mehrotra]

On Mon, Oct 17, 2016 at 11:04 AM, Carsten Ziegeler  wrote:
>>
>>> I would rather go with a "property names filter hint" - and if we detect
>> other use cases we add another property for that.
>>
>> Makes sense. This would keep things simple and precise
>
> I plan to release API early this week :) So it would be great if we
> could get this in, or we move it to the next api release.

Would ObserverConfiguration be the place to expose this hint part?

Chetan Mehrotra

Re: What all changes does ResourceChangeListener are interested in

[Carsten Ziegeler]

Chetan Mehrotra wrote
> Oak would be collecting the name of the properties (OAK-4907) which
> got changed in a given commit and that can be used for filtering. This
> I think should help the MapEntries case and probably the resourceType
> case also. They can just hint for those property names and then
> listener would only be called if there is some change in those
> properties
> 
>> I would rather go with a "property names filter hint" - and if we detect
> other use cases we add another property for that.
> 
> Makes sense. This would keep things simple and precise

I plan to release API early this week :) So it would be great if we
could get this in, or we move it to the next api release.

> 
>> Yes, and as mentioned a check based on a resource type is not that easy,
> 
> Can you provide some details on what kind of change the impl is
> interested in. Is it change in the value of resourceType property? Any
> pseudo code of the kind of logic that listener would be performing.
> That would help in determining if we can make this case faster
> 

The code is usually interested in the change of a resource of a specific
type,
for example, if a page is added, removed or changed. This does not mean
that the sling:resourceType property changed, but something else on that
node. In addition that node could have a resourceType property set to
myapp:MyPage which inherits through five levels from sling:Page (let's
assume it would exist) and the listener would be registered for being
interested in sling:Page.
The logic to calculate the resource type hierarchy is quiet complex and
requires access to a resource resolver to do so as potentially several
resources in the resource tree must be traversed to get the result.

Carsten

 

-- 
Carsten Ziegeler
Adobe Research Switzerland
cziege...@apache.org

Re: What all changes does ResourceChangeListener are interested in

[Chetan Mehrotra]

Oak would be collecting the name of the properties (OAK-4907) which
got changed in a given commit and that can be used for filtering. This
I think should help the MapEntries case and probably the resourceType
case also. They can just hint for those property names and then
listener would only be called if there is some change in those
properties

> I would rather go with a "property names filter hint" - and if we detect
other use cases we add another property for that.

Makes sense. This would keep things simple and precise

> Yes, and as mentioned a check based on a resource type is not that easy,

Can you provide some details on what kind of change the impl is
interested in. Is it change in the value of resourceType property? Any
pseudo code of the kind of logic that listener would be performing.
That would help in determining if we can make this case faster

Chetan Mehrotra


On Sat, Oct 15, 2016 at 7:51 PM, Carsten Ziegeler  wrote:
> Dominik Süß wrote
>> For me one of the most interesting filters for "native" filtering would be
>> based on ResourceType (& RT changes). Optimally we would even have the
>> option to filter by ResourceType and its SubTypes - but detecting the
>> subtypes is something that would go beyond a hint (basically preprocessing
>> a hint to "unfold" it into all of the subtypes to watch out for).
>>
>> I guess within the ResourceTypes the best practice would go into the
>> direction of SubTyping whenever the group covered is too generic and needs
>> to be treated differently.
>>
> Yes, and as mentioned a check based on a resource type is not that easy,
> you need to check the resource type property, the super type property,
> maybe the jcr primary node type and then also walk up the resource type
> hierarchy.
> If Oak would provide a hook for registering these additional checks then
> we "maybe" could do it.
>
> Carsten
>
>
>
> --
> Carsten Ziegeler
> Adobe Research Switzerland
> cziege...@apache.org
>

Re: What all changes does ResourceChangeListener are interested in

[Carsten Ziegeler]

Dominik Süß wrote
> For me one of the most interesting filters for "native" filtering would be
> based on ResourceType (& RT changes). Optimally we would even have the
> option to filter by ResourceType and its SubTypes - but detecting the
> subtypes is something that would go beyond a hint (basically preprocessing
> a hint to "unfold" it into all of the subtypes to watch out for).
> 
> I guess within the ResourceTypes the best practice would go into the
> direction of SubTyping whenever the group covered is too generic and needs
> to be treated differently.
> 
Yes, and as mentioned a check based on a resource type is not that easy,
you need to check the resource type property, the super type property,
maybe the jcr primary node type and then also walk up the resource type
hierarchy.
If Oak would provide a hook for registering these additional checks then
we "maybe" could do it.

Carsten

 

-- 
Carsten Ziegeler
Adobe Research Switzerland
cziege...@apache.org

Re: What all changes does ResourceChangeListener are interested in

[Dominik Süß]

For me one of the most interesting filters for "native" filtering would be
based on ResourceType (& RT changes). Optimally we would even have the
option to filter by ResourceType and its SubTypes - but detecting the
subtypes is something that would go beyond a hint (basically preprocessing
a hint to "unfold" it into all of the subtypes to watch out for).

I guess within the ResourceTypes the best practice would go into the
direction of SubTyping whenever the group covered is too generic and needs
to be treated differently.

Cheers
Dominik


On Sat, Oct 15, 2016 at 8:54 AM, Carsten Ziegeler 
wrote:

> Chetan Mehrotra wrote
> > On Thu, Oct 13, 2016 at 6:18 PM, Carsten Ziegeler 
> wrote:
> >> The listeners we have are really just interested in paths
> >
> > Path is one aspect. There must be other aspect like change in specific
> > property name like sling:alias etc.
> >
>
> I think the most expensive one at the moment is the MapEntries listener
> in resource resolver which checks for vanity path and alias properties.
> So this is listening on add/remove/change of those properties.
>
> The other use case is where a listener is interested in a resource type
> - but that can't be done on Oak level as you would need to know the
> resource type hierarchy and other logic to detect the resource type.
>
> So, if we want to do this, we might add a hint for property names.
> I wouldn't go with a general filter string, like LDAP filter - as this
> creates problems on both sides: these filters are not rocket science but
> it's a little bit hard to write them, especially if they get long. And a
> provider implementation needs to parse and handle them.
> I would rather go with a "property names filter hint" - and if we detect
> other use cases we add another property for that.
>
>  Carsten
>
> --
> Carsten Ziegeler
> Adobe Research Switzerland
> cziege...@apache.org
>
>

Re: What all changes does ResourceChangeListener are interested in

[Carsten Ziegeler]

Chetan Mehrotra wrote
> On Thu, Oct 13, 2016 at 6:18 PM, Carsten Ziegeler  
> wrote:
>> The listeners we have are really just interested in paths
> 
> Path is one aspect. There must be other aspect like change in specific
> property name like sling:alias etc.
> 

I think the most expensive one at the moment is the MapEntries listener
in resource resolver which checks for vanity path and alias properties.
So this is listening on add/remove/change of those properties.

The other use case is where a listener is interested in a resource type
- but that can't be done on Oak level as you would need to know the
resource type hierarchy and other logic to detect the resource type.

So, if we want to do this, we might add a hint for property names.
I wouldn't go with a general filter string, like LDAP filter - as this
creates problems on both sides: these filters are not rocket science but
it's a little bit hard to write them, especially if they get long. And a
provider implementation needs to parse and handle them.
I would rather go with a "property names filter hint" - and if we detect
other use cases we add another property for that.

 Carsten

-- 
Carsten Ziegeler
Adobe Research Switzerland
cziege...@apache.org

Re: What all changes does ResourceChangeListener are interested in

[Chetan Mehrotra]

On Thu, Oct 13, 2016 at 6:18 PM, Carsten Ziegeler  wrote:
> The listeners we have are really just interested in paths

Path is one aspect. There must be other aspect like change in specific
property name like sling:alias etc.

Chetan Mehrotra

Re: What all changes does ResourceChangeListener are interested in

[Carsten Ziegeler]

Chetan Mehrotra wrote
> Sure. So can you provide some details on what all the listeners are
> interested in.

The listeners we have are really just interested in paths, in Sling so
far we don't have any special needs I think :)

Carsten

> 
> We can then decide what type of filtering we should expose. At
> simplest level it can be just a ldap filter like string as used in
> service filtering.
> Chetan Mehrotra
> 

 

-- 
Carsten Ziegeler
Adobe Research Switzerland
cziege...@apache.org

Re: What all changes does ResourceChangeListener are interested in

[Chetan Mehrotra]

Sure. So can you provide some details on what all the listeners are
interested in.

We can then decide what type of filtering we should expose. At
simplest level it can be just a ldap filter like string as used in
service filtering.
Chetan Mehrotra


On Thu, Oct 13, 2016 at 5:01 PM, Carsten Ziegeler  wrote:
> Chetan Mehrotra wrote
>> On Thu, Oct 13, 2016 at 2:53 PM, Carsten Ziegeler  
>> wrote:
>>> For now, we limit the functionality as we have to support every feature
>>> across all resource providers.
>>
>> Okie. But may be we can add support for filtering hint. If the
>> underlying implementation can optimize for that it would be useful for
>> such deployments. If not then underlying implementation can resort to
>> hard work.
>>
>> For e.g. some impls cannot do sort of query result but that does not
>> mean to not have sort support. Just that those implementations would
>> do in memory sorting. Not performant but works. Similar can be the
>> case for filtering criteria
>>
> Ok, if we go with a hint, we still should not go crazy and add too many
> options.
> Let's just focus on things that are of use today.
>
> We'll probably soon release the API, so if we want to have something
> like that it would be great if someone can come up with a proposal :)
>
>  Carsten
>
> --
> Carsten Ziegeler
> Adobe Research Switzerland
> cziege...@apache.org
>

Re: What all changes does ResourceChangeListener are interested in

[Carsten Ziegeler]

Chetan Mehrotra wrote
> On Thu, Oct 13, 2016 at 2:53 PM, Carsten Ziegeler  
> wrote:
>> For now, we limit the functionality as we have to support every feature
>> across all resource providers.
> 
> Okie. But may be we can add support for filtering hint. If the
> underlying implementation can optimize for that it would be useful for
> such deployments. If not then underlying implementation can resort to
> hard work.
> 
> For e.g. some impls cannot do sort of query result but that does not
> mean to not have sort support. Just that those implementations would
> do in memory sorting. Not performant but works. Similar can be the
> case for filtering criteria
> 
Ok, if we go with a hint, we still should not go crazy and add too many
options.
Let's just focus on things that are of use today.

We'll probably soon release the API, so if we want to have something
like that it would be great if someone can come up with a proposal :)

 Carsten

-- 
Carsten Ziegeler
Adobe Research Switzerland
cziege...@apache.org

Re: What all changes does ResourceChangeListener are interested in

[Chetan Mehrotra]

On Thu, Oct 13, 2016 at 2:53 PM, Carsten Ziegeler  wrote:
> For now, we limit the functionality as we have to support every feature
> across all resource providers.

Okie. But may be we can add support for filtering hint. If the
underlying implementation can optimize for that it would be useful for
such deployments. If not then underlying implementation can resort to
hard work.

For e.g. some impls cannot do sort of query result but that does not
mean to not have sort support. Just that those implementations would
do in memory sorting. Not performant but works. Similar can be the
case for filtering criteria

Chetan Mehrotra

Re: What all changes does ResourceChangeListener are interested in

[Carsten Ziegeler]

Chetan Mehrotra wrote
> Hi Team,
> 
> As part of OAK-4796 the observation handling is being improved. Focus
> here is to make use of pre filtering to avoid expensive diff operation
> depending on what all changes the listener is interested in.
> 
> So if listener can tell Oak precisely what all changes it is
> interested in (filtering criteria) then Oak can determine wether given
> change is relevant for the listener or not.
> 
> With OAK-4907 following filter criteria can be supported. Note some of
> them are yet to be exposed via Jackrabbit API [1]
> 
> - Name of properties which got changed
> - Node names which got changed
> - NodeType of nodes which got changed. This includes the nodetype of
> ancestor nodes also
> - Set of paths which got changed
> 
> Given in Sling we recently modified quite a few EventListener to
> ResourceChangeListener API can you provide some inputs on what all
> those listeners were interested in and can above set of filtering
> criteria cover them.
> 
> Basically looking for observation pattern
> 
> - What is the resource change listener interested in (path/property or
> anything else)
> - Is it interested only in cluster local changes or its interested in
> all changed
> 

For now, we limit the functionality as we have to support every feature
across all resource providers.

So resource change listeners can filter on the type of the change
(add/remove/change of a resource/node), path, and path with glob patterns.

And that's all for now.


Carsten

-- 
Carsten Ziegeler
Adobe Research Switzerland
cziege...@apache.org

What all changes does ResourceChangeListener are interested in

[Chetan Mehrotra]

Hi Team,

As part of OAK-4796 the observation handling is being improved. Focus
here is to make use of pre filtering to avoid expensive diff operation
depending on what all changes the listener is interested in.

So if listener can tell Oak precisely what all changes it is
interested in (filtering criteria) then Oak can determine wether given
change is relevant for the listener or not.

With OAK-4907 following filter criteria can be supported. Note some of
them are yet to be exposed via Jackrabbit API [1]

- Name of properties which got changed
- Node names which got changed
- NodeType of nodes which got changed. This includes the nodetype of
ancestor nodes also
- Set of paths which got changed

Given in Sling we recently modified quite a few EventListener to
ResourceChangeListener API can you provide some inputs on what all
those listeners were interested in and can above set of filtering
criteria cover them.

Basically looking for observation pattern

- What is the resource change listener interested in (path/property or
anything else)
- Is it interested only in cluster local changes or its interested in
all changed


Chetan Mehrotra
[1] 
https://docs.adobe.com/docs/en/aem/6-0/develop/ref/javadoc/org/apache/jackrabbit/api/observation/JackrabbitEventFilter.html