Re: Scalabilty of EventListener

2013-10-28 Thread Alexander Klimetschek
On 26.10.2013, at 09:49, Justin Edelson jus...@justinedelson.com wrote: Why does the workflow engine have to make that decision itself? Can't that decision be expressed declaratively? They are, after all, being defined declaratively by the administrator, so it should just be a matter of

Re: Scalabilty of EventListener

2013-10-26 Thread Justin Edelson
Hi, On Fri, Oct 25, 2013 at 4:54 PM, Alexander Klimetschek aklim...@adobe.com wrote: On 24.10.2013, at 16:55, Justin Edelson jus...@justinedelson.com wrote: Given that there's already an expression syntax defined in the EventAdmin specification, wouldn't it make more sense for Oak to just

Re: Scalabilty of EventListener

2013-10-25 Thread Alexander Klimetschek
On 24.10.2013, at 16:55, Justin Edelson jus...@justinedelson.com wrote: Given that there's already an expression syntax defined in the EventAdmin specification, wouldn't it make more sense for Oak to just use OSGi Events? That's the only way I personally could forsee us being in a position to

Re: Scalabilty of EventListener

2013-10-24 Thread Carsten Ziegeler
As stated, so far the whole discussion is very theoretical - apart from the point that right now the jcr listener does a read for each and every changed node to get the resource type. This could definitely be improved with Oak as Oak is doing a content diff and therefore it should be possible to

Re: Scalabilty of EventListener

2013-10-24 Thread Alexander Klimetschek
I don't think it's so theoretical: a generic observation listener such as the jcr listener in sling, that handles all events all the time with no option to reduce the scope in any way, seems like a barrier to scalability. The goal should be that any kind of observation should always be as

Re: Scalabilty of EventListener

2013-10-24 Thread Justin Edelson
Hi Alex, Given that there's already an expression syntax defined in the EventAdmin specification, wouldn't it make more sense for Oak to just use OSGi Events? That's the only way I personally could forsee us being in a position to deprecate/remote the JCR Observation/OSGi EventAdmin bridge which

Re: Scalabilty of EventListener

2013-10-23 Thread Dominik Süß
I just realized that there might be the need to dynamicaly add paths to the Listener. I have at least 2 cases in mind where the paths cannot be determined at deployment time which is the workflow launcher in CQ (where a user does define a ruleset at runtime) which might be changed to individual

Re: Scalabilty of EventListener

2013-10-23 Thread Bertrand Delacretaz
Hi, On Wed, Oct 23, 2013 at 11:30 AM, Dominik Süß dominik.su...@gmail.com wrote: ...I have at least 2 cases in mind where the paths cannot be determined at deployment time... We studied observation usage patterns [1] earlier this year on a few Sling-based apps and it turns out that a lot of

Re: Scalabilty of EventListener

2013-10-23 Thread Bertrand Delacretaz
Hi, On Tue, Oct 22, 2013 at 5:48 PM, Carsten Ziegeler cziege...@apache.org wrote: ...The mapping handler in the resource resolver is probably the most interesting one as it changes for nodes with some well defined properties, basically scanning the whole repository... This is one example

Re: Scalabilty of EventListener

2013-10-23 Thread Carsten Ziegeler
So you mean instead of doing observation, doing a query periodically? This would mean that we basically say, one of the main features of JCR, observation, is not usable. Carsten 2013/10/23 Bertrand Delacretaz bdelacre...@apache.org Hi, On Tue, Oct 22, 2013 at 5:48 PM, Carsten Ziegeler

Re: Scalabilty of EventListener

2013-10-23 Thread Ian Boston
On 23 October 2013 11:25, Carsten Ziegeler cziege...@apache.org wrote: So you mean instead of doing observation, doing a query periodically? This would mean that we basically say, one of the main features of JCR, observation, is not usable. Past experience says that global observation of all

Re: Scalabilty of EventListener

2013-10-23 Thread Carsten Ziegeler
I get this with global observation, but we're now talking about observation usage patterns and replacing them with queries. And these usage patterns are usually only observing partial parts of the repository. Carsten 2013/10/23 Ian Boston i...@tfd.co.uk On 23 October 2013 11:25, Carsten

Re: Scalabilty of EventListener

2013-10-23 Thread Bertrand Delacretaz
On Wed, Oct 23, 2013 at 1:37 PM, Carsten Ziegeler cziege...@apache.org wrote: ...I get this with global observation, but we're now talking about observation usage patterns and replacing them with queries. And these usage patterns are usually only observing partial parts of the repository

Re: Scalabilty of EventListener

2013-10-23 Thread Carsten Ziegeler
Ok, so I suggest we identify these parts then first concretely before we speculate that there is a problem and replace it with something where we don't know the real impact. So far, I heard that the global listener which translates jcr events into OSGi events is a problem - however as far as I

Re: Scalabilty of EventListener

2013-10-23 Thread Bertrand Delacretaz
On Wed, Oct 23, 2013 at 2:10 PM, Carsten Ziegeler cziege...@apache.org wrote: So far, I heard that the global listener which translates jcr events into OSGi events is a problem - however as far as I followed the discussion, this isn't a problem right now but might lead to problems in some

Re: Scalabilty of EventListener

2013-10-23 Thread Dominik Süß
The analysis looks pretty good to me but does not provide answers of how to solve this. The first two topics Cached Content and Content Export, Replication to Remote Systems are the ones where I don't see an option to get rid of the content change triggers. This might not apply to the whole tree,

Re: Scalabilty of EventListener

2013-10-23 Thread Bertrand Delacretaz
Hi, On Wed, Oct 23, 2013 at 4:01 PM, Dominik Süß dominik.su...@gmail.com wrote: ...This might not apply to the whole tree, but does this really matter when the tree that needs to be watched contains over 90% of the data?... What's important is the frequency of observation events - if that 90%

Re: Scalabilty of EventListener

2013-10-23 Thread Dominik Süß
Hi Bertrand, I'm not so sure about the seldom changes - it is not about the amound of changes but about the amount of changes in a specific timeframe. The initial discussion started with the asumtion to have to deal with tons of changes within a second, e.g. for huge imports. In some cases those

Re: Scalabilty of EventListener

2013-10-23 Thread Bertrand Delacretaz
On Wed, Oct 23, 2013 at 4:42 PM, Dominik Süß dominik.su...@gmail.com wrote: ...Regarding the options you were stating: - service properties: my initial thought, but static and can therefore not adress all scenarios... Service properties for a given service are static, but a service that has

Scalabilty of EventListener

2013-10-22 Thread Dominik Süß
In a discussion [0] within the Oak mailinglist it became clear that the way Sling listens zu JCR Repository Changes and transforms all of them to events will not scale well in some big scale scenarios that oak is aiming to enable. Therefore the question was posted if it would be feasible and/or

Re: Scalabilty of EventListener

2013-10-22 Thread Carsten Ziegeler
From the use cases I know, the listeners register for a specific path and are rarely interested in anything else. Some do check the resource type as well. For example the script engines check for changes under /libs and /apps (the resource paths actually) for changes to scripts, the job engine