Re: Oak JCR Observation scalability aspects and concerns

2013-10-22 Thread Thomas Mueller
Hi, I think it's quite clear that global observation listeners are not scalable. (Observation listeners that listen for all events below root, from all cluster nodes.) It is needed for backward compatibility, but we need to find a solution to make this obsolete. It's not enough to just reduce the

jackrabbit-oak build #2425: Fixed

2013-10-22 Thread Travis CI
Build Update for apache/jackrabbit-oak - Build: #2425 Status: Fixed Duration: 1559 seconds Commit: c9710c3482c997cdf49e6903e781273e75f9 (trunk) Author: Thomas Mueller Message: OAK-825 Unnecessary performance overhead (too many child nodes are traversed;

Re: Oak JCR Observation scalability aspects and concerns

2013-10-22 Thread Chetan Mehrotra
On Mon, Oct 21, 2013 at 6:47 PM, Jukka Zitting jukka.zitt...@gmail.com wrote: -1 This introduces the problem where a single JCR event listener can block or slow down all other listeners. That can be mitigated upto an extent by using some sort of Black List (OAK-1084). However current approach

Re: Oak JCR Observation scalability aspects and concerns

2013-10-22 Thread Chetan Mehrotra
On Mon, Oct 21, 2013 at 11:39 PM, Jukka Zitting jukka.zitt...@gmail.com wrote: 3) The Observer mechanism allows a listener to look at repository changes in variable granularity and frequency depending on application needs and current repository load. Thus an Oak Observer can potentially

Re: Oak JCR Observation scalability aspects and concerns

2013-10-22 Thread Felix Meschberger
Hi Am 22.10.2013 um 11:17 schrieb Chetan Mehrotra: On Mon, Oct 21, 2013 at 11:39 PM, Jukka Zitting jukka.zitt...@gmail.com wrote: 3) The Observer mechanism allows a listener to look at repository changes in variable granularity and frequency depending on application needs and current

Re: Oak JCR Observation scalability aspects and concerns

2013-10-22 Thread Angela Schreiber
hi felix from what i see Sling heavily relies on jackrabbit-core functionality... i would be very pleased if it would just rely on public API such as JCR, Jackrabbit API and in the future OAK; it doesn't and this is causing a lot of troubles. kind regards angela On 10/22/13 11:21 AM, Felix

Re: Enable MongoDB on travis-ci again

2013-10-22 Thread Jukka Zitting
Hi, On Tue, Oct 22, 2013 at 4:08 AM, Marcel Reutegger mreut...@adobe.com wrote: Any objections? None, +1! BR, Jukka Zitting

Re: Oak JCR Observation scalability aspects and concerns

2013-10-22 Thread Jukka Zitting
Hi, On Tue, Oct 22, 2013 at 5:21 AM, Felix Meschberger fmesc...@adobe.com wrote: Am 22.10.2013 um 11:17 schrieb Chetan Mehrotra: I think in Sling case it would make sense for it to be implemented as an Observer. And I had a look at implementation of some of the listener implementations of [1]

Re: Oak JCR Observation scalability aspects and concerns

2013-10-22 Thread Felix Meschberger
Hi Am 22.10.2013 um 15:27 schrieb Jukka Zitting: Hi, On Tue, Oct 22, 2013 at 5:21 AM, Felix Meschberger fmesc...@adobe.com wrote: Am 22.10.2013 um 11:17 schrieb Chetan Mehrotra: I think in Sling case it would make sense for it to be implemented as an Observer. And I had a look at

Re: Oak JCR Observation scalability aspects and concerns

2013-10-22 Thread Carsten Ziegeler
Sling's jcr listener provides an API/contract - so this is an infrastructure component used by application code (or other infrastructure code). And as the listener is delegating the promotion of events to the EventAdmin, the jcr listener does not know if there are event listeners at all or what

Re: Oak JCR Observation scalability aspects and concerns

2013-10-22 Thread Felix Meschberger
Hi Am 22.10.2013 um 15:52 schrieb Jukka Zitting: Hi, On Tue, Oct 22, 2013 at 9:41 AM, Felix Meschberger fmesc...@adobe.com wrote: The JcrResourceListener just gets JCR Observation events, creates the OSGi Event objects and hands them over for distribution by the OSGi EventAdmin service.

Re: Oak JCR Observation scalability aspects and concerns

2013-10-22 Thread Jukka Zitting
Hi, On Tue, Oct 22, 2013 at 9:43 AM, Carsten Ziegeler cziege...@apache.org wrote: This is the contract we have to maintain in Sling. I repeat from my earlier post: Right, it just means that a deployment with such an observer will have a built-in scalability limit as at some point the

Re: Oak JCR Observation scalability aspects and concerns

2013-10-22 Thread Thomas Mueller
Hi, That's one Event object per event -- not one event per listener per event. This is completely different to JCR. Well, this still doesn't scale, if every cluster node needs all events. It doesn't matter how many observation listeners you have. To block scalability, it's enough to have one

Re: Oak JCR Observation scalability aspects and concerns

2013-10-22 Thread Thomas Mueller
Hi, This is the contract we have to maintain in Sling. Well, we can't maintain this contract, because it blocks scalability. Regards, Thomas

Re: Oak JCR Observation scalability aspects and concerns

2013-10-22 Thread Jukka Zitting
Hi, On Tue, Oct 22, 2013 at 9:59 AM, Felix Meschberger fmesc...@adobe.com wrote: That's one Event object per event -- not one event per listener per event. This is completely different to JCR. You're mistaking the problem here, it's not the number of listeners, it's the number of events *per

Re: Oak JCR Observation scalability aspects and concerns

2013-10-22 Thread Bertrand Delacretaz
On Tue, Oct 22, 2013 at 4:19 PM, Thomas Mueller muel...@adobe.com wrote: This is the contract we have to maintain in Sling. Well, we can't maintain this contract, because it blocks scalability I tend to agree, and OTOH not everybody will need the kind of scalability that we're discussing

Re: Oak JCR Observation scalability aspects and concerns

2013-10-22 Thread Carsten Ziegeler
I really would like to have a constructive discussion here. I think the Sling use case is pretty well explained now - that's an api Sling offers and which is used by a lot of code out there (a great part of Sling is based on the OSGi events and layers on top of Sling are using it as well). That's

Re: Oak JCR Observation scalability aspects and concerns

2013-10-22 Thread Dominik Süß
Hi :) Speaking as developer using the Sling eventing I just wanted to add that in most cases there are restrictions on Paths (most times not just one but multiple searchpaths) and on a resourceType (not just exact match but a set or pattern to identify a set of resourceTypes) and in some

Re: Oak JCR Observation scalability aspects and concerns

2013-10-22 Thread Bertrand Delacretaz
On Tue, Oct 22, 2013 at 4:30 PM, Carsten Ziegeler cziege...@apache.org wrote: ...4. same as 3. but keep the old Sling API with a bold marker when it's used that this does not scale... That's my favorite choice - along with providing a way for users of those Sling events to specify more

Re: Oak JCR Observation scalability aspects and concerns

2013-10-22 Thread Dominik Süß
+1 on 4 since I fear 3 will create some overhead for existing solutions that won't need this kind of scalabilty (and therefore create uncessary efforts for migration). This is the old compat pattern seen so often. IMHO this should be an extension that can be installed but is not available by

Re: Oak JCR Observation scalability aspects and concerns

2013-10-22 Thread Carsten Ziegeler
Just to reiterate :) if we go with 3 or 4, someone has to do the work in Sling (and other places) and adapt the code. As obviously as soon as a single listener is using the old pattern, the whole mechanism is mood. Carsten 2013/10/22 Dominik Süß dominik.su...@gmail.com +1 on 4 since I fear 3

jackrabbit-oak build #2429: Broken

2013-10-22 Thread Travis CI
Build Update for apache/jackrabbit-oak - Build: #2429 Status: Broken Duration: 1269 seconds Commit: 5b0260a4ca87418e0630a816b21b3cdeeb8eec15 (trunk) Author: Marcel Reutegger Message: Re-enable MongoDB for tests git-svn-id:

Re: Oak JCR Observation scalability aspects and concerns

2013-10-22 Thread Thomas Mueller
Hi, I really would like to have a constructive discussion here. Sure. If we want a fully scalable solution, Sling needs to be changed. 1. we leave everything as is 2. we maintain the API as-is in Sling and try to make the implementation as fast as possible This will limit scalability so I

RE: jackrabbit-oak build #2429: Broken

2013-10-22 Thread Marcel Reutegger
hmm, I didn't see this when I ran the TCK tests on my machine, but the test sequence is different on my windows machine. I will disable mongodb again if this is a reoccurring issue... regards marcel -Original Message- From: Alex Parvulescu [mailto:alex.parvule...@gmail.com] Sent:

Re: Oak JCR Observation scalability aspects and concerns

2013-10-22 Thread Dominik Süß
I just opened a thread at sling-dev for further discussion about api and implementation changes on sling side [0] For discussions around usage of this api within sling please use this linked thread [0]. Best regards Dominik [0] markmail.org/thread/plb7ledhsna33r3g On Tue, Oct 22, 2013 at

Improving the benchmark suite

2013-10-22 Thread Tobias Bocanegra
Hi, I'd like to make the following changes to the way how the benchmarks work: 1. add support for executing several benchmarks within the same suite. currently each benchmark has it's own 'setUp()' code that might be expensive to execute. e.g. import a large structure, create nodes, etc. 2.

jackrabbit-oak build #2430: Still Failing

2013-10-22 Thread Travis CI
Build Update for apache/jackrabbit-oak - Build: #2430 Status: Still Failing Duration: 1841 seconds Commit: 461d4fdc6986688069eac4d93fa80c09e004a637 (trunk) Author: Angela Schreiber Message: OAK-527: benchmark tests git-svn-id:

Re: Improving the benchmark suite

2013-10-22 Thread Jukka Zitting
Hi, On Tue, Oct 22, 2013 at 1:34 PM, Tobias Bocanegra tri...@apache.org wrote: WDYT? Any improvements are of course welcome. On the other hand I believe we are reaching the limits of what the benchmark suite was originally designed for, i.e. a quick and simple mechanism for running basic

Re: Improving the benchmark suite

2013-10-22 Thread Tobias Bocanegra
On Tue, Oct 22, 2013 at 11:39 AM, Jukka Zitting jukka.zitt...@gmail.com wrote: Hi, On Tue, Oct 22, 2013 at 1:34 PM, Tobias Bocanegra tri...@apache.org wrote: WDYT? Any improvements are of course welcome. On the other hand I believe we are reaching the limits of what the benchmark suite

jackrabbit-oak build #2431: Still Failing

2013-10-22 Thread Travis CI
Build Update for apache/jackrabbit-oak - Build: #2431 Status: Still Failing Duration: 1329 seconds Commit: 2bd7b1443cbb93043493b44a1f99a0f43b6888d8 (trunk) Author: Tobias Bocanegra Message: OAK-527 Implement Permission evaluation - calculate number of