On Wed, May 2, 2012 at 4:20 PM, Pierre-Arnaud Marcelot <[email protected]> wrote: > On 2 mai 2012, at 12:24, Kiran Ayyagari wrote: >> this method has two extreme cases, >> 1. best performance when no attributes are requested >> 2. same overhead as cloning when all attributes are requested >> >> cause in real world scenario 1 will never be used, > > Actually in Studio, scenario 1 is used a lot. > That's exactly what happens do when opening a node and browsing the DIT in > the UI (almost, because we do ask for the objectClass attribute). > yeap, I know, but this is a very limited usage, or shall I say just for tooling (essentially helpful for paging etc, else dOOM is certain the moment I expand a 100k node tree (provided my limit is set to > 100k ;)) > Regards, > Pierre-Arnaud > >> we might end up >> copying some attributes anyway >> but still better than cloning all >> On Wed, May 2, 2012 at 3:19 PM, Emmanuel Lécharny <[email protected]> >> wrote: >>> Le 5/2/12 9:53 AM, Alex Karasulu a écrit : >>>> >>>> On Wed, May 2, 2012 at 2:43 AM, Emmanuel >>>> Lécharny<[email protected]>wrote: >>>> >>>>> Le 5/1/12 3:05 PM, Alex Karasulu a écrit : >>>>> >>>>>> On Tue, May 1, 2012 at 4:08 AM, Emmanuel Lécharny<[email protected]>** >>>>>> >>>>>> wrote: >>>>>> >>>>>> o Object scope search (lookup) : 49 880 req/s compared to 23 081 on the >>>>>> previous trunk >>>>>> o One Level scope search (5 entries returned) : 68 715 entries returned >>>>>> per second, compared to 33 120/s >>>>>> o Sub Level scope search (10 entries returned ) : 70 830 entries >>>>>> returned >>>>>> per second, compared to 18 910/s >>>>>> >>>>>> >>>>>> This is great work Emmanuel. Nicely done! >>>>>> >>>>> I have some even better results, as of today : >>>>> >>>>> o Object scope search (lookup) : 52 712 req/s compared to 23 081 on the >>>>> previous trunk >>>>> o One Level scope search (5 entries returned) : 72 635 entries returned >>>>> per second, compared to 33 120/s >>>>> o Sub Level scope search (10 entries returned ) : 75 100 entries returned >>>>> per second, compared to 18 910/s >>>>> >>>>> >>>> This is just sick man! You've more than doubled the performance. >>> >>> >>> Some new idea this morning : >>> >>> atm, we do clone the entries we fetch from the server, then we filter the >>> Attributes and the values, modifying the cloned entries. This leads to >>> useless create of the removed Attributes and Values. We suggested to >>> accumulate the modifications and to apply them at the end, avoiding the >>> cloning of AT which will not be returned. >>> >>> First of all, we can avoid cloning the Values. The Value implementation are >>> immutable classes. This save around 7% of the time. >>> >>> But this is not all we can do : we can simply avoid the accumulation of >>> modifications *and* avoid cloning the entry ! >>> >>> The idea is simple : when we get an entry in the cursor we have got back, we >>> create a new empty entry, then we iterate over all the original entry's >>> attributes and values, and for each one of those attributes and values, we >>> check the filters, which will just tell if the Attribute/Value must be >>> ditched or kept. This way, we don't do anything useless, like storing the >>> modification or creating useless Attributes. >>> >>> It will work to the extent we deal with the CollectiveAttributes which must >>> be injected somewhere, before we enter the loop (a connectiveAttribute might >>> perfectly be removed by the ACI filter...). But we can also inject those >>> added collective attributes into the loop of filters. >>> >>> I may miss something, but I do think that this solution is a clear winner, >>> even in term of implementation... >>> >>> thoughts ? >>> >>>> >>> >>> >>> -- >>> Regards, >>> Cordialement, >>> Emmanuel Lécharny >>> www.iktek.com >>> >> >> >> >> -- >> Kiran Ayyagari >
-- Kiran Ayyagari
