On 2 mai 2012, at 12:55, Kiran Ayyagari wrote: > On Wed, May 2, 2012 at 4:20 PM, Pierre-Arnaud Marcelot <[email protected]> > wrote: >> On 2 mai 2012, at 12:24, Kiran Ayyagari wrote: >>> this method has two extreme cases, >>> 1. best performance when no attributes are requested >>> 2. same overhead as cloning when all attributes are requested >>> >>> cause in real world scenario 1 will never be used, >> >> Actually in Studio, scenario 1 is used a lot. >> That's exactly what happens do when opening a node and browsing the DIT in >> the UI (almost, because we do ask for the objectClass attribute). >> > yeap, I know, but this is a very limited usage, or shall I say just > for tooling (essentially helpful for paging etc, else dOOM is certain > the moment I expand a 100k node tree (provided my limit is set to > > 100k ;))
Yeah, that's for sure. We agree that directories are not meant to be browse by GUI as primary usage… ;) Regards, Pierre-Arnaud >> Regards, >> Pierre-Arnaud >> >>> we might end up >>> copying some attributes anyway >>> but still better than cloning all >>> On Wed, May 2, 2012 at 3:19 PM, Emmanuel Lécharny <[email protected]> >>> wrote: >>>> Le 5/2/12 9:53 AM, Alex Karasulu a écrit : >>>>> >>>>> On Wed, May 2, 2012 at 2:43 AM, Emmanuel >>>>> Lécharny<[email protected]>wrote: >>>>> >>>>>> Le 5/1/12 3:05 PM, Alex Karasulu a écrit : >>>>>> >>>>>>> On Tue, May 1, 2012 at 4:08 AM, Emmanuel Lécharny<[email protected]>** >>>>>>> >>>>>>> wrote: >>>>>>> >>>>>>> o Object scope search (lookup) : 49 880 req/s compared to 23 081 on the >>>>>>> previous trunk >>>>>>> o One Level scope search (5 entries returned) : 68 715 entries returned >>>>>>> per second, compared to 33 120/s >>>>>>> o Sub Level scope search (10 entries returned ) : 70 830 entries >>>>>>> returned >>>>>>> per second, compared to 18 910/s >>>>>>> >>>>>>> >>>>>>> This is great work Emmanuel. Nicely done! >>>>>>> >>>>>> I have some even better results, as of today : >>>>>> >>>>>> o Object scope search (lookup) : 52 712 req/s compared to 23 081 on the >>>>>> previous trunk >>>>>> o One Level scope search (5 entries returned) : 72 635 entries returned >>>>>> per second, compared to 33 120/s >>>>>> o Sub Level scope search (10 entries returned ) : 75 100 entries returned >>>>>> per second, compared to 18 910/s >>>>>> >>>>>> >>>>> This is just sick man! You've more than doubled the performance. >>>> >>>> >>>> Some new idea this morning : >>>> >>>> atm, we do clone the entries we fetch from the server, then we filter the >>>> Attributes and the values, modifying the cloned entries. This leads to >>>> useless create of the removed Attributes and Values. We suggested to >>>> accumulate the modifications and to apply them at the end, avoiding the >>>> cloning of AT which will not be returned. >>>> >>>> First of all, we can avoid cloning the Values. The Value implementation are >>>> immutable classes. This save around 7% of the time. >>>> >>>> But this is not all we can do : we can simply avoid the accumulation of >>>> modifications *and* avoid cloning the entry ! >>>> >>>> The idea is simple : when we get an entry in the cursor we have got back, >>>> we >>>> create a new empty entry, then we iterate over all the original entry's >>>> attributes and values, and for each one of those attributes and values, we >>>> check the filters, which will just tell if the Attribute/Value must be >>>> ditched or kept. This way, we don't do anything useless, like storing the >>>> modification or creating useless Attributes. >>>> >>>> It will work to the extent we deal with the CollectiveAttributes which must >>>> be injected somewhere, before we enter the loop (a connectiveAttribute >>>> might >>>> perfectly be removed by the ACI filter...). But we can also inject those >>>> added collective attributes into the loop of filters. >>>> >>>> I may miss something, but I do think that this solution is a clear winner, >>>> even in term of implementation... >>>> >>>> thoughts ? >>>> >>>>> >>>> >>>> >>>> -- >>>> Regards, >>>> Cordialement, >>>> Emmanuel Lécharny >>>> www.iktek.com >>>> >>> >>> >>> >>> -- >>> Kiran Ayyagari >> > > > > -- > Kiran Ayyagari
