Hi Emmanuel, Interesting idea!
A great combination of both original solutions into a single one... I like the idea of lazy evaluation and the fact that the administrator can, if he wants it, go back to solution #1 by simply searching (objectClass=*) under the newly created/updated AP after it has been added/modified (It can be an option of a dedicated tool to manage the APs in Studio). The best of both worlds... +1 Regards, Pierre-Arnaud On 13 déc. 2010, at 11:41, Emmanuel Lecharny wrote: > Hi guys, > > I'm resuming the APs implementation. I have had a new idea this morning about > the best way to handle them. > > Currently, we had devised about too possible options > > 1) Processing the entries while adding a new AP > The idea is that when we add a new AP (or delete an existing one, or > modifying an existing one), we evaluate all the underlying entries and update > each one which is selected by the associated SubtreeSpecification by adding a > reference to its associated AP in the entry. > This is done once, then when a user grab an entry, we don't need to evaluate > it again, as this work has already been done, so we accept to pay the price > once, but it's free for every operation after this initial update. > > The price to pay is pretty expensive, if we consider a huge base with hundred > of thousands entries, as we have to update many, and as an update is slow (we > currently can process 600 per second of such updates - depending on the > server we are running on, of course -, so for 1 000 000 entries it might take > up to 30 minutes ). > > 2) Processing the evaluation on the fly > The second idea is to evaluate the entry when a user requests it, to avoid > paying the price for the initial update. > > The big problem with this approach is that if we have many APs > SubtreeSpecifications to evaluate, this evaluation is not free, plus it has > to occur *everytime* a user access the entry. That could slow down the server > a lot (probably double the time it takes to return an entry). > > So far, we have decider to go for option #1, simply because adding an AP is > an administrative task, which does not occur very often. We all realize that > it has drawbacks, but it's far better than option #2 anyway. > > Now, I'd like to propose a third option, which might be acceptable, > considering that we will pay a small price compared to both option #1 and #2. > > 3) Processing on the fly and store the result > The idea is to process the evaluation when someone fetches an entry, but to > store the evaluation result in the entry. If the fetched entry has already > been evaluated, then no need to process it again. > > The direct benefit will be that we won't have this huge initial update > required by option #1, and the entry evaluation wil be just a matter to check > if an AT has been set instead of fully evalute the entry as in option #2. > > How will it works ? Simple : when an AP is added, a Timestamp (TS) is > updated. This TS will be incremented every time we add a new AP. We don't > update any entry. > Then when a user fetch some entries, for every one of the selected entries, > we check if it has a TS AT present. If not, then we evaluate the entry > against the AP it depends upon, and when done, we add the last TS into the > entry, save it and return (or not) the entry to the user. > If the entry has a TS AT, then we have two cases > - The TS equals the last TS : the entry has already been evaluated, and we > can return it (or not, depending on the evaluation result) > - The TS is below, so we reevaluate the entry. > > Now, how will we distinguish between an entry evaluating to true or false ? > Easy : if the entry is selected by a SubtreeSpecification, we will have a ref > to the associated AP in it, otherwise no. > > I do think that this third option has great advantages, and not only from the > user or admin POV : the implementation will be greatly improved, as we don't > have to deal with all the cases when handling an AP > addition/modification/deletion. > > One last thing : one may object that the on the fly evaluation is not > acceptable, as it will cost a lot for the first access to an entry. True. But > I claim that it's the administrator to deal with this problem, and then its > not anymore a user issue. When the admin add an new AP, he also can fetch all > the entries below the added AP, and they will be updated on the fly. The > users won't be impacted anymore. > > thoughts ? > > -- > Regards, > Cordialement, > Emmanuel Lécharny > www.iktek.com >
