Interesting analysis... Thanks for sharing this!
Regards, Pierre-Arnaud On 3 juil. 2011, at 22:20, Emmanuel Lecharny wrote: > Hi guys, > > I had time yesterday and today to think a bit more about the reverse table we > use in index. I thought we were able to remove them and saving the time it > costs to generate them. > > However, one day and a half without coding make me realize that those reverse > table are a critical part of an efficient evaluation engine. I initially > thought that using some Sets computation could allow us to remove those > table, and I was right, put to the point we don't have to manipulate big sets > of IDs. For instance, with a huge database with millions of entries, a > OneLevel or SubLevel index will potentially contains millions of IDs > associated with one ID. In this case, doing an intersection operation on this > huge set will simply be overkilling in term of memory (as we wlll have to > keep the resulting set in memory). > > OTOH, the reverse table on a OneLevel or SubLevel index will only contain a > few IDs associated with one entry, unless the tree is extremely deep. > > This is an issue not only for the OneLevel or SubLevel index, but also for > the ObjectClass index. > > As of now, I don't really think that continuing any further is worthily. The > best is probably to keep going with trunk, and eventually re-inject the few > fixes I have applied i my experimental branch. > > In any case, this was an interesting exercise : it helped me to get deep into > this part of the code I was not familiar with, and at the end, validate the > initial approach. > > -- > Regards, > Cordialement, > Emmanuel Lécharny > www.iktek.com >
