Thank you Howard (and openldap core team) for this synthesis of the
current and future state of dynamic membership devs.
Indeed, some of my customers were suffering from dynlist performances,
and the decision was taken to go back to more static features (like
memberOf).
For now I did not test nestgroup overlay, but I'll give a look for next
installations. Do you have any idea in which version of OpenLDAP
nestgroup will be officially released?
Regards,
Le 26/02/2024 à 16:54, Howard Chu a écrit :
The recent work on expanding dynamic group functionality in the dynlist overlay
seems to have been
a bad idea. It makes an already fairly complex overlay even more complicated,
and it puts a lot more
work into the read side of operations, which adds up to quite noticeable
slowdowns in search performance
on deployments that make heavy use of dynamic groups with large memberships.
It appears that in these situations, the approach used in the autogroup overlay
(which has been
in contrib since 2007) is better. It moves all of the membership management
into the write side
of operations, updating memberships whenever dynamic group definitions are
modified, or when
member entries are written/updated/etc. As such, it allows dynamically defined
group memberships
to be read/searched at full speed, as if they were static groups. The search
performance difference
between autogroup and the dynlist approach is pretty drastic when large groups
and large numbers
of groups are in use. As such I believe autogroup should be recommended, going
forward, and we
should move it from contrib into the core code.
For similar reasons, dynamically managing memberOf attributes in dynlist also
has a major
impact on search performance. And as alluded to before, adding that
functionality to dynlist has
made the code quite a bit more complex. Again for performance reasons, it's
better to manage
this attribute on the write side, updating a real attribute in the DB when
groups and memberships
are modified, instead of doing the lookup work on the read/search side. As
such, we should
reverse the decision to deprecate the memberof overlay. There were a couple
problems that the
overlay presented in replication environments before, that prompted us to
deprecate it, but I
believe those problems have now been resolved. The first one, referenced in
ITS#7400, had to do
with the actual memberof attributes getting replicated even though they weren't
meant to be.
The solution for that should have been to specify that memberof should be
excluded from replication
using "exattr" in a syncrepl consumer config. The exattr option wasn't behaving
as intended in
the past, and using it would cause data desyncs, but that problem was fixed
long ago.
The other problem was simply due to the lack of ordering guarantees in syncrepl
refreshes,
which prevented the memberof overlay from updating an member's memberof
attribute if the group
entry got replicated before the member entry. That problem has been solved in
ITS#10167, simply
adding a check whenever new entries are added, to see if they're already
members of any existing groups.
As such, the memberof overlay should be perfectly fine for use in replication
scenarios now.
More testing of those scenarios is welcome.
With statically managed member and memberof attributes, the major hits to
search performance have
been removed. We're still trying dynamically managed nesting of groups though,
which is what the
new nestgroup overlay (ITS#10161) is for. Again, we need testing to see how
much performance impact
there is from this much-reduced overhead in the read/search side of things. The
config is also a
lot easier to understand in its own overlay, instead of shoe-horned into the
dynlist config.
Testing and feedback appreciated.
--
David Coutadeur | IAM integrator
david.coutad...@worteks.com
+33 7 88 46 85 34
16 avenue Hoche, Paris 75008
Worteks | https://www.worteks.com
