Hi all,
Splitting ipausers helped ipa user-add speed a lot. Two other things helped:
1) Setting nsslapd-cachememsize much larger than the size of the
id2entry file
2) Increasing the size of the DN cache size, nsslapd-ndn-cache-max-size
I've got 60,000+ users now and user-add only takes slightly over three
seconds.
Because we will continue to add large number of users at the start of
the University's fall term and the new userids will be sequential I
chose to use a prime number, 211, of ipauser groups and hash the
userid. Thus the new ones will spread evenly over the groups and we
should end up with less than 1,000 users per group.
Thanks again for your help,
Daryl
-------- Forwarded Message --------
Subject: Re: [Freeipa-users] ipa user-add slows down as more users are
added
Date: Thu, 5 Nov 2015 16:37:29 -0600
From: Daryl Fonseca-Holt <daryl.fonseca-h...@umanitoba.ca>
To: Rich Megginson <rmegg...@redhat.com>
On 11/04/15 17:25, Rich Megginson wrote:
On 11/04/2015 04:07 PM, Rob Crittenden wrote:
Daryl Fonseca-Holt wrote:
Hi All,
I am testing migration from NIS with a custom MySQL backend to IPA. In
our testing ipa user-add starts out at around 12 seconds per user but
slows down as more users are add. By 5000+ users it is taking 90+
seconds. We have 120,000+ users. I'm looking at 155 days to load all
the
users :(
Per some performance tuning documentation I've increased
nsslapd-cachememsize to 35,651,584 and am currently getting pretty high
hit ratios (see below).
Use dbmon.sh instead of db_stat - it will give you the entry cache
information as well as a summary of the db cache information (instead
of the quite verbose db_stat output).
Thanks for the tip. I've got some tuning to do. My dbcachefree is
negative and the roevicts are high.
However, one thread of ns-slapd pegs out core at
100% and I can't get get it to add users any faster. I'm not seeing any
I/O or memory swapping.
The problem is most likely the default IPA users group. As it gets
humongous adding new members slows it down.
So could he disable the automember and memberof plugins? Then have
those work offline, after the users are added?
Suggestions would be appreciated. Multi-master will probably help but
with that many accounts it would take a lot of masters to get additions
done to a resonable (45 seconds or less?) time. Is there any guideline
for number of users per master?
IPA is multi-master. All users are on all masters.
If anything I'd expect that running imports on different masters would
slow things down as changes on multiple masters would need to get merged
together, particularly the default group.
Right.
Certainly bumping up the caches to match what the final expected sizes
is probably a good idea but I don't see it influencing import speed all
that much.
Right.
One idea I've had is to add the users in batches of 1000. What you'd do
is create 120 non-POSIX user groups, ipausers1..ipausers120, and add
them as members of ipausers.
Then for each batch change the default ipausers group with:
$ ipa config-mod --defaultgroup=<name>
This should keep the user-add command fairly peppy and keep the ipausers
group somewhat in check via the nesting.
I'm doing this but with a twist. I have an existing user base so I'm
going to use
the existing UID to hash to one of the ipauser<n> groups. Will do
round-robin
for future adds. Being an education institution we get a lot of new
accounts each fall.
I imagine that the UI would blow up if you tried to view the ipausers
group as it tried to dereference 120k users.
You'll probably also want to disable the compat module for the import.
I'll give this a try too.
I assume you've already done some amount of testing with a smaller batch
of users to ensure they migrate ok, passwords work, etc?
rob
Thanks to both of you. It will be a while before I can report the fruits
of this
exercise. I'm reluctant to give up the 5000+ that I've already added so I'm
deleting them from ipauser and adding them to the appropriate ipauser<n>.
Meantime I've got a bunch of other scripts to write for some of the
custom stuff
we did with PAM and the old MySQL backend.
Regards,
Daryl
--
--
Daryl Fonseca-Holt
IST/CNS/Unix Server Team
University of Manitoba
204.480.1079
--
Manage your subscription for the Freeipa-users mailing list:
https://www.redhat.com/mailman/listinfo/freeipa-users
Go to http://freeipa.org for more info on the project
