On 06/05/2016 10:45 AM, Martin Basti wrote:



On 03.06.2016 17:49, thierry bordaz wrote:
Hello,

A performance bottleneck during provisioning was described http://www.freeipa.org/page/V4/Performance_Improvements#typical_provisioning:_ldapadd_entries.2C_migrate-ds...

I wrote the attached script that is following http://www.freeipa.org/page/V4/Performance_Improvements#Algorithm

This is a preliminary script that needs to be improved but just to check it matches basic requirements I am sending it to the alias to get some feedback.

The output of a provisioning session is below. The steps are:

  * install freeipa-server
  * create a provisioning file using
    https://github.com/freeipa/freeipa-tools/blob/master/create-test-data.py
  * ipa-provision.py prepare
  * ipa-provision.py import
  * ipa-provision.py finish

Note: you will likely need to define the DM password with the option '-w <password>'

regards
thierry


/tmp/ipa-provision.py prepare -f /tmp/1K_users_800_hosts_groups_hostgroups_sudorules_hbac_rules.ldif -d 1
entrycache = 10485760
dncache    = 10485760
dbcache    = 209715200
dblock     = 10000
Preparation of the bulk import is now completed
If you want to continue:
- run /tmp/ipa-provision.py import -f /tmp/1K_users_800_hosts_groups_hostgroups_sudorules_hbac_rules.ldif
  - run /tmp/ipa-provision.py finish -b <suffix_dn>

If you want to not continue:
  - run /tmp/ipa-provision.py abort



/tmp/ipa-provision import -f /tmp/1K_users_800_hosts_groups_hostgroups_sudorules_hbac_rules.ldif -debug 2 adding new entry "uid=user0,cn=users,cn=accounts,dc=abc,dc=idm,dc=lab,dc=eng,dc=brq,dc=redhat,dc=com"

adding new entry "uid=user1,cn=users,cn=accounts,dc=abc,dc=idm,dc=lab,dc=eng,dc=brq,dc=redhat,dc=com"

adding new entry "uid=user2,cn=users,cn=accounts,dc=abc,dc=idm,dc=lab,dc=eng,dc=brq,dc=redhat,dc=com"
...
adding new entry "ipaUniqueID=autogenerate,cn=hbac,dc=abc,dc=idm,dc=lab,dc=eng,dc=brq,dc=redhat,dc=com"

adding new entry "ipaUniqueID=autogenerate,cn=hbac,dc=abc,dc=idm,dc=lab,dc=eng,dc=brq,dc=redhat,dc=com"

Bulk import is now completed
You need to run:
  - run /tmp/ipa-provision.py finish -b <suffix_dn>



/tmp/ipa-provision.py finish

Waiting for (objectClass=inetorgperson) fixup to complete...
Completed filter (objectClass=inetorgperson)

Waiting for (objectClass=ipausergroup) fixup to complete...
Completed filter (objectClass=ipausergroup)

Waiting for (objectClass=ipahost) fixup to complete...
Completed filter (objectClass=ipahost)

Waiting for (objectClass=ipahostgroup) fixup to complete...
Completed filter (objectClass=ipahostgroup)

Waiting for (objectClass=ipasudorule) fixup to complete...
Completed filter (objectClass=ipasudorule)

Waiting for (objectClass=ipahbacrule) fixup to complete...
Completed filter (objectClass=ipahbacrule)

Bulk import and fixup tasks are now completed




Thank you for provisioning script:

* should be this part of freeIPA or is it just script for testing purposes? * the code isn't pythonic much, if it is just for testing I don't care, but if this will be part of freeIPA I care a lot (I can help in this case :) ) * maybe some parts of IPA code can be reused for this script, is has to run on installed server anyway

Martin^2


Hi Martin,

Thanks for your review of that script.. over the weekend :-)
It should be part of freeipa as a way to support provisioning of large number of entries. I am not really surprised it is not pythonic, I tried first to use IPA code but got lost and finally decided to do it that easy way. I am fine if it needs to be rewritten and would really appreciate your help :-)

Yes it has to run on an installed server. But even being part of Freeipa deliver, this script would preferably run with freeipa being stop and only DS running. It saves memory and limits DS ops.

Thanks
thierry
-- 
Manage your subscription for the Freeipa-devel mailing list:
https://www.redhat.com/mailman/listinfo/freeipa-devel
Contribute to FreeIPA: http://www.freeipa.org/page/Contribute/Code

Reply via email to