subject:"\[Freeipa\-devel\] Provisioning throughput"

Re: [Freeipa-devel] Provisioning throughput

2016-05-31 Thread thierry bordaz




On 05/31/2016 02:02 PM, Petr Vobornik wrote:

On 05/04/2016 02:20 PM, thierry bordaz wrote:

Hello,

 I have been doing some tests/measures using
 https://github.com/freeipa/freeipa-tools/blob/master/create-test-data.py.
 The tool creates a set of typical users/hosts/groups... to import with a
 ldapadd.

 I wrote down some finding in
 
http://www.freeipa.org/page/V4/Performance_Improvements#Provisioning_throughput_and_DS_plugins.
 I still have to do some cleanup around the performance but the basic of a
 possible improvement is to do provisioning in several steps (disabling
 plugins, provisioning, enabling plugin, running fixup tasks).

 Before going further in the design I wanted to share those ideas and know 
if
 it raise any concern.

 thanks
 thierry


Let me conclude the previous sub-thread and also continue with
discussion we had over phone.

The subthread ended with proposal to go with offline provisioning by
disabling LDAP ports similar to ipa-server-upgrade tool and then using
LDAPI to add records by using ipalib in server mode e.g., as in
ipa_otptoken_import.py

So these are the tasks to solve/investigate:

1. Provide guidance/write script which would disable memberof plugin and
other plugins. Disable ldap and ldaps port


I started describing the operations 
http://www.freeipa.org/page/V4/Performance_Improvements#Algorithm.
It needs to be updated with disabling regular ports and accessing 
through ldapi


2. Provide guidance/script how to use ipalib in server mode and how to
import date. This could be even script which would load file in some
format(e.g. JSON) and executed commands from the file. Basically what
was Alexander proposing and I was against it. After some thought, I
agree that the tool could be easy to write but I'd rather avoid adding
it to 4.4 release maybe even to future releases. Because:
  - It's almost the same as `[RFE] run multiple CLI commands in a batch`
 With the distinction that
it connects directly LDAPI and not public API. First I'd like to see
#5821 and then we can think of using same logic(input) to work in
"migration" mode.
- I don't want to include a quickly baked tool so late in 4.4
development. It will have design flaws which will be harder to fix
later. I don't want to loose time with discussion about design of the
tool in this phase of 4.4.
Investigating a ldap bulk load, there is a quite limited set of 
operations to prepare the instance, run a ldap bulk load and run fixup.
However, starting yesterday looking at ipa-otptoken-import I am still 
unable to connect through ldapi and do those really simple operations... 
so even it could be easy to write tool my progress start slowly


Regarding the batch commands, it looks quite different than bulk import 
because batch commands (like user-add) run  several ldap ops 
(SRCH/ADD/MOD) and also batch commands does expect that DS plugins like 
memberof are enabled.




3. Provide guidance/script to revert #1 and run memberof fixup task.

4. Investigate how replication of so many records with member attributes
will affect other replicas. I.e. if it would not brick entire topology.

*Right now #4 seems to be the most important*.


This was shortly described in 
http://www.freeipa.org/page/V4/Performance_Improvements#Memberof_plugin 
and 
http://www.freeipa.org/page/V4/Performance_Improvements#Replication_being_late.


The topology will slowly converge and eventually all provisioned entries 
will be available everywhere.
But slowly can mean hours or even days before it converges. During that 
period, a provisioned rules can grant some rights on one replica (where 
it was imported) but will not grant it on an other where the rule is not 
yet replicated.


If the topology is a production topology, it can be better to rely on 
total init of the replicas than on replication.






Then #1, #2, #3 could be delivered as sample script included in
documentation and/or blog post/github. It would allow us to change it
later. I would not focus on #2 before other core 4.4 items are finished.
A lot of guidance is already written in tree design page but it will
need to be transform to easily consumable form for admins. The scripts
can be prepared even when 4.4 is out.

In other words I would not create any official new ipa utility yet.


--
Manage your subscription for the Freeipa-devel mailing list:
https://www.redhat.com/mailman/listinfo/freeipa-devel
Contribute to FreeIPA: http://www.freeipa.org/page/Contribute/Code

Re: [Freeipa-devel] Provisioning throughput

2016-05-31 Thread Petr Vobornik

On 05/04/2016 02:20 PM, thierry bordaz wrote:
> Hello,
> 
> I have been doing some tests/measures using
> https://github.com/freeipa/freeipa-tools/blob/master/create-test-data.py.
> The tool creates a set of typical users/hosts/groups... to import with a
> ldapadd.
> 
> I wrote down some finding in
> 
> http://www.freeipa.org/page/V4/Performance_Improvements#Provisioning_throughput_and_DS_plugins.
> I still have to do some cleanup around the performance but the basic of a
> possible improvement is to do provisioning in several steps (disabling
> plugins, provisioning, enabling plugin, running fixup tasks).
> 
> Before going further in the design I wanted to share those ideas and know 
> if
> it raise any concern.
> 
> thanks
> thierry
> 

Let me conclude the previous sub-thread and also continue with
discussion we had over phone.

The subthread ended with proposal to go with offline provisioning by
disabling LDAP ports similar to ipa-server-upgrade tool and then using
LDAPI to add records by using ipalib in server mode e.g., as in
ipa_otptoken_import.py

So these are the tasks to solve/investigate:

1. Provide guidance/write script which would disable memberof plugin and
other plugins. Disable ldap and ldaps port

2. Provide guidance/script how to use ipalib in server mode and how to
import date. This could be even script which would load file in some
format(e.g. JSON) and executed commands from the file. Basically what
was Alexander proposing and I was against it. After some thought, I
agree that the tool could be easy to write but I'd rather avoid adding
it to 4.4 release maybe even to future releases. Because:
 - It's almost the same as `[RFE] run multiple CLI commands in a batch`
 With the distinction that
it connects directly LDAPI and not public API. First I'd like to see
#5821 and then we can think of using same logic(input) to work in
"migration" mode.
- I don't want to include a quickly baked tool so late in 4.4
development. It will have design flaws which will be harder to fix
later. I don't want to loose time with discussion about design of the
tool in this phase of 4.4.

3. Provide guidance/script to revert #1 and run memberof fixup task.

4. Investigate how replication of so many records with member attributes
will affect other replicas. I.e. if it would not brick entire topology.

*Right now #4 seems to be the most important*.

Then #1, #2, #3 could be delivered as sample script included in
documentation and/or blog post/github. It would allow us to change it
later. I would not focus on #2 before other core 4.4 items are finished.
A lot of guidance is already written in tree design page but it will
need to be transform to easily consumable form for admins. The scripts
can be prepared even when 4.4 is out.

In other words I would not create any official new ipa utility yet.
-- 
Petr Vobornik

-- 
Manage your subscription for the Freeipa-devel mailing list:
https://www.redhat.com/mailman/listinfo/freeipa-devel
Contribute to FreeIPA: http://www.freeipa.org/page/Contribute/Code

Re: [Freeipa-devel] Provisioning throughput

2016-05-26 Thread thierry bordaz




On 05/26/2016 12:23 PM, Alexander Bokovoy wrote:

On Thu, 26 May 2016, thierry bordaz wrote:
The limitation would be to run the provisioning on IPA master. 
During provisioning, membership attribute will be invalid (memberof 
not computed). Is it acceptable that IPA master contains invalid 
membership for some time ?

Consider provisioning to be at the same level as running
ipa-server-upgrade -- access via 389/636 ports is not allowed, LDAPI is
the only interface enabled which implies there would be no problem 
if we

set expectations right: provisioning mode is offline.


Yes I agree, provisioning mode is offline.
My concern is about side effects on the rest of the topology if we 
are putting IPA master offline (is password update possible on 
replica ?).

Sure, update on replica would be queued in replication queue. Password
changes are local anyway, they result in updates of few password
attributes and that's all. These attributes replicated in the same way
as anything else.

Yes that is right.
I remember a discussion about the master key that was only available on 
IPA master and I thought that IPA master had a specific role around krb 
attributes. But if provisioning can be done on IPA master, it is then a 
good idea to use root/ldapi to avoid getting DM password.


thanks for all your feedback and help

thierry

--
Manage your subscription for the Freeipa-devel mailing list:
https://www.redhat.com/mailman/listinfo/freeipa-devel
Contribute to FreeIPA: http://www.freeipa.org/page/Contribute/Code

Re: [Freeipa-devel] Provisioning throughput

2016-05-26 Thread Alexander Bokovoy


On Thu, 26 May 2016, thierry bordaz wrote:
The limitation would be to run the provisioning on IPA master. 
During provisioning, membership attribute will be invalid 
(memberof not computed). Is it acceptable that IPA master contains 
invalid membership for some time ?

Consider provisioning to be at the same level as running
ipa-server-upgrade -- access via 389/636 ports is not allowed, LDAPI is
the only interface enabled which implies there would be no problem if we
set expectations right: provisioning mode is offline.


Yes I agree, provisioning mode is offline.
My concern is about side effects on the rest of the topology if we are 
putting IPA master offline (is password update possible on replica ?).

Sure, update on replica would be queued in replication queue. Password
changes are local anyway, they result in updates of few password
attributes and that's all. These attributes replicated in the same way
as anything else.
--
/ Alexander Bokovoy

--
Manage your subscription for the Freeipa-devel mailing list:
https://www.redhat.com/mailman/listinfo/freeipa-devel
Contribute to FreeIPA: http://www.freeipa.org/page/Contribute/Code

Re: [Freeipa-devel] Provisioning throughput

2016-05-26 Thread thierry bordaz




On 05/26/2016 11:26 AM, Martin Basti wrote:



On 26.05.2016 11:24, thierry bordaz wrote:



On 05/26/2016 11:12 AM, Alexander Bokovoy wrote:

On Thu, 26 May 2016, thierry bordaz wrote:



On 05/25/2016 09:31 PM, Rob Crittenden wrote:

thierry bordaz wrote:



On 05/25/2016 08:49 PM, Rob Crittenden wrote:

thierry bordaz wrote:


Hello,

Thanks for all the feedbacks. I updated the design accordingly 
and with

additional tests results
(http://www.freeipa.org/page/V4/Performance_Improvements#Proposed_improvements) 




Several improvements can be done, in particular in DS plugins 
(memberof,
retroCL), but for "easy" benefit provisioning will be done with 
memberof

disabled followed by fixup.

It remains some aspects that are not clear to me:

 * For best performance, DS tuning and provisioning/fixup would
   preferably be done under 'directory manager'
   That means prompting DM password and writing it into 
temporary file.

   Is that a concern ?
 * Fixup requires that we know the filters matching the 
provisioned

   entries. For example :
 o (objectClass=inetorgperson)
 o (objectClass=ipausergroup)
 o (objectClass=ipahost)
 o (objectClass=ipahostgroup)
 o (objectClass=ipasudorule)
 o (objectClass=ipahbacrule)

   The set of objectclass could be hardcode or provided in the
   provisioning CLI option
   What to do if an entry in in the provision file does not 
match

   any of those filter ? Should it stop without starting the
   provisioning ?
 * The CLI doing the provisioning could be something like 'ipa
   provision ' or should it be a separated command e.g.
   ipa-bulk-load ?


It depends. There is a migration command now, ipa migrate-ds, that
adds records and is impacted by this. There is also the 
possibility of

looping calls to ipa [user|group|etc]-add.


I agree that migration and bulk load can be linked. If migration
dump/update a set of entries before filling them into a new 
instance it

could use bulk load.
For set loop of ipa -add, I think they add many others 
direct

operations (mainly SRCH) before doing the ADD in order to check
coherency. bulk load looks more straightforward.


I just wonder if some (all) of this could be done manually. 
Document how to turn off memberof, do the import whatever way is 
appropriate, then run the fixup? I'm not sure what you had in mind.


I don't want to think small but do we expect to be importing a 
slew of hosts, sudorules, etc? I guess the potential is there but 
would it be on the same scale as users? If you focus only on 
users/groups does that change the use case at all?


In fact, I am using such small scripts to prepare and run/monitor 
the provisioning.
If providing a set of scripts and document a procedure is enough I 
am fine with this.



Would it be reasonable to require bulk import to be done on an IPA
master so we can leverage the ldapi socket?

Do you mean using ldapi to reduce network latency or automember or
something else ?


To avoid the DM password issues. ldapi autobinds to DM when the id 
is root.


Yes I said automember but was thinking to autobind.
That is nice idea to avoid prompting DM password.
In addition, slapi-nis participating to slowing down the 
provisioning if it is using ldapi/DM slapi-nis will be offline by 
itself.


The limitation would be to run the provisioning on IPA master. 
During provisioning, membership attribute will be invalid (memberof 
not computed). Is it acceptable that IPA master contains invalid 
membership for some time ?

Consider provisioning to be at the same level as running
ipa-server-upgrade -- access via 389/636 ports is not allowed, LDAPI is
the only interface enabled which implies there would be no problem 
if we

set expectations right: provisioning mode is offline.


Yes I agree, provisioning mode is offline.
My concern is about side effects on the rest of the topology if we 
are putting IPA master offline (is password update possible on 
replica ?).



thierry



How long it takes until memberof data are recreated using replication? 
(IIRC and memberof attributes are not replicated)


I have not precise data on what will be the replication latency.
IMHO provisioning will be "fast" on the server where we run the 
provisioning. Then the entries will be replicated to replicas where 
memberof is enabled. replication latency of 
users/usergroups/host/hostgroup should also be quite low, let say few 
minutes behind. Now the replication latency for entries like 
sudorules/hbacrules would be important, probably several hours.


thierry


--
Manage your subscription for the Freeipa-devel mailing list:
https://www.redhat.com/mailman/listinfo/freeipa-devel
Contribute to FreeIPA: http://www.freeipa.org/page/Contribute/Code

Re: [Freeipa-devel] Provisioning throughput

2016-05-26 Thread Martin Basti




On 26.05.2016 11:24, thierry bordaz wrote:



On 05/26/2016 11:12 AM, Alexander Bokovoy wrote:

On Thu, 26 May 2016, thierry bordaz wrote:



On 05/25/2016 09:31 PM, Rob Crittenden wrote:

thierry bordaz wrote:



On 05/25/2016 08:49 PM, Rob Crittenden wrote:

thierry bordaz wrote:


Hello,

Thanks for all the feedbacks. I updated the design accordingly 
and with

additional tests results
(http://www.freeipa.org/page/V4/Performance_Improvements#Proposed_improvements) 




Several improvements can be done, in particular in DS plugins 
(memberof,
retroCL), but for "easy" benefit provisioning will be done with 
memberof

disabled followed by fixup.

It remains some aspects that are not clear to me:

 * For best performance, DS tuning and provisioning/fixup would
   preferably be done under 'directory manager'
   That means prompting DM password and writing it into 
temporary file.

   Is that a concern ?
 * Fixup requires that we know the filters matching the provisioned
   entries. For example :
 o (objectClass=inetorgperson)
 o (objectClass=ipausergroup)
 o (objectClass=ipahost)
 o (objectClass=ipahostgroup)
 o (objectClass=ipasudorule)
 o (objectClass=ipahbacrule)

   The set of objectclass could be hardcode or provided in the
   provisioning CLI option
   What to do if an entry in in the provision file does not 
match

   any of those filter ? Should it stop without starting the
   provisioning ?
 * The CLI doing the provisioning could be something like 'ipa
   provision ' or should it be a separated command e.g.
   ipa-bulk-load ?


It depends. There is a migration command now, ipa migrate-ds, that
adds records and is impacted by this. There is also the 
possibility of

looping calls to ipa [user|group|etc]-add.


I agree that migration and bulk load can be linked. If migration
dump/update a set of entries before filling them into a new 
instance it

could use bulk load.
For set loop of ipa -add, I think they add many others direct
operations (mainly SRCH) before doing the ADD in order to check
coherency. bulk load looks more straightforward.


I just wonder if some (all) of this could be done manually. 
Document how to turn off memberof, do the import whatever way is 
appropriate, then run the fixup? I'm not sure what you had in mind.


I don't want to think small but do we expect to be importing a slew 
of hosts, sudorules, etc? I guess the potential is there but would 
it be on the same scale as users? If you focus only on users/groups 
does that change the use case at all?


In fact, I am using such small scripts to prepare and run/monitor 
the provisioning.
If providing a set of scripts and document a procedure is enough I 
am fine with this.



Would it be reasonable to require bulk import to be done on an IPA
master so we can leverage the ldapi socket?

Do you mean using ldapi to reduce network latency or automember or
something else ?


To avoid the DM password issues. ldapi autobinds to DM when the id 
is root.


Yes I said automember but was thinking to autobind.
That is nice idea to avoid prompting DM password.
In addition, slapi-nis participating to slowing down the 
provisioning if it is using ldapi/DM slapi-nis will be offline by 
itself.


The limitation would be to run the provisioning on IPA master. 
During provisioning, membership attribute will be invalid (memberof 
not computed). Is it acceptable that IPA master contains invalid 
membership for some time ?

Consider provisioning to be at the same level as running
ipa-server-upgrade -- access via 389/636 ports is not allowed, LDAPI is
the only interface enabled which implies there would be no problem if we
set expectations right: provisioning mode is offline.


Yes I agree, provisioning mode is offline.
My concern is about side effects on the rest of the topology if we are 
putting IPA master offline (is password update possible on replica ?).



thierry



How long it takes until memberof data are recreated using replication? 
(IIRC and memberof attributes are not replicated)


--
Manage your subscription for the Freeipa-devel mailing list:
https://www.redhat.com/mailman/listinfo/freeipa-devel
Contribute to FreeIPA: http://www.freeipa.org/page/Contribute/Code

Re: [Freeipa-devel] Provisioning throughput

2016-05-26 Thread thierry bordaz




On 05/26/2016 11:12 AM, Alexander Bokovoy wrote:

On Thu, 26 May 2016, thierry bordaz wrote:



On 05/25/2016 09:31 PM, Rob Crittenden wrote:

thierry bordaz wrote:



On 05/25/2016 08:49 PM, Rob Crittenden wrote:

thierry bordaz wrote:


Hello,

Thanks for all the feedbacks. I updated the design accordingly 
and with

additional tests results
(http://www.freeipa.org/page/V4/Performance_Improvements#Proposed_improvements) 




Several improvements can be done, in particular in DS plugins 
(memberof,
retroCL), but for "easy" benefit provisioning will be done with 
memberof

disabled followed by fixup.

It remains some aspects that are not clear to me:

 * For best performance, DS tuning and provisioning/fixup would
   preferably be done under 'directory manager'
   That means prompting DM password and writing it into temporary 
file.

   Is that a concern ?
 * Fixup requires that we know the filters matching the provisioned
   entries. For example :
 o (objectClass=inetorgperson)
 o (objectClass=ipausergroup)
 o (objectClass=ipahost)
 o (objectClass=ipahostgroup)
 o (objectClass=ipasudorule)
 o (objectClass=ipahbacrule)

   The set of objectclass could be hardcode or provided in the
   provisioning CLI option
   What to do if an entry in in the provision file does not 
match

   any of those filter ? Should it stop without starting the
   provisioning ?
 * The CLI doing the provisioning could be something like 'ipa
   provision ' or should it be a separated command e.g.
   ipa-bulk-load ?


It depends. There is a migration command now, ipa migrate-ds, that
adds records and is impacted by this. There is also the 
possibility of

looping calls to ipa [user|group|etc]-add.


I agree that migration and bulk load can be linked. If migration
dump/update a set of entries before filling them into a new 
instance it

could use bulk load.
For set loop of ipa -add, I think they add many others direct
operations (mainly SRCH) before doing the ADD in order to check
coherency. bulk load looks more straightforward.


I just wonder if some (all) of this could be done manually. Document 
how to turn off memberof, do the import whatever way is appropriate, 
then run the fixup? I'm not sure what you had in mind.


I don't want to think small but do we expect to be importing a slew 
of hosts, sudorules, etc? I guess the potential is there but would 
it be on the same scale as users? If you focus only on users/groups 
does that change the use case at all?


In fact, I am using such small scripts to prepare and run/monitor the 
provisioning.
If providing a set of scripts and document a procedure is enough I am 
fine with this.



Would it be reasonable to require bulk import to be done on an IPA
master so we can leverage the ldapi socket?

Do you mean using ldapi to reduce network latency or automember or
something else ?


To avoid the DM password issues. ldapi autobinds to DM when the id 
is root.


Yes I said automember but was thinking to autobind.
That is nice idea to avoid prompting DM password.
In addition, slapi-nis participating to slowing down the provisioning 
if it is using ldapi/DM slapi-nis will be offline by itself.


The limitation would be to run the provisioning on IPA master. During 
provisioning, membership attribute will be invalid (memberof not 
computed). Is it acceptable that IPA master contains invalid 
membership for some time ?

Consider provisioning to be at the same level as running
ipa-server-upgrade -- access via 389/636 ports is not allowed, LDAPI is
the only interface enabled which implies there would be no problem if we
set expectations right: provisioning mode is offline.


Yes I agree, provisioning mode is offline.
My concern is about side effects on the rest of the topology if we are 
putting IPA master offline (is password update possible on replica ?).



thierry

--
Manage your subscription for the Freeipa-devel mailing list:
https://www.redhat.com/mailman/listinfo/freeipa-devel
Contribute to FreeIPA: http://www.freeipa.org/page/Contribute/Code

Re: [Freeipa-devel] Provisioning throughput

2016-05-26 Thread Martin Basti




On 26.05.2016 11:12, Alexander Bokovoy wrote:

On Thu, 26 May 2016, thierry bordaz wrote:



On 05/25/2016 09:31 PM, Rob Crittenden wrote:

thierry bordaz wrote:



On 05/25/2016 08:49 PM, Rob Crittenden wrote:

thierry bordaz wrote:


Hello,

Thanks for all the feedbacks. I updated the design accordingly 
and with

additional tests results
(http://www.freeipa.org/page/V4/Performance_Improvements#Proposed_improvements) 




Several improvements can be done, in particular in DS plugins 
(memberof,
retroCL), but for "easy" benefit provisioning will be done with 
memberof

disabled followed by fixup.

It remains some aspects that are not clear to me:

 * For best performance, DS tuning and provisioning/fixup would
   preferably be done under 'directory manager'
   That means prompting DM password and writing it into temporary 
file.

   Is that a concern ?
 * Fixup requires that we know the filters matching the provisioned
   entries. For example :
 o (objectClass=inetorgperson)
 o (objectClass=ipausergroup)
 o (objectClass=ipahost)
 o (objectClass=ipahostgroup)
 o (objectClass=ipasudorule)
 o (objectClass=ipahbacrule)

   The set of objectclass could be hardcode or provided in the
   provisioning CLI option
   What to do if an entry in in the provision file does not 
match

   any of those filter ? Should it stop without starting the
   provisioning ?
 * The CLI doing the provisioning could be something like 'ipa
   provision ' or should it be a separated command e.g.
   ipa-bulk-load ?


It depends. There is a migration command now, ipa migrate-ds, that
adds records and is impacted by this. There is also the 
possibility of

looping calls to ipa [user|group|etc]-add.


I agree that migration and bulk load can be linked. If migration
dump/update a set of entries before filling them into a new 
instance it

could use bulk load.
For set loop of ipa -add, I think they add many others direct
operations (mainly SRCH) before doing the ADD in order to check
coherency. bulk load looks more straightforward.


I just wonder if some (all) of this could be done manually. Document 
how to turn off memberof, do the import whatever way is appropriate, 
then run the fixup? I'm not sure what you had in mind.


I don't want to think small but do we expect to be importing a slew 
of hosts, sudorules, etc? I guess the potential is there but would 
it be on the same scale as users? If you focus only on users/groups 
does that change the use case at all?


In fact, I am using such small scripts to prepare and run/monitor the 
provisioning.
If providing a set of scripts and document a procedure is enough I am 
fine with this.



Would it be reasonable to require bulk import to be done on an IPA
master so we can leverage the ldapi socket?

Do you mean using ldapi to reduce network latency or automember or
something else ?


To avoid the DM password issues. ldapi autobinds to DM when the id 
is root.


Yes I said automember but was thinking to autobind.
That is nice idea to avoid prompting DM password.
In addition, slapi-nis participating to slowing down the provisioning 
if it is using ldapi/DM slapi-nis will be offline by itself.


The limitation would be to run the provisioning on IPA master. During 
provisioning, membership attribute will be invalid (memberof not 
computed). Is it acceptable that IPA master contains invalid 
membership for some time ?

Consider provisioning to be at the same level as running
ipa-server-upgrade -- access via 389/636 ports is not allowed, LDAPI is
the only interface enabled which implies there would be no problem if we
set expectations right: provisioning mode is offline.

I agree

Martin^2

--
Manage your subscription for the Freeipa-devel mailing list:
https://www.redhat.com/mailman/listinfo/freeipa-devel
Contribute to FreeIPA: http://www.freeipa.org/page/Contribute/Code

Re: [Freeipa-devel] Provisioning throughput

2016-05-26 Thread Alexander Bokovoy


On Thu, 26 May 2016, thierry bordaz wrote:



On 05/25/2016 09:31 PM, Rob Crittenden wrote:

thierry bordaz wrote:



On 05/25/2016 08:49 PM, Rob Crittenden wrote:

thierry bordaz wrote:


Hello,

Thanks for all the feedbacks. I updated the design accordingly 
and with

additional tests results
(http://www.freeipa.org/page/V4/Performance_Improvements#Proposed_improvements)


Several improvements can be done, in particular in DS plugins 
(memberof,
retroCL), but for "easy" benefit provisioning will be done 
with memberof

disabled followed by fixup.

It remains some aspects that are not clear to me:

 * For best performance, DS tuning and provisioning/fixup would
   preferably be done under 'directory manager'
   That means prompting DM password and writing it into 
temporary file.

   Is that a concern ?
 * Fixup requires that we know the filters matching the provisioned
   entries. For example :
 o (objectClass=inetorgperson)
 o (objectClass=ipausergroup)
 o (objectClass=ipahost)
 o (objectClass=ipahostgroup)
 o (objectClass=ipasudorule)
 o (objectClass=ipahbacrule)

   The set of objectclass could be hardcode or provided in the
   provisioning CLI option
   What to do if an entry in in the provision file does not match
   any of those filter ? Should it stop without starting the
   provisioning ?
 * The CLI doing the provisioning could be something like 'ipa
   provision ' or should it be a separated command e.g.
   ipa-bulk-load ?


It depends. There is a migration command now, ipa migrate-ds, that
adds records and is impacted by this. There is also the possibility of
looping calls to ipa [user|group|etc]-add.


I agree that migration and bulk load can be linked. If migration
dump/update a set of entries before filling them into a new instance it
could use bulk load.
For set loop of ipa -add, I think they add many others direct
operations (mainly SRCH) before doing the ADD in order to check
coherency. bulk load looks more straightforward.


I just wonder if some (all) of this could be done manually. Document 
how to turn off memberof, do the import whatever way is appropriate, 
then run the fixup? I'm not sure what you had in mind.


I don't want to think small but do we expect to be importing a slew 
of hosts, sudorules, etc? I guess the potential is there but would 
it be on the same scale as users? If you focus only on users/groups 
does that change the use case at all?


In fact, I am using such small scripts to prepare and run/monitor the 
provisioning.
If providing a set of scripts and document a procedure is enough I am 
fine with this.



Would it be reasonable to require bulk import to be done on an IPA
master so we can leverage the ldapi socket?

Do you mean using ldapi to reduce network latency or automember or
something else ?


To avoid the DM password issues. ldapi autobinds to DM when the id 
is root.


Yes I said automember but was thinking to autobind.
That is nice idea to avoid prompting DM password.
In addition, slapi-nis participating to slowing down the provisioning 
if it is using ldapi/DM slapi-nis will be offline by itself.


The limitation would be to run the provisioning on IPA master. During 
provisioning, membership attribute will be invalid (memberof not 
computed). Is it acceptable that IPA master contains invalid 
membership for some time ?

Consider provisioning to be at the same level as running
ipa-server-upgrade -- access via 389/636 ports is not allowed, LDAPI is
the only interface enabled which implies there would be no problem if we
set expectations right: provisioning mode is offline.
--
/ Alexander Bokovoy

--
Manage your subscription for the Freeipa-devel mailing list:
https://www.redhat.com/mailman/listinfo/freeipa-devel
Contribute to FreeIPA: http://www.freeipa.org/page/Contribute/Code

Re: [Freeipa-devel] Provisioning throughput

2016-05-26 Thread thierry bordaz




On 05/26/2016 09:32 AM, Alexander Bokovoy wrote:

On Wed, 25 May 2016, Rob Crittenden wrote:

thierry bordaz wrote:



On 05/25/2016 08:49 PM, Rob Crittenden wrote:

thierry bordaz wrote:


Hello,

Thanks for all the feedbacks. I updated the design accordingly and 
with

additional tests results
(http://www.freeipa.org/page/V4/Performance_Improvements#Proposed_improvements) 



Several improvements can be done, in particular in DS plugins 
(memberof,
retroCL), but for "easy" benefit provisioning will be done with 
memberof

disabled followed by fixup.

It remains some aspects that are not clear to me:

 * For best performance, DS tuning and provisioning/fixup would
   preferably be done under 'directory manager'
   That means prompting DM password and writing it into temporary 
file.

   Is that a concern ?
 * Fixup requires that we know the filters matching the provisioned
   entries. For example :
 o (objectClass=inetorgperson)
 o (objectClass=ipausergroup)
 o (objectClass=ipahost)
 o (objectClass=ipahostgroup)
 o (objectClass=ipasudorule)
 o (objectClass=ipahbacrule)

   The set of objectclass could be hardcode or provided in the
   provisioning CLI option
   What to do if an entry in in the provision file does not match
   any of those filter ? Should it stop without starting the
   provisioning ?
 * The CLI doing the provisioning could be something like 'ipa
   provision ' or should it be a separated command e.g.
   ipa-bulk-load ?


It depends. There is a migration command now, ipa migrate-ds, that
adds records and is impacted by this. There is also the possibility of
looping calls to ipa [user|group|etc]-add.


I agree that migration and bulk load can be linked. If migration
dump/update a set of entries before filling them into a new instance it
could use bulk load.
For set loop of ipa -add, I think they add many others direct
operations (mainly SRCH) before doing the ADD in order to check
coherency. bulk load looks more straightforward.


I just wonder if some (all) of this could be done manually. Document 
how to turn off memberof, do the import whatever way is appropriate, 
then run the fixup? I'm not sure what you had in mind.


I don't want to think small but do we expect to be importing a slew 
of hosts, sudorules, etc? I guess the potential is there but would it 
be on the same scale as users? If you focus only on users/groups does 
that change the use case at all?

I tend to agree with Rob on this. Maybe we should have a simple
script/update file that does preparatory work and another one that
returns configuration into the right state and document how to use them.


Ok.
rereading the thread I realize we are talking of 
user/usergroups/host/hostgroup.


Provisioning such entries is not that bad.
For example 5Kusers/hosts are provisioned in 5min without memberof and 
19min with memberof


The real problem is provisioning sudorules and hbacrules where the 
impact of memberof is very important.
For example 100 sudorules are provisioned in 30s without memberof and 2h 
with memberof.


Do you think provisioning should also considere sudorules/hbac or only 
user/usergroups/host/hostgroup ?


--
Manage your subscription for the Freeipa-devel mailing list:
https://www.redhat.com/mailman/listinfo/freeipa-devel
Contribute to FreeIPA: http://www.freeipa.org/page/Contribute/Code

Re: [Freeipa-devel] Provisioning throughput

2016-05-26 Thread thierry bordaz




On 05/25/2016 09:31 PM, Rob Crittenden wrote:

thierry bordaz wrote:



On 05/25/2016 08:49 PM, Rob Crittenden wrote:

thierry bordaz wrote:


Hello,

Thanks for all the feedbacks. I updated the design accordingly and 
with

additional tests results
(http://www.freeipa.org/page/V4/Performance_Improvements#Proposed_improvements) 



Several improvements can be done, in particular in DS plugins 
(memberof,
retroCL), but for "easy" benefit provisioning will be done with 
memberof

disabled followed by fixup.

It remains some aspects that are not clear to me:

  * For best performance, DS tuning and provisioning/fixup would
preferably be done under 'directory manager'
That means prompting DM password and writing it into temporary 
file.

Is that a concern ?
  * Fixup requires that we know the filters matching the provisioned
entries. For example :
  o (objectClass=inetorgperson)
  o (objectClass=ipausergroup)
  o (objectClass=ipahost)
  o (objectClass=ipahostgroup)
  o (objectClass=ipasudorule)
  o (objectClass=ipahbacrule)

The set of objectclass could be hardcode or provided in the
provisioning CLI option
What to do if an entry in in the provision file does not match
any of those filter ? Should it stop without starting the
provisioning ?
  * The CLI doing the provisioning could be something like 'ipa
provision ' or should it be a separated command e.g.
ipa-bulk-load ?


It depends. There is a migration command now, ipa migrate-ds, that
adds records and is impacted by this. There is also the possibility of
looping calls to ipa [user|group|etc]-add.


I agree that migration and bulk load can be linked. If migration
dump/update a set of entries before filling them into a new instance it
could use bulk load.
For set loop of ipa -add, I think they add many others direct
operations (mainly SRCH) before doing the ADD in order to check
coherency. bulk load looks more straightforward.


I just wonder if some (all) of this could be done manually. Document 
how to turn off memberof, do the import whatever way is appropriate, 
then run the fixup? I'm not sure what you had in mind.


I don't want to think small but do we expect to be importing a slew of 
hosts, sudorules, etc? I guess the potential is there but would it be 
on the same scale as users? If you focus only on users/groups does 
that change the use case at all?


In fact, I am using such small scripts to prepare and run/monitor the 
provisioning.
If providing a set of scripts and document a procedure is enough I am 
fine with this.



Would it be reasonable to require bulk import to be done on an IPA
master so we can leverage the ldapi socket?

Do you mean using ldapi to reduce network latency or automember or
something else ?


To avoid the DM password issues. ldapi autobinds to DM when the id is 
root.


Yes I said automember but was thinking to autobind.
That is nice idea to avoid prompting DM password.
In addition, slapi-nis participating to slowing down the provisioning if 
it is using ldapi/DM slapi-nis will be offline by itself.


The limitation would be to run the provisioning on IPA master. During 
provisioning, membership attribute will be invalid (memberof not 
computed). Is it acceptable that IPA master contains invalid membership 
for some time ?




rob



--
Manage your subscription for the Freeipa-devel mailing list:
https://www.redhat.com/mailman/listinfo/freeipa-devel
Contribute to FreeIPA: http://www.freeipa.org/page/Contribute/Code

Re: [Freeipa-devel] Provisioning throughput

2016-05-26 Thread Alexander Bokovoy


On Wed, 25 May 2016, Rob Crittenden wrote:

thierry bordaz wrote:



On 05/25/2016 08:49 PM, Rob Crittenden wrote:

thierry bordaz wrote:


Hello,

Thanks for all the feedbacks. I updated the design accordingly and with
additional tests results
(http://www.freeipa.org/page/V4/Performance_Improvements#Proposed_improvements)

Several improvements can be done, in particular in DS plugins (memberof,
retroCL), but for "easy" benefit provisioning will be done with memberof
disabled followed by fixup.

It remains some aspects that are not clear to me:

 * For best performance, DS tuning and provisioning/fixup would
   preferably be done under 'directory manager'
   That means prompting DM password and writing it into temporary file.
   Is that a concern ?
 * Fixup requires that we know the filters matching the provisioned
   entries. For example :
 o (objectClass=inetorgperson)
 o (objectClass=ipausergroup)
 o (objectClass=ipahost)
 o (objectClass=ipahostgroup)
 o (objectClass=ipasudorule)
 o (objectClass=ipahbacrule)

   The set of objectclass could be hardcode or provided in the
   provisioning CLI option
   What to do if an entry in in the provision file does not match
   any of those filter ? Should it stop without starting the
   provisioning ?
 * The CLI doing the provisioning could be something like 'ipa
   provision ' or should it be a separated command e.g.
   ipa-bulk-load ?


It depends. There is a migration command now, ipa migrate-ds, that
adds records and is impacted by this. There is also the possibility of
looping calls to ipa [user|group|etc]-add.


I agree that migration and bulk load can be linked. If migration
dump/update a set of entries before filling them into a new instance it
could use bulk load.
For set loop of ipa -add, I think they add many others direct
operations (mainly SRCH) before doing the ADD in order to check
coherency. bulk load looks more straightforward.


I just wonder if some (all) of this could be done manually. Document 
how to turn off memberof, do the import whatever way is appropriate, 
then run the fixup? I'm not sure what you had in mind.


I don't want to think small but do we expect to be importing a slew of 
hosts, sudorules, etc? I guess the potential is there but would it be 
on the same scale as users? If you focus only on users/groups does 
that change the use case at all?

I tend to agree with Rob on this. Maybe we should have a simple
script/update file that does preparatory work and another one that
returns configuration into the right state and document how to use them.

--
/ Alexander Bokovoy

--
Manage your subscription for the Freeipa-devel mailing list:
https://www.redhat.com/mailman/listinfo/freeipa-devel
Contribute to FreeIPA: http://www.freeipa.org/page/Contribute/Code

Re: [Freeipa-devel] Provisioning throughput

2016-05-25 Thread thierry bordaz

On 05/25/2016 08:49 PM, Rob Crittenden wrote:

thierry bordaz wrote:

Hello,

Thanks for all the feedbacks. I updated the design accordingly and with
additional tests results
(http://www.freeipa.org/page/V4/Performance_Improvements#Proposed_improvements)

Several improvements can be done, in particular in DS plugins (memberof,
retroCL), but for "easy" benefit provisioning will be done with memberof
disabled followed by fixup.

It remains some aspects that are not clear to me:

* For best performance, DS tuning and provisioning/fixup would
preferably be done under 'directory manager'
That means prompting DM password and writing it into temporary file.
Is that a concern ?
* Fixup requires that we know the filters matching the provisioned
entries. For example :
o (objectClass=inetorgperson)
o (objectClass=ipausergroup)
o (objectClass=ipahost)
o (objectClass=ipahostgroup)
o (objectClass=ipasudorule)
o (objectClass=ipahbacrule)

The set of objectclass could be hardcode or provided in the
provisioning CLI option
What to do if an entry in in the provision file does not match
any of those filter ? Should it stop without starting the
provisioning ?
* The CLI doing the provisioning could be something like 'ipa
provision ' or should it be a separated command e.g.
ipa-bulk-load ?

It depends. There is a migration command now, ipa migrate-ds, that
adds records and is impacted by this. There is also the possibility of
looping calls to ipa [user|group|etc]-add.

I agree that migration and bulk load can be linked. If migration
dump/update a set of entries before filling them into a new instance it
could use bulk load.
For set loop of ipa -add, I think they add many others direct
operations (mainly SRCH) before doing the ADD in order to check
coherency. bulk load looks more straightforward.

Would it be reasonable to require bulk import to be done on an IPA
master so we can leverage the ldapi socket?
Do you mean using ldapi to reduce network latency or automember or
something else ?

thanks
theirry

rob

thanks
thierry

On 05/13/2016 10:18 AM, Ludwig Krispenz wrote:

On 05/13/2016 09:42 AM, Petr Spacek wrote:

On 13.5.2016 09:26, Martin Kosek wrote:

On 05/12/2016 04:16 PM, Ludwig Krispenz wrote:

On 05/12/2016 03:45 PM, Ludwig Krispenz wrote:

On 05/12/2016 02:16 PM, Petr Vobornik wrote:

On 05/10/2016 05:50 PM, thierry bordaz wrote:

On 05/05/2016 03:44 PM, Petr Vobornik wrote:

On 05/04/2016 02:20 PM, thierry bordaz wrote:

Hello,

I have been doing some tests/measures using
https://github.com/freeipa/freeipa-tools/blob/master/create-test-data.py.

The tool creates a set of typical
users/hosts/groups... to

import with a
ldapadd.

I wrote down some finding in
http://www.freeipa.org/page/V4/Performance_Improvements#Provisioning_throughput_and_DS_plugins.

I still have to do some cleanup around the performance
but the
basic of a
possible improvement is to do provisioning in several
steps
(disabling
plugins, provisioning, enabling plugin, running fixup
tasks).

Before going further in the design I wanted to share
those ideas
and know if
it raise any concern.

thanks
thierry

Hi Thierry,

Thanks for the analysis. Very nice.

Knowing this will help us suggesting workarounds also for old
releases.

Couple questions:

Have you tested retrCL disabled with memberOf enabled. It seems
that it
would eliminate 550K adds and 0.8M searches. What would be the
time
improvement?

Do you know what is the time when memberof is enabled but
slapi-nis and
retroCL are disabled?
The culprit of the performance issue is very likely related to
SRCH

(internal) triggered by memberof.

If retroCL is disabled and memberof enabled, #SRCH is 13.8M.
If retroCL is disabled, slapi-nis disabled and memberof enabled
#SRCH is
14.8
When all of them are enabled the #SRCH is 15M.

You are right if retroCL is disabled the #ADD drops but it has no
significant effect on the duration.

ok, thanks for the analysis

Regarding the duration of the provisioning, values are not
really stable
as performance of VM fluctuates. But as soon as memberof is
enabled the
provisioning lasts > 4hours where the same provisioning lasts
6mins as
soon as memberof is disabled.

I need to confirm the average time for internal searches but
assuming
1ms per SRCH it consumes >90% of the provisioning.

From the text it was not clear to me, if you find or
investigate
possible improvements in memberof plugin which would improve the
performance without stopping and starting DS.
As was discussed at mtg, have you tried if the DS restart is
really

necessary?

Re: [Freeipa-devel] Provisioning throughput

2016-05-25 Thread Rob Crittenden


thierry bordaz wrote:



On 05/25/2016 08:49 PM, Rob Crittenden wrote:

thierry bordaz wrote:


Hello,

Thanks for all the feedbacks. I updated the design accordingly and with
additional tests results
(http://www.freeipa.org/page/V4/Performance_Improvements#Proposed_improvements)

Several improvements can be done, in particular in DS plugins (memberof,
retroCL), but for "easy" benefit provisioning will be done with memberof
disabled followed by fixup.

It remains some aspects that are not clear to me:

  * For best performance, DS tuning and provisioning/fixup would
preferably be done under 'directory manager'
That means prompting DM password and writing it into temporary file.
Is that a concern ?
  * Fixup requires that we know the filters matching the provisioned
entries. For example :
  o (objectClass=inetorgperson)
  o (objectClass=ipausergroup)
  o (objectClass=ipahost)
  o (objectClass=ipahostgroup)
  o (objectClass=ipasudorule)
  o (objectClass=ipahbacrule)

The set of objectclass could be hardcode or provided in the
provisioning CLI option
What to do if an entry in in the provision file does not match
any of those filter ? Should it stop without starting the
provisioning ?
  * The CLI doing the provisioning could be something like 'ipa
provision ' or should it be a separated command e.g.
ipa-bulk-load ?


It depends. There is a migration command now, ipa migrate-ds, that
adds records and is impacted by this. There is also the possibility of
looping calls to ipa [user|group|etc]-add.


I agree that migration and bulk load can be linked. If migration
dump/update a set of entries before filling them into a new instance it
could use bulk load.
For set loop of ipa -add, I think they add many others direct
operations (mainly SRCH) before doing the ADD in order to check
coherency. bulk load looks more straightforward.


I just wonder if some (all) of this could be done manually. Document how 
to turn off memberof, do the import whatever way is appropriate, then 
run the fixup? I'm not sure what you had in mind.


I don't want to think small but do we expect to be importing a slew of 
hosts, sudorules, etc? I guess the potential is there but would it be on 
the same scale as users? If you focus only on users/groups does that 
change the use case at all?



Would it be reasonable to require bulk import to be done on an IPA
master so we can leverage the ldapi socket?

Do you mean using ldapi to reduce network latency or automember or
something else ?


To avoid the DM password issues. ldapi autobinds to DM when the id is root.

rob

--
Manage your subscription for the Freeipa-devel mailing list:
https://www.redhat.com/mailman/listinfo/freeipa-devel
Contribute to FreeIPA: http://www.freeipa.org/page/Contribute/Code

Re: [Freeipa-devel] Provisioning throughput

2016-05-25 Thread Rob Crittenden

thierry bordaz wrote:

Hello,

Thanks for all the feedbacks. I updated the design accordingly and with
additional tests results
(http://www.freeipa.org/page/V4/Performance_Improvements#Proposed_improvements)
Several improvements can be done, in particular in DS plugins (memberof,
retroCL), but for "easy" benefit provisioning will be done with memberof
disabled followed by fixup.

It remains some aspects that are not clear to me:

It depends. There is a migration command now, ipa migrate-ds, that adds
records and is impacted by this. There is also the possibility of
looping calls to ipa [user|group|etc]-add.

Would it be reasonable to require bulk import to be done on an IPA
master so we can leverage the ldapi socket?

rob

thanks
thierry

On 05/13/2016 10:18 AM, Ludwig Krispenz wrote:

On 05/13/2016 09:42 AM, Petr Spacek wrote:

On 13.5.2016 09:26, Martin Kosek wrote:

On 05/12/2016 04:16 PM, Ludwig Krispenz wrote:

On 05/12/2016 03:45 PM, Ludwig Krispenz wrote:

On 05/12/2016 02:16 PM, Petr Vobornik wrote:

On 05/10/2016 05:50 PM, thierry bordaz wrote:

On 05/05/2016 03:44 PM, Petr Vobornik wrote:

On 05/04/2016 02:20 PM, thierry bordaz wrote:

Hello,

I have been doing some tests/measures using
https://github.com/freeipa/freeipa-tools/blob/master/create-test-data.py.

The tool creates a set of typical users/hosts/groups... to
import with a
ldapadd.

I wrote down some finding in
http://www.freeipa.org/page/V4/Performance_Improvements#Provisioning_throughput_and_DS_plugins.

Before going further in the design I wanted to share
those ideas
and know if
it raise any concern.

thanks
thierry

Hi Thierry,

Thanks for the analysis. Very nice.

Knowing this will help us suggesting workarounds also for old
releases.

Couple questions:

Have you tested retrCL disabled with memberOf enabled. It seems
that it
would eliminate 550K adds and 0.8M searches. What would be the
time
improvement?

Do you know what is the time when memberof is enabled but
slapi-nis and
retroCL are disabled?

The culprit of the performance issue is very likely related to SRCH
(internal) triggered by memberof.

If retroCL is disabled and memberof enabled, #SRCH is 13.8M.
If retroCL is disabled, slapi-nis disabled and memberof enabled
#SRCH is
14.8
When all of them are enabled the #SRCH is 15M.

You are right if retroCL is disabled the #ADD drops but it has no
significant effect on the duration.

ok, thanks for the analysis

I need to confirm the average time for internal searches but
assuming
1ms per SRCH it consumes >90% of the provisioning.

From the text it was not clear to me, if you find or
investigate
possible improvements in memberof plugin which would improve the
performance without stopping and starting DS.

As was discussed at mtg, have you tried if the DS restart is really
necessary?

memberof plugin can be enabled and disabled while the server is
running, BUT
to achieve this the "enable-dynamic-plugins" feature has to be
turned on. And
then any enable/disable of a plugin would try to do it dynamically
an dnot
wait for the restart.
And I think not all plugins are able to handle this, TomasB was
once working
on it for IPA plugins, but it was not completed as far as I know

but enabling dynamic plugins can be done without restart, so what
can be done is.
- enable dynamic plugins
- disable memberof
- do some work
- enable memberof
- disable dynamic plugins

Please see
https://fedorahosted.org/freeipa/ticket/4203#comment:9
I do not think this will be that easy. We would first need to invest
into

Re: [Freeipa-devel] Provisioning throughput

2016-05-25 Thread thierry bordaz

Hello,

It remains some aspects that are not clear to me:

thanks
thierry

On 05/13/2016 10:18 AM, Ludwig Krispenz wrote:

On 05/13/2016 09:42 AM, Petr Spacek wrote:

On 13.5.2016 09:26, Martin Kosek wrote:

On 05/12/2016 04:16 PM, Ludwig Krispenz wrote:

On 05/12/2016 03:45 PM, Ludwig Krispenz wrote:

On 05/12/2016 02:16 PM, Petr Vobornik wrote:

On 05/10/2016 05:50 PM, thierry bordaz wrote:

On 05/05/2016 03:44 PM, Petr Vobornik wrote:

On 05/04/2016 02:20 PM, thierry bordaz wrote:

Hello,

I have been doing some tests/measures using
https://github.com/freeipa/freeipa-tools/blob/master/create-test-data.py.

The tool creates a set of typical users/hosts/groups... to
import with a
ldapadd.

I wrote down some finding in
http://www.freeipa.org/page/V4/Performance_Improvements#Provisioning_throughput_and_DS_plugins.

I still have to do some cleanup around the performance
but the

basic of a
possible improvement is to do provisioning in several
steps

(disabling
plugins, provisioning, enabling plugin, running fixup
tasks).

Before going further in the design I wanted to share
those ideas

and know if
it raise any concern.

thanks
thierry

Hi Thierry,

Thanks for the analysis. Very nice.

Knowing this will help us suggesting workarounds also for old
releases.

Couple questions:

Have you tested retrCL disabled with memberOf enabled. It seems
that it
would eliminate 550K adds and 0.8M searches. What would be the
time

improvement?

Do you know what is the time when memberof is enabled but
slapi-nis and

retroCL are disabled?

The culprit of the performance issue is very likely related to SRCH
(internal) triggered by memberof.

If retroCL is disabled and memberof enabled, #SRCH is 13.8M.
If retroCL is disabled, slapi-nis disabled and memberof enabled
#SRCH is

14.8
When all of them are enabled the #SRCH is 15M.

You are right if retroCL is disabled the #ADD drops but it has no
significant effect on the duration.

ok, thanks for the analysis

soon as memberof is disabled.

I need to confirm the average time for internal searches but
assuming

1ms per SRCH it consumes >90% of the provisioning.

From the text it was not clear to me, if you find or
investigate

possible improvements in memberof plugin which would improve the
performance without stopping and starting DS.

wait for the restart.
And I think not all plugins are able to handle this, TomasB was
once working

on it for IPA plugins, but it was not completed as far as I know
but enabling dynamic plugins can be done without restart, so what
can be done is.

- enable dynamic plugins
- disable memberof
- do some work
- enable memberof
- disable dynamic plugins

Please see
https://fedorahosted.org/freeipa/ticket/4203#comment:9
I do not think this will be that easy. We would first need to invest
into
updating FreeIPA plugins to work with dynamic plugins setting and
then we could

do things alike above.

It looks like that for FreeIPA 4.4, we will need to live with DS
restart unless

there is some workaround...
couldn't the scenario I outline above with enabling dynamic plugins
only temporary work, are

Re: [Freeipa-devel] Provisioning throughput

2016-05-17 Thread thierry bordaz

On 05/13/2016 10:18 AM, Ludwig Krispenz wrote:

On 05/13/2016 09:42 AM, Petr Spacek wrote:

On 13.5.2016 09:26, Martin Kosek wrote:

On 05/12/2016 04:16 PM, Ludwig Krispenz wrote:

On 05/12/2016 03:45 PM, Ludwig Krispenz wrote:

On 05/12/2016 02:16 PM, Petr Vobornik wrote:

On 05/10/2016 05:50 PM, thierry bordaz wrote:

On 05/05/2016 03:44 PM, Petr Vobornik wrote:

On 05/04/2016 02:20 PM, thierry bordaz wrote:

Hello,

I have been doing some tests/measures using
https://github.com/freeipa/freeipa-tools/blob/master/create-test-data.py.

The tool creates a set of typical users/hosts/groups... to
import with a
ldapadd.

I wrote down some finding in
http://www.freeipa.org/page/V4/Performance_Improvements#Provisioning_throughput_and_DS_plugins.

I still have to do some cleanup around the performance
but the

basic of a
possible improvement is to do provisioning in several
steps

(disabling
plugins, provisioning, enabling plugin, running fixup
tasks).

Before going further in the design I wanted to share
those ideas

and know if
it raise any concern.

thanks
thierry

Hi Thierry,

Thanks for the analysis. Very nice.

Knowing this will help us suggesting workarounds also for old
releases.

Couple questions:

Have you tested retrCL disabled with memberOf enabled. It seems
that it
would eliminate 550K adds and 0.8M searches. What would be the
time

improvement?

Do you know what is the time when memberof is enabled but
slapi-nis and

retroCL are disabled?

The culprit of the performance issue is very likely related to SRCH
(internal) triggered by memberof.

If retroCL is disabled and memberof enabled, #SRCH is 13.8M.
If retroCL is disabled, slapi-nis disabled and memberof enabled
#SRCH is

14.8
When all of them are enabled the #SRCH is 15M.

You are right if retroCL is disabled the #ADD drops but it has no
significant effect on the duration.

ok, thanks for the analysis

soon as memberof is disabled.

I need to confirm the average time for internal searches but
assuming

1ms per SRCH it consumes >90% of the provisioning.

From the text it was not clear to me, if you find or
investigate

possible improvements in memberof plugin which would improve the
performance without stopping and starting DS.

wait for the restart.
And I think not all plugins are able to handle this, TomasB was
once working

on it for IPA plugins, but it was not completed as far as I know
but enabling dynamic plugins can be done without restart, so what
can be done is.

- enable dynamic plugins
- disable memberof
- do some work
- enable memberof
- disable dynamic plugins

do things alike above.

It looks like that for FreeIPA 4.4, we will need to live with DS
restart unless

there is some workaround...
couldn't the scenario I outline above with enabling dynamic plugins
only temporary work, are there any attempts to enable/disable plugins
during provisioning ? If that would be the case that would also
require a restart

One more thing:

How does it affect topologies with replicas?

I might be wrong, but if memberOf is always computed locally then we
have to

disable it on *all* replicas.

If we disabled it only on one replica and not others, the chosen
replica would
be way faster than rest of the topology and I'm not sure what would
happen

later on.
good point. we exclude memberof from replication as it is regenerated
on every server, so each replica would suffer from the performance
problem

Right, that is a very good point.
Provisioning will be slow (through direct update or replicated update)
as soon as memberof is enabled.
An option is to disable memberof only on the server receiving direct
upates, provisioning will be fast, then let the topology converge with
slow replication of the provisioned entries (fixup updates will not be
replicated).
An other option is to disable memberof on all replica, do the
provisioning. Then run fixup on all the replicas. That means we have a
mechanism to detect that all provisioned entries have been replicated
before running fixup.

Thierry, Ludwig, can you comment on this?

--
Manage your subscription for the Freeipa-devel mailing list:

Re: [Freeipa-devel] Provisioning throughput

2016-05-13 Thread Ludwig Krispenz

On 05/13/2016 09:42 AM, Petr Spacek wrote:

On 13.5.2016 09:26, Martin Kosek wrote:

On 05/12/2016 04:16 PM, Ludwig Krispenz wrote:

On 05/12/2016 03:45 PM, Ludwig Krispenz wrote:

On 05/12/2016 02:16 PM, Petr Vobornik wrote:

On 05/10/2016 05:50 PM, thierry bordaz wrote:

On 05/05/2016 03:44 PM, Petr Vobornik wrote:

On 05/04/2016 02:20 PM, thierry bordaz wrote:

Hello,

I have been doing some tests/measures using
https://github.com/freeipa/freeipa-tools/blob/master/create-test-data.py.

The tool creates a set of typical users/hosts/groups... to
import with a
ldapadd.

I wrote down some finding in
http://www.freeipa.org/page/V4/Performance_Improvements#Provisioning_throughput_and_DS_plugins.

I still have to do some cleanup around the performance but the
basic of a
possible improvement is to do provisioning in several steps
(disabling
plugins, provisioning, enabling plugin, running fixup tasks).

Before going further in the design I wanted to share those ideas
and know if
it raise any concern.

thanks
thierry

Hi Thierry,

Thanks for the analysis. Very nice.

Knowing this will help us suggesting workarounds also for old releases.

Couple questions:

Have you tested retrCL disabled with memberOf enabled. It seems that it
would eliminate 550K adds and 0.8M searches. What would be the time
improvement?

Do you know what is the time when memberof is enabled but slapi-nis and
retroCL are disabled?

The culprit of the performance issue is very likely related to SRCH
(internal) triggered by memberof.

If retroCL is disabled and memberof enabled, #SRCH is 13.8M.
If retroCL is disabled, slapi-nis disabled and memberof enabled #SRCH is
14.8
When all of them are enabled the #SRCH is 15M.

You are right if retroCL is disabled the #ADD drops but it has no
significant effect on the duration.

ok, thanks for the analysis

Regarding the duration of the provisioning, values are not really stable
as performance of VM fluctuates. But as soon as memberof is enabled the
provisioning lasts > 4hours where the same provisioning lasts 6mins as
soon as memberof is disabled.

I need to confirm the average time for internal searches but assuming
1ms per SRCH it consumes >90% of the provisioning.

From the text it was not clear to me, if you find or investigate
possible improvements in memberof plugin which would improve the
performance without stopping and starting DS.

As was discussed at mtg, have you tried if the DS restart is really
necessary?

memberof plugin can be enabled and disabled while the server is running, BUT
to achieve this the "enable-dynamic-plugins" feature has to be turned on. And
then any enable/disable of a plugin would try to do it dynamically an dnot
wait for the restart.
And I think not all plugins are able to handle this, TomasB was once working
on it for IPA plugins, but it was not completed as far as I know

but enabling dynamic plugins can be done without restart, so what can be done
is.
- enable dynamic plugins
- disable memberof
- do some work
- enable memberof
- disable dynamic plugins

Please see
https://fedorahosted.org/freeipa/ticket/4203#comment:9
I do not think this will be that easy. We would first need to invest into
updating FreeIPA plugins to work with dynamic plugins setting and then we could
do things alike above.

It looks like that for FreeIPA 4.4, we will need to live with DS restart unless
there is some workaround...
couldn't the scenario I outline above with enabling dynamic plugins only
temporary work, are there any attempts to enable/disable plugins during
provisioning ? If that would be the case that would also require a restart

One more thing:

How does it affect topologies with replicas?

I might be wrong, but if memberOf is always computed locally then we have to
disable it on *all* replicas.

If we disabled it only on one replica and not others, the chosen replica would
be way faster than rest of the topology and I'm not sure what would happen
later on.
good point. we exclude memberof from replication as it is regenerated on
every server, so each replica would suffer from the performance problem

Thierry, Ludwig, can you comment on this?

--
Red Hat GmbH, http://www.de.redhat.com/, Registered seat: Grasbrunn,
Commercial register: Amtsgericht Muenchen, HRB 153243,
Managing Directors: Paul Argiry, Charles Cachera, Michael Cunningham, Michael
O'Neill

--
Manage your subscription for the Freeipa-devel mailing list:
https://www.redhat.com/mailman/listinfo/freeipa-devel
Contribute to FreeIPA: http://www.freeipa.org/page/Contribute/Code

Re: [Freeipa-devel] Provisioning throughput

2016-05-13 Thread Petr Spacek

On 13.5.2016 09:26, Martin Kosek wrote:
> On 05/12/2016 04:16 PM, Ludwig Krispenz wrote:
>>
>> On 05/12/2016 03:45 PM, Ludwig Krispenz wrote:
>>>
>>> On 05/12/2016 02:16 PM, Petr Vobornik wrote:
 On 05/10/2016 05:50 PM, thierry bordaz wrote:
>
> On 05/05/2016 03:44 PM, Petr Vobornik wrote:
>> On 05/04/2016 02:20 PM, thierry bordaz wrote:
>>> Hello,
>>>
>>>   I have been doing some tests/measures using
>>> https://github.com/freeipa/freeipa-tools/blob/master/create-test-data.py.
>>>
>>>   The tool creates a set of typical users/hosts/groups... to
>>> import with a
>>>   ldapadd.
>>>
>>>   I wrote down some finding in
>>> http://www.freeipa.org/page/V4/Performance_Improvements#Provisioning_throughput_and_DS_plugins.
>>>
>>>
>>>   I still have to do some cleanup around the performance but the
>>> basic of a
>>>   possible improvement is to do provisioning in several steps
>>> (disabling
>>>   plugins, provisioning, enabling plugin, running fixup tasks).
>>>
>>>   Before going further in the design I wanted to share those ideas
>>> and know if
>>>   it raise any concern.
>>>
>>>   thanks
>>>   thierry
>>>
>> Hi Thierry,
>>
>> Thanks for the analysis. Very nice.
>>
>> Knowing this will help us suggesting workarounds also for old releases.
>>
>> Couple questions:
>>
>> Have you tested retrCL disabled with memberOf enabled. It seems that it
>> would eliminate 550K adds and 0.8M searches. What would be the time
>> improvement?
>>
>> Do you know what is the time when memberof is enabled but slapi-nis and
>> retroCL are disabled?
> The culprit of the performance issue is very likely related to SRCH
> (internal) triggered by memberof.
>
> If retroCL is disabled and memberof enabled, #SRCH is 13.8M.
> If retroCL is disabled, slapi-nis disabled and memberof enabled #SRCH is
> 14.8
> When all of them are enabled the #SRCH is 15M.
>
> You are right if retroCL is disabled the #ADD drops but it has no
> significant effect on the duration.
 ok, thanks for the analysis

> Regarding the duration of the provisioning, values are not really stable
> as performance of VM fluctuates. But as soon as memberof is enabled the
> provisioning lasts > 4hours where the same provisioning lasts 6mins as
> soon as memberof is disabled.
>
> I need to confirm the average time for internal searches but assuming
> 1ms per SRCH it consumes >90% of the provisioning.
>
>
>>   From the text it was not clear to me, if you find or investigate
>> possible improvements in memberof plugin which would improve the
>> performance without stopping and starting DS.
 As was discussed at mtg, have you tried if the DS restart is really
 necessary?
>>> memberof plugin can be enabled and disabled while the server is running, BUT
>>> to achieve this the "enable-dynamic-plugins" feature has to be turned on. 
>>> And
>>> then any enable/disable of a plugin would try to do it dynamically an dnot
>>> wait for the restart.
>>> And I think not all plugins are able to handle this, TomasB was once working
>>> on it for IPA plugins, but it was not completed as far as I know
>> but enabling dynamic plugins can be done without restart, so what can be 
>> done is.
>> - enable dynamic plugins
>> - disable memberof
>> - do some work
>> - enable memberof
>> - disable dynamic plugins
> 
> Please see
> https://fedorahosted.org/freeipa/ticket/4203#comment:9
> I do not think this will be that easy. We would first need to invest into
> updating FreeIPA plugins to work with dynamic plugins setting and then we 
> could
> do things alike above.
> 
> It looks like that for FreeIPA 4.4, we will need to live with DS restart 
> unless
> there is some workaround...

One more thing:

How does it affect topologies with replicas?

I might be wrong, but if memberOf is always computed locally then we have to
disable it on *all* replicas.

If we disabled it only on one replica and not others, the chosen replica would
be way faster than rest of the topology and I'm not sure what would happen
later on.

Thierry, Ludwig, can you comment on this?

-- 
Petr^2 Spacek

-- 
Manage your subscription for the Freeipa-devel mailing list:
https://www.redhat.com/mailman/listinfo/freeipa-devel
Contribute to FreeIPA: http://www.freeipa.org/page/Contribute/Code

Re: [Freeipa-devel] Provisioning throughput

2016-05-13 Thread Martin Kosek

On 05/12/2016 04:16 PM, Ludwig Krispenz wrote:
> 
> On 05/12/2016 03:45 PM, Ludwig Krispenz wrote:
>>
>> On 05/12/2016 02:16 PM, Petr Vobornik wrote:
>>> On 05/10/2016 05:50 PM, thierry bordaz wrote:

 On 05/05/2016 03:44 PM, Petr Vobornik wrote:
> On 05/04/2016 02:20 PM, thierry bordaz wrote:
>> Hello,
>>
>>   I have been doing some tests/measures using
>> https://github.com/freeipa/freeipa-tools/blob/master/create-test-data.py.
>>
>>   The tool creates a set of typical users/hosts/groups... to
>> import with a
>>   ldapadd.
>>
>>   I wrote down some finding in
>> http://www.freeipa.org/page/V4/Performance_Improvements#Provisioning_throughput_and_DS_plugins.
>>
>>
>>   I still have to do some cleanup around the performance but the
>> basic of a
>>   possible improvement is to do provisioning in several steps
>> (disabling
>>   plugins, provisioning, enabling plugin, running fixup tasks).
>>
>>   Before going further in the design I wanted to share those ideas
>> and know if
>>   it raise any concern.
>>
>>   thanks
>>   thierry
>>
> Hi Thierry,
>
> Thanks for the analysis. Very nice.
>
> Knowing this will help us suggesting workarounds also for old releases.
>
> Couple questions:
>
> Have you tested retrCL disabled with memberOf enabled. It seems that it
> would eliminate 550K adds and 0.8M searches. What would be the time
> improvement?
>
> Do you know what is the time when memberof is enabled but slapi-nis and
> retroCL are disabled?
 The culprit of the performance issue is very likely related to SRCH
 (internal) triggered by memberof.

 If retroCL is disabled and memberof enabled, #SRCH is 13.8M.
 If retroCL is disabled, slapi-nis disabled and memberof enabled #SRCH is
 14.8
 When all of them are enabled the #SRCH is 15M.

 You are right if retroCL is disabled the #ADD drops but it has no
 significant effect on the duration.
>>> ok, thanks for the analysis
>>>
 Regarding the duration of the provisioning, values are not really stable
 as performance of VM fluctuates. But as soon as memberof is enabled the
 provisioning lasts > 4hours where the same provisioning lasts 6mins as
 soon as memberof is disabled.

 I need to confirm the average time for internal searches but assuming
 1ms per SRCH it consumes >90% of the provisioning.


>   From the text it was not clear to me, if you find or investigate
> possible improvements in memberof plugin which would improve the
> performance without stopping and starting DS.
>>> As was discussed at mtg, have you tried if the DS restart is really
>>> necessary?
>> memberof plugin can be enabled and disabled while the server is running, BUT
>> to achieve this the "enable-dynamic-plugins" feature has to be turned on. And
>> then any enable/disable of a plugin would try to do it dynamically an dnot
>> wait for the restart.
>> And I think not all plugins are able to handle this, TomasB was once working
>> on it for IPA plugins, but it was not completed as far as I know
> but enabling dynamic plugins can be done without restart, so what can be done 
> is.
> - enable dynamic plugins
> - disable memberof
> - do some work
> - enable memberof
> - disable dynamic plugins

Please see
https://fedorahosted.org/freeipa/ticket/4203#comment:9
I do not think this will be that easy. We would first need to invest into
updating FreeIPA plugins to work with dynamic plugins setting and then we could
do things alike above.

It looks like that for FreeIPA 4.4, we will need to live with DS restart unless
there is some workaround...

Martin

-- 
Manage your subscription for the Freeipa-devel mailing list:
https://www.redhat.com/mailman/listinfo/freeipa-devel
Contribute to FreeIPA: http://www.freeipa.org/page/Contribute/Code

Re: [Freeipa-devel] Provisioning throughput

2016-05-12 Thread Ludwig Krispenz

On 05/12/2016 03:45 PM, Ludwig Krispenz wrote:

On 05/12/2016 02:16 PM, Petr Vobornik wrote:

On 05/10/2016 05:50 PM, thierry bordaz wrote:

On 05/05/2016 03:44 PM, Petr Vobornik wrote:

On 05/04/2016 02:20 PM, thierry bordaz wrote:

Hello,

I have been doing some tests/measures using
https://github.com/freeipa/freeipa-tools/blob/master/create-test-data.py.

The tool creates a set of typical users/hosts/groups... to
import with a
ldapadd.

I wrote down some finding in
http://www.freeipa.org/page/V4/Performance_Improvements#Provisioning_throughput_and_DS_plugins.

I still have to do some cleanup around the performance but the
basic of a
possible improvement is to do provisioning in several steps
(disabling
plugins, provisioning, enabling plugin, running fixup tasks).

Before going further in the design I wanted to share those
ideas

and know if
it raise any concern.

thanks
thierry

Hi Thierry,

Thanks for the analysis. Very nice.

Knowing this will help us suggesting workarounds also for old
releases.

Couple questions:

Have you tested retrCL disabled with memberOf enabled. It seems
that it

would eliminate 550K adds and 0.8M searches. What would be the time
improvement?

Do you know what is the time when memberof is enabled but slapi-nis
and

retroCL are disabled?

The culprit of the performance issue is very likely related to SRCH
(internal) triggered by memberof.

If retroCL is disabled and memberof enabled, #SRCH is 13.8M.
If retroCL is disabled, slapi-nis disabled and memberof enabled
#SRCH is

14.8
When all of them are enabled the #SRCH is 15M.

You are right if retroCL is disabled the #ADD drops but it has no
significant effect on the duration.

ok, thanks for the analysis

Regarding the duration of the provisioning, values are not really
stable

as performance of VM fluctuates. But as soon as memberof is enabled the
provisioning lasts > 4hours where the same provisioning lasts 6mins as
soon as memberof is disabled.

I need to confirm the average time for internal searches but assuming
1ms per SRCH it consumes >90% of the provisioning.

From the text it was not clear to me, if you find or investigate
possible improvements in memberof plugin which would improve the
performance without stopping and starting DS.

As was discussed at mtg, have you tried if the DS restart is really
necessary?
memberof plugin can be enabled and disabled while the server is
running, BUT
to achieve this the "enable-dynamic-plugins" feature has to be turned
on. And then any enable/disable of a plugin would try to do it
dynamically an dnot wait for the restart.
And I think not all plugins are able to handle this, TomasB was once
working on it for IPA plugins, but it was not completed as far as I know
but enabling dynamic plugins can be done without restart, so what can be
done is.

- enable dynamic plugins
- disable memberof
- do some work
- enable memberof
- disable dynamic plugins

And if it is required, what would be needed to not require restart.

The workaround should be easy to use.

--
Manage your subscription for the Freeipa-devel mailing list:
https://www.redhat.com/mailman/listinfo/freeipa-devel
Contribute to FreeIPA: http://www.freeipa.org/page/Contribute/Code

Re: [Freeipa-devel] Provisioning throughput

2016-05-12 Thread Ludwig Krispenz

On 05/12/2016 02:16 PM, Petr Vobornik wrote:

On 05/10/2016 05:50 PM, thierry bordaz wrote:

On 05/05/2016 03:44 PM, Petr Vobornik wrote:

On 05/04/2016 02:20 PM, thierry bordaz wrote:

Hello,

I have been doing some tests/measures using

https://github.com/freeipa/freeipa-tools/blob/master/create-test-data.py.

The tool creates a set of typical users/hosts/groups... to
import with a
ldapadd.

I wrote down some finding in

http://www.freeipa.org/page/V4/Performance_Improvements#Provisioning_throughput_and_DS_plugins.

I still have to do some cleanup around the performance but the
basic of a
possible improvement is to do provisioning in several steps
(disabling
plugins, provisioning, enabling plugin, running fixup tasks).

Before going further in the design I wanted to share those ideas
and know if
it raise any concern.

thanks
thierry

Hi Thierry,

Thanks for the analysis. Very nice.

Knowing this will help us suggesting workarounds also for old releases.

Couple questions:

Have you tested retrCL disabled with memberOf enabled. It seems that it
would eliminate 550K adds and 0.8M searches. What would be the time
improvement?

Do you know what is the time when memberof is enabled but slapi-nis and
retroCL are disabled?

The culprit of the performance issue is very likely related to SRCH
(internal) triggered by memberof.

If retroCL is disabled and memberof enabled, #SRCH is 13.8M.
If retroCL is disabled, slapi-nis disabled and memberof enabled #SRCH is
14.8
When all of them are enabled the #SRCH is 15M.

You are right if retroCL is disabled the #ADD drops but it has no
significant effect on the duration.

ok, thanks for the analysis

Regarding the duration of the provisioning, values are not really stable
as performance of VM fluctuates. But as soon as memberof is enabled the
provisioning lasts > 4hours where the same provisioning lasts 6mins as
soon as memberof is disabled.

I need to confirm the average time for internal searches but assuming
1ms per SRCH it consumes >90% of the provisioning.

From the text it was not clear to me, if you find or investigate
possible improvements in memberof plugin which would improve the
performance without stopping and starting DS.

As was discussed at mtg, have you tried if the DS restart is really
necessary?

memberof plugin can be enabled and disabled while the server is running, BUT
to achieve this the "enable-dynamic-plugins" feature has to be turned
on. And then any enable/disable of a plugin would try to do it
dynamically an dnot wait for the restart.
And I think not all plugins are able to handle this, TomasB was once
working on it for IPA plugins, but it was not completed as far as I know

And if it is required, what would be needed to not require restart.

The workaround should be easy to use.

--
Manage your subscription for the Freeipa-devel mailing list:
https://www.redhat.com/mailman/listinfo/freeipa-devel
Contribute to FreeIPA: http://www.freeipa.org/page/Contribute/Code

Re: [Freeipa-devel] Provisioning throughput

2016-05-12 Thread Petr Vobornik

On 05/10/2016 05:50 PM, thierry bordaz wrote:
> 
> 
> On 05/05/2016 03:44 PM, Petr Vobornik wrote:
>> On 05/04/2016 02:20 PM, thierry bordaz wrote:
>>> Hello,
>>>
>>>  I have been doing some tests/measures using
>>> 
>>> https://github.com/freeipa/freeipa-tools/blob/master/create-test-data.py.
>>>
>>>  The tool creates a set of typical users/hosts/groups... to
>>> import with a
>>>  ldapadd.
>>>
>>>  I wrote down some finding in
>>> 
>>> http://www.freeipa.org/page/V4/Performance_Improvements#Provisioning_throughput_and_DS_plugins.
>>>
>>>  I still have to do some cleanup around the performance but the
>>> basic of a
>>>  possible improvement is to do provisioning in several steps
>>> (disabling
>>>  plugins, provisioning, enabling plugin, running fixup tasks).
>>>
>>>  Before going further in the design I wanted to share those ideas
>>> and know if
>>>  it raise any concern.
>>>
>>>  thanks
>>>  thierry
>>>
>> Hi Thierry,
>>
>> Thanks for the analysis. Very nice.
>>
>> Knowing this will help us suggesting workarounds also for old releases.
>>
>> Couple questions:
>>
>> Have you tested retrCL disabled with memberOf enabled. It seems that it
>> would eliminate 550K adds and 0.8M searches. What would be the time
>> improvement?
>>
>> Do you know what is the time when memberof is enabled but slapi-nis and
>> retroCL are disabled?
> The culprit of the performance issue is very likely related to SRCH
> (internal) triggered by memberof.
> 
> If retroCL is disabled and memberof enabled, #SRCH is 13.8M.
> If retroCL is disabled, slapi-nis disabled and memberof enabled #SRCH is
> 14.8
> When all of them are enabled the #SRCH is 15M.
> 
> You are right if retroCL is disabled the #ADD drops but it has no
> significant effect on the duration.

ok, thanks for the analysis

> 
> Regarding the duration of the provisioning, values are not really stable
> as performance of VM fluctuates. But as soon as memberof is enabled the
> provisioning lasts > 4hours where the same provisioning lasts 6mins as
> soon as memberof is disabled.
> 
> I need to confirm the average time for internal searches but assuming
> 1ms per SRCH it consumes >90% of the provisioning.
> 
> 
>>
>>  From the text it was not clear to me, if you find or investigate
>> possible improvements in memberof plugin which would improve the
>> performance without stopping and starting DS.
> 

As was discussed at mtg, have you tried if the DS restart is really
necessary?

And if it is required, what would be needed to not require restart.

The workaround should be easy to use.
-- 
Petr Vobornik

-- 
Manage your subscription for the Freeipa-devel mailing list:
https://www.redhat.com/mailman/listinfo/freeipa-devel
Contribute to FreeIPA: http://www.freeipa.org/page/Contribute/Code

Re: [Freeipa-devel] Provisioning throughput

2016-05-10 Thread thierry bordaz

On 05/05/2016 03:44 PM, Petr Vobornik wrote:

On 05/04/2016 02:20 PM, thierry bordaz wrote:

Hello,

I have been doing some tests/measures using
https://github.com/freeipa/freeipa-tools/blob/master/create-test-data.py.
The tool creates a set of typical users/hosts/groups... to import with a
ldapadd.

I wrote down some finding in

http://www.freeipa.org/page/V4/Performance_Improvements#Provisioning_throughput_and_DS_plugins.
I still have to do some cleanup around the performance but the basic of a
possible improvement is to do provisioning in several steps (disabling
plugins, provisioning, enabling plugin, running fixup tasks).

Before going further in the design I wanted to share those ideas and know
if
it raise any concern.

thanks
thierry

Hi Thierry,

Thanks for the analysis. Very nice.

Knowing this will help us suggesting workarounds also for old releases.

Couple questions:

Have you tested retrCL disabled with memberOf enabled. It seems that it
would eliminate 550K adds and 0.8M searches. What would be the time
improvement?

Do you know what is the time when memberof is enabled but slapi-nis and
retroCL are disabled?
The culprit of the performance issue is very likely related to SRCH
(internal) triggered by memberof.

If retroCL is disabled and memberof enabled, #SRCH is 13.8M.
If retroCL is disabled, slapi-nis disabled and memberof enabled #SRCH is
14.8

When all of them are enabled the #SRCH is 15M.

You are right if retroCL is disabled the #ADD drops but it has no
significant effect on the duration.

Regarding the duration of the provisioning, values are not really stable
as performance of VM fluctuates. But as soon as memberof is enabled the
provisioning lasts > 4hours where the same provisioning lasts 6mins as
soon as memberof is disabled.

I need to confirm the average time for internal searches but assuming
1ms per SRCH it consumes >90% of the provisioning.

From the text it was not clear to me, if you find or investigate
possible improvements in memberof plugin which would improve the
performance without stopping and starting DS.

--
Manage your subscription for the Freeipa-devel mailing list:
https://www.redhat.com/mailman/listinfo/freeipa-devel
Contribute to FreeIPA: http://www.freeipa.org/page/Contribute/Code

Re: [Freeipa-devel] Provisioning throughput

2016-05-09 Thread thierry bordaz

On 05/05/2016 03:44 PM, Petr Vobornik wrote:

On 05/04/2016 02:20 PM, thierry bordaz wrote:

Hello,

I have been doing some tests/measures using
https://github.com/freeipa/freeipa-tools/blob/master/create-test-data.py.
The tool creates a set of typical users/hosts/groups... to import with a
ldapadd.

I wrote down some finding in

Before going further in the design I wanted to share those ideas and know
if
it raise any concern.

thanks
thierry

Hi Thierry,

Thanks for the analysis. Very nice.

Knowing this will help us suggesting workarounds also for old releases.

Couple questions:

Have you tested retrCL disabled with memberOf enabled. It seems that it
would eliminate 550K adds and 0.8M searches. What would be the time
improvement?

Basically provisioning/migrate does not scale well because of memberOf.

Each time you add a member to a group you need to reevalute its
membership (15M search) and finally update the member (that is a MOD for
the member + ADD in retroCL). If a same entry is added to a group and
later added to an other group, the first evaluation/updates are useless.

Disabling memberof drops SRCH from 15M -> ~1M. You can get an additional
gain ~1M -> ~0.1M if you disable retroCL.

I tested retroCL disabled / memberOf enabled but on a different dataset.
It gaves low improvement in term of duration.

I think MODs are here more expensive that ADD.
I will test your suggestion to confirm the numbers.

Do you know what is the time when memberof is enabled but slapi-nis and
retroCL are disabled?

No. I will test it.

From the text it was not clear to me, if you find or investigate
possible improvements in memberof plugin which would improve the
performance without stopping and starting DS.

memberof does not scale well because of #SRCH and #MOD.
Ludwig suggested a group caching that is looking close to other
in-memory implementations of memberof.
Such cache would reduce the #SRCH possibly from 15M->1M, but the
implementation is not immediate.
Also note that filling the cache has a cost that can delay startup (like
we saw in slapi-nis) and a memory footprint.

Regarding the MODs we can make memberof attribute a virtual attribute
and stop updating the entries. This is also not immediate to implement:
rewrite memberof plugin.

From 7.3 schedule, I think we can get an important improvement if DS
start-stop is acceptable.

Improving memberof is more a longer term option.

thanks
thierry

--
Manage your subscription for the Freeipa-devel mailing list:
https://www.redhat.com/mailman/listinfo/freeipa-devel
Contribute to FreeIPA: http://www.freeipa.org/page/Contribute/Code

Re: [Freeipa-devel] Provisioning throughput

2016-05-05 Thread Petr Vobornik

On 05/04/2016 02:20 PM, thierry bordaz wrote:
> Hello,
> 
> I have been doing some tests/measures using
> https://github.com/freeipa/freeipa-tools/blob/master/create-test-data.py.
> The tool creates a set of typical users/hosts/groups... to import with a
> ldapadd.
> 
> I wrote down some finding in
> 
> http://www.freeipa.org/page/V4/Performance_Improvements#Provisioning_throughput_and_DS_plugins.
> I still have to do some cleanup around the performance but the basic of a
> possible improvement is to do provisioning in several steps (disabling
> plugins, provisioning, enabling plugin, running fixup tasks).
> 
> Before going further in the design I wanted to share those ideas and know 
> if
> it raise any concern.
> 
> thanks
> thierry
> 

Hi Thierry,

Thanks for the analysis. Very nice.

Knowing this will help us suggesting workarounds also for old releases.

Couple questions:

Have you tested retrCL disabled with memberOf enabled. It seems that it
would eliminate 550K adds and 0.8M searches. What would be the time
improvement?

Do you know what is the time when memberof is enabled but slapi-nis and
retroCL are disabled?

>From the text it was not clear to me, if you find or investigate
possible improvements in memberof plugin which would improve the
performance without stopping and starting DS.
-- 
Petr Vobornik

-- 
Manage your subscription for the Freeipa-devel mailing list:
https://www.redhat.com/mailman/listinfo/freeipa-devel
Contribute to FreeIPA: http://www.freeipa.org/page/Contribute/Code

[Freeipa-devel] Provisioning throughput

2016-05-04 Thread thierry bordaz


Hello,

   I have been doing some tests/measures using
   https://github.com/freeipa/freeipa-tools/blob/master/create-test-data.py.
   The tool creates a set of typical users/hosts/groups... to import
   with a ldapadd.

   I wrote down some finding in
   
http://www.freeipa.org/page/V4/Performance_Improvements#Provisioning_throughput_and_DS_plugins.
   I still have to do some cleanup around the performance but the basic
   of a possible improvement is to do provisioning in several steps
   (disabling plugins, provisioning, enabling plugin, running fixup tasks).

   Before going further in the design I wanted to share those ideas and
   know if it raise any concern.

   thanks
   thierry

-- 
Manage your subscription for the Freeipa-devel mailing list:
https://www.redhat.com/mailman/listinfo/freeipa-devel
Contribute to FreeIPA: http://www.freeipa.org/page/Contribute/Code

Re: [Freeipa-devel] Provisioning throughput

Re: [Freeipa-devel] Provisioning throughput

Re: [Freeipa-devel] Provisioning throughput

Re: [Freeipa-devel] Provisioning throughput

Re: [Freeipa-devel] Provisioning throughput

Re: [Freeipa-devel] Provisioning throughput

Re: [Freeipa-devel] Provisioning throughput

Re: [Freeipa-devel] Provisioning throughput

Re: [Freeipa-devel] Provisioning throughput

Re: [Freeipa-devel] Provisioning throughput

Re: [Freeipa-devel] Provisioning throughput

Re: [Freeipa-devel] Provisioning throughput

Re: [Freeipa-devel] Provisioning throughput

Re: [Freeipa-devel] Provisioning throughput

Re: [Freeipa-devel] Provisioning throughput

Re: [Freeipa-devel] Provisioning throughput

Re: [Freeipa-devel] Provisioning throughput

Re: [Freeipa-devel] Provisioning throughput

Re: [Freeipa-devel] Provisioning throughput

Re: [Freeipa-devel] Provisioning throughput

Re: [Freeipa-devel] Provisioning throughput

Re: [Freeipa-devel] Provisioning throughput

Re: [Freeipa-devel] Provisioning throughput

Re: [Freeipa-devel] Provisioning throughput

Re: [Freeipa-devel] Provisioning throughput

Re: [Freeipa-devel] Provisioning throughput

[Freeipa-devel] Provisioning throughput

27 matches

Site Navigation

Mail list logo

Footer information