Re: [Freeipa-devel] [PATCH] test speedup patch
On 01/19/2011 04:17 AM, Rob Crittenden wrote: Rob Crittenden wrote: Rob Crittenden wrote: Attached is a rough cut of a patch to try to speed up the cli a little bit. Basically in production mode it will skip some things during initialization. My concept is that we develop in mode != production and release in mode == production. I managed to knock a second or so off time to do a user-show on average. There may be some other things we can do to speed things up, I'm still looking. Some feedback on the approach would be appreciated. Note that I've completely ruled out SSL/Negotiate. I did my testing on lite-server which doesn't use SSL or Negotiate and it was STILL taking on average 3-4+ seconds per command. The server side was consistently taking 1 second to complete. rob oh, and the patch. I ran a couple of moderate tests this evening that executed 42 separate operations like add, delete, and managing group membership. I ran this 10 times each on 2 identical VMs, one with a bit older code and one with this patch then averaged the times. With the patch the average was 1.3 seconds per operation, without 2.6. A 50% improvement is more than I expected, I saw a 33% improvement on individual runs. I'll keep at it but this seems promising. I was also a bit surprised that the average time without the patch was so low, I was expecting something over 3 seconds. Specifically what this patch does is it avoids doing some self-validation. There is some amount of risk that the framework could blow up but in a deployed situation I think the risk is rather low. A side-effect of the API tester makeapi is that it loads the framework. We can force it to be run in production mode so the product shouldn't be buildable if it has inconsistencies. rob I find it hard to believe this patch causes such a big improvement in performance. Especially the parts skipping asserts, that shouldn't be significantly slower than your average ifs. Instance locking shouldn't be a time consuming operation either. Bypassing check routines for parameter namespaces might provide a performance boost as it is called for every single plugin we have (~250). On the other hand, it is only used for positional arguments and most plugins only have 1 or 2 of those. Personally, I would do some more tests on a single machine, because there's no guarantee, that two VMs with an identical image have the same performance. If it really provides a significant improvement, than it's awesome, because I like the philosophy of this patch. It removes self-checking and instance locking, that is completely useless in a production environment and kind of limiting in non-production. I think there's more places likes this in the framework. Long story short: It's improbable, but not impossible, for the changes introduced by this patch to cause such a big performance improvement. Even if it doesn't, the patch is still good. Pavel ___ Freeipa-devel mailing list Freeipa-devel@redhat.com https://www.redhat.com/mailman/listinfo/freeipa-devel
Re: [Freeipa-devel] [PATCH] test speedup patch
Pavel Zuna wrote: On 01/19/2011 04:17 AM, Rob Crittenden wrote: Rob Crittenden wrote: Rob Crittenden wrote: Attached is a rough cut of a patch to try to speed up the cli a little bit. Basically in production mode it will skip some things during initialization. My concept is that we develop in mode != production and release in mode == production. I managed to knock a second or so off time to do a user-show on average. There may be some other things we can do to speed things up, I'm still looking. Some feedback on the approach would be appreciated. Note that I've completely ruled out SSL/Negotiate. I did my testing on lite-server which doesn't use SSL or Negotiate and it was STILL taking on average 3-4+ seconds per command. The server side was consistently taking 1 second to complete. rob oh, and the patch. I ran a couple of moderate tests this evening that executed 42 separate operations like add, delete, and managing group membership. I ran this 10 times each on 2 identical VMs, one with a bit older code and one with this patch then averaged the times. With the patch the average was 1.3 seconds per operation, without 2.6. A 50% improvement is more than I expected, I saw a 33% improvement on individual runs. I'll keep at it but this seems promising. I was also a bit surprised that the average time without the patch was so low, I was expecting something over 3 seconds. Specifically what this patch does is it avoids doing some self-validation. There is some amount of risk that the framework could blow up but in a deployed situation I think the risk is rather low. A side-effect of the API tester makeapi is that it loads the framework. We can force it to be run in production mode so the product shouldn't be buildable if it has inconsistencies. rob I find it hard to believe this patch causes such a big improvement in performance. Especially the parts skipping asserts, that shouldn't be significantly slower than your average ifs. Instance locking shouldn't be a time consuming operation either. I'm equally unbelieving! In my dev tree I was seeing 3.9 second+ times without the patch. With the patch I was still often seeing 3+ seconds but sometimes it would be significantly lower. This is a VM so who knows what other stuff is going on. I agree that locking shouldn't be that slow but cumulatively it is a bit of a drag. Similarly the assert p.instance.api is self is quite slow. Bypassing check routines for parameter namespaces might provide a performance boost as it is called for every single plugin we have (~250). On the other hand, it is only used for positional arguments and most plugins only have 1 or 2 of those. I believe that bypassing the check routines is where the biggest boost comes from. check_name() is also skipped and that does a regex match. Personally, I would do some more tests on a single machine, because there's no guarantee, that two VMs with an identical image have the same performance. Using 2 machines lets me test code without having to constantly re-install the bits or apply patches. It is easier to track the state so I don't end up with meaningless data :-) I'll stick the meager tests I have into a while loop and let it run for longer than 10 iterations to get a better idea of what is going on. If it really provides a significant improvement, than it's awesome, because I like the philosophy of this patch. It removes self-checking and instance locking, that is completely useless in a production environment and kind of limiting in non-production. I think there's more places likes this in the framework. Long story short: It's improbable, but not impossible, for the changes introduced by this patch to cause such a big performance improvement. Even if it doesn't, the patch is still good. Well, this is why I sent the patch out for a pre-review :-) rob ___ Freeipa-devel mailing list Freeipa-devel@redhat.com https://www.redhat.com/mailman/listinfo/freeipa-devel
Re: [Freeipa-devel] [PATCH] test speedup patch
Just tested. I do see a performance increase of ~30% Without the Patch time ipa user-find -- 1 user matched -- User login: admin Last name: Administrator Home directory: /home/admin Login shell: /bin/bash Account activation status: False Member of groups: admins Number of entries returned 1 real 0m1.558s user 0m0.810s sys 0m0.165s With the Patch time ipa user-find -- 1 user matched -- User login: admin Last name: Administrator Home directory: /home/admin Login shell: /bin/bash Account disabled: False Member of groups: admins Number of entries returned 1 real 0m1.067s user 0m0.841s sys 0m0.157s On 1/18/11 7:17 PM, Rob Crittenden rcrit...@redhat.com wrote: Rob Crittenden wrote: Rob Crittenden wrote: Attached is a rough cut of a patch to try to speed up the cli a little bit. Basically in production mode it will skip some things during initialization. My concept is that we develop in mode != production and release in mode == production. I managed to knock a second or so off time to do a user-show on average. There may be some other things we can do to speed things up, I'm still looking. Some feedback on the approach would be appreciated. Note that I've completely ruled out SSL/Negotiate. I did my testing on lite-server which doesn't use SSL or Negotiate and it was STILL taking on average 3-4+ seconds per command. The server side was consistently taking 1 second to complete. rob oh, and the patch. I ran a couple of moderate tests this evening that executed 42 separate operations like add, delete, and managing group membership. I ran this 10 times each on 2 identical VMs, one with a bit older code and one with this patch then averaged the times. With the patch the average was 1.3 seconds per operation, without 2.6. A 50% improvement is more than I expected, I saw a 33% improvement on individual runs. I'll keep at it but this seems promising. I was also a bit surprised that the average time without the patch was so low, I was expecting something over 3 seconds. Specifically what this patch does is it avoids doing some self-validation. There is some amount of risk that the framework could blow up but in a deployed situation I think the risk is rather low. A side-effect of the API tester makeapi is that it loads the framework. We can force it to be run in production mode so the product shouldn't be buildable if it has inconsistencies. rob ___ Freeipa-devel mailing list Freeipa-devel@redhat.com https://www.redhat.com/mailman/listinfo/freeipa-devel ___ Freeipa-devel mailing list Freeipa-devel@redhat.com https://www.redhat.com/mailman/listinfo/freeipa-devel
Re: [Freeipa-devel] [PATCH] test speedup patch
On 01/19/2011 10:26 AM, JR Aquino wrote: Just tested. I do see a performance increase of ~30% Without the Patch time ipa user-find -- 1 user matched -- User login: admin Last name: Administrator Home directory: /home/admin Login shell: /bin/bash Account activation status: False Member of groups: admins Number of entries returned 1 real 0m1.558s user 0m0.810s sys 0m0.165s With the Patch time ipa user-find -- 1 user matched -- User login: admin Last name: Administrator Home directory: /home/admin Login shell: /bin/bash Account disabled: False Member of groups: admins Number of entries returned 1 real 0m1.067s user 0m0.841s sys 0m0.157s On 1/18/11 7:17 PM, Rob Crittendenrcrit...@redhat.com wrote: Rob Crittenden wrote: Rob Crittenden wrote: Attached is a rough cut of a patch to try to speed up the cli a little bit. Basically in production mode it will skip some things during initialization. My concept is that we develop in mode != production and release in mode == production. I managed to knock a second or so off time to do a user-show on average. There may be some other things we can do to speed things up, I'm still looking. Some feedback on the approach would be appreciated. Note that I've completely ruled out SSL/Negotiate. I did my testing on lite-server which doesn't use SSL or Negotiate and it was STILL taking on average 3-4+ seconds per command. The server side was consistently taking 1 second to complete. rob oh, and the patch. I ran a couple of moderate tests this evening that executed 42 separate operations like add, delete, and managing group membership. I ran this 10 times each on 2 identical VMs, one with a bit older code and one with this patch then averaged the times. With the patch the average was 1.3 seconds per operation, without 2.6. A 50% improvement is more than I expected, I saw a 33% improvement on individual runs. I'll keep at it but this seems promising. I was also a bit surprised that the average time without the patch was so low, I was expecting something over 3 seconds. Specifically what this patch does is it avoids doing some self-validation. There is some amount of risk that the framework could blow up but in a deployed situation I think the risk is rather low. A side-effect of the API tester makeapi is that it loads the framework. We can force it to be run in production mode so the product shouldn't be buildable if it has inconsistencies. rob ___ Freeipa-devel mailing list Freeipa-devel@redhat.com https://www.redhat.com/mailman/listinfo/freeipa-devel ___ Freeipa-devel mailing list Freeipa-devel@redhat.com https://www.redhat.com/mailman/listinfo/freeipa-devel ACK ___ Freeipa-devel mailing list Freeipa-devel@redhat.com https://www.redhat.com/mailman/listinfo/freeipa-devel
[Freeipa-devel] [PATCH] test speedup patch
Attached is a rough cut of a patch to try to speed up the cli a little bit. Basically in production mode it will skip some things during initialization. My concept is that we develop in mode != production and release in mode == production. I managed to knock a second or so off time to do a user-show on average. There may be some other things we can do to speed things up, I'm still looking. Some feedback on the approach would be appreciated. Note that I've completely ruled out SSL/Negotiate. I did my testing on lite-server which doesn't use SSL or Negotiate and it was STILL taking on average 3-4+ seconds per command. The server side was consistently taking 1 second to complete. rob ___ Freeipa-devel mailing list Freeipa-devel@redhat.com https://www.redhat.com/mailman/listinfo/freeipa-devel
Re: [Freeipa-devel] [PATCH] test speedup patch
Rob Crittenden wrote: Attached is a rough cut of a patch to try to speed up the cli a little bit. Basically in production mode it will skip some things during initialization. My concept is that we develop in mode != production and release in mode == production. I managed to knock a second or so off time to do a user-show on average. There may be some other things we can do to speed things up, I'm still looking. Some feedback on the approach would be appreciated. Note that I've completely ruled out SSL/Negotiate. I did my testing on lite-server which doesn't use SSL or Negotiate and it was STILL taking on average 3-4+ seconds per command. The server side was consistently taking 1 second to complete. rob oh, and the patch. From 75dcb2e51defdfed9d5adc44c70e9fdb23995ef3 Mon Sep 17 00:00:00 2001 From: Rob Crittenden rcrit...@redhat.com Date: Tue, 18 Jan 2011 17:23:54 -0500 Subject: [PATCH] Attempt #1 at speeding up cli --- ipalib/frontend.py |7 --- ipalib/plugable.py |8 ++-- 2 files changed, 10 insertions(+), 5 deletions(-) diff --git a/ipalib/frontend.py b/ipalib/frontend.py index eeed398..51fa524 100644 --- a/ipalib/frontend.py +++ b/ipalib/frontend.py @@ -351,9 +351,10 @@ class HasParam(Plugin): self._filter_param_by_context(name, env), sort=False ) -check = getattr(self, 'check_' + name, None) -if callable(check): -check(namespace) +if self.env.mode != 'production': +check = getattr(self, 'check_' + name, None) +if callable(check): +check(namespace) setattr(self, name, namespace) diff --git a/ipalib/plugable.py b/ipalib/plugable.py index 264bb68..93fc177 100644 --- a/ipalib/plugable.py +++ b/ipalib/plugable.py @@ -207,6 +207,8 @@ class Plugin(ReadOnly): def finalize(self): +if self.env.mode == 'production': +return lock(self) def set_api(self, api): @@ -609,11 +611,13 @@ class API(DictProxy): for p in plugins.itervalues(): p.instance.set_api(self) -assert p.instance.api is self +if self.env.mode != 'production': +assert p.instance.api is self for p in plugins.itervalues(): p.instance.finalize() -assert islocked(p.instance) is True +if self.env.mode != 'production': +assert islocked(p.instance) is True object.__setattr__(self, '_API__finalized', True) tuple(PluginInfo(p) for p in plugins.itervalues()) object.__setattr__(self, 'plugins', -- 1.7.3.4 ___ Freeipa-devel mailing list Freeipa-devel@redhat.com https://www.redhat.com/mailman/listinfo/freeipa-devel
Re: [Freeipa-devel] [PATCH] test speedup patch
On 01/18/2011 05:35 PM, Rob Crittenden wrote: Rob Crittenden wrote: Attached is a rough cut of a patch to try to speed up the cli a little bit. Basically in production mode it will skip some things during initialization. My concept is that we develop in mode != production and release in mode == production. I managed to knock a second or so off time to do a user-show on average. There may be some other things we can do to speed things up, I'm still looking. Some feedback on the approach would be appreciated. Note that I've completely ruled out SSL/Negotiate. I did my testing on lite-server which doesn't use SSL or Negotiate and it was STILL taking on average 3-4+ seconds per command. The server side was consistently taking 1 second to complete. rob oh, and the patch. ___ Freeipa-devel mailing list Freeipa-devel@redhat.com https://www.redhat.com/mailman/listinfo/freeipa-devel I don't feel I understand the scope yet enough to ACK. What are the ramifications of the checks that you are bypassing? Do we really want to do them in development mode, or are they more appropriate to be run as part of a test suite? ___ Freeipa-devel mailing list Freeipa-devel@redhat.com https://www.redhat.com/mailman/listinfo/freeipa-devel
