Dne 26.10.2011 16:50, Jan Cholasta napsal(a):
Dne 26.10.2011 16:39, Martin Kosek napsal(a):
On Wed, 2011-10-26 at 15:52 +0200, Jan Cholasta wrote:
Dne 26.10.2011 15:41, Martin Kosek napsal(a):
On Wed, 2011-10-26 at 11:39 +0200, Jan Cholasta wrote:
Dne 25.10.2011 22:30, Rob Crittenden napsal(a):
Ondrej Hamada wrote:
On 10/25/2011 04:01 PM, Martin Kosek wrote:
On Tue, 2011-10-25 at 15:29 +0200, Ondrej Hamada wrote:
Lazy initialization of ipalib plugins is used under all
only when context = cli. Every loaded plugin is pre-finalized -
is set, which means, that the plugin needs to be finalized.
call of plugin's __gettattr__ checks the flag and finalizes the
if necessary. The code was implemented by jcholast. Time
commands execution is quite markable:
patch [s] | normal [s] | command
1.468 | 2.287 | ipa user-add jsmith --firt=john
1.658 | 2.235 | ipa user-del jsmith
1.624 | 2.204 | ipa dnsrecord-find example.com
Thanks for submitting the patch. Ondra, just please provide the
proper format (exported via command `git format-patch -M -C
--full-index -1' which I sent you earlier).
Sorry, correct version attached
Wow, this is very impressive, great job Ondra and Honza!
Martin, ACK from me but I'd like a second opinion. The patch is very
straightforward and clean, just want to make sure we're not missing a
I ran the self-tests and didn't see any problem there.
Before pushing please add the ticket # to the commit.
I've just remembered that special methods aren't looked up through
__getattribute__ (see the note at
That might possibly cause problems in Command and subclasses,
This should be investigated. All our unit tests are OK though.
I am also thinking if we shouldn't do this optimization on CLI side
as the original ticket suggests. As server side loads and finalizes all
plugins just once at httpd start, performance should be OK. Plus, we
would know right after the start that we are ready and there is no
problem with finalizing any component.
As we discussed earlier - are you sure it works that way?
I did a few
Django-based applications before and I'm pretty sure all mod_wsgi does
is caching the bytecode, which is run from the beginning for each
request, like what CLI does.
I did few Django application myself too. Its all about how mod_wsgi is
configured. You may want to learn more about WSGIDaemonProcess and
In IPA, we have API already prepared and finalized for every incoming
If it does work the way you described, the worst thing that could happen
is that the plugins would be finalized the first time they are used and
then stay initialized until the server terminates.
I know, I was just being conservative. As this optimization has no
effect on server performance I considered it a little bit safer to load
everything at start and be sure we are OK. But its not that hard
OK, thanks, just wanted to be sure.
Added finalization for __call__ and the check for CLI. Patch attached.
>From 083f5d555d9e652463c2ed670bc08830f075d52f Mon Sep 17 00:00:00 2001
From: Jan Cholasta <jchol...@redhat.com>
Date: Mon, 31 Oct 2011 06:25:58 -0400
Subject: [PATCH] Finalize plugins on-demand in CLI.
ipalib/frontend.py | 1 +
ipalib/plugable.py | 27 ++++++++++++++++++++++++---
2 files changed, 25 insertions(+), 3 deletions(-)
diff --git a/ipalib/frontend.py b/ipalib/frontend.py
index 61e7f49..285049c 100644
@@ -411,6 +411,7 @@ class Command(HasParam):
If not in a server context, the call will be forwarded over
XML-RPC and the executed an the nearest IPA server.
params = self.args_options_2_params(*args, **options)
'raw: %s(%s)', self.name, ', '.join(self._repr_iter(**params))
diff --git a/ipalib/plugable.py b/ipalib/plugable.py
index b0e4156..9f8a40e 100644
@@ -173,6 +173,7 @@ class Plugin(ReadOnly):
label = None
+ __finalize_later = False
self.__api = None
@@ -217,12 +218,34 @@ class Plugin(ReadOnly):
api = property(__get_api)
+ def try_finalize(self):
+ if self.__api.env.context == 'cli':
+ self.__finalize_later = True
if not is_production_mode(self):
+ def __finalize_and_check(self):
+ if not is_production_mode(self.__api):
+ assert islocked(self) is True
+ def finalize_late(self):
+ if not self.__finalize_later:
+ self.__finalize_later = False
+ def __getattribute__(self, name):
+ if not name.startswith('_Plugin__') and not name.startswith('_ReadOnly__') and name != 'finalize_late':
+ return object.__getattribute__(self, name)
def set_api(self, api):
Set reference to `API` instance.
@@ -638,9 +661,7 @@ class API(DictProxy):
assert p.instance.api is self
for p in plugins.itervalues():
- if not production_mode:
- assert islocked(p.instance) is True
object.__setattr__(self, '_API__finalized', True)
tuple(PluginInfo(p) for p in plugins.itervalues())
Freeipa-devel mailing list