Dne 26.10.2011 16:50, Jan Cholasta napsal(a):
Dne 26.10.2011 16:39, Martin Kosek napsal(a):
On Wed, 2011-10-26 at 15:52 +0200, Jan Cholasta wrote:
Dne 26.10.2011 15:41, Martin Kosek napsal(a):
On Wed, 2011-10-26 at 11:39 +0200, Jan Cholasta wrote:
Dne 25.10.2011 22:30, Rob Crittenden napsal(a):
Ondrej Hamada wrote:
On 10/25/2011 04:01 PM, Martin Kosek wrote:
On Tue, 2011-10-25 at 15:29 +0200, Ondrej Hamada wrote:
https://fedorahosted.org/freeipa/ticket/1336

Lazy initialization of ipalib plugins is used under all
contexts, not
only when context = cli. Every loaded plugin is pre-finalized -
a flag
is set, which means, that the plugin needs to be finalized.
Then every
call of plugin's __gettattr__ checks the flag and finalizes the
plugin
if necessary. The code was implemented by jcholast. Time
reduction of
commands execution is quite markable:

patch [s] | normal [s] | command
-----------------------------------------------------------------------

1.468 | 2.287 | ipa user-add jsmith --firt=john
--last=smith
1.658 | 2.235 | ipa user-del jsmith
1.624 | 2.204 | ipa dnsrecord-find example.com

Thanks for submitting the patch. Ondra, just please provide the
patch in
proper format (exported via command `git format-patch -M -C
--patience
--full-index -1' which I sent you earlier).

Martin


Sorry, correct version attached

Wow, this is very impressive, great job Ondra and Honza!

Martin, ACK from me but I'd like a second opinion. The patch is very
straightforward and clean, just want to make sure we're not missing a
corner case.

I ran the self-tests and didn't see any problem there.

Before pushing please add the ticket # to the commit.

rob


I've just remembered that special methods aren't looked up through
__getattribute__ (see the note at
http://docs.python.org/reference/datamodel.html#more-attribute-access-for-new-style-classes).

That might possibly cause problems in Command and subclasses,
because it
uses __call__.

Honza


This should be investigated. All our unit tests are OK though.

I am also thinking if we shouldn't do this optimization on CLI side
only
as the original ticket suggests. As server side loads and finalizes all
plugins just once at httpd start, performance should be OK. Plus, we
would know right after the start that we are ready and there is no
problem with finalizing any component.

Martin


As we discussed earlier - are you sure it works that way?

Yes.

I did a few
Django-based applications before and I'm pretty sure all mod_wsgi does
is caching the bytecode, which is run from the beginning for each
request, like what CLI does.

I did few Django application myself too. Its all about how mod_wsgi is
configured. You may want to learn more about WSGIDaemonProcess and
WSGIImportScript directives.

In IPA, we have API already prepared and finalized for every incoming
request.


If it does work the way you described, the worst thing that could happen
is that the plugins would be finalized the first time they are used and
then stay initialized until the server terminates.

Honza


I know, I was just being conservative. As this optimization has no
effect on server performance I considered it a little bit safer to load
everything at start and be sure we are OK. But its not that hard
requirement.

Martin


OK, thanks, just wanted to be sure.

Honza


Added finalization for __call__ and the check for CLI. Patch attached.

Honza

--
Jan Cholasta
>From 083f5d555d9e652463c2ed670bc08830f075d52f Mon Sep 17 00:00:00 2001
From: Jan Cholasta <jchol...@redhat.com>
Date: Mon, 31 Oct 2011 06:25:58 -0400
Subject: [PATCH] Finalize plugins on-demand in CLI.

ticket 1336
---
 ipalib/frontend.py |    1 +
 ipalib/plugable.py |   27 ++++++++++++++++++++++++---
 2 files changed, 25 insertions(+), 3 deletions(-)

diff --git a/ipalib/frontend.py b/ipalib/frontend.py
index 61e7f49..285049c 100644
--- a/ipalib/frontend.py
+++ b/ipalib/frontend.py
@@ -411,6 +411,7 @@ class Command(HasParam):
         If not in a server context, the call will be forwarded over
         XML-RPC and the executed an the nearest IPA server.
         """
+        self.finalize_late()
         params = self.args_options_2_params(*args, **options)
         self.debug(
             'raw: %s(%s)', self.name, ', '.join(self._repr_iter(**params))
diff --git a/ipalib/plugable.py b/ipalib/plugable.py
index b0e4156..9f8a40e 100644
--- a/ipalib/plugable.py
+++ b/ipalib/plugable.py
@@ -173,6 +173,7 @@ class Plugin(ReadOnly):
     """
 
     label = None
+    __finalize_later = False
 
     def __init__(self):
         self.__api = None
@@ -217,12 +218,34 @@ class Plugin(ReadOnly):
         return self.__api
     api = property(__get_api)
 
+    def try_finalize(self):
+        if self.__api.env.context == 'cli':
+            self.__finalize_later = True
+        else:
+            self.__finalize_and_check()
+
     def finalize(self):
         """
         """
         if not is_production_mode(self):
             lock(self)
 
+    def __finalize_and_check(self):
+        self.finalize()
+        if not is_production_mode(self.__api):
+            assert islocked(self) is True
+
+    def finalize_late(self):
+        if not self.__finalize_later:
+            return
+        self.__finalize_later = False
+        self.__finalize_and_check()
+
+    def __getattribute__(self, name):
+        if not name.startswith('_Plugin__') and not name.startswith('_ReadOnly__') and name != 'finalize_late':
+            self.finalize_late()
+        return object.__getattribute__(self, name)
+
     def set_api(self, api):
         """
         Set reference to `API` instance.
@@ -638,9 +661,7 @@ class API(DictProxy):
                 assert p.instance.api is self
 
         for p in plugins.itervalues():
-            p.instance.finalize()
-            if not production_mode:
-                assert islocked(p.instance) is True
+            p.instance.try_finalize()
         object.__setattr__(self, '_API__finalized', True)
         tuple(PluginInfo(p) for p in plugins.itervalues())
         object.__setattr__(self, 'plugins',
-- 
1.7.6.4

_______________________________________________
Freeipa-devel mailing list
Freeipa-devel@redhat.com
https://www.redhat.com/mailman/listinfo/freeipa-devel

Reply via email to