Re: DJblets _check_expired() issues
I opened another topic: https://groups.google.com/forum/#!topic/reviewboard/kS23n4m5aNw These above issues seem to be caused by the Apache configuration i used (MPM worker instead of MPM fork). Le mardi 10 décembre 2013 17:18:46 UTC+1, Vincent G. a écrit : Hi, Before to explain the problem, below the configuration: ReviewBoard version: 1.7.6 Django version: 1.4.5 Djblets version: 0.7.11 Apache used Memcache used (version 1.48, python lib) I developed some extensions for Reviewboard which are using TemplateHook and UrlHook, to add some HTML code on existing Reviewboard web pages. When i installed and enabled these extensions through administration web page, and users use the Reviewboard, some errors as below have been notified: (even after restarting apache, memcache, and users have refresh their web browser). *Traceback (most recent call last): File /usr/local/lib/python2.7/dist-packages/Django-1.4.5-py2.7.egg/django/core/handlers/base.py, line 89, in get_responseresponse = middleware_method(request) File /usr/local/lib/python2.7/dist-packages/Djblets-0.7.11-py2.7.egg/djblets/extensions/middleware.py, line 32, in process_requestself._check_expired() File /usr/local/lib/python2.7/dist-packages/Djblets-0.7.11-py2.7.egg/djblets/extensions/middleware.py, line 50, in _check_expiredextension_manager.load(full_reload=True) File /usr/local/lib/python2.7/dist-packages/Djblets-0.7.11-py2.7.egg/djblets/extensions/base.py, line 453, in loadself._clear_extensions() File /usr/local/lib/python2.7/dist-packages/Djblets-0.7.11-py2.7.egg/djblets/extensions/base.py, line 542, in _clear_extensionsself._uninit_extension(extension) File /usr/local/lib/python2.7/dist-packages/Djblets-0.7.11-py2.7.egg/djblets/extensions/base.py, line 595, in _uninit_extensionextension.shutdown() File /usr/local/lib/python2.7/dist-packages/Djblets-0.7.11-py2.7.egg/djblets/extensions/base.py, line 170, in shutdownhook.shutdown() File /usr/local/lib/python2.7/dist-packages/Djblets-0.7.11-py2.7.egg/djblets/extensions/hooks.py, line 46, in shutdownsuper(URLHook, self).shutdown() File /usr/local/lib/python2.7/dist-packages/Djblets-0.7.11-py2.7.egg/djblets/extensions/base.py, line 246, in shutdownself.__class__.remove_hook(self) File /usr/local/lib/python2.7/dist-packages/Djblets-0.7.11-py2.7.egg/djblets/extensions/base.py, line 274, in remove_hookcls.hooks.remove(hook)ValueError: list.remove(x): x not in list* or *Traceback (most recent call last): File /usr/local/lib/python2.7/dist-packages/Django-1.4.5-py2.7.egg/django/core/handlers/base.py, line 89, in get_responseresponse = middleware_method(request) File /usr/local/lib/python2.7/dist-packages/Djblets-0.7.11-py2.7.egg/djblets/extensions/middleware.py, line 32, in process_requestself._check_expired() File /usr/local/lib/python2.7/dist-packages/Djblets-0.7.11-py2.7.egg/djblets/extensions/middleware.py, line 50, in _check_expiredextension_manager.load(full_reload=True) File /usr/local/lib/python2.7/dist-packages/Djblets-0.7.11-py2.7.egg/djblets/extensions/base.py, line 511, in loadself._init_extension(ext_class) File /usr/local/lib/python2.7/dist-packages/Djblets-0.7.11-py2.7.egg/djblets/extensions/base.py, line 580, in _init_extensionextension.info.installed = extension.registration.installedAttributeError: 'MetricsExtension' object has no attribute 'registration'* I tried to find the problem and it seems to come from a cache issue. As i understood (please tell me if i well understood or not), for each HTTP request, the Djblet middleware (located in middleware.py file) checks if the extension manager is expired (_check_expired() method). In *djblets/extensions/middleware.py* file: *class ExtensionsMiddleware(object):...def process_request(self, request):self._check_expired()def _check_expired(self):...for extension_manager in get_extension_managers():if extension_manager.is_expired(): extension_manager.load(full_reload=True)* In *djblets/extensions/base.py* file: *class ExtensionManager(object):...def is_expired(self): sync_gen = cache.get(self._sync_key) return (sync_gen is None or(type(sync_gen) is int and sync_gen != self._last_sync_gen))* If is *_expired()* returns True, all extensions are reloaded (extension_manager.load(full_reload=True) in _check_expired() method. And the problem seems to come during this reloading (see above error messages). So, is it possible to not execute the* extension_manager.load(full_reload=True)* method after each HTTP Request ? As to keep all the time the *_last_sync_gen* key of the extension manager in the cache. I don't understand why you need to put the key
Re: Which recommended Apache MPM to use with Reviewboard + extensions
Hi Christian,
Thank you to answer so fast.
I would like to notify that the behavior of the Reviewboard, specially when
extensions are used is very 'strange' when Apache server is in MPM worker
mode instead of MPM Prefork.
When in MPM Worker, i noticed that the HTTP request are setup as:
wsgi.multithread : True and wsgi.multiprocess : True.
Ok. It's normal according to the mod_wsgi document ( read
http://code.google.com/p/modwsgi/wiki/ProcessesAndThreading )
The issues i found (described in the topic
https://groups.google.com/forum/#!topic/reviewboard/XL0E9Sb22w0 ) is we
have sometimes a lot of errors ('[Review Board] ERROR (EXTERNAL IP)'
emails) because some object does not exist for example. I tried to analyze
the problem and perhaps i found the cause of these issues.
When the wsgi.multithread is set as True (so for example when Apache MPM
Worker is used), the HTTP requests received by the Reviewboard middleware
(as for example, the djblet middleware which checks the expiration of the
extension manager) can be processed in parallel in each thread BUT also in
the same process. So, when an expiration is detected (by comparing key with
cache) the extension manager is reloaded (so means that ALL extensions have
been reloaded). The problem is that if an other HTTP request is processed
at the same time in other thread BUT in the same process (so normally
memory is shared between threads), this HTTP request can have some issues
because the loading of the extension is not finished. That's why, i think,
i received a lot of '[Review Board] ERROR (EXTERNAL IP)' errors. The
process_request() method of the ExtensionsMiddleware class (in djblet
package) seems to not include synchronization mechanism.
I din't check yet, if you updated the Djblet library (we used the 0.7.11
version, and Reviewboard 1.7.6), but i think it's very important to check
the Apache configuration and use the Apache Prefork MPM to avoid the issues
we encountered. Because in this case, the multithread mechanism is not used
(wsgi.multithread= False) or synchronization mechanism shall be included in
the process_request() of each middleware.
That's why i asked you the recommended Apache MPM to use with Reviewboard.
Don't worry about the other topic i created.
I can understand you have no time. A new released of Reviewboard has been
out recently.
We planned to upgrade our Reviewboard application, but i saw you have
changed some things, especially the extension functionality.
So i need to check if our extensions are always compatible.
Regards,
Vincent
--
Get the Review Board Power Pack at http://www.reviewboard.org/powerpack/
---
Sign up for Review Board hosting at RBCommons: https://rbcommons.com/
---
Happy user? Let us know at http://www.reviewboard.org/users/
---
You received this message because you are subscribed to the Google Groups
reviewboard group.
To unsubscribe from this group and stop receiving emails from it, send an email
to reviewboard+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.
Which recommended Apache MPM to use with Reviewboard + extensions
Hi, I would like to know which Apache MPM is recommended to use with Reviewboard ? MPM Fork or MPM Worker ? Because i have some issues when i use Apache MPM worker with Reviewboard Extensions (read my other topic Djblets _isexpired() issues) in case of *wsgi.multithread = True* wsgi.multiprocess = True If wsgi.multithread is set to False (by using Apache MPM Fork example), no problem. Best regards, Vincent G. -- Get the Review Board Power Pack at http://www.reviewboard.org/powerpack/ --- Sign up for Review Board hosting at RBCommons: https://rbcommons.com/ --- Happy user? Let us know at http://www.reviewboard.org/users/ --- You received this message because you are subscribed to the Google Groups reviewboard group. To unsubscribe from this group and stop receiving emails from it, send an email to reviewboard+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/groups/opt_out.
DJblets _check_expired() issues
of ReviewBoard at the same time is trying to display
a web page where the extension is used, is it not a problem ?
Because perhaps the extension is not yet full reloaded, which can create
perhaps the above errors.
Or is it an issue how i wrote the extensions ?
Below one of the extension i created:
*class MetricsExtension(Extension):...is_configurable = True
has_admin_site = True...def __init__(self, *args, **kwargs):
super(MetricsExtension, self).__init__(*args, **kwargs)
URLHook(self, patterns('',(r'^rbmetrics/',
include('rbmetrics.urls'TemplateHook(self,
base-scripts-post,
rbmetrics/metrics_init.html,
apply_to=['view_diff', 'review-request-detail'])*
Hope you understand my problem, because not very easy to explain and my
english is so poor ... ;-)
Regards,
Vincent G.
--
Get the Review Board Power Pack at http://www.reviewboard.org/powerpack/
---
Sign up for Review Board hosting at RBCommons: https://rbcommons.com/
---
Happy user? Let us know at http://www.reviewboard.org/users/
---
You received this message because you are subscribed to the Google Groups
reviewboard group.
To unsubscribe from this group and stop receiving emails from it, send an email
to reviewboard+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.
DJblets _check_expired() issues
of ReviewBoard at the same time is trying to display
a web page where the extension is used, is it not a problem ?
Because perhaps the extension is not yet full reloaded, which can create
perhaps the above errors.
Or is it an issue how i wrote the extensions ?
Below one of the extension i created:
class MetricsExtension(Extension):
...
is_configurable = True
has_admin_site = True
...
def __init__(self, *args, **kwargs):
super(MetricsExtension, self).__init__(*args, **kwargs)
URLHook(self, patterns('',
(r'^rbmetrics/', include('rbmetrics.urls'
TemplateHook(self, base-scripts-post,
rbmetrics/metrics_init.html,
apply_to=['view_diff',
'review-request-detail'])
Hope you understand my problem, because not very easy to explain and my
english is so poor ... ;-)
Regards,
Vincent G.
--
Get the Review Board Power Pack at http://www.reviewboard.org/powerpack/
---
Sign up for Review Board hosting at RBCommons: https://rbcommons.com/
---
Happy user? Let us know at http://www.reviewboard.org/users/
---
You received this message because you are subscribed to the Google Groups
reviewboard group.
To unsubscribe from this group and stop receiving emails from it, send an email
to reviewboard+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.
