Hi Graham, My scripts were loaded successfully when mod_wsgi creates processes. I am using Daemon mode, and WSGIImportScript is declared inside of VirtualHost. Thank you a lot.
I got another problem, which may be a little bit off topic in this group. I used wrapt.when_imported to register a function when importing flask.app, while it doesn't work. In my application I printed out sys.modules, and found that wrapt and related modules were not there. I also checked sys.meta_path. At the end of the pre-loaded scripts (let's call it patch.py), sys.meta_path had one ImportHookFinder instance, while in my app, sys.meta_path was empty. Looks like after loading patch.py, modules were unloaded and finder objects were cleared. I also found patch.py and the application were running in the same process and both were in the main thread, as what I expected. Do you know any solution to it? Looking forward to your reply. Thanks, Yang On Friday, July 4, 2014 3:14:04 PM UTC-7, Graham Dumpleton wrote: > > Does it strictly need a WSGI middleware? > > What I have done isn't strictly a WSGI middleware as it just passes things > through and doesn't need to wrap access to wsgi.input nor do anything with > the response. > > It could easily be modified to do that though if needed. > > You can find an example of a quite deeming logging middleware at: > > > http://code.google.com/p/modwsgi/wiki/DebuggingTechniques#Tracking_Request_and_Response > > Instead of just having: > > return wrapped(*args, **kwargs) > > you would have something like: > > return LoggingMiddleware(wrapped, '/tmp/wsgi')(*args, **kwargs) > > Graham > > On 05/07/2014, at 8:07 AM, theliuy <[email protected] <javascript:>> wrote: > > Hi Graham, > > I tried to implement it as you described. It really helps. Thanks a lot. > > What I am trying to do is, I want to listen and log every requests coming > to my applications. I have already made some scripts to analyze the log and > find some interesting requests from specific user, and also built a tool to > mock what he/she did. It is used to reproduce some bugs caused by > consequent calls. I am looking for a way to implement this procedure into > other projects. Since log analyzer and mock tool are offline and flexible, > one of the difficulties is how to log coming requests. Registering it to > every flask app or wsgis needs lots of work, that's why I am looking for a > way to implement a "middleware" between apache/wsgi_mod and flask app. > > Thanks, > Yang > > > On Friday, July 4, 2014 5:14:49 AM UTC-7, Graham Dumpleton wrote: >> >> Okay, seems you can now use WSGIImportScript inside of VirtualHost. >> >> A long long time ago this wasn't the case. Seems I fixed it at some point >> so that it is now possible but the docs weren't updated. I did look at the >> code to try and work it out when I responded but was looking in the wrong >> place for what dictates whether the directive can be used in a VirtualHost. >> >> So you don't need to worry about using WSGIScriptAlias at all. You can >> always use WSGIImportScript, placing it after the WSGIDaemonProcess >> directive for the daemon process group that the WSGIImportScript needs to >> refer to, even inside of the VirtualHost. >> >> Graham >> >> On 04/07/2014, at 9:25 PM, Graham Dumpleton <[email protected]> >> wrote: >> >> >> On 04/07/2014, at 7:58 PM, Graham Dumpleton <[email protected]> >> wrote: >> >> On 04/07/2014, at 5:52 PM, theliuy <[email protected]> wrote: >> >> I have several web services using Apache + mod_wsgi + Flask. For some >> debug purpose, I want to listen the incoming requests and log them. I don't >> want to modify flask applications, or wsgi scripts neither. Because there >> are too many existing projects, I can't do it one by one. And they are >> going to be deployed in multiple environment, I just need this feature in >> one or two of them. >> >> One possible solution come to my mind is to modify Flask framework. >> Implement a "logger" inside flask. I think it will be better if I can find >> a way to register a script in mod_wsgi. Let mod_wsgi run it whenever >> requests are coming. >> >> I don't know if there is any better way to make it. Please give me some >> hint. >> >> >> What about the request are you wanting to log exactly and for what >> purpose? >> >> I can explain a way of doing what you want which avoids you needing to >> change either the code of your application, Flask or any other package, but >> the context of what you are trying to capture and why will help as I will >> then know what sort of WSGI middleware I will need to employ to capture >> what you need. >> >> So if you can respond with that extra information and in the mean time I >> will validate that my idea for how to do it will work. >> >> >> I would still like to know what you are trying to capture, but if you >> really want to avoid making changes to any existing code or third party >> modules, you can do the following. The technique uses monkey patching, >> employing the `wrapt` library to simplify the process and ensure how the >> monkey patching is done is correct. >> >> First up, you need to have the `wrapt` module installed. >> >> pip install wrapt >> >> >> Next, create a file called 'patch.py' which contains: >> >> from __future__ import print_function >> >> from wrapt import when_imported, wrap_function_wrapper >> >> def flask_wsgi_app_wrapper(wrapped, instance, args, kwargs): >> def bind_call_args(environ, *args, **kwargs): >> return environ >> >> environ = bind_call_args(*args, **kwargs) >> >> print(10*'>') >> for key in sorted(environ.keys()): >> print('%s: %s' % (key, repr(environ[key]))) >> print(10*'<') >> >> return wrapped(*args, **kwargs) >> >> @when_imported('flask.app') >> def instrument_flask_app(module): >> wrap_function_wrapper(module, 'Flask.wsgi_app', >> flask_wsgi_app_wrapper) >> >> >> What needs to be done now is that this 'patch.py' file has to be imported >> somehow and it ideally needs to be imported before the actual WSGI >> application script file is imported. It preferably would just have been >> imported at the start of the specific WSGI script file for the application >> you want to add debugging to. But since you don't want to do that, then you >> will need to modify the Apache configuration file and use mod_wsgi to >> preload it. >> >> What exactly you need to do here may depend on whether you are using >> mod_wsgi embedded mode or daemon mode. If using daemon mode, it may also >> depend on whether the WSGIDaemonProcess directives are specified inside of >> the context of a VirtualHost, or outside at global server scope. >> >> The problem here is that the WSGIImportScript can only be defined at >> global server scope. That is, it cannot be specified inside of a >> VirtualHost, but has to be outside. >> >> Right now I can't remember the rules about whether a WSGIImportScript can >> refer to a daemon process group specified using WSGIDaemonProcess directive >> inside of a VirtualHost. >> >> If it can, then the WSGIImportScript directive will at least have to be >> added after the VirtualHost containing the WSGIDaemonProcess. >> >> Anyway, if you are using embedded mode at least, then you need to use the >> WSGIImportScript directive as: >> >> WSGIImportScript /some/path/patch.py process-group=%{GLOBAL} >> application-group=%{GLOBAL} >> >> >> The application-group option should be set to the specific application >> group context that the WSGI application you want to debug, is running in. >> >> What will happen is that when the Apache child worker processes are >> started up and Python gets initialised, mod_wsgi will preload the patch.py >> file into the Python interpreter. >> >> In being imported, the @with_imported decorator will register that the >> instrument_flask_app() function should be called when the 'flask.app' >> module is imported by the WSGI application. >> >> When the module is the flask.app module is import, the >> instrument_flask_app() function is executed at that point, it will be >> passed the flask.app module before the import even returns the module to >> the code that imported it. The Flask.wsgi_app() method will then have a >> function wrapper applied using wrap_function_wrapper(). >> >> Later when handling a WSGI request, the Flask.wsgi_app() will be called. >> At that point the flask_wsgi_app_wrapper() wrapper will actually be called. >> Inside of that we bind the function arguments to extract the 'environ' >> argument passed to the WSGI application entry point for Flask. The wrapper >> function can then print out what it wants from 'environ'. Finally, the >> wrapper will call the original wrapped Flask.wsgi_app() method so the WSGI >> application can handle the request. >> >> So that is the principle of how it works, but that Apache configuration >> only works with embedded mode for sure. If using daemon mode we may not be >> able to use WSGIImportScript however and may have to use another trick. >> >> For daemon mode therefore, if the WSGIDaemonProcess directive is actually >> specified outside of any VirtualHost, then we can still use >> WSGIImportScript. Thus you would use: >> >> WSGIImportScript /some/path/patch.py process-group=group >> application-group=%{GLOBAL} >> >> >> The process-group option would be set to the daemon process group name, >> and application-group option again set to the specific application group >> context that the WSGI application you want to debug, is running in. >> >> The WSGIImportScript in this case should come after the WSGIDaemonProcess >> directive it is referring to. >> >> Now if the WSGIDaemonProcess group directive is inside of the >> VirtualHost, I am not sure if one can do: >> >> <VirtualHost *:80> >> ... >> WSGIDaemonProcess group >> ... >> </VirtualHost> >> >> WSGIImportScript /some/path/patch.py process-group=group >> application-group=%{GLOBAL} >> >> >> If that doesn't work, we have to cheat a bit and do: >> >> <VirtualHost *:80> >> ... >> WSGIDaemonProcess group >> WSGIScriptAlias /.patch /some/path/patch.py process-group=group >> application-group=%{GLOBAL} >> <Location /.patch> >> Deny from all >> </Location> >> ... >> </VirtualHost> >> >> >> In other words, we actually use WSGIScriptAlias and the fact that if both >> process-group and application-group are specified, that the WSGI script >> file would normally be preloaded. >> >> Since we have to specify a URL as the mount point and there will be no >> actual WSGI application in the script file, then we use a Location block to >> block access to the URL to force a forbidden HTTP response if someone tries >> to access that URL. >> >> That therefore represents a technical solution for what I believe you >> want. The question now is if you really need to do what you think you do. >> >> Note that if you are really after a monitoring solution for evaluating >> application performance, then there are better ways of doing what you want. >> >> If you do try it and have the case of using WSGIDaemonProcess inside of a >> VirtualHost, do tell me if WSGIImportScript outside of the VirtualHost can >> still refer to it. I will try and test it myself, but it isn't convenient >> to do so right now. >> >> Graham >> >> >> > -- > You received this message because you are subscribed to the Google Groups > "modwsgi" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to [email protected] <javascript:>. > To post to this group, send email to [email protected] <javascript:> > . > Visit this group at http://groups.google.com/group/modwsgi. > For more options, visit https://groups.google.com/d/optout. > > > -- You received this message because you are subscribed to the Google Groups "modwsgi" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To post to this group, send email to [email protected]. Visit this group at http://groups.google.com/group/modwsgi. For more options, visit https://groups.google.com/d/optout.
