Add debug printing out: import mod_wsgi print mod_wsgi.process_group print mod_wsgi.application_group
This will verify the Python sub interpreter that the patch.py is being loaded in. It should be the same as your WSGI application. Graham On 12/07/2014, at 3:37 PM, theliuy <[email protected]> wrote: > Hi Graham, > > My scripts were loaded successfully when mod_wsgi creates processes. I am > using Daemon mode, and WSGIImportScript is declared inside of VirtualHost. > Thank you a lot. > > I got another problem, which may be a little bit off topic in this group. I > used wrapt.when_imported to register a function when importing flask.app, > while it doesn't work. In my application I printed out sys.modules, and found > that wrapt and related modules were not there. I also checked sys.meta_path. > At the end of the pre-loaded scripts (let's call it patch.py), sys.meta_path > had one ImportHookFinder instance, while in my app, sys.meta_path was empty. > Looks like after loading patch.py, modules were unloaded and finder objects > were cleared. I also found patch.py and the application were running in the > same process and both were in the main thread, as what I expected. > > Do you know any solution to it? Looking forward to your reply. > > Thanks, > Yang > > > On Friday, July 4, 2014 3:14:04 PM UTC-7, Graham Dumpleton wrote: > Does it strictly need a WSGI middleware? > > What I have done isn't strictly a WSGI middleware as it just passes things > through and doesn't need to wrap access to wsgi.input nor do anything with > the response. > > It could easily be modified to do that though if needed. > > You can find an example of a quite deeming logging middleware at: > > > http://code.google.com/p/modwsgi/wiki/DebuggingTechniques#Tracking_Request_and_Response > > Instead of just having: > > return wrapped(*args, **kwargs) > > you would have something like: > > return LoggingMiddleware(wrapped, '/tmp/wsgi')(*args, **kwargs) > > Graham > > On 05/07/2014, at 8:07 AM, theliuy <[email protected]> wrote: > >> Hi Graham, >> >> I tried to implement it as you described. It really helps. Thanks a lot. >> >> What I am trying to do is, I want to listen and log every requests coming to >> my applications. I have already made some scripts to analyze the log and >> find some interesting requests from specific user, and also built a tool to >> mock what he/she did. It is used to reproduce some bugs caused by consequent >> calls. I am looking for a way to implement this procedure into other >> projects. Since log analyzer and mock tool are offline and flexible, one of >> the difficulties is how to log coming requests. Registering it to every >> flask app or wsgis needs lots of work, that's why I am looking for a way to >> implement a "middleware" between apache/wsgi_mod and flask app. >> >> Thanks, >> Yang >> >> >> On Friday, July 4, 2014 5:14:49 AM UTC-7, Graham Dumpleton wrote: >> Okay, seems you can now use WSGIImportScript inside of VirtualHost. >> >> A long long time ago this wasn't the case. Seems I fixed it at some point so >> that it is now possible but the docs weren't updated. I did look at the code >> to try and work it out when I responded but was looking in the wrong place >> for what dictates whether the directive can be used in a VirtualHost. >> >> So you don't need to worry about using WSGIScriptAlias at all. You can >> always use WSGIImportScript, placing it after the WSGIDaemonProcess >> directive for the daemon process group that the WSGIImportScript needs to >> refer to, even inside of the VirtualHost. >> >> Graham >> >> On 04/07/2014, at 9:25 PM, Graham Dumpleton <[email protected]> wrote: >> >>> >>> On 04/07/2014, at 7:58 PM, Graham Dumpleton <[email protected]> wrote: >>>> On 04/07/2014, at 5:52 PM, theliuy <[email protected]> wrote: >>>>> I have several web services using Apache + mod_wsgi + Flask. For some >>>>> debug purpose, I want to listen the incoming requests and log them. I >>>>> don't want to modify flask applications, or wsgi scripts neither. Because >>>>> there are too many existing projects, I can't do it one by one. And they >>>>> are going to be deployed in multiple environment, I just need this >>>>> feature in one or two of them. >>>>> >>>>> One possible solution come to my mind is to modify Flask framework. >>>>> Implement a "logger" inside flask. I think it will be better if I can >>>>> find a way to register a script in mod_wsgi. Let mod_wsgi run it whenever >>>>> requests are coming. >>>>> >>>>> I don't know if there is any better way to make it. Please give me some >>>>> hint. >>>> >>>> What about the request are you wanting to log exactly and for what purpose? >>>> >>>> I can explain a way of doing what you want which avoids you needing to >>>> change either the code of your application, Flask or any other package, >>>> but the context of what you are trying to capture and why will help as I >>>> will then know what sort of WSGI middleware I will need to employ to >>>> capture what you need. >>>> >>>> So if you can respond with that extra information and in the mean time I >>>> will validate that my idea for how to do it will work. >>> >>> I would still like to know what you are trying to capture, but if you >>> really want to avoid making changes to any existing code or third party >>> modules, you can do the following. The technique uses monkey patching, >>> employing the `wrapt` library to simplify the process and ensure how the >>> monkey patching is done is correct. >>> >>> First up, you need to have the `wrapt` module installed. >>> >>> pip install wrapt >>> >>> Next, create a file called 'patch.py' which contains: >>> >>> from __future__ import print_function >>> >>> from wrapt import when_imported, wrap_function_wrapper >>> >>> def flask_wsgi_app_wrapper(wrapped, instance, args, kwargs): >>> def bind_call_args(environ, *args, **kwargs): >>> return environ >>> >>> environ = bind_call_args(*args, **kwargs) >>> >>> print(10*'>') >>> for key in sorted(environ.keys()): >>> print('%s: %s' % (key, repr(environ[key]))) >>> print(10*'<') >>> >>> return wrapped(*args, **kwargs) >>> >>> @when_imported('flask.app') >>> def instrument_flask_app(module): >>> wrap_function_wrapper(module, 'Flask.wsgi_app', flask_wsgi_app_wrapper) >>> >>> What needs to be done now is that this 'patch.py' file has to be imported >>> somehow and it ideally needs to be imported before the actual WSGI >>> application script file is imported. It preferably would just have been >>> imported at the start of the specific WSGI script file for the application >>> you want to add debugging to. But since you don't want to do that, then you >>> will need to modify the Apache configuration file and use mod_wsgi to >>> preload it. >>> >>> What exactly you need to do here may depend on whether you are using >>> mod_wsgi embedded mode or daemon mode. If using daemon mode, it may also >>> depend on whether the WSGIDaemonProcess directives are specified inside of >>> the context of a VirtualHost, or outside at global server scope. >>> >>> The problem here is that the WSGIImportScript can only be defined at global >>> server scope. That is, it cannot be specified inside of a VirtualHost, but >>> has to be outside. >>> >>> Right now I can't remember the rules about whether a WSGIImportScript can >>> refer to a daemon process group specified using WSGIDaemonProcess directive >>> inside of a VirtualHost. >>> >>> If it can, then the WSGIImportScript directive will at least have to be >>> added after the VirtualHost containing the WSGIDaemonProcess. >>> >>> Anyway, if you are using embedded mode at least, then you need to use the >>> WSGIImportScript directive as: >>> >>> WSGIImportScript /some/path/patch.py process-group=%{GLOBAL} >>> application-group=%{GLOBAL} >>> >>> The application-group option should be set to the specific application >>> group context that the WSGI application you want to debug, is running in. >>> >>> What will happen is that when the Apache child worker processes are started >>> up and Python gets initialised, mod_wsgi will preload the patch.py file >>> into the Python interpreter. >>> >>> In being imported, the @with_imported decorator will register that the >>> instrument_flask_app() function should be called when the 'flask.app' >>> module is imported by the WSGI application. >>> >>> When the module is the flask.app module is import, the >>> instrument_flask_app() function is executed at that point, it will be >>> passed the flask.app module before the import even returns the module to >>> the code that imported it. The Flask.wsgi_app() method will then have a >>> function wrapper applied using wrap_function_wrapper(). >>> >>> Later when handling a WSGI request, the Flask.wsgi_app() will be called. At >>> that point the flask_wsgi_app_wrapper() wrapper will actually be called. >>> Inside of that we bind the function arguments to extract the 'environ' >>> argument passed to the WSGI application entry point for Flask. The wrapper >>> function can then print out what it wants from 'environ'. Finally, the >>> wrapper will call the original wrapped Flask.wsgi_app() method so the WSGI >>> application can handle the request. >>> >>> So that is the principle of how it works, but that Apache configuration >>> only works with embedded mode for sure. If using daemon mode we may not be >>> able to use WSGIImportScript however and may have to use another trick. >>> >>> For daemon mode therefore, if the WSGIDaemonProcess directive is actually >>> specified outside of any VirtualHost, then we can still use >>> WSGIImportScript. Thus you would use: >>> >>> WSGIImportScript /some/path/patch.py process-group=group >>> application-group=%{GLOBAL} >>> >>> The process-group option would be set to the daemon process group name, and >>> application-group option again set to the specific application group >>> context that the WSGI application you want to debug, is running in. >>> >>> The WSGIImportScript in this case should come after the WSGIDaemonProcess >>> directive it is referring to. >>> >>> Now if the WSGIDaemonProcess group directive is inside of the VirtualHost, >>> I am not sure if one can do: >>> >>> <VirtualHost *:80> >>> ... >>> WSGIDaemonProcess group >>> ... >>> </VirtualHost> >>> >>> WSGIImportScript /some/path/patch.py process-group=group >>> application-group=%{GLOBAL} >>> >>> If that doesn't work, we have to cheat a bit and do: >>> >>> <VirtualHost *:80> >>> ... >>> WSGIDaemonProcess group >>> WSGIScriptAlias /.patch /some/path/patch.py process-group=group >>> application-group=%{GLOBAL} >>> <Location /.patch> >>> Deny from all >>> </Location> >>> ... >>> </VirtualHost> >>> >>> In other words, we actually use WSGIScriptAlias and the fact that if both >>> process-group and application-group are specified, that the WSGI script >>> file would normally be preloaded. >>> >>> Since we have to specify a URL as the mount point and there will be no >>> actual WSGI application in the script file, then we use a Location block to >>> block access to the URL to force a forbidden HTTP response if someone tries >>> to access that URL. >>> >>> That therefore represents a technical solution for what I believe you want. >>> The question now is if you really need to do what you think you do. >>> >>> Note that if you are really after a monitoring solution for evaluating >>> application performance, then there are better ways of doing what you want. >>> >>> If you do try it and have the case of using WSGIDaemonProcess inside of a >>> VirtualHost, do tell me if WSGIImportScript outside of the VirtualHost can >>> still refer to it. I will try and test it myself, but it isn't convenient >>> to do so right now. >>> >>> Graham >>> >> >> >> -- >> You received this message because you are subscribed to the Google Groups >> "modwsgi" group. >> To unsubscribe from this group and stop receiving emails from it, send an >> email to [email protected]. >> To post to this group, send email to [email protected]. >> Visit this group at http://groups.google.com/group/modwsgi. >> For more options, visit https://groups.google.com/d/optout. > > > -- > You received this message because you are subscribed to the Google Groups > "modwsgi" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to [email protected]. > To post to this group, send email to [email protected]. > Visit this group at http://groups.google.com/group/modwsgi. > For more options, visit https://groups.google.com/d/optout. -- You received this message because you are subscribed to the Google Groups "modwsgi" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To post to this group, send email to [email protected]. Visit this group at http://groups.google.com/group/modwsgi. For more options, visit https://groups.google.com/d/optout.
