Oh! Thanks. They were using different application groups. I found the
default value of WSGIApplicationGroup is %{RESOURCE}, while I tried
WSGIImportScript /some/path/patch.py process-group=some_process_gronp
application-group=%{RESOURCE}
Looks like the application-group of patch.py was assigned to '%{RESOURCE}'.
And I have to declare WSGIApplicationGroup explicitly. Do we have any other
way to make it?
Thanks,
Yang
On Saturday, July 12, 2014 3:51:13 PM UTC-7, Graham Dumpleton wrote:
>
> Add debug printing out:
>
> import mod_wsgi
> print mod_wsgi.process_group
> print mod_wsgi.application_group
>
> This will verify the Python sub interpreter that the patch.py is being
> loaded in. It should be the same as your WSGI application.
>
> Graham
>
> On 12/07/2014, at 3:37 PM, theliuy <[email protected] <javascript:>> wrote:
>
> Hi Graham,
>
> My scripts were loaded successfully when mod_wsgi creates processes. I am
> using Daemon mode, and WSGIImportScript is declared inside of VirtualHost.
> Thank you a lot.
>
> I got another problem, which may be a little bit off topic in this group.
> I used wrapt.when_imported to register a function when importing flask.app,
> while it doesn't work. In my application I printed out sys.modules, and
> found that wrapt and related modules were not there. I also checked
> sys.meta_path. At the end of the pre-loaded scripts (let's call it
> patch.py), sys.meta_path had one ImportHookFinder instance, while in my
> app, sys.meta_path was empty. Looks like after loading patch.py, modules
> were unloaded and finder objects were cleared. I also found patch.py and
> the application were running in the same process and both were in the main
> thread, as what I expected.
>
> Do you know any solution to it? Looking forward to your reply.
>
> Thanks,
> Yang
>
>
> On Friday, July 4, 2014 3:14:04 PM UTC-7, Graham Dumpleton wrote:
>>
>> Does it strictly need a WSGI middleware?
>>
>> What I have done isn't strictly a WSGI middleware as it just passes
>> things through and doesn't need to wrap access to wsgi.input nor do
>> anything with the response.
>>
>> It could easily be modified to do that though if needed.
>>
>> You can find an example of a quite deeming logging middleware at:
>>
>>
>> http://code.google.com/p/modwsgi/wiki/DebuggingTechniques#Tracking_Request_and_Response
>>
>> Instead of just having:
>>
>> return wrapped(*args, **kwargs)
>>
>> you would have something like:
>>
>> return LoggingMiddleware(wrapped, '/tmp/wsgi')(*args, **kwargs)
>>
>> Graham
>>
>> On 05/07/2014, at 8:07 AM, theliuy <[email protected]> wrote:
>>
>> Hi Graham,
>>
>> I tried to implement it as you described. It really helps. Thanks a lot.
>>
>> What I am trying to do is, I want to listen and log every requests coming
>> to my applications. I have already made some scripts to analyze the log and
>> find some interesting requests from specific user, and also built a tool to
>> mock what he/she did. It is used to reproduce some bugs caused by
>> consequent calls. I am looking for a way to implement this procedure into
>> other projects. Since log analyzer and mock tool are offline and flexible,
>> one of the difficulties is how to log coming requests. Registering it to
>> every flask app or wsgis needs lots of work, that's why I am looking for a
>> way to implement a "middleware" between apache/wsgi_mod and flask app.
>>
>> Thanks,
>> Yang
>>
>>
>> On Friday, July 4, 2014 5:14:49 AM UTC-7, Graham Dumpleton wrote:
>>>
>>> Okay, seems you can now use WSGIImportScript inside of VirtualHost.
>>>
>>> A long long time ago this wasn't the case. Seems I fixed it at some
>>> point so that it is now possible but the docs weren't updated. I did look
>>> at the code to try and work it out when I responded but was looking in the
>>> wrong place for what dictates whether the directive can be used in a
>>> VirtualHost.
>>>
>>> So you don't need to worry about using WSGIScriptAlias at all. You can
>>> always use WSGIImportScript, placing it after the WSGIDaemonProcess
>>> directive for the daemon process group that the WSGIImportScript needs to
>>> refer to, even inside of the VirtualHost.
>>>
>>> Graham
>>>
>>> On 04/07/2014, at 9:25 PM, Graham Dumpleton <[email protected]>
>>> wrote:
>>>
>>>
>>> On 04/07/2014, at 7:58 PM, Graham Dumpleton <[email protected]>
>>> wrote:
>>>
>>> On 04/07/2014, at 5:52 PM, theliuy <[email protected]> wrote:
>>>
>>> I have several web services using Apache + mod_wsgi + Flask. For some
>>> debug purpose, I want to listen the incoming requests and log them. I don't
>>> want to modify flask applications, or wsgi scripts neither. Because there
>>> are too many existing projects, I can't do it one by one. And they are
>>> going to be deployed in multiple environment, I just need this feature in
>>> one or two of them.
>>>
>>> One possible solution come to my mind is to modify Flask framework.
>>> Implement a "logger" inside flask. I think it will be better if I can find
>>> a way to register a script in mod_wsgi. Let mod_wsgi run it whenever
>>> requests are coming.
>>>
>>> I don't know if there is any better way to make it. Please give me some
>>> hint.
>>>
>>>
>>> What about the request are you wanting to log exactly and for what
>>> purpose?
>>>
>>> I can explain a way of doing what you want which avoids you needing to
>>> change either the code of your application, Flask or any other package, but
>>> the context of what you are trying to capture and why will help as I will
>>> then know what sort of WSGI middleware I will need to employ to capture
>>> what you need.
>>>
>>> So if you can respond with that extra information and in the mean time I
>>> will validate that my idea for how to do it will work.
>>>
>>>
>>> I would still like to know what you are trying to capture, but if you
>>> really want to avoid making changes to any existing code or third party
>>> modules, you can do the following. The technique uses monkey patching,
>>> employing the `wrapt` library to simplify the process and ensure how the
>>> monkey patching is done is correct.
>>>
>>> First up, you need to have the `wrapt` module installed.
>>>
>>> pip install wrapt
>>>
>>>
>>> Next, create a file called 'patch.py' which contains:
>>>
>>> from __future__ import print_function
>>>
>>> from wrapt import when_imported, wrap_function_wrapper
>>>
>>> def flask_wsgi_app_wrapper(wrapped, instance, args, kwargs):
>>> def bind_call_args(environ, *args, **kwargs):
>>> return environ
>>>
>>> environ = bind_call_args(*args, **kwargs)
>>>
>>> print(10*'>')
>>> for key in sorted(environ.keys()):
>>> print('%s: %s' % (key, repr(environ[key])))
>>> print(10*'<')
>>>
>>> return wrapped(*args, **kwargs)
>>>
>>> @when_imported('flask.app')
>>> def instrument_flask_app(module):
>>> wrap_function_wrapper(module, 'Flask.wsgi_app',
>>> flask_wsgi_app_wrapper)
>>>
>>>
>>> What needs to be done now is that this 'patch.py' file has to be
>>> imported somehow and it ideally needs to be imported before the actual WSGI
>>> application script file is imported. It preferably would just have been
>>> imported at the start of the specific WSGI script file for the application
>>> you want to add debugging to. But since you don't want to do that, then you
>>> will need to modify the Apache configuration file and use mod_wsgi to
>>> preload it.
>>>
>>> What exactly you need to do here may depend on whether you are using
>>> mod_wsgi embedded mode or daemon mode. If using daemon mode, it may also
>>> depend on whether the WSGIDaemonProcess directives are specified inside of
>>> the context of a VirtualHost, or outside at global server scope.
>>>
>>> The problem here is that the WSGIImportScript can only be defined at
>>> global server scope. That is, it cannot be specified inside of a
>>> VirtualHost, but has to be outside.
>>>
>>> Right now I can't remember the rules about whether a WSGIImportScript
>>> can refer to a daemon process group specified using WSGIDaemonProcess
>>> directive inside of a VirtualHost.
>>>
>>> If it can, then the WSGIImportScript directive will at least have to be
>>> added after the VirtualHost containing the WSGIDaemonProcess.
>>>
>>> Anyway, if you are using embedded mode at least, then you need to use
>>> the WSGIImportScript directive as:
>>>
>>> WSGIImportScript /some/path/patch.py process-group=%{GLOBAL}
>>> application-group=%{GLOBAL}
>>>
>>>
>>> The application-group option should be set to the specific application
>>> group context that the WSGI application you want to debug, is running in.
>>>
>>> What will happen is that when the Apache child worker processes are
>>> started up and Python gets initialised, mod_wsgi will preload the patch.py
>>> file into the Python interpreter.
>>>
>>> In being imported, the @with_imported decorator will register that the
>>> instrument_flask_app() function should be called when the 'flask.app'
>>> module is imported by the WSGI application.
>>>
>>> When the module is the flask.app module is import, the
>>> instrument_flask_app() function is executed at that point, it will be
>>> passed the flask.app module before the import even returns the module to
>>> the code that imported it. The Flask.wsgi_app() method will then have a
>>> function wrapper applied using wrap_function_wrapper().
>>>
>>> Later when handling a WSGI request, the Flask.wsgi_app() will be called.
>>> At that point the flask_wsgi_app_wrapper() wrapper will actually be called.
>>> Inside of that we bind the function arguments to extract the 'environ'
>>> argument passed to the WSGI application entry point for Flask. The wrapper
>>> function can then print out what it wants from 'environ'. Finally, the
>>> wrapper will call the original wrapped Flask.wsgi_app() method so the WSGI
>>> application can handle the request.
>>>
>>> So that is the principle of how it works, but that Apache configuration
>>> only works with embedded mode for sure. If using daemon mode we may not be
>>> able to use WSGIImportScript however and may have to use another trick.
>>>
>>> For daemon mode therefore, if the WSGIDaemonProcess directive is
>>> actually specified outside of any VirtualHost, then we can still use
>>> WSGIImportScript. Thus you would use:
>>>
>>> WSGIImportScript /some/path/patch.py process-group=group
>>> application-group=%{GLOBAL}
>>>
>>>
>>> The process-group option would be set to the daemon process group name,
>>> and application-group option again set to the specific application group
>>> context that the WSGI application you want to debug, is running in.
>>>
>>> The WSGIImportScript in this case should come after the
>>> WSGIDaemonProcess directive it is referring to.
>>>
>>> Now if the WSGIDaemonProcess group directive is inside of the
>>> VirtualHost, I am not sure if one can do:
>>>
>>> <VirtualHost *:80>
>>> ...
>>> WSGIDaemonProcess group
>>> ...
>>> </VirtualHost>
>>>
>>> WSGIImportScript /some/path/patch.py process-group=group
>>> application-group=%{GLOBAL}
>>>
>>>
>>> If that doesn't work, we have to cheat a bit and do:
>>>
>>> <VirtualHost *:80>
>>> ...
>>> WSGIDaemonProcess group
>>> WSGIScriptAlias /.patch /some/path/patch.py process-group=group
>>> application-group=%{GLOBAL}
>>> <Location /.patch>
>>> Deny from all
>>> </Location>
>>> ...
>>> </VirtualHost>
>>>
>>>
>>> In other words, we actually use WSGIScriptAlias and the fact that if
>>> both process-group and application-group are specified, that the WSGI
>>> script file would normally be preloaded.
>>>
>>> Since we have to specify a URL as the mount point and there will be no
>>> actual WSGI application in the script file, then we use a Location block to
>>> block access to the URL to force a forbidden HTTP response if someone tries
>>> to access that URL.
>>>
>>> That therefore represents a technical solution for what I believe you
>>> want. The question now is if you really need to do what you think you do.
>>>
>>> Note that if you are really after a monitoring solution for evaluating
>>> application performance, then there are better ways of doing what you want.
>>>
>>> If you do try it and have the case of using WSGIDaemonProcess inside of
>>> a VirtualHost, do tell me if WSGIImportScript outside of the VirtualHost
>>> can still refer to it. I will try and test it myself, but it isn't
>>> convenient to do so right now.
>>>
>>> Graham
>>>
>>>
>>>
>> --
>> You received this message because you are subscribed to the Google Groups
>> "modwsgi" group.
>> To unsubscribe from this group and stop receiving emails from it, send an
>> email to [email protected].
>> To post to this group, send email to [email protected].
>> Visit this group at http://groups.google.com/group/modwsgi.
>> For more options, visit https://groups.google.com/d/optout.
>>
>>
>>
> --
> You received this message because you are subscribed to the Google Groups
> "modwsgi" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to [email protected] <javascript:>.
> To post to this group, send email to [email protected] <javascript:>
> .
> Visit this group at http://groups.google.com/group/modwsgi.
> For more options, visit https://groups.google.com/d/optout.
>
>
>
--
You received this message because you are subscribed to the Google Groups
"modwsgi" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to [email protected].
To post to this group, send email to [email protected].
Visit this group at http://groups.google.com/group/modwsgi.
For more options, visit https://groups.google.com/d/optout.
