Re: [modwsgi] Merged WWW-Authenticate response headers
This issue has been address in mod_wsgi 4.0 trunk in source code repository if you can check that it now works as required. Graham On 28 October 2011 09:43, Graham Dumpleton graham.dumple...@gmail.com wrote: Here is the reason. For daemon mode, partially out of convenience as could reuse some routines provided by Apache, the format of the response on the socket returned from daemon process to Apache child process is the same as that which would be returned by a CGI script. The code then uses an existing function in Apache to parse that CGI type response and back fill it into the Apache request structure. In doing that the code has: /* The HTTP specification says that it is legal to merge duplicate * headers into one. Some browsers that support Cookies don't like * merged headers and prefer that each Set-Cookie header is sent * separately. Lets humour those browsers by not merging. * Oh what a pain it is. */ So, it is mindful of joining headers, but only for the Cookie case. Normally the WWW-Authenticate case wouldn't be a problem because the original Authorization header can never be passed to a CGI script and it wouldn't be able to handle authorisation and so generate WWW-Authenticate. To fix it would mean making a copy of the existing function in Apache I am using and never have it merge headers. This is probably doable and would simplify things. There are extra protections in that function around squashing duplicates of other response headers as well, however that might also being done on daemon mode side anyway so may not be a problem. Duplicating the function would also mean that can generate more specific errors messages related to mod_wsgi when timeout occurs rather than the current cryptic CGI heritage message. Can you create a ticket on the mod_wsgi issue tracker with information from these posts? I am busy coming up to a GA release of product at work at the moment, but after that intend to come back to do some work on mod_wsgi and try and get a 4.0 release out with some new features specifically related to product doing at work. So sorry, not sure I have a quick solution right now. Don't know if mod_headers is flexible enough to allow you to effectively split a response header into two. http://httpd.apache.org/docs/current/mod/mod_headers.html It may be possible using some convoluted sequence of directives. Not sure that mod_rewrite is of any help as don't think it works on responses. Graham On 28 October 2011 09:02, Mark Nevill mark.nev...@gmail.com wrote: Hi On 27 October 2011 23:38, Graham Dumpleton graham.dumple...@gmail.com wrote: Can you verify this is only the case for mod_wsgi daemon mode. Confirmed. Following responses were generated with identical code but toggling of a WSGIProcessGroup wsgidev directive: [begin] $ curl --include [...]/test.wsgi/auth HTTP/1.1 401 Unauthorized Date: Thu, 27 Oct 2011 21:58:19 GMT Server: Apache/2.2.16 (Debian) WWW-Authenticate: Negotiate, Basic realm=Internal Area Content-Length: 43 Vary: Accept-Encoding Content-Type: text/plain DAEMON MODE: process_group=wsgidev [end] vs [begin] $ curl --include [...]/test.wsgi/auth HTTP/1.1 401 Unauthorized Date: Thu, 27 Oct 2011 21:58:51 GMT Server: Apache/2.2.16 (Debian) WWW-Authenticate: Negotiate WWW-Authenticate: Basic realm=Internal Area Content-Length: 40 Vary: Accept-Encoding Content-Type: text/plain EMBEDDED MODE: no process_group [end] Regards, Mark -- You received this message because you are subscribed to the Google Groups modwsgi group. To post to this group, send email to modwsgi@googlegroups.com. To unsubscribe from this group, send email to modwsgi+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/modwsgi?hl=en. -- You received this message because you are subscribed to the Google Groups modwsgi group. To post to this group, send email to modwsgi@googlegroups.com. To unsubscribe from this group, send email to modwsgi+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/modwsgi?hl=en.
[modwsgi] Controlling the upload of large files
I'm trying to implement a limit on the size of uploaded files which depends
on a user's status, i.e. a user who pays gets to upload larger files than
one who doesn't. I want to check the size of the upload BEFORE reading the
data because I don't want to read 27 gigabytes only to find that this user
is limited to 1MB. I can't user Apache's LimitRequestBody directive. The
filtering has to be done within the application because it's the only part
of the system that knows the user's status.
I found this:
http://mail.python.org/pipermail/web-sig/2008-November/003639.html
which says:
If you use embedded mode, so long as your WSGI application doesn't
read the input and just returns the error response, the request
content wouldn't be read at all.
So I tried this:
def application(environ, start_response):
status = '200 OK'
headers = [('Content-type', 'text/html'),('Connection','close')]
out = start_response(status, headers)
out('--%s--br' % environ.get('CONTENT_LENGTH'))
return ['''form method=post enctype=multipart/form-data
input name=f type=file
input type=submit/form''']
and ran it using:
WSGIScriptAlias /wsgitest /path/to/driver.wsgi
location /wsgitest
WSGIApplicationGroup %{GLOBAL}
/Location
which as far as I can tell should result in running in embedded mode. This
seems like it *should* display the size of the uploaded file without
reading the file data. However, empirically, the data is being read. If I
upload a large file, there's a long delay before getting a response.
So what am I doing wrong?
Thanks, and Happy Thanksgiving!
--
You received this message because you are subscribed to the Google Groups
modwsgi group.
To view this discussion on the web visit
https://groups.google.com/d/msg/modwsgi/-/rEYG3K-zj74J.
To post to this group, send email to modwsgi@googlegroups.com.
To unsubscribe from this group, send email to
modwsgi+unsubscr...@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/modwsgi?hl=en.
Re: [modwsgi] Controlling the upload of large files
If I recall correctly, when I wrote the code to read uploaded files, it was
one chunk at a time (eg 4096 bytes). Wouldn't this give you the
opportunity to stop at the right point?
take a look at github.com/appcove/AppStruct ...
Python/AppStruct/WSGI/Lib.py (or related, maybe werkseug.py) to see the
code.
Take care.
On Nov 24, 2011 1:48 PM, Lisper ron.gar...@gmail.com wrote:
I'm trying to implement a limit on the size of uploaded files which
depends on a user's status, i.e. a user who pays gets to upload larger
files than one who doesn't. I want to check the size of the upload BEFORE
reading the data because I don't want to read 27 gigabytes only to find
that this user is limited to 1MB. I can't user Apache's LimitRequestBody
directive. The filtering has to be done within the application because
it's the only part of the system that knows the user's status.
I found this:
http://mail.python.org/pipermail/web-sig/2008-November/003639.html
which says:
If you use embedded mode, so long as your WSGI application doesn't
read the input and just returns the error response, the request
content wouldn't be read at all.
So I tried this:
def application(environ, start_response):
status = '200 OK'
headers = [('Content-type', 'text/html'),('Connection','close')]
out = start_response(status, headers)
out('--%s--br' % environ.get('CONTENT_LENGTH'))
return ['''form method=post enctype=multipart/form-data
input name=f type=file
input type=submit/form''']
and ran it using:
WSGIScriptAlias /wsgitest /path/to/driver.wsgi
location /wsgitest
WSGIApplicationGroup %{GLOBAL}
/Location
which as far as I can tell should result in running in embedded mode.
This seems like it *should* display the size of the uploaded file without
reading the file data. However, empirically, the data is being read. If I
upload a large file, there's a long delay before getting a response.
So what am I doing wrong?
Thanks, and Happy Thanksgiving!
--
You received this message because you are subscribed to the Google Groups
modwsgi group.
To view this discussion on the web visit
https://groups.google.com/d/msg/modwsgi/-/rEYG3K-zj74J.
To post to this group, send email to modwsgi@googlegroups.com.
To unsubscribe from this group, send email to
modwsgi+unsubscr...@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/modwsgi?hl=en.
--
You received this message because you are subscribed to the Google Groups
modwsgi group.
To post to this group, send email to modwsgi@googlegroups.com.
To unsubscribe from this group, send email to
modwsgi+unsubscr...@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/modwsgi?hl=en.
Re: [modwsgi] Controlling the upload of large files
No, the problem is that the entire file is (apparently) read by modwsgi before the application code is run at all. -- You received this message because you are subscribed to the Google Groups modwsgi group. To view this discussion on the web visit https://groups.google.com/d/msg/modwsgi/-/tQ3BVEA08bAJ. To post to this group, send email to modwsgi@googlegroups.com. To unsubscribe from this group, send email to modwsgi+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/modwsgi?hl=en.
Re: [modwsgi] Controlling the upload of large files
I am pretty sure this is not the case... Looking forward to Graham's comments. Look at docs on environ['wsgi.input']. Thanks! On Nov 24, 2011 3:00 PM, Lisper ron.gar...@gmail.com wrote: No, the problem is that the entire file is (apparently) read by modwsgi before the application code is run at all. -- You received this message because you are subscribed to the Google Groups modwsgi group. To view this discussion on the web visit https://groups.google.com/d/msg/modwsgi/-/tQ3BVEA08bAJ. To post to this group, send email to modwsgi@googlegroups.com. To unsubscribe from this group, send email to modwsgi+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/modwsgi?hl=en. -- You received this message because you are subscribed to the Google Groups modwsgi group. To post to this group, send email to modwsgi@googlegroups.com. To unsubscribe from this group, send email to modwsgi+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/modwsgi?hl=en.
Re: [modwsgi] Controlling the upload of large files
Do you mean sent rather than read? Except for Opera, browsers don't implement 100-continue and so the browser will always send the huge upload anyway. If the browser said it is using HTTP/1.1 then Apache doesn't have a choice but to still read the entire request content to throw it away if you return a 200 response. This is because there may be another request following the first over the same connection. So the problem is the browsers that send the data anyway. Try your test again but don't return a 200 response, instead return a 413 request entity too large error response. When a non 200 response is returned Apache will know it is an error and should just send the response and not also try and consume the request content as when it is an error response browsers aren't supposed to send a subsequent request over the same connection. Graham On 25 November 2011 07:00, Lisper ron.gar...@gmail.com wrote: No, the problem is that the entire file is (apparently) read by modwsgi before the application code is run at all. -- You received this message because you are subscribed to the Google Groups modwsgi group. To view this discussion on the web visit https://groups.google.com/d/msg/modwsgi/-/tQ3BVEA08bAJ. To post to this group, send email to modwsgi@googlegroups.com. To unsubscribe from this group, send email to modwsgi+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/modwsgi?hl=en. -- You received this message because you are subscribed to the Google Groups modwsgi group. To post to this group, send email to modwsgi@googlegroups.com. To unsubscribe from this group, send email to modwsgi+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/modwsgi?hl=en.
Re: [modwsgi] Controlling the upload of large files
BTW, don't expect to see your nice message in the browser even when using 413. Because a browser not implementing 100-continue as it should, they usually don't try and deal with a response before sending all data. Because Apache will close input side of connection, the browser will fail sending all the content and general throws up a nasty connection error rather than read the error response and show it. Graham On 25 November 2011 07:28, Graham Dumpleton graham.dumple...@gmail.com wrote: Do you mean sent rather than read? Except for Opera, browsers don't implement 100-continue and so the browser will always send the huge upload anyway. If the browser said it is using HTTP/1.1 then Apache doesn't have a choice but to still read the entire request content to throw it away if you return a 200 response. This is because there may be another request following the first over the same connection. So the problem is the browsers that send the data anyway. Try your test again but don't return a 200 response, instead return a 413 request entity too large error response. When a non 200 response is returned Apache will know it is an error and should just send the response and not also try and consume the request content as when it is an error response browsers aren't supposed to send a subsequent request over the same connection. Graham On 25 November 2011 07:00, Lisper ron.gar...@gmail.com wrote: No, the problem is that the entire file is (apparently) read by modwsgi before the application code is run at all. -- You received this message because you are subscribed to the Google Groups modwsgi group. To view this discussion on the web visit https://groups.google.com/d/msg/modwsgi/-/tQ3BVEA08bAJ. To post to this group, send email to modwsgi@googlegroups.com. To unsubscribe from this group, send email to modwsgi+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/modwsgi?hl=en. -- You received this message because you are subscribed to the Google Groups modwsgi group. To post to this group, send email to modwsgi@googlegroups.com. To unsubscribe from this group, send email to modwsgi+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/modwsgi?hl=en.
Re: [modwsgi] Controlling the upload of large files
Wow, that really sucks. It does indeed appear to be the case that the server is doing the Right Thing and it's the browser that's screwing this up. Bummer. I guess it's time to break out the client-side Javascript filefrobber object :-( Thanks, Graham! -- You received this message because you are subscribed to the Google Groups modwsgi group. To view this discussion on the web visit https://groups.google.com/d/msg/modwsgi/-/hCysmYE5y0YJ. To post to this group, send email to modwsgi@googlegroups.com. To unsubscribe from this group, send email to modwsgi+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/modwsgi?hl=en.
[modwsgi] I use WSGIDaemonProcess server user=demon group=demon but it still use www-data user
I user ubuntu 10.04 Apache/2.2.14 mod_wsgi 2.8.
I use WSGIDaemonProcess server user=demon group=demon ,but when I
print sys.stderr,os.environ,I see the result:
'''
[Fri Nov 25 15:23:12 2011] [error] {'LANG': 'C', 'SHLVL': '1',
'APACHE_RUN_USER': 'www-data', 'APACHE_PID_FILE': '/var/run/
apache2.pid', 'PWD': '/', 'APACHE_RUN_GROUP': 'www-data', 'PATH': '/
usr/local/bin:/usr/bin:/bin', '_': '/usr/sbin/apache2'}
'''
How can I change the user wsgi running?
Thanks
--
You received this message because you are subscribed to the Google Groups
modwsgi group.
To post to this group, send email to modwsgi@googlegroups.com.
To unsubscribe from this group, send email to
modwsgi+unsubscr...@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/modwsgi?hl=en.
Re: [modwsgi] I use WSGIDaemonProcess server user=demon group=demon but it still use www-data user
On 25 November 2011 18:28, Chunlin Zhang zhangchun...@gmail.com wrote:
I user ubuntu 10.04 Apache/2.2.14 mod_wsgi 2.8.
I use WSGIDaemonProcess server user=demon group=demon ,but when I
print sys.stderr,os.environ,I see the result:
'''
[Fri Nov 25 15:23:12 2011] [error] {'LANG': 'C', 'SHLVL': '1',
'APACHE_RUN_USER': 'www-data', 'APACHE_PID_FILE': '/var/run/
apache2.pid', 'PWD': '/', 'APACHE_RUN_GROUP': 'www-data', 'PATH': '/
usr/local/bin:/usr/bin:/bin', '_': '/usr/sbin/apache2'}
'''
How can I change the user wsgi running?
You can't tell from APACHE_RUN_USER variable that will only say what
Apache child worker processes run as, not mod_wsgi daemon processes.
You need to use:
import os
os.geteuid()
501
Then see what the uid matches to in /etc/passwd file.
Also make sure you have a WSGIProcessGroup directive that tells your
app to run in daemon process group configured by WSGIDaemonProcess.
Graham
--
You received this message because you are subscribed to the Google Groups
modwsgi group.
To post to this group, send email to modwsgi@googlegroups.com.
To unsubscribe from this group, send email to
modwsgi+unsubscr...@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/modwsgi?hl=en.
[modwsgi] Re: I use WSGIDaemonProcess server user=demon group=demon but it still use www-data user
After I add WSGIProcessGroup directive,my app work ok now,but the
print sys.stderr,os.environ still www-data.
It works now but weird.
On Fri, Nov 25, 2011 at 3:28 PM, Chunlin Zhang zhangchun...@gmail.com wrote:
I user ubuntu 10.04 Apache/2.2.14 mod_wsgi 2.8.
I use WSGIDaemonProcess server user=demon group=demon ,but when I
print sys.stderr,os.environ,I see the result:
'''
[Fri Nov 25 15:23:12 2011] [error] {'LANG': 'C', 'SHLVL': '1',
'APACHE_RUN_USER': 'www-data', 'APACHE_PID_FILE': '/var/run/
apache2.pid', 'PWD': '/', 'APACHE_RUN_GROUP': 'www-data', 'PATH': '/
usr/local/bin:/usr/bin:/bin', '_': '/usr/sbin/apache2'}
'''
How can I change the user wsgi running?
Thanks
--
You received this message because you are subscribed to the Google Groups
modwsgi group.
To post to this group, send email to modwsgi@googlegroups.com.
To unsubscribe from this group, send email to
modwsgi+unsubscr...@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/modwsgi?hl=en.
