[issue21472] Fix wsgiref handling of absolute HTTP Request-URI
mouad added the comment: Hi Guido, If I understand this correctly, the HOST header was added only in HTTP1.1 and setting the absolute URI was the right behavior client should follow if they are behind a proxy for HTTP1.0, but the same behavior was kept in HTTP1.1 for backward compatibility. In any case I would worry (a bit) that this might cause security issues if implemented as naively as shown in your patch, the other components of the URL should probably be validated against the configuration of the server. Also I am wondering whether specifying a different port or protocol (e.g. HTTPS) should be allowed or not. If there should be a validation, I think it should be done in BaseHTTPRequestHandler, FWIW this later doesn't validate HOST header neither, just tested sending a request to python -mhttp.server which succeeded. $ telnet 127.0.0.1 8000 GET /dummy HTTP/1.1 HOST google.com One thing to note here for future work for the validation part, is that according to http://www.w3.org/Protocols/rfc2616/rfc2616-sec5.html#sec5.2, first point: If Request-URI is an absoluteURI, the host is part of the Request-URI. Any Host header field value in the request MUST be ignored. and what to do with extraneous parts of the path (in particular the #fragment identifier) AFAIK clients are not supposed to send fragments to servers, and I didn't find in the HTTP spec what should happen if they do, CherryPy for example (link of the code is in the footer) will raise 400 if request URI include the #fragment part. An other thing that CherryPy guys also did, is that ``HTTPRequest(...).path`` will always return a relativ path, which is IMHO the right behavior but for backward compatibility I hesitate fixing this problem directly in BaseHTTPRequestHandler or should we ? You should probably also be careful with path-less domain names -- IIRC some URL parsers produce as the path for e.g. http://python.org;. According to the PEP-0333 the PATH_INFO can be empty: PATH_INFO The remainder of the request URL's path, designating the virtual location of the request's target within the application. This may be an empty string, if the request URL targets the application root and does not have a trailing slash. Have you asked for the status of this particular feature? I sent an email to python WEB-SIG mailing list trying to gather information about this behavior, but not luck yet :( At this point I would like to link to both CherryPy and Werkzeug that already handle this behavior. Werkzeug: https://github.com/mitsuhiko/werkzeug/blob/0.9.6/werkzeug/serving.py#L75-81 CherryPy: https://github.com/cherrypy/cherrypy/blob/cherrypy-3.2.2rc1/cherrypy/wsgiserver/wsgiserver3.py#L633-638. As a side note, I have already said in my email to the WEB-sig kudos to cherrypy guys, the funny part is that I am starting to think that they succeeded in both my tests because they didn't rely on core python implementation of BaseHTTPRequestHandler (I guess reinventing the wheel sometime works :)), at the opposite to other frameworks that showed this problems :). Cheers, -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue21472 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue21472] Fix wsgiref handling of absolute HTTP Request-URI
New submission from mouad: Hi, As most of you know, working behind a HTTP proxy raise all this corner cases that no one think about until he has to, one of them i had to deal with some months ago is absolute request URI in HTTP request, that some client will send when they detect that they are behind an HTTP proxy, to be more precise i am talking about this section of rfc2616 (https://gist.github.com/mouadino/7930974), i.e. GET http://domain.com/path HTTP/1.1 As my surprise i found that most WSGI server that are out there doesn't support very well this feature including wsgiref, to test that i have created this simple python script https://gist.github.com/mouadino/7930974 that include also status of this bug in different other projects. In term of bug impact, if a WSGI server doesn't support this, the WSGI environment variable PATH_INFO will end up containing an absolute URI instead of relative as the standard require (http://legacy.python.org/dev/peps/pep-0333/), which mean that most uri routing that web framework do end up failing (Returning 404) b/c most of them will try to much PATH_INFO with there different routes (which are relative paths) to find which controller action should they executed e.g. https://pypi.python.org/pypi/Routes . FWIW i found this bug while trying to debug boto library (that use absolute uri if proxy is detected https://github.com/mouadino/boto/blob/v2.13.2/boto/connection.py#L955) that was communicating with an OpenStack API that use eventlet wsgi server (That doesn't handle absolute URI https://github.com/eventlet/eventlet/blob/v0.14/eventlet/wsgi.py#L471) and routes (that match PATH_INFO with action path https://github.com/bbangert/routes/blob/master/routes/mapper.py#L678). -- components: Library (Lib) messages: 218258 nosy: mouad, pje priority: normal severity: normal status: open title: Fix wsgiref handling of absolute HTTP Request-URI type: behavior versions: Python 2.7, Python 3.1, Python 3.2, Python 3.3, Python 3.4, Python 3.5 ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue21472 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue21472] Fix wsgiref handling of absolute HTTP Request-URI
Changes by mouad mouad...@gmail.com: -- keywords: +patch Added file: http://bugs.python.org/file35214/issue21472.patch ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue21472 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue12038] assertEqual doesn't display newline differences quite well
mouad mouad...@gmail.com added the comment:
I took on consideration the two suggestion that was in the issue and create
this patch that basically show only until the last character that differ and
truncate the rest.
I just included one test case, other test case if they should be included (and
if my solution got accepted :)), should be more *complex* and check using Regex
the correctness of the assertion message, but here is the output that we can
see when applying the patch:
tc.assertEqual(x\n * 40 + \n, x\n * 40 + \r\n)
'x\nx\nx\nx\nx\nx\nx\nx\nx\nx\nx\nx\nx\n...nx\nx\nx\nx\nx\nx\nx\nx\nx\nx\nx\nx\n\n
[truncated]... !=
'x\nx\nx\nx\nx\nx\nx\nx\nx\nx\nx\nx\nx\n...\nx\nx\nx\nx\nx\nx\nx\nx\nx\nx\nx\n\r
[truncated]...
x
x
x
x
x
x
x
x
x
x
x
x
x
x
x
x
x
x
x
x
x
x
x
x
x
x
x
x
x
x
x
x
x
x
x
x
x
x
x
x
-
+
tc.assertEqual('', 'abbb')
AssertionError: 'aa [truncated]... != 'ab [truncated]...
-
+ abbb
tc.assertEqual('', '')
AssertionError: 'a [truncated]... != 'b [truncated]...
-
+
tc.assertEqual(x\n * 80 + \n, x\n * 80 + \r)
AssertionError:
'x\nx\nx\nx\nx\nx\nx\nx\nx\nx\nx\nx\nx\n...nx\nx\nx\nx\nx\nx\nx\nx\nx\nx\nx\nx\n\n
[truncated]... !=
'x\nx\nx\nx\nx\nx\nx\nx\nx\nx\nx\nx\nx\n...nx\nx\nx\nx\nx\nx\nx\nx\nx\nx\nx\nx\n\r
[truncated]...
--
keywords: +patch
nosy: +mouad
Added file: http://bugs.python.org/file22486/issue-12038.patch
___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue12038
___
___
Python-bugs-list mailing list
Unsubscribe:
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue12157] join method of multiprocessing Pool object hangs if iterable argument of pool.map is empty
Changes by mouad mouad...@gmail.com: -- keywords: +patch Added file: http://bugs.python.org/file22459/pool.patch ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue12157 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue12157] join method of multiprocessing Pool object hangs if iterable argument of pool.map is empty
Changes by mouad mouad...@gmail.com: Added file: http://bugs.python.org/file22460/test_multiprocess.patch ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue12157 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue12157] join method of multiprocessing Pool object hangs if iterable argument of pool.map is empty
Changes by mouad mouad...@gmail.com: Added file: http://bugs.python.org/file22461/support.patch ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue12157 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue12157] join method of multiprocessing Pool object hangs if iterable argument of pool.map is empty
mouad mouad...@gmail.com added the comment: Hello, This is my first patch to cpython, hope it will be accepted :) The fix that i did is to remove the ResultMap instance from the pool cache when the iterable is empty. In general here is what happen: The map method create a MapResult instance, which add it self automatically to the pool._cache and this ResultMap instance will be used by the task that will be created and added after in the pool._taskqueue to communicate the task result, but in case of an empty iterable the task will not be created and we will end up with a MapResult with no task and when we will try to join the pool, it will hang waiting for the task to set the result in the MapResult instance. For the test i created a new helper `operation_timeout` that is used as a contextmanager to make sure that the test will not hang for ever, i don't know if it's useful maybe just running the test without checking for any timeout is more *realistic*. -- nosy: +mouad ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue12157 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue12157] join method of multiprocessing Pool object hangs if iterable argument of pool.map is empty
Changes by mouad mouad...@gmail.com: Removed file: http://bugs.python.org/file22459/pool.patch ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue12157 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue12157] join method of multiprocessing Pool object hangs if iterable argument of pool.map is empty
Changes by mouad mouad...@gmail.com: Removed file: http://bugs.python.org/file22460/test_multiprocess.patch ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue12157 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue12157] join method of multiprocessing Pool object hangs if iterable argument of pool.map is empty
Changes by mouad mouad...@gmail.com: Removed file: http://bugs.python.org/file22461/support.patch ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue12157 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue12157] join method of multiprocessing Pool object hangs if iterable argument of pool.map is empty
Changes by mouad mouad...@gmail.com: Added file: http://bugs.python.org/file22466/issue-12157.patch ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue12157 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue12410] Create a new helper function that enable to test that an operation don't hang more than a given timeout.
New submission from mouad mouad...@gmail.com: While working on issue #12157 [http://bugs.python.org/issue12157], I needed a function that make sure that an operation will not hang forever, for this reason i have create this helper function that support the context manager protocol and accept a timeout as an argument and raise an IOError if the operation didn't terminate before that timeout. -- components: Tests files: operation_timeout.patch keywords: patch messages: 139075 nosy: mouad priority: normal severity: normal status: open title: Create a new helper function that enable to test that an operation don't hang more than a given timeout. type: feature request versions: Python 3.2 Added file: http://bugs.python.org/file22467/operation_timeout.patch ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue12410 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue12157] join method of multiprocessing Pool object hangs if iterable argument of pool.map is empty
mouad mouad...@gmail.com added the comment: The test case use a helper function in test/support.py that i have proposed in issue #12410. I'm dropping this comment here because i don't have the rights to edit the issue dependency. cheers; -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue12157 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue12410] Create a new helper function that enable to test that an operation don't hang more than a given timeout.
Changes by mouad mouad...@gmail.com: Removed file: http://bugs.python.org/file22467/operation_timeout.patch ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue12410 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue12410] Create a new helper function that enable to test that an operation don't hang more than a given timeout.
Changes by mouad mouad...@gmail.com: Added file: http://bugs.python.org/file22469/operation_timeout.patch ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue12410 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue12410] Create a new helper function that enable to test that an operation don't hang more than a given timeout.
mouad mouad...@gmail.com added the comment: Thanks for the instructive feedback :) I totally agree i guess there is a lot of issues that i didn't think of :-(, my first thinking was to use Pool.join timeout argument but it was removed in 3.2 (by the way i didn't find the issue or the rational that lead to this change). And now that i know about faulthandler module i guess that will make also my life easier :), i will rewrite the patch in the issue #12157 to not use any *fancy* way to check if it will hang. cheers, -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue12410 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue12157] join method of multiprocessing Pool object hangs if iterable argument of pool.map is empty
Changes by mouad mouad...@gmail.com: Added file: http://bugs.python.org/file22475/issue-12157.patch ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue12157 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue12157] join method of multiprocessing Pool object hangs if iterable argument of pool.map is empty
mouad mouad...@gmail.com added the comment: Here is a new patch that in the opposite of the first one, it don't try to check if the pool.join() will hang or no, after a discussion with neologix in issue #12410 . -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue12157 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
