[issue4953] cgi module cannot handle POST with multipart/form-data in 3.x
Pierre Quentel pierre.quen...@gmail.com added the comment: @Victor Thanks for the comments - I don't understand why FieldStorage changes sys.stdout and sys.stderr (see remarks about IOMix above): please remove the charset argument (it is also confusing to have two encoding arguments). it should be done somewhere else done please remove the O_BINARY hack: the patch is for Python 3.2 and I closed issue #10841. If you would like a backport, another patch should be written later done encoding = 'latin-1' # ?: write a real comment or remove it I removed this part 'self.fp.read(...) # bytes': you should add a test on the type if you are not sure that fp.read() gives bytes added tests in read_urlencoded(), read_multi() and read_binary() file: the file(-like) object from which you can read the data *as bytes*: you should mention that TextIOWrapper is also tolerated (accepted?) not done : here file is not the argument passed to the FieldStorage constructor, but the attribute of values returned from calls to FieldStorage. In the new implementation, its read() method always returns bytes you may set fp directly to sys.stdin.buffer (instead of sys.stdin) if fp is None (it will be easier after removing the O_BINARY thing) done the patch adds a tab in an empty line, please don't do that :-) done (hopefully :-) you should add a (private?) attribute to FieldStorage to decide if it works on bytes or unicode, instead of using self.filename is not None test (eg. self._use_bytes = (self.filename is not None) done i don't like the idea of having a generic self.__write() method supporting bytes and unicode. it would prefer two methods, eg. self.__write_text() and self.__write_binary() (they can share a third private method) not done, the argument of __write is always bytes i don't like stream_encoding name: what is the stream here? do you process a file, a string or a stream? why not just self.encoding? done - import email.parser,email.feedparser one import is useless here. I prefer from email.feedparser import FeedParser because you get directly a ImportError if the symbol is missing. And it's already faster to get FeedParser instead of email.feedparser.FeedParser in a loop (dummy micro-optimization) done even I like the following change, please do it in a separated patch: -if type(value) is type([]): +if isinstance(value,list): not done I really don't like the IOMix thing: removed I vote +0 to commit the patch now (if the release manager agrees), and +1 if all of my remarks are fixed. should be close to +0.8 now ;-) -- Added file: http://bugs.python.org/file20402/cgi_32.patch ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue4953 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue4953] cgi module cannot handle POST with multipart/form-data in 3.x
Changes by Pierre Quentel pierre.quen...@gmail.com: Removed file: http://bugs.python.org/file20387/cgi_32.patch ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue4953 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue4953] cgi module cannot handle POST with multipart/form-data in 3.x
Etienne Robillard e...@gthcfoundation.org added the comment: +1 thanks for this input. I agree for the most part. However if the io semantics in python 3 is radically different than on python 2, I could have expected that WSGI scripts would similarly depend on a newer type of file descriptor access using the ``sys`` module. cheers! -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue4953 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue4953] cgi module cannot handle POST with multipart/form-data in 3.x
Glenn Linderman v+pyt...@g.nevcal.com added the comment: Pierre, Thank you for the new patch, with the philosophy of it's broke, so let's produce something the committers like to get it fixed. I see you overlooked removing the second use of O_BINARY. Locally, I removed that also, and tested your newest patch, and it still functions great for me. -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue4953 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue4953] cgi module cannot handle POST with multipart/form-data in 3.x
Pierre Quentel pierre.quen...@gmail.com added the comment: Glenn, you read my mind ;-) Thanks for mentioning the O_BINARY thing. New (last !) patch attached -- Added file: http://bugs.python.org/file20403/cgi_32.patch ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue4953 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue4953] cgi module cannot handle POST with multipart/form-data in 3.x
Changes by Pierre Quentel pierre.quen...@gmail.com: Removed file: http://bugs.python.org/file20402/cgi_32.patch ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue4953 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue4953] cgi module cannot handle POST with multipart/form-data in 3.x
STINNER Victor victor.stin...@haypocalc.com added the comment: r87996+r87997 adds encoding and errors argument to parse_qs() and parse_qsl() of urllib.parse. It is needed to decoded correctly %XX syntax in cgi. r87998 is the patch on the cgi module. Changes with cgi_32.patch: * Use TextIOWrapper instead of TextIOBase, because TextIOBase has no buffer attribute * typo in a docstring: it must must = must * (docstring) default: sys.stdin = default: sys.stdin.buffer * PEP 8: hasattr(a,b) = hasattr(a, b) (same for isinstance) and encoding = 'utf-8' = encoding='utf-8' (in the argument list) * xxx.decode(...) # str: remove useless # str comment. decode() always give unicode in Python 3 (same change for .encode() # bytes) * Rename next variables to next_boundary because next is a builtin function in Python 3 (unrelated change). * FieldStorage.innerboundary and FieldStorage.outerboundary are bytes objects: encode innerboundary in the constructor, and raise an error if outerboundary is not a bytes object * Rename _use_bytes to _binary_file * isinstance(bytes) test: write the type, not the value, in the error message * Replace line[:2] == b'--' by line.startswith(b'--'), and then replace line.strip() by line.rstrip() * test_fieldstorage_multipart() uses ASCII (and specifiy the encoding to FieldStorage) * add FieldStorage.errors attribute: pass it to parse_qsl() * add errors attribute to FieldStorage: same default value than urllib.parse.unquote(): 'replace' * parse(): pass encoding argument to parse_qs() * FieldStorage: pass encoding and errors arguments to parse_qsl() Because the patch on TextIOBase, it patched the docstring: --- fp : file pointer; default: sys.stdin.buffer (not used when the request method is GET) Can be : 1. an instance of (a subclass of) TextIOWrapper, in this case it must provide an attribute buffer = the binary layer that returns bytes from its read() method, and preferably an attribute encoding (defaults to latin-1) 2. an object whose read() and readline() methods return bytes --- becomes --- fp : file pointer; default: sys.stdin.buffer (not used when the request method is GET) Can be : 1. a TextIOWrapper object 2. an object whose read() and readline() methods return bytes --- Replace type(value) is type([]) is done in another commit: r87999. I consider that the work on this issue is done, and so I close it. If I am wrong, explain why and repoen the issue. Please test the cgi as much as possible before Python 3.2 final: reopen the issue if it doesn't work. -- resolution: - fixed status: open - closed ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue4953 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue4953] cgi module cannot handle POST with multipart/form-data in 3.x
STINNER Victor victor.stin...@haypocalc.com added the comment: Oh, I forgot to credit the author(s): who wrote the patch? -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue4953 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue4953] cgi module cannot handle POST with multipart/form-data in 3.x
Pierre Quentel pierre.quen...@gmail.com added the comment: Thanks a lot Victor ! I wrote the patch : Pierre Quentel (pierre.quen...@gmail.com) with many inputs by Glenn Linderman 2011/1/14 STINNER Victor rep...@bugs.python.org STINNER Victor victor.stin...@haypocalc.com added the comment: Oh, I forgot to credit the author(s): who wrote the patch? -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue4953 ___ -- title: cgi module cannot handle POST with multipart/form-data in3.x - cgi module cannot handle POST with multipart/form-data in 3.x Added file: http://bugs.python.org/file20405/unnamed ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue4953 ___Thanks a lot Victor !brbrI wrote the patch : Pierre Quentel (a href=mailto:pierre.quen...@gmail.com;pierre.quen...@gmail.com/a) with many inputs by Glenn Lindermanbrbrdiv class=gmail_quote2011/1/14 STINNER Victor span dir=ltrlt;a href=mailto:rep...@bugs.python.org;rep...@bugs.python.org/agt;/spanbr blockquote class=gmail_quote style=margin: 0pt 0pt 0pt 0.8ex; border-left: 1px solid rgb(204, 204, 204); padding-left: 1ex;div class=imbr STINNER Victor lt;a href=mailto:victor.stin...@haypocalc.com;victor.stin...@haypocalc.com/agt; added the comment:br br /divOh, I forgot to credit the author(s): who wrote the patch?br br --br divdiv/divdiv class=h5br ___br Python tracker lt;a href=mailto:rep...@bugs.python.org;rep...@bugs.python.org/agt;br lt;a href=http://bugs.python.org/issue4953; target=_blankhttp://bugs.python.org/issue4953/agt;br ___br /div/div/blockquote/divbrdiv class=ogc-tooltip-class id=ogc_tooltip_id/div ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue4953] cgi module cannot handle POST with multipart/form-data in 3.x
STINNER Victor victor.stin...@haypocalc.com added the comment: TODO: Add more tests to test_cgi. What is the latest patch for test_cgi? -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue4953 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue4953] cgi module cannot handle POST with multipart/form-data in 3.x
Changes by Pierre Quentel pierre.quen...@gmail.com: Removed file: http://bugs.python.org/file20405/unnamed ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue4953 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue4953] cgi module cannot handle POST with multipart/form-data in 3.x
Pierre Quentel pierre.quen...@gmail.com added the comment: My latest patch for test_cgi is in cgi_32.patch I will try to add more tests later -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue4953 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue4953] cgi module cannot handle POST with multipart/form-data in 3.x
STINNER Victor victor.stin...@haypocalc.com added the comment: haypo What is the latest patch for test_cgi? quentel My latest patch for test_cgi is in cgi_32.patch Ok, but cgi_32.patch doesn't add any test. I only adapt existing tests for your other changes. I remove cgi_32.patch because it was commited. -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue4953 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue4953] cgi module cannot handle POST with multipart/form-data in 3.x
Changes by STINNER Victor victor.stin...@haypocalc.com: Removed file: http://bugs.python.org/file20403/cgi_32.patch ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue4953 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue4953] cgi module cannot handle POST with multipart/form-data in 3.x
STINNER Victor victor.stin...@haypocalc.com added the comment: Remove cgi_plus_tests.diff: it looks to be an old version of cgi_32.patch. @r.david.murray: Did you write cgi_plus_tests.diff, or is it based on the work on Pierre Quentel? -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue4953 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue4953] cgi module cannot handle POST with multipart/form-data in 3.x
Changes by STINNER Victor victor.stin...@haypocalc.com: Removed file: http://bugs.python.org/file20269/cgi_plus_tests.diff ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue4953 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue4953] cgi module cannot handle POST with multipart/form-data in 3.x
Changes by STINNER Victor victor.stin...@haypocalc.com: Removed file: http://bugs.python.org/file12750/tmpav1vve.html ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue4953 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue4953] cgi module cannot handle POST with multipart/form-data in 3.x
Changes by STINNER Victor victor.stin...@haypocalc.com: Removed file: http://bugs.python.org/file12751/tmpy44zj7.html ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue4953 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue4953] cgi module cannot handle POST with multipart/form-data in 3.x
STINNER Victor victor.stin...@haypocalc.com added the comment: Remove tmpy44zj7.html and tmpav1vve.html: a similar file is included in full_source_and_error.zip. -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue4953 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue4953] cgi module cannot handle POST with multipart/form-data in 3.x
Changes by Antoine Pitrou pit...@free.fr: -- nosy: -pitrou ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue4953 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue4953] cgi module cannot handle POST with multipart/form-data in 3.x
R. David Murray rdmur...@bitdance.com added the comment: Victor: we normally leave the patch file that was committed attached to the issue for future reference. The _plus_tests file was just the original patch plus the existing cgi tests adjusted to pass in bytes instead of strings to cgi, if I recall correctly. Nor original code of mine in either part :) -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue4953 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue4953] cgi module cannot handle POST with multipart/form-data in 3.x
Changes by Glenn Linderman v+pyt...@g.nevcal.com: -- versions: +Python 3.2 -Python 3.3 ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue4953 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue4953] cgi module cannot handle POST with multipart/form-data in 3.x
Glenn Linderman v+pyt...@g.nevcal.com added the comment: Thanks to Pierre for producing patch after patch and testing testing testing, and to Victor for committing it, as well as others that contributed in smaller ways, as I tried to. I look forward to 3.2 rc1 so I can discard all my temporary patched copies of cgi.py -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue4953 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue4953] cgi module cannot handle POST with multipart/form-data in 3.x
STINNER Victor victor.stin...@haypocalc.com added the comment: Because I'm unable to read the whole history and analyze each file attached to this issue, I opened #10911 to ask to write more tests for the cgi module. -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue4953 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue4953] cgi module cannot handle POST with multipart/form-data in 3.x
STINNER Victor victor.stin...@haypocalc.com added the comment: Le vendredi 14 janvier 2011 à 19:11 +, R. David Murray a écrit : Victor: we normally leave the patch file that was committed attached to the issue for future reference. Sorry, but there were too much files. I was trying to figure out if there is something useful in the files. The _plus_tests file was just the original patch plus the existing cgi tests adjusted to pass in bytes instead of strings to cgi, if I recall correctly. Nor original code of mine in either part :) Ok. -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue4953 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue4953] cgi module cannot handle POST with multipart/form-data in 3.x
Éric Araujo mer...@netwok.org added the comment: Small tip: To ease review, I recommend you work from a checkout of the Subversion py3k branch, using svn add if you have new files and then producing one svn diff of the whole checkout. It’s easier than looking at multiple files, even more so if they’re hidden in a zip. We programmers like text :) You can also remove outdated patches from this page to make it clearer. -- title: cgi module cannot handle POST with multipart/form-data in3.0 - cgi module cannot handle POST with multipart/form-data in 3.x ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue4953 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue4953] cgi module cannot handle POST with multipart/form-data in 3.x
Changes by Pierre Quentel pierre.quen...@gmail.com: Removed file: http://bugs.python.org/file20229/cgi_diff.txt ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue4953 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue4953] cgi module cannot handle POST with multipart/form-data in 3.x
Changes by Pierre Quentel pierre.quen...@gmail.com: Removed file: http://bugs.python.org/file20235/cgi_diff.txt ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue4953 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue4953] cgi module cannot handle POST with multipart/form-data in 3.x
Changes by Pierre Quentel pierre.quen...@gmail.com: Removed file: http://bugs.python.org/file20322/cgi_diff_20110109.txt ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue4953 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue4953] cgi module cannot handle POST with multipart/form-data in 3.x
Changes by Pierre Quentel pierre.quen...@gmail.com: Removed file: http://bugs.python.org/file20323/cgi_tests.zip ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue4953 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue4953] cgi module cannot handle POST with multipart/form-data in 3.x
Changes by Pierre Quentel pierre.quen...@gmail.com: Removed file: http://bugs.python.org/file20356/cgi_diff_20110111.txt ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue4953 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue4953] cgi module cannot handle POST with multipart/form-data in 3.x
Changes by Pierre Quentel pierre.quen...@gmail.com: Removed file: http://bugs.python.org/file20382/cgi_diff_20110112.txt ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue4953 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue4953] cgi module cannot handle POST with multipart/form-data in 3.x
Changes by Pierre Quentel pierre.quen...@gmail.com: Removed file: http://bugs.python.org/file20383/cgi_20110113.diff ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue4953 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue4953] cgi module cannot handle POST with multipart/form-data in 3.x
Changes by Pierre Quentel pierre.quen...@gmail.com: Removed file: http://bugs.python.org/file20384/test_cgi_20111013.diff ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue4953 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue4953] cgi module cannot handle POST with multipart/form-data in 3.x
Changes by Pierre Quentel pierre.quen...@gmail.com: Removed file: http://bugs.python.org/file20244/cgi_diff.txt ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue4953 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue4953] cgi module cannot handle POST with multipart/form-data in 3.x
Pierre Quentel pierre.quen...@gmail.com added the comment: Ok Eric, thanks for the tips I attach the diff for the 2 modified modules (cgi.py and test_cgi.py). For the other tests, they are not in the branch and there are many test files so I leave the zip file I removed outdated diffs -- Added file: http://bugs.python.org/file20387/cgi_32.patch ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue4953 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue4953] cgi module cannot handle POST with multipart/form-data in 3.x
R. David Murray rdmur...@bitdance.com added the comment: It getting in to 3.2 would be a release manager call, so I've set it to release blocker so Georg can make the call. My opinion is that while I would *really* like to see this fixed in 3.2, the changes really should have a thorough *design* review as well as a code review. The argument for putting it in would be that it is broken as is (at least for binary file upload, possibly in other ways as well), and if we can get agreement on the API changes, we can fix any remaining bugs in 3.2.1. However, making API changes at this point (post-beta) requires a significant exception to our normal development rules, and I don't like doing things this rushed and last minute. But I also don't like the thought of having FieldStorage be broken in 3.2. Georg, I'm really busy this week, and don't have time to do a review, unfortunately. If you think it worth considering putting it in, I can try to take a look at the API changes tomorrow, but unfortunately can make no promise to do so. Hopefully others can, if needed. -- nosy: +georg.brandl priority: high - release blocker ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue4953 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue4953] cgi module cannot handle POST with multipart/form-data in 3.x
Pierre Quentel pierre.quen...@gmail.com added the comment: Ok, thanks. Here is a summary of the API changes : - the argument fp passed to FieldStorage is either an instance of (a subclass of) io.TextIOBase with a buffer attribute for the underlying binary layer (thus, it can't be a StringIO instance) ; or an object with read() and readline() methods that return bytes Defaults to sys.stdin - 2 additional arguments can be passed to the FieldStorage constructor : . stream_encoding : the encoding used by the user agent to encode submitted data. It must be the same as the content-type of the HTML page where the form stands. Defaults to utf-8 . charset : the encoding used by the CGI script (the one used by the print() function to encode and send to sys.stdout). It must be the same as the charset in the content-type header sent by this script. Defaults to None, in which case the default encoding of sys.stdout is used - the only change in the object returned by FieldStorage() is that, if a field represents a file (its argument filename is not None), the read() method on this field returns bytes, and its attribute value is a bytestring, not a string -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue4953 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue4953] cgi module cannot handle POST with multipart/form-data in 3.x
Andy Harrington ahar...@luc.edu added the comment: I found a similar issue. If you want more simple files demonstrating the issue, I have attached some. If I start my localCGIServer.py, then I can use adder.html fine (uses get), but with adderpost.html (uses post) the cgi action file, adder.cgi (that worked fine with the get version) hangs. -- nosy: +andyharrington Added file: http://bugs.python.org/file20392/localCGIServer.py ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue4953 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue4953] cgi module cannot handle POST with multipart/form-data in 3.x
Changes by Andy Harrington ahar...@luc.edu: Removed file: http://bugs.python.org/file20392/localCGIServer.py ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue4953 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue4953] cgi module cannot handle POST with multipart/form-data in 3.x
Changes by Andy Harrington ahar...@luc.edu: Added file: http://bugs.python.org/file20393/adder.html ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue4953 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue4953] cgi module cannot handle POST with multipart/form-data in 3.x
Changes by Andy Harrington ahar...@luc.edu: Added file: http://bugs.python.org/file20394/adderpost.html ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue4953 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue4953] cgi module cannot handle POST with multipart/form-data in 3.x
Changes by Antoine Pitrou pit...@free.fr: -- nosy: -pitrou ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue4953 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue4953] cgi module cannot handle POST with multipart/form-data in 3.x
Changes by Andy Harrington ahar...@luc.edu: Added file: http://bugs.python.org/file20395/localCGIServer.py ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue4953 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue4953] cgi module cannot handle POST with multipart/form-data in 3.x
Changes by Andy Harrington ahar...@luc.edu: Added file: http://bugs.python.org/file20396/adder.cgi ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue4953 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue4953] cgi module cannot handle POST with multipart/form-data in 3.x
Antoine Pitrou pit...@free.fr added the comment: Ok, there are 10+ files attached, 20+ comments, no up-to-date patch. It's really too late for 3.2 IMO. -- nosy: +pitrou priority: release blocker - normal stage: patch review - needs patch versions: -Python 3.2 ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue4953 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue4953] cgi module cannot handle POST with multipart/form-data in 3.x
Pierre Quentel pierre.quen...@gmail.com added the comment: Comment ça, no up to date patch ? cgi_32.patch is up to date, the API changes are documented, the unittests work, what else do you want ? -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue4953 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue4953] cgi module cannot handle POST with multipart/form-data in 3.x
Antoine Pitrou pit...@free.fr added the comment: Comment ça, no up to date patch ? cgi_32.patch is up to date, the API changes are documented, the unittests work, what else do you want ? The O_BINARY stuff looks obsolete to me. -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue4953 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue4953] cgi module cannot handle POST with multipart/form-data in 3.x
Glenn Linderman v+pyt...@g.nevcal.com added the comment: The O_BINARY stuff was probably necessary because issue 10841 is not yet in the build Pierre was using? I agree it in not necessary with the fix for that issue, but neither does it hurt. It could be stripped out, if you think that is best, Antoine. But there is a working patch. -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue4953 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue4953] cgi module cannot handle POST with multipart/form-data in 3.x
Éric Araujo mer...@netwok.org added the comment: Can one person please 1) Sum up the discussion and its outcome briefly 2) Remove all patches and replace them with one diff with docs, tests and code (even if you have new files, you don’t have to put them in a zip, use svn add and they will show up in the svn diff, which is really easier to review) -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue4953 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue4953] cgi module cannot handle POST with multipart/form-data in 3.x
Changes by Éric Araujo mer...@netwok.org: Removed file: http://bugs.python.org/file20288/unnamed ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue4953 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue4953] cgi module cannot handle POST with multipart/form-data in 3.x
Etienne Robillard e...@gthcfoundation.org added the comment: +1 -- title: cgi module cannot handle POST with multipart/form-data in 3.x - cgi module cannot handle POST with multipart/form-data in 3.x ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue4953 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue4953] cgi module cannot handle POST with multipart/form-data in 3.x
STINNER Victor victor.stin...@haypocalc.com added the comment:
I tested cgi_32.patch on Windows with Apache:
- a test with a binary file works: I get a binary file instead of a text file
- a test with a non-ASCII character (a\xe9b) works: the text is correctly
decoded
I used the test script from full_source_and_error.zip.
Comments on cgi_32.patch:
- I don't understand why FieldStorage changes sys.stdout and sys.stderr (see
remarks about IOMix above): please remove the charset argument (it is also
confusing to have two encoding arguments). it should be done somewhere else
- please remove the O_BINARY hack: the patch is for Python 3.2 and I closed
issue #10841. If you would like a backport, another patch should be written
later
- encoding = 'latin-1' # ?: write a real comment or remove it
- 'self.fp.read(...) # bytes': you should add a test on the type if you are
not sure that fp.read() gives bytes
- file: the file(-like) object from which you can read the data *as bytes*:
you should mention that TextIOWrapper is also tolerated (accepted?)
- you may set fp directly to sys.stdin.buffer (instead of sys.stdin) if fp is
None (it will be easier after removing the O_BINARY thing)
- the patch adds a tab in an empty line, please don't do that :-)
- you should add a (private?) attribute to FieldStorage to decide if it works
on bytes or unicode, instead of using self.filename is not None test (eg.
self._use_bytes = (self.filename is not None)
- i don't like the idea of having a generic self.__write() method supporting
bytes and unicode. it would prefer two methods, eg. self.__write_text() and
self.__write_binary() (they can share a third private method)
- i don't like stream_encoding name: what is the stream here? do you
process a file, a string or a stream? why not just self.encoding?
- import email.parser,email.feedparser one import is useless here. I prefer
from email.feedparser import FeedParser because you get directly a
ImportError if the symbol is missing. And it's already faster to get FeedParser
instead of email.feedparser.FeedParser in a loop (dummy micro-optimization)
- even I like the following change, please do it in a separated patch:
-if type(value) is type([]):
+if isinstance(value,list):
I really don't like the IOMix thing:
- sys.stdout.write() should not accept bytes
- FieldStorage should not replace sys.stdout and sys.stderr: if you want to
set the encoding of these files, set PYTHONIOENCODING environment variable
before running your program (it changes also the encoding of sys.stdio)
- IOMix should not accept bytes *and* unicode. It's better to have an explicit
API like stdout.write('unicode') and stdout.buffer.write(b'bytes)
Most parts of the patch are correct and fix real bugs. Since cgi is broken
currently (eg. it doesn't handle binary files correctly), anything making the
situation better would be nice.
I vote +0 to commit the patch now (if the release manager agrees), and +1 if
all of my remarks are fixed.
--
___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue4953
___
___
Python-bugs-list mailing list
Unsubscribe:
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue4953] cgi module cannot handle POST with multipart/form-data in 3.x
Glenn Linderman v+pyt...@g.nevcal.com added the comment: Victor, thanks for your comments, and interest in this bug. Other than the existence of the charset parameter, and whether or not to include IOMix, I think all of the others could be fixed later, and do not hurt at present. So I will just comment on those two comments. I would prefer to see FieldStorage not have the charset attribute also, but I don't have the practice to produce an alternate patch, and I can see that it would be a convenience for some CGI scripts to specify that parameter, and have one API call do all the work necessary to adjust the IO streams, and read all the parameters, and then the rest of the logic of the web app can follow. Personally, I adjust the stdout/stderr streams earlier in my scripts, and only optionally call FieldStorage, if I determine the request needs such. I've been using IOMix for some months (I have a version for both Python 2 and 3), and it solves a real problem in generating web page data streams... the data stream should be bytes, but a lot of the data is manipulated using str, which would then need to be decoded. The default encoding of stdout is usually wrong, so must somehow be changed. And when you have chunks of bytes (in my experience usually from a database or file) to copy to the output stream, if your prior write was str, and then you write bytes to sys.stdout.binary, you have to also remember to flush the TextIOBuffer first. IOMix provides a convenient solution to all these problems, doing the flushing for you automatically, and just taking what comes and doing the right thing. If I hadn't already invented IOMix to help write web pages, I would want to :) -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue4953 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue4953] cgi module cannot handle POST with multipart/form-data in 3.x
Graham Dumpleton graham.dumple...@gmail.com added the comment: FWIW, keep in mind that cgi.FieldStorage is also quite often used in WSGI scripts in arbitrary WSGI servers which have got nothing to do with CGI. Having cgi.FieldStorage muck around with stdout/stderr under WSGI, even where using a CGI/WSGI bridge, would potentially be a bad thing to do, especially in embedded systems like mod_wsgi where sys.stdout and sys.stderr are replaced with file like objects that map onto Apache error logging. Even in non embedded systems, you could very well screw up any application logging done via stdout/stderr and break the application. So, the default or common code paths should never play with sys.stdout or sys.stderr. It is already a PITA that the implementation falls back to using sys.argv when QUERY_STRING isn't defined which also could produce strange results under a WSGI server. In other words, please don't go adding any more code which makes the wrong assumption that this is only used in CGI scripts. -- nosy: +grahamd ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue4953 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue4953] cgi module cannot handle POST with multipart/form-data in 3.x
Glenn Linderman v+pyt...@g.nevcal.com added the comment: Graham, Thanks for your comments. Fortunately, if the new charset parameter is not supplied, no mucking with stdout or stderr is done, which is the only reason I cannot argue strongly against the feature, which I would have implemented as a separate API... it doesn't get in the way if you don't use it. I would be happy to see the argv code removed, but it has been there longer than I have been a Python user, so I just live with it ... and don't pass arguments to my CGI scripts anyway. I've assumed that is some sort of a debug feature, but I also saw some code in the HTTPCGIServer and http.server that apparently, on some platforms, actually do pass parameters to CGI on the command lines. I would be happy to see that code removed too, but it also predates my Python experience. And no signs of if debug: by either of them! -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue4953 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
